All loops
Claude CodeHard 11 min· claude-haiku-4-5

Flaky test triager

Auto-quarantines tests that fail >3× in 7 days and opens an issue.

Read the full Claude Code build guide →
NOT DEPLOYEDNOT DEPLOYED
0146ms
Trigger
POST https://hooks.claudeloops.run/flaky-test-triager · event=slack.slash:/loop
02926ms
Agent
claude-haiku-4-5 · in 826 tok · out 545 tok
03216ms
Tools
github-mcp/github:issues.createComment → 200 OK · 266ms
0436ms
Verify
schema check · pydantic v2 passed
0566ms
Output
changelog updated · v1.4.2
0626ms
Notify
audit log written · runbook link attached
SUCCESS
0%
0 runs
P50
0ms
median
P95
0ms
tail
AVG COST
per run
LAST OK
never
LAST FAIL
never
none
Latency · last 30 runs0 samples
no runs yet
Latest output · what your users see
Openacme/api #1846
feat: add /health endpoint
9 files+206-16
  • Ran full test suite → 218 passed, 0 failed (12.3s)
  • Coverage delta: +0.4% on `src/lib/*`
  • No breaking changes detected in public API
// press Test to run once · Watch live to keep streaming · Deploy to make it real
The problem

ci,tests

The outcome

Flaky tests poison CI signal and people stop trusting red builds.

Ingredients & skills

Secrets
  • ANTHROPIC_API_KEY
  • GITHUB_TOKEN
Providers
  • Anthropic
  • GitHub
MCP servers
  • github-mcp
#claude-code#github#automation

How it works

Auto-quarantines tests that fail >3× in 7 days and opens an issue.

Step 1

1 — Create the locker

Locker `flaky-test-triager` holds the GitHub token (repo scope) and the Anthropic key.

bash
locker create flaky-test-triager
locker set $_ ANTHROPIC_API_KEY=sk-ant-...
locker set $_ GITHUB_TOKEN=ghp_...
Step 2

2 — Wire the GitHub event

Subscribe to the relevant GitHub webhook through the `github-mcp` server. Claude only sees the event payload and the files it requests.

Step 3

3 — Ship the agent

Single TS file in `agents/`. Stateless; the locker is bound at runtime so no env editing for new repos.

One-line deploy

The button above runs the same command with your saved config. This is the raw CLI form.

bash
npx claudeloops deploy flaky-test-triager
https://hooks.claudeloops.run/flaky-test-triager

Related loops