Agent verify loops

An agent that cannot tell pass from fail is improvising. Give it a pass/fail signal — tests, types, linters, or a scripted repro — and it can close a loop: edit, run, read the failure, edit again. You define "done." You still review.

Walking through the loop

Plan first for multi-file work. Ask for a short plan (files touched, risks, test plan). Approve the plan, then implement in a fresh context if the exploration filled the window.

Small steps. One vertical slice at a time. Giant one-shot generations are hard to review and hard to revert.

Tests as guardrails. Write or generate failing tests that encode the behavior, then let the agent make them pass.

Verify by running. Prefer pnpm test, go test, a repro script, or a screenshot diff — not "the model said it works."

Task: rename UserService methods to match a new API across 40 call sites. You write a characterization test for one critical flow, give the agent the rename map and rg / codemod constraints, require pnpm test green, then spot-check auth call sites yourself. Mechanical work: agent. Judgment on auth: you.

Context windows fill with command output and failed attempts. When quality drops — repeated wrong files, ignored instructions, looping fixes — start a new session with a tighter brief and the files that matter. Throwing more tokens at a confused thread is how you get a confident mess. If two resets fail on the same task, the problem is usually underspecified product intent. Write the invariants yourself, then retry with those as the acceptance tests.

As of when this was written, people run several agent sessions on different branches or worktrees. That only works if branches are isolated, you still review the merged result as one change, and you watch cost. Start with one agent and a clean verify loop before you orchestrate a swarm.

Common mistakes

• "The model said it works." Run the test.
• One giant generation. Hard to review, hard to revert.
• Two agents editing the same files. Isolation first.
• A confused thread that you keep feeding. Reset with a tighter brief.

Check yourself

1. What is the pass/fail signal on a 40-file rename? A characterization test on a critical flow, plus pnpm test green — and your review of auth sites.

2. The agent loops on the same wrong file. What do you do? New session, tighter brief, only the files that matter. Two failed resets means you underspecified done.

Key takeaways

• The loop is edit → verify → edit. You define done.
• Plan, then implement, in small slices.
• Reset a wandering thread instead of adding tokens.
• One agent with a clean signal beats a swarm.

Further learning

Next, load the right constraints in prompting AI tools.