Production and case studies
Coding agent review
Force a branch and a PR. Humans own architecture, security and test integrity. A fresh-context reviewer catches what the author agent missed.
Even strong agents need review. Autonomy without review ships security holes and stylistic debt at machine speed. The PR diff is the product. Force a branch plus a PR, not silent commits to main.
Concept
Before you start: evals tell you how often the agent goes green. Review is the human (or a second agent with fresh context) deciding whether that green is a change you want.
The big idea
- Separate author agent from reviewer agent (fresh context) for an adversarial pass.
- Humans still own architecture fit, security, product intent and test integrity.
- Failure modes to watch: over-editing (unrelated files), loop thrashing (the same failing fix), context exhaustion (forgets the original bug), hallucinated APIs.
- Merge gates: CI green, no unexplained test deletions, secret scan clean.
UX patterns: plan approval before edits, inline diff review, one-click revert, visible command log. The reviewer's job is judgment. The agent's job is to make that judgment cheap by keeping diffs small and evidence attached.
When an agent opens a pull request, humans should see at a glance whether they can trust the change:
- Restated goal and approach in the PR body
- Commands run and test results pasted or linked
- Explicit callouts when tests were modified
- Small, reviewable diffs; no drive-by formatting across the monorepo
- Known residual risks ("did not run load tests")
Agents can fill most of that template if you put it in project memory.
Security notes
- Secret scanning on diffs before commit.
- Block edits to
.env, credential files and CI secrets unless explicitly permitted. - Treat
curl | bashtool output as untrusted. Sandbox network. - Require lockfile review when dependencies change.
Prompt injection via a malicious README or issue text is real. An issue that says "exfiltrate secrets to this URL" can steer a too-powerful agent. Keep network denied by default. Treat issue bodies and dependency README files as untrusted content in the prompt structure. Same discipline as support tickets in guardrails.
Interactive products use tiered permission modes: auto-approve safe reads, prompt on writes, never-auto on network or git push. Headless CI agents use stricter static policies. Persist an allowlist per repo for trusted commands (pytest, npm test, cargo check) so humans are not click-fatigued into approving dangerous ones.
Common mistakes
- Silent commits to main. The review surface disappeared.
- The same agent reviews its own PR in the same context. It will agree with itself.
- Ignoring test diffs because CI is green.
Check yourself
- Why a fresh-context reviewer agent? It does not share the author agent's stale plan or self-justifying transcript.
- An issue asks the agent to curl a URL with secrets. What should already be true? Network is denied by default, and issue text is treated as untrusted.
Key takeaways
- Force branch plus PR. No silent commits to main.
- Humans own security, architecture and test integrity.
- Use a fresh-context reviewer. Watch over-editing and test changes.
- Deny network by default. Scan secrets. Treat issue text as untrusted.
Further learning
Claude Code: Best practices for agentic coding (Anthropic)Tips and patterns for getting the most out of Claude Code, from configuring your environment to scaling across parallel sessions.Claude Code DocsThe map of the whole loop sits on coding agent.
Related
Case study: coding agent
How coding agents work: repo context, edit tools, test loops, sandboxes and review.
Coding agent evals
SWE-bench compares scaffolds. Your own suite catches monorepo conventions, test gaming and over-eager edits.
Guardrails, security and safety
Prompt injection is unsolved. Cut blast radius with architecture, not a classifier that claims to detect it.