Case study: coding agent

Coding agents — Claude Code, Cursor agents, Codex-style loops, open SWE-bench harnesses — are LLM systems where the product is the agent loop: gather context, edit code, run checks, iterate. They work unusually well because code has verifiers (parsers, typecheckers, tests). Compared to support agents, coding agents usually get more autonomy because a failing test is an automatic critic. Humans still sit at plan approval, PR review and permission boundaries.

The same agent and infrastructure ideas apply, with sharper tools and sandboxes. Using existing tools well is covered in working with AI tools.

while not done and under budget:
  observe (files, test output, errors)
  think / plan
  act (search, read, edit, run)

Each iteration should append tool results as ordinary messages so the model sees ground truth. When context grows, compact older turns into a structured summary (files touched, tests run, open questions) rather than dropping the beginning silently. Lost requirements are a top failure mode.

Anthropic's SWE-bench scaffold philosophy: minimal scaffolding, strong tools, let the model decide the trajectory until it stops or hits context limits. Your job is to make tools reliable and feedback loud, then get out of the way.

Phases you still want as guidance, even if they are not a hard-coded state machine: understand, plan, edit, verify, iterate.

Concepts in this section

Read them in order. Each one builds on the one before.

Step	Concept	What it answers
1	Agentic code search	Grep and glob, or dump embeddings?
2	Patch edit tools	How does the agent change a file safely?
3	Coding agent evals	How do you know the scaffold got better?
4	Coding agent review	What must a human still own?

Local "yolo mode" that auto-approves everything is fine on a throwaway branch in a container. It is malpractice on a laptop with production cloud credentials in the environment. Minimum bar: OS-level sandbox or container, network off by default, workspace-scoped filesystem, permission prompts for high-impact actions, timeouts and max tool-call budgets.

Common mistakes

A clever prompt and weak tools. Harness quality moves SWE-bench as much as the model.
Open-ended credentials on a laptop. The sandbox is the product.
Reviewing only product code while the agent rewrote the tests.

Key takeaways

Coding agents are model plus scaffold: observe → act → verify, with stop budgets.
Prefer grep, glob and read over a blind embedding dump.
Unique-match patches with clear errors beat sloppy full-file rewrites.
Run tests in a sandbox with permission tiers.
Evaluate with SWE-bench-style tasks plus internal suites. Watch for test gaming.
Human review owns security, architecture and test integrity.