Agent-Assisted Code Review · ~9 min
Ten lessons, one reflex: name the review pathology in front of you, then reach for the move that resolves it — not a vaguer instruction to "review harder."
The through-line of the whole course: review throughput, not generation, is now the bottleneck — so spend reviewer attention where signal says it matters and stay silent everywhere else. Every move below is a way to buy signal back from volume.
| Symptom you're staring at | Move | From |
|---|---|---|
| PRs are piling up faster than anyone can read them | Treat review throughput as the bottleneck — measure value merged, not PRs opened; cut volume before refining the checklist | L1 |
| The AI reviewer floods every PR with nits | Set a confidence floor (e.g. ≥90%) and let silence be a valid output — 29% of Copilot reviews say nothing | L2 |
| "Find bugs" prompt gives shallow, random findings | Build a pipeline over the diff — classify, retrieve, reason, reproduce — not one LLM call | L3 |
| Every PR gets the same deep (or shallow) pass | Tiered review — route depth by risk and tune effort; calibrate the threshold, don't review everything hard | L4 |
| The reviewer wastes attention on cosmetic hunks | Label each hunk by what it touches so attention lands where structure says it matters | L5 |
| One big reviewer misses whole defect classes | Run several specialized reviewers in parallel, then synthesize one verdict — diversity beats one prompt | L6 |
| Plausible findings turn out false on inspection | Gate every finding behind a reproduction step so fabricated ones never reach the human | L7 |
| Humans and agents step on each other's work | Split by comparative advantage — agents triage breadth; humans judge intent and trade-offs | L8 |
| A 600-line agent PR with an empty description | Author for reviewability — slice small, describe the why, predict where review will choke; request a smaller PR past ~400 lines | L9 |
| The same nit gets written on every PR | Turn recurring comments into learned rules so the nit is never written twice | L10 |
The early lessons set the economics: AI raises PR volume faster than value, so the scarce resource is the reviewer's fixed budget of attention per PR. The middle lessons spend that budget well — an inspection pipeline that runs CI changes first (the cheapest hard stop) and demands evidence last, structure labels and a committee that point attention at the right hunks, and a reproduction gate that throws out plausible-but- false findings before a human ever sees them. The late lessons protect the budget over time — author PRs to be reviewable, and fold every recurring comment back into a rule. One discipline, applied at each layer: signal over volume.
Every backfire in this course rhymes. A wall of low-confidence comments trains you to ignore all AI output, burying the one critical finding. Each extra reviewer comment on an agent PR correlates with a 2.8-point drop in merge probability — read as required corrections, not productive alignment — so a checklist that surfaces ten findings amplifies ghosting. Past ~400 lines per diff, sustained review degrades to surface scanning regardless of checklist quality. When the checklist becomes a comfort blanket, removing PRs from the queue beats refining it.
Mixed review — across the whole course
Question 1 · from L7Reviewing an agent-authored PR mainly verifies…
Question 2 · from the playbookIn the ten-minute inspection order, the step that comes first is…
Question 3 · from L9When a diff exceeds roughly 400 lines, the right move is to…
Question 4 · from the tableOn an agent PR, each extra reviewer comment correlates with…
Question 5 · spaced recall from Signal Over VolumeAn AI reviewer that has nothing high-confidence to add should…