How Much Human Review Is Enough When Agents Write the Code?
A careful reviewer can read fewer than 500 lines of code an hour. If they go faster than that, or keep reviewing for much longer than an hour, they start to miss more defects. These limits come from SmartBear's study of code review at a Cisco programming team, which was done long before coding agents existed.
That limit is the same today, but the amount of code waiting for review is much larger. Suppose each person on a team of six spends one focused hour a day on review. At that pace the whole team can read 3,000 lines a day at most with real attention. One engineer running a few agents in parallel can produce more code than that alone. I worked through that math in the parallel agents post.
Most teams still have a written rule that a human reviews every change. In practice that rule is no longer being followed. Faros AI's 2026 report, built on telemetry from 22,000 developers, found a 31.3% rise in pull requests merged with no review as teams moved from low to high AI adoption. Its follow-up six months later says that figure has grown to 76.3%. LinearB's 2026 benchmarks, covering 8.1 million pull requests, found AI-assisted PRs to be 2.6x larger than unassisted ones: over 400 lines against 157 at the 75th percentile. These numbers point the same way: there is more code in the review queue than reviewers can read. Since some changes will merge without a careful human read anyway, I would rather decide in advance which ones those are.
Treat review as a budget
In a team that uses agents heavily, reviewer attention is the resource that runs out first. You can't add more of it quickly, and its quality drops when people review for too long. I find it useful to treat it like a budget and decide where it gets spent.
Reviewing every change the same way is also a spending decision, and not a good one. A README typo and a change to the permission check each get the same ten minutes, which is more than the typo needs and much less than the permission check needs.
This gives a practical answer to "how much review is enough?" The amount of review should match how much damage the change would do if it were wrong. I'd estimate that from three things, and all three can be checked before reading any code.
- How far the damage spreads (often called blast radius). If the change is wrong, does it affect a page of docs, one internal tool, or every customer's invoice?
- How easy it is to undo. Can I undo it in a minute with a feature flag or a revert, or is it a data migration that cannot be rolled back?
- Evidence that does not come from the author. Is there proof that the change works, other than tests the same agent wrote for its own code? Examples are tests I wrote myself, type checks, contract tests, or a real run against staging.
When a change affects little, is easy to undo, and has strong evidence behind it, a human reading every line adds very little. When it affects a lot, is hard to undo, and has weak evidence, it should not merge on automated checks alone, however good those checks are.
Three lanes
The system I'd set up has three lanes. CI assigns the lane from the file paths a change touches, so the author, human or agent, does not get to choose how much review their own PR receives.
| Lane 1: Evidence only | Lane 2: Review the intent | Lane 3: Read every line | |
|---|---|---|---|
| Typical changes | Docs, copy, formatting, code behind a disabled flag | Feature work, bug fixes, dependency updates, refactors inside one service | Auth, payments, data migrations, infra, CI config, the review rules themselves |
| If it's wrong | Small, easy to notice, undone in minutes | Limited to one service, undone with a revert | Costly, hard to notice, or impossible to undo |
| What gates it | CI, types, tests, automated reviewer | All of lane 1, plus one human | All of lane 2, plus a second human who knows that system |
| What the human does | Audits a sample each week | Checks the goal, the approach, and the evidence | Reads every line of the diff |
The lanes are defined in a file of path patterns that CI reads. If a change matches more than one lane, it goes to the strictest one. For example:
# review-lanes.yml - a change takes the highest lane any of its files match
lane3:
- "services/auth/**"
- "services/billing/**"
- "db/migrations/**"
- "infra/**"
- ".github/**"
- "CLAUDE.md"
- ".claude/**"
- "review-lanes.yml"
lane1:
- "docs/**"
- "**/*.md"
default: lane2
A change that touches one file in services/auth and forty in docs is a lane 3 change. Any path that is not listed goes to lane 2. So if I forget to classify a path, the worst result is one extra human review, and the change is still reviewed.
Lane 1 is the lane people worry about most, and it is also where most of the saved time comes from. When reviewers are busy, these changes already get a quick look and an approval. Once the lane has a name, you can put proper automated checks on it and move the hours it saves to lane 3. The automated reviewer is the main check here, so its instructions need care. I'd tell it to compare the change with the original ticket instead of the PR description, because the author wrote the description.
Dependency updates are not in lane 1, even though they look mechanical. They bring in code that nobody on the team wrote, and passing tests say little about what else that code does. I'd keep them in lane 2.
In lane 2 the reviewer reads fewer lines and spends more time on judgment. They check that the change solves the right problem, that the approach makes sense, and that the evidence shows it works. In practice that means reading the tests closely and reading only the parts of the implementation that look risky.
Lane 3 should get more attention than it gets today, and the time saved in lane 1 is what pays for it. I'd keep these diffs small, give the reviewer time to read them without rushing, and pick a reviewer who has been on call for that system.
The file that defines the lanes has to be in lane 3 too, which is why review-lanes.yml lists itself. Otherwise an agent could edit the path list and move its own change into the lane that merges without a human. The agent's instruction files are in lane 3 for the same reason. CLAUDE.md is a markdown file, and without that entry the **/*.md pattern would treat a change to the agent's rules as a docs edit.
A lighter lane does not change who is responsible. In the earlier review post I argued that whoever opens a PR owns every line of it, and that still holds in lane 1. The person who started the agent owns the change after it merges, including when it breaks.
Two habits that reduce the review load
The lanes decide which changes get human attention. Two habits reduce how much attention each change needs once it reaches a reviewer. A third, reviewing the agent's plan before any code exists, is covered in the spec-driven development post.
Make the PR carry its own evidence
A diff shows what changed, but it does not show whether anyone checked that the change works. So I want a PR written by an agent to include:
- The commands that were run, and their actual output
- A screenshot or recording for anything visual
- What was not verified, stated plainly
- Anything the agent was unsure about
The last two items are the most useful to me. If the PR says "I did not test the retry path", I know where to spend my review time. If it only says that everything passed, I have no idea where to look.
Stop generating more than you can review
If the review queue grows every day, the answer is to run fewer agents at once. Reviewing faster only means reviewing less carefully. A change that waits for review starts to conflict with other changes, and the longer it waits, the more likely the reviewer is to approve it quickly just to clear it. The test I use is the one from the parallel agents post: if an agent PR has been open for more than a day, I'm already running too many agents.
Check that a human is actually in the loop
The repository for this site runs a small version of the lanes. Hooks check every action an agent takes before it happens. A new post only has to pass an editorial lint. A commit has to pass lint and typecheck first. Editing a post that is already published needs a person to approve that specific change. The policy files, the CI config and the lockfile can't be edited by an agent at all. I described the full setup in the policy post.
When I tested it, the rule that failed was the one that depended on a person. The hook for published posts works by escalating the edit to an approval prompt. I pointed a worker agent at a published post and told it to change the title. The hook did its job and escalated twice, but the edit went through anyway, and the title of my first post was changed in the working tree.
The session was running in a mode where approval prompts can be answered by an automated classifier, so the approval I thought came from a person came from software. The logs showed that the hook escalated, but they did not record who or what approved the edit.
I suspect a lot of human oversight fails the same way. The approval step is still there, but it no longer means that someone looked. Either software answers it, or a tired person clicks Approve on a long queue of PRs without opening them. On paper the process still has a human in it.
Two checks worth running on your own setup:
- Who can answer each approval? If a bot, a classifier, or the author of the change can approve it, then it is an automated check and should be called one. That can be fine for lane 1, as long as nobody counts it as human review.
- Is the answer recorded? Log who approved and how long they spent. If an approval arrives eight seconds after a 900-line diff was opened, the log should make it obvious that nobody read it.
How to know it's enough
The first version of the lanes will be wrong in places. I'd adjust them using four numbers, tracked separately for each lane:
- Escaped defects. Bugs that reached production, counted by the lane their change went through.
- Revert rate. How often a merged change gets undone.
- Time to undo. How long it takes from noticing a bad change to having it reverted or switched off.
- Review catch rate. How often human review in that lane finds something real.
If lane 1 goes a full quarter with clean audits and no bugs reaching production, add more paths to it. If an audit finds a problem, remove that kind of change from the lane and add the check that would have caught it. If lane 3 reviews never find anything, then either that code is safer than I thought or the reviews are not careful enough, and I'd want to know which one it is.
I don't have target values for these numbers that would fit every team, because they depend on how much a production bug costs you. What I'd look at is the trend. Review is enough when few bugs reach production, the ones that do are cheap to fix, and most of them come from the lanes where you decided to accept more risk.
Where to start
I'd start with three things, and each one can be done this week.
- Write your lane 3 list. These are the paths where a wrong change is costly or cannot be undone. Give each path a named person who owns it.
- Change the PR template for agent work. Add "what was run," "what was not verified," and "where I'm unsure."
- Look at one week of approvals. Count how many were given less than a minute after the PR was opened. Those changes were approved but probably not read, so this count shows how much review is really happening today.
If you only do one of these, do the third. Agents have made code much cheaper to write, but a wrong change costs as much as it did before. So I think the job now is to decide which changes a person has to read, and then to check that a person really reads them.
References
- SmartBear: Best Practices for Peer Code Review - the Cisco study: under 500 lines an hour, and performance drops after about 60 minutes
- Faros AI: The Speed Trap - the 31.3% and 76.3% figures on PRs merged without review, from 22,000 developers
- LinearB: 8 million pull requests - AI-assisted PRs 2.6x larger, over 400 lines against 157 at the 75th percentile
- Code Review When the Author Is an AI Agent - how to read a single agent-authored diff
- Policy as Code for Coding Agents - the hooks and the approval gate that got auto-answered
- Running Coding Agents in Parallel - why review throughput caps how many agents are worth running
- Spec-Driven Development, Without the Hype - reviewing the plan before the code
Keep reading
Code Review When the Author Is an AI Agent
When writing code is cheap, reviewing it is where all the value goes. Most teams are still running a review process built for a world that no longer exists. Here's what changes when a machine wrote the diff.
Policy as Code for Coding Agents
CLAUDE.md is a suggestion. Permission rules and hooks are enforcement. I built a policy layer for this repo in an afternoon, then pointed six agents at it with instructions to break it. Here's every file, what held, what didn't, the seven times it blocked me, and the layer above the project that refused to let an agent weaken its own rules.
Running Coding Agents in Parallel: The Bottleneck Moved
Git worktrees made it trivial to run five agents at once, and every tool now ships a fleet mode. But the numbers say parallel agents make teams slower, not faster. Here's what actually breaks at N > 1, and how to size a fleet to the only limit that matters.