How the Claude Code Team Does Code Review
Anthropic webinar, 18 November 2026, now on demand
Table of Contents
- Event
- The claim
- Questions to bring
- How does a check know it covered anything?
- What stops a tuned reviewer from converging on what it already finds?
- Is risk computed from the diff, or from what the diff touches?
- What reviews the artifact that is not in the diff?
- How are incidents turned into evals without overfitting to the incident?
- Is revert rate a measurement or a target?
- Related research on this site
- Notes
Event
| Field | Value |
|---|---|
| Webinar | How the Claude Code team does code review |
| Host | Anthropic |
| Speaker | Morgan Lunt, PM on Claude Code |
| Date | EST |
| Format | Virtual; on demand afterwards, behind a registration form |
| URL | claude.com/resources/webinars |
The claim
"When agents write the code, pull requests pile up faster than anyone can read them." The headline number: Anthropic cut its revert rate by more than 40% while merged pull requests more than doubled, with most of the reading moved to machines and human attention aimed where the risk is.
Five things the page says it covers:
- verifying work before a pull request exists — checks Claude can run, plus a fresh-context review of the diff
- tuning the AI reviewer with
CLAUDE.mdandREVIEW.mdso findings stay accurate and review rounds converge - scaling human approval by risk, with tiers set by the change rather than the author
- turning past incidents and reverts into an eval that improves the reviewer over time
- measuring human review time, with revert and incident rates as the guardrail
Questions to bring
Each of these comes from a failure in this repo rather than from the abstract, so the answers are checkable here.
How does a check know it covered anything?
The first layer is "checks Claude can run" before a pull request exists. The
failure mode is not a check that breaks, it is a check that passes while
covering nothing. In one session here, cljs-check inspected 2 of 24
bundles because the other 22 were never in its list; deploy-check printed
✗ Mismatch and exited 0; and verify-canonical discarded its own verdict
with || true. All three were named as gates. What makes a check in this
layer report what it did not look at?
What stops a tuned reviewer from converging on what it already finds?
REVIEW.md tuning promises findings that "stay accurate" and rounds that
"converge". Convergence is also what a reviewer does when it has learned the
shape of past findings and stopped seeing anything else. Is there a negative
control — a fixture each rule must flag, failing the suite if it does
not? Two linters here have that; no deploy check did.
Is risk computed from the diff, or from what the diff touches?
Tiers "set by the change rather than the author" is the right axis. But the
riskiest change in a recent week here was one character — adding css to
a FilesMatch in .htaccess, which governs cache policy for the whole
site — and the most damaging was a change with no diff at all: a
committed build artifact that quietly stopped matching its sources. A
line-count or file-count proxy sees neither.
What reviews the artifact that is not in the diff?
Four times in one week a committed CLJS bundle was stale: make reported
the target current because a rebase gave the sources a newer mtime than the
build. The diff was clean every time. Does the review see generated or
committed artifacts, and can it tell "unchanged because correct" from
"unchanged because never rebuilt"?
How are incidents turned into evals without overfitting to the incident?
This is the most interesting of the five, and the hardest. A revert becomes a regression test easily; becoming an eval that catches the next member of the same class is a different thing. What does the generalisation step look like, and who does it?
Is revert rate a measurement or a target?
A 40% drop is the headline, and revert rate is offered as the guardrail on review time. It also falls when people stop reverting — when a fix-forward norm takes hold, or when reverting a large agent-authored change becomes costly enough that nobody does it. What is the counter-metric that would distinguish the two? This is the proxy problem Terence Tao put at the centre of Math 2.0: a measure that was a good proxy under one regime becomes misaligned under another, and optimising it then does harm.
Notes
Notes pending.