AI

AI Is Changing Code Review. Engineering Leaders Need to Catch Up.

AI has made writing code cheaper. But most engineering organizations still use processes designed for a world where writing code was the bottleneck.

Will Larson’s experiment at Imprint illustrates the problem: engineers are merging four times as many pull requests as two years ago, yet median review time has grown from 33 to 77 minutes.
Meanwhile, reported incidents have declined.

More code, longer reviews, fewer incidents.
The bottleneck has moved.

Stop reviewing everything the same way

Larson’s solution is straightforward: introduce a fast path for low-risk changes that meet strict, machine-verified criteria. Small, isolated changes with passing checks can qualify for automated approval; sensitive or complex changes still require human review.

Only about 10% of changes qualified in the early experiment.
That’s fine.
The point isn’t to eliminate code review.
It’s to stop spending expensive human attention where it adds little value.

This is a broader lesson about AI adoption: don’t just automate existing workflows. Redesign them around the new economics.

Invest in verification, not more review

Adding AI reviewers that comment on code, followed by other agents that respond to those comments, can create an elaborate simulation of human review without necessarily improving software quality.

The better investment is in automated verification:
* strong tests
* static analysis
* security checks
* reproducible builds
* explicit risk boundaries.

AI can generate code.
Your engineering system must establish whether that code is safe to ship.

Human judgment remains essential, but its highest-value role shifts toward defining policies, handling ambiguous decisions, and improving the system when failures expose gaps.

What to do

  • Measure the bottleneck.
    Track review latency, PR throughput, escaped defects, incidents, and engineering time spent reviewing AI-generated code.
  • Create a low-risk fast path.
    Define explicit eligibility rules for small, isolated changes. Automate approval only when required checks pass.
  • Strengthen verification.
    Invest in test coverage, static analysis, security scanning, and independent checks that provide evidence beyond an AI reviewer’s opinion.
  • Keep humans on high-risk changes.
    Require deeper scrutiny for security-sensitive code, shared infrastructure, critical business logic, and broad-impact changes.
  • Run a controlled experiment.
    Start with one team or repository. Measure cycle time and quality before expanding the policy.
  • Learn from failures.
    When an automated change causes an incident, improve the checks and eligibility rules instead of relying solely on more human vigilance.
  • Measure outcomes, not AI activity.
    Optimize for safely shipped software, shorter lead times, and fewer defects—not lines of AI-generated code or PRs merged.

The bottom line: AI won’t deliver its full productivity gains if your engineering organization keeps the same bottlenecks.
The competitive advantage belongs to teams that redesign the entire software delivery system around fast, verifiable change.

Be strong.


Discover more from Ido Green

Subscribe to get the latest posts sent to your email.

Standard

Leave a Reply