The security review that can't keep up

Most of what gets said about AI authorship and code review assumes a reviewer being asked, for the first time, to vouch for code they did not write — to read unfamiliar work, by an author whose reasoning they can’t reconstruct, and put their name under it. Security review starts from the far end of that discomfort and lives there. Reading code you didn’t write, by an author whose competence you cannot assume, hunting for the failure the author never intended — that was the job before any of this. The application-security function is the one review discipline that was always adversarial, always reading code as guilty until cleared, always assuming the diff in front of it was hiding something. By the logic that makes AI authorship hard for everyone else, security review is the part of the org that should barely notice.

It is instead the part that breaks first. Not because the work got harder in kind — the security reviewer was already doing the most paranoid version of review there is — but because three conditions that made the work possible at the current size of the team went away at the same time, and none of them was ever written into the job description. Security review did not survive on the reviewer being tougher than everyone else. It survived on a throttle, an interlocutor, and a gradient. AI removed all three at once, and left the function with the same headcount pointed at a different problem.

1. The throttle that’s gone

Security review was never sized to the codebase. It was sized to the rate the codebase changed, and that rate had a governor bolted to it: code was produced by people typing, and people typing is slow and expensive. However large the appsec function was, it was large relative to a flow of new code that moved at human-authorship speed. The ratio held because both sides of it were tied to the same quantity — more engineers meant more code to review and more budget to review it, and the two grew together. Nobody designed this coupling. It was just true, the way the water level is the same on both sides of an open lock.

AI broke the coupling on one side only. Production throughput is now decoupled from headcount; audit throughput is not. A security reviewer reads at the same speed they always did, and there are the same number of them, and the volume arriving at their queue is set by a different equation than the one that sizes their team. The function becomes the system bottleneck — and a bottleneck under load has exactly one lever it can pull without hiring: review a smaller fraction of what comes through.

That lever has respectable names. Sampling. Risk-tiering. Spot checks. Reserving deep review for diffs that touch flagged surfaces. For almost every other review discipline this is a defensible degradation — you inspect a representative slice and the misses are distributed like the average, so a sample tells you something true about the whole. Security is the one domain where the misses are not distributed like the average, because there is an adversary selecting them. This is the shrinking sample, and it fails in a way no other sampled process does. A codebase that is ninety percent audited is not ninety percent secure. It is as secure as the least-secure thing in the unaudited tenth, because that is precisely where the attack goes. Coverage and security are the same number only when coverage is total. Below total, an adversary opens the gap between them as wide as the unreviewed surface allows, and does it deliberately, because finding the one diff you didn’t look at is the job on the other side.

So the throughput mismatch does not surface as an honest backlog that everyone can see growing. It surfaces as a coverage figure that quietly stops meaning what it used to, on a dashboard that looks healthier than ever — fewer findings per diff, faster turnaround, a cleaner trend line — for the exact reason that less is being looked at. The throttle that made the staffing ratio work is gone, and nothing on the reporting surface is shaped to show its absence.

2. Plausible-and-exploitable

Security review was never only a reading. It was an interrogation with a respondent. The reviewer asked what the author assumed about this input, why this boundary was treated as trusted, what happens when the value arrives hostile instead of well-formed — and the author, who had made those calls deliberately or could at least be pinned to them, answered. The answers were where the threat model actually lived. The diff showed what the code did; the conversation exposed what the author believed, and vulnerabilities live in the gap between the two. You cannot read a false trust assumption off a diff, because correct-looking code and code resting on a wrong belief about its inputs are the same text. You could, however, ask.

The respondent is gone. The code’s author optimized for one property — output that resembles the code that usually solves this — and adversarial resistance is not visible anywhere in that objective. Worse than invisible: it is frequently opposed to it. Security is almost definitionally a deviation from the convenient construction. The string-concatenated query is the normal-looking one; the parameterized version is what a careful author adds on top. The endpoint that trusts its caller is the shorter one; the check is the extra step. The deserializer that accepts the whole object is the idiomatic one; the allowlist is the deliberate narrowing. The training distribution is the average case, and the average case is repeatedly the insecure case, because securing it was always the additional move a disciplined author made against the path of least resistance.

This is a different axis from whether the code is correct, and the difference is the whole point. A diff can do exactly what was intended, pass every test, satisfy the spec to the letter, and hand an adversary precisely what they came for — those are independent properties of the same change. Call it plausible-and-exploitable: output that is functionally right and idiomatically clean and carries a vulnerability not as an error the model made but as the median solution to the problem it was given. The flaw is not a mistake in the work. It is the work, done the common way. A reviewer who has cleared the diff as correct has not begun the security review — and the cleanliness that earned the correctness pass is the same cleanliness that will keep the exploit from showing. The smooth surface is not covering for a sloppy author who got unlucky. There is no author, and the smoothness is the default, including over the hole.

3. The unauditable diff

A security reviewer does not read every line with equal attention. Nobody can, and at any real diff size the attempt collapses into reading none of it carefully. The reviewer triages — scarce suspicion gets aimed, and what aims it is a gradient. The constructions that look unusual, hand-rolled, improvised past the author’s depth are where the instinct says stop here. Hand-rolled crypto draws the eye. A manual parse where a library should be draws the eye. A bespoke escaping routine, an off-pattern session check, a clever bit of pointer arithmetic — the texture of someone working past their competence is the texture a reviewer is trained to slow down on. That instinct is not really reading the code. It is reading the author through the code, and it works because human error has signatures: people are weakest where they improvise, and improvisation looks different from fluency on the page.

AI output has no such signatures, or rather it has precisely one — uniform idiomatic competence, laid down as evenly over the part the model got wrong as over the part it got right. The crypto reads as clean as the logging. The vulnerable boundary looks as practiced as the safe one. The dangerous deserialization is written with the same unhesitating fluency as the trivial getter beside it. The gradient the reviewer used to triage a large diff is flat. Every line is equally plausible, which means every line is equally suspect, which at volume is operationally identical to no line being suspect at all — because attention that cannot concentrate is attention that cannot be spent.

This is the unauditable diff, and the name is doing something specific. It does not mean a diff containing a line no human could understand. Any single line, given an afternoon, can be understood. It means a diff that offers no signal about which lines deserve the afternoon, arriving at a rate that forbids giving the afternoon to all of them. The reviewer’s competence is intact; their tools are intact; what’s gone is the thing upstream of both — the read on where the danger probably is. Strip the targeting out of a function whose entire economy was the targeting of a scarce resource, and you have not made it slower. You have made it a process that runs and produces a verdict and is no longer connected to the thing it was verifying.

Closing

Security review looked immune to the AI-authorship problem because it had already metabolized the parts that everyone else was just now choking on — the hostile stance, the unfamiliar code, the missing benefit of the doubt. What it had not metabolized, because nothing had ever threatened all three at once, were the three quiet affordances underneath the job: a throttle that kept production inside audit range, an interlocutor who could be made to account for the threat model, and a gradient that told scarce suspicion where to land. None was written down. None was understood as load-bearing. AI removed all three in a single motion and left the function staffed, tooled, and mandated exactly as before, with a firehose where the throttle used to be and no author at the other end of the line.

The teams that see this stop trying to review their way out of it, because none of the three losses is reviewable. The volume cannot be out-read; the missing author cannot be subpoenaed; the flat gradient cannot be re-steepened by concentrating harder. The move is to push security down into the substrate, where it holds regardless of who or what produced the diff — invariants enforced by construction instead of caught by inspection, analysis that assumes generated code and does not wait on a human to smell the flaw, trust boundaries that fail closed with no reviewer in the loop. Security becomes a property of the ground the code stands on rather than a verdict a person renders on each change, because the person-per-diff model is the specific thing that broke.

The teams that don’t will keep reporting a security review that is, on every visible measure, performing better than it ever has — faster, cleaner, fewer findings per merge — and they will be telling the truth about the measures while the measures stop describing the thing they were built to track. The most dangerous number in an AI-authored codebase is a falling vulnerability count that everyone reads as a rising bar. Fewer findings is what a more secure codebase looks like. It is also what an unaudited one looks like. The dashboard cannot tell you which you have. The adversary can, and will.