Look, the Agile Manifesto has a nice phrase: “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.” But no one followed it. We ended up measuring the “velocity” of the team with what we had on hand: estimation points at the end of the sprint, accumulated PRs, features delivered per week. And I’ve heard more than one technical leader swear that they measure ”commits per day” and complain that their team is “slow.” Slow. With a CI that takes fifteen minutes and reviews that are left until Friday afternoon.
The intention is noble, don’t get me wrong. We want rhythm, we want predictability, we want a sign that things are moving forward. The problem is that none of these measurements tell you what we think they tell us. What we end up measuring rarely has to do with delivered value and almost always has to do with observable activity. And observable activity, in 2026, with language models capable of generating 800 lines in thirty seconds, is about to become a completely empty metric.
#.The mirage of productivity
Software engineering has an unhealthy relationship with vanity metrics. We call them that because they feel good, they look good on a slide, but they don’t tell you anything about the actual health of the system. LOC was the first. Then the commits per day, the story points, the velocity… each decade invents its own and, after a while, leaves it gamified and empty.
The interesting thing is that AI arrived and, instead of abandoning the habit, we put new clothes on it. Now we measure “Copilot acceptance rate”, “AI-generated LOC”, “prompts per developer”, “time saved by AI” estimated by the tool we are evaluating itself — even tokens consumed. It’s the same mechanics: a number that goes up is a number that calms you.
There is a law that we should frame on the wall of every office: Goodhart’s law. In its original formulation, by economist Charles Goodhart in 1975, it says something like “any observed statistical regularity tends to collapse once pressure is put on it for control purposes” fn-1. We owe the popular, more scathing version to the anthropologist Marilyn Strathern in 1997: “When a measure becomes a target, it ceases to be a good measure” fn-2. When the measure becomes the objective, it is no longer a good measure. Period. Fred Brooks, long before, had already put his finger on the same issue when talking about the man-month in The Mythical Man-Month: “The man-month as a unit for measuring the size of a job is a dangerous and deceptive myth” fn-3. A dangerous and misleading myth. He didn’t write that with LOC in mind, but the spirit is the same: reducing engineering work to a mechanical unit of counting is to confuse the map with the territory.
#.What DORA had already told us
If there is one book that should have killed the debate over what to measure, it is Accelerate, by Nicole Forsgren, Jez Humble and Gene Kim, published in 2018 fn-4. Years of research on thousands of teams, analyzed with statistical rigor, condensed into a handful of metrics that do correlate with organizational performance.
The famous DORA. Originally there were four: change lead time, deployment frequency, change failure rate and time to restore service. That quartet became canonical. What many people haven’t noticed is that DORAs evolved. The current guide published by dora.dev fn-5 refines the names of the classic quartet and adds an additional metric focused on the rework of deployments: the proportion of deployments that end up requiring correction or repetition. The body of knowledge is still alive, and that is healthy.
What hasn’t changed is the core idea: these metrics don’t measure how much you write, they measure how well the entire system delivers. And there is one property of DORA that I find especially valuable today: they are resistant to throughput inflation. You can double commits per day without moving a single one of these metrics in the right direction. In fact, you will most likely make them worse.
In my previous post, Piano piano si va lontano, I talked about sustainable speed: that constant rhythm, indefinitely, that the eighth principle of the Agile Manifesto talked about. DORA is, in a sense, the instrumental that tells you if you are going at that pace or if you are accelerating full throttle only to come to a screeching halt three months later.
#.Why AI makes DORA matter ten times more
Here comes the awkward part. The AI raises the throughput. A lot. Code generation, which used to be the bottleneck, is almost no longer the bottleneck. But the other steps in the flow—integration, review, deployment, recovery—don’t scale at the same pace. They don’t climb, in fact, almost at all. Reviewing a well-made PR takes the same time as it did five years ago. In fact, it may take even longer, because you have to be more careful with what the AI wrote: in some cases what seems like a reasonable solution is actually a detour, an alternative route that works for the happy case and silently fails in all the others. Understanding an incident in production at 3 AM takes the same time as it did fifteen minutes ago. The pattern is what one would expect: apparent velocity goes up, activity metrics improve, and everything looks good at the quarterly meeting. What happens in parallel, silently, is that the metrics that do matter — stability, failure rate, recovery time — start moving in the wrong direction. Not catastrophically, but little by little. And when something breaks, the archeology of the PR that introduced it usually reveals the same thing: one reviewer, few minutes, zero comments.
The worrying pattern could be summarized as “the 800-line PR passed in three minutes.” The mechanics are simple: someone uses an AI assistant, gets a diff that touches many files, the tests are green (the assistant also wrote them, against the same mental model), CI passes, the reviewer rolls his eyes over them, passes. The loop is closed: the implementation and the tests encode the same blind spots. Error paths that the model did not consider are neither implemented nor tested. The regression appears as a “bug in production” a week later.
The change failure rate is the metric that best captures this. The formula is simple: CFR = (deployments that caused incident / rollback / hotfix) divided by (total deployments), in a rolling window of 30 or 90 days. DORA’s directional bands: Elite below 5%, High around 10%, Medium between 10% and 15%, Low above 15%. This metric is beautiful because it doesn’t get better if you send more: it only gets better if you send better.
The DORA 2025 report, with thousands of respondents, has a phrase that I am going to steal until I wear it out
“AI doesn’t fix a team; it amplifies what’s already there.”
AI does not fix a team; it amplifies what already exists. If the team had review discipline, good testing, a reasonable architecture, AI is going to give it superpowers. If you had chronic shortcuts, performative reviews, and smoke tests disguised as a suite, the AI is going to give you superpowers too, but in the opposite direction. And here it is convenient to be honest with the evidence. That GitHub Copilot study that everyone is talking about, the “55.8% faster” fn-7 — yes, the one published by GitHub — measured a single task, an HTTP server in JavaScript, with 95 participants and a 95% confidence interval between 21% and 89%. It’s a task, not a job. As a counterbalance, the 2025 METR study fn-8 did an RCT on 16 experienced open source maintainers, 246 real tasks on large repositories that the participants already knew. The result was uncomfortable: 19% slower with AI tools, while they perceived to be 20% faster. The difference between measured productivity and perceived productivity is part of the problem we are trying to solve.
I’m not telling you that AI-generated code is objectively worse. The evidence is mixed and continues to emerge. What I am telling you is that, in a world where throughput is trivially amplifiable, the metrics that matter are the ones that measure everything else.
#.The missing layer: understanding metrics
DORA is essential, but it is not enough. DORA measures system delivery; It does not measure whether the team understands what it delivers. And that, I think, is the new front.
In 2021, Forsgren returned to the fray with Storey, Maddila, Zimmermann, Houck and Butler, publishing “The SPACE of Developer Productivity” on ACM Queue fn-9. The central thesis of the paper is that productivity is not captured with a single dimension, much less with an activity metric. They propose five: Ssatisfaction and well-being, Performance, Activity, Communication and collaboration, and Efficiency and flow. The idea is not to measure all five exhaustively, but to choose at least one per dimension and resist the temptation to reduce everything to activity, which is the dimension that is most similar to the old LOC.
More recently, in December 2024, Abi Noda published DX Core 4 fn-10, a framework that attempts to unify DORA, SPACE, and DevEx into four practical dimensions: Speed, Effectiveness, Quality, and Impact. SPACE and DORA do not replace each other: they complement each other. DX Core 4 is an honest attempt to give leaders a reasonable dashboard without having to choose between rival frameworks.
Against that background, these are the signs of understanding that I’ve been seeing lately. It is not a closed catalog — I am almost certainly forgetting some and I am finishing some of them in my head:1. Short-lived code and churn. What percentage of the code is removed or rewritten in the first N days after being merged? In healthy repositories, with N = 30 days, 10-20% early churn is normal. What begins to be symptomatic is when the churn triggers. The 2025 GitClear report fn-11 (with the caveat that it is a commercial vendor and should be read with a grain of salt) shows that code reviewed within two weeks went from 5.5% in 2020 to 7.9% in 2024, and lines copied/pasted from 8.3% to 12.3%. Anecdotally, I’ve seen it: AI-generated code tends to churn because the first version solves the prompt, the second solves the real requirement when reality hits, and the third refactors because the previous two diverged in style. Three rewrites of the same function in six weeks, without the code taking “shape”.
2. Depth of review. Are the PRs being reviewed, or are they simply being approved? The warning signs are clear: PRs of more than 500 LOC approved in less than five minutes, comments per PR trending towards zero, approval rate with a single reviewer going up, time-to-first-review going down while PR size goes up. That last pattern is the most telling: when the system approves larger and larger code faster and faster, something broke.
3. Distribution of knowledge. How many people on the team could answer, without help, a question about this module? The old bus factor, but applied to specific pieces of the system. Teams with impeccable DORA metrics can have a bus factor of one in critical areas, and no one notices until that person goes on vacation.
4. Confidence in recovery. Not just MTTR, but a qualitative question: did we understand the root cause well enough to explain it in one sentence, without embellishment? This was a change I proposed a while ago in a team: after each incident, the post-mortem begins with a sentence explaining what happened. If it doesn’t fit into a clear sentence, we don’t understand. Teams with impeccable DORA can have diagnostic times tripling without anyone noticing, because no one fully understands the code they are sending.
And now, the obligatory warning: Goodhart applies to all of these too. Deployment frequency is played by artificially fragmenting PRs. CFR is played by reclassifying incidents as “demotions.” MTTR is played by auto-resolving flappy alerts. churn is played by leaving dead code instead of deleting it. Review comments are played with performative nits. If you make them individual targets, you kill them.
#.How to implement this without going back to dashboard hellOne of the things that made me hate LOC reporting fifteen years ago was the culture they built around it. The manager looked at the dashboard on Friday, the engineer looked at the dashboard on Thursday to make sure Friday looked good. It was a surveillance mechanism, not a learning mechanism. I don’t want to go back there. I suspect you don’t either.
The guiding principle is simple: instrument the flow, not the individual. The metrics belong to the team and the system. There is no leader board. There is no ranking. There is no number per engineer next to their photo in the Slack channel. If your instrumentation ends up becoming that, you made the same mistake as in 2010.
The book Software Engineering at Google, in chapter 7 fn-12, describes a useful framework very well: GSM — Goals, Signals, Metrics. You start by defining the goal (for example: “we want to deliver with a low rate of regressions”). Then you identify signals, which are the observable things that would indicate progress toward that goal (low CFR, low churn, substantive review comments). Then, and only then, do you choose the concrete metrics that approximate those signals. The key is that the metrics are at the service of the goal, not the other way around. If the metric becomes useless or perverse, you change it.
Technically, there is no need to reinvent anything. A modest pipeline of Postgres + Metabase or Grafana fed from the GitHub API, your CI and your incident tracker gives you 80% of what you need in a weekend. If you’d rather not build it, tools like LinearB, DX, Swarmia or Jellyfish make this out of the box. Any path works; What doesn’t work is not implementing anything and governing by intuition.
And the minimum viable ritual: a biweekly team retrospective where these metrics are looked at as a whole, questions are asked about what is seen, and small things are changed. Not a monthly executive dashboard that looks down on people. The first creates conversation; the second creates fear.
#.Closing — fast to where?
In Piano piano si va lontano he closed with the Italian proverb: slowly you go far. It wasn’t an invitation to go slow, it was a reminder that sustainable speed requires solid foundations.
Today I want to add a twist. AI gave us one thing, and one thing only: generation speed. He didn’t give us direction. He didn’t give us criteria. He didn’t give us understanding. Those are still ours, the human ones, and they are precisely the ones that the old metrics do not know how to measure and the new ones, if we are not careful, neither do. The question “how much code did we produce?” was never interesting. The interesting question was always “do we understand what we built? Can we change it tomorrow without fear? Can we explain it to someone new?” The only thing that changed is that now, with AI, the first question answers itself — and that answer, converted into a metric, is worthless.
The metrics we choose in the next two or three years will silently decide what kind of engineers we will be in ten. If we measure generation, we will have generators. If we measure sustainable delivery and understanding, perhaps we have engineers. It’s not obvious which way is going to win. But it is obvious that the LOC, finally, died.
Rest in peace. The least we owe him is not to resurrect him with another name. And yet, look around: we are already doing it.
- GOODHART, Charles (1975). “Problems of Monetary Management: The UK Experience.” Papers in Monetary Economics, Reserve Bank of Australia.↩
- STRATHERN, Marilyn (1997). “‘Improving ratings’: audit in the British University system”. European Review, 5(3), pp. 305–321.↩
- BROOKS, Frederick P. (1975). The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley, Chapter 2.↩
- FORSGREN, Nicole; HUMBLE, Jez; KIM, Gene (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution. https://itrevolution.com/product/accelerate/↩
- DORA (2024–2026). “DORA Metrics: The Four Keys”. https://dora.dev/guides/dora-metrics-four-keys/↩
- PENG, Sida; KALLIAMVAKOU, Eirini; CIHON, Peter; DEMIRER, Mert (2023). “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot.” arXiv:2302.06590.↩
- METR (2025). “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity.” arXiv:2507.09089. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/↩
- FORSGREN, Nicole; STOREY, Margaret-Anne; MADDILA, Chandra; ZIMMERMANN, Thomas; HOUCK, Brian; BUTLER, Jenna (2021). “The SPACE of Developer Productivity.” ACM Queue, 19(1). https://queue.acm.org/detail.cfm?id=3454124↩
- NODA, Abi (2024). “Introducing DX Core 4”. https://newsletter.getdx.com/p/introducing-the-dx-core-4↩
- GitClear (2025). “AI Copilot Code Quality: 2025 Look Back at 12 Months of Data.” https://www.gitclear.com/ai_assistant_code_quality_2025_research↩
- WINTERS, Titus; MANSHRECK, Tom; WRIGHT, Hyrum (eds.) (2020). Software Engineering at Google. O’Reilly, Chapter 7: “Measuring Engineering Productivity” (Ciera Jaspan). https://abseil.io/resources/swe-book/html/ch07.html↩