GoKawiilIt’s fifteen years ago (bear with me, I’ve been in this industry since the late 90s, most of my good stories start this way), and you’ve got two senior developers at a SaaS company. One of them writes 40% more lines of code than the other. Is that developer better? More impactful for the business? Should the other one be polishing their CV?
Of course not. You’d want to know what actually shipped. What it did for customers, for revenue, for reliability. Lines of code, PR counts… we spent a couple of decades learning these are stereotypically bad ways to measure a developer, to the point where suggesting them today is laughable.
Sooooo… Here’s what the industry put on the billboard this year:
Every single one is a volume claim. “Percent of code written by AI” is just lines of code with a better publicist. (The sceptic in me editing this draft would like to point out that it’s no coincidence that all of these are AI vendors of some kind, so pumping adoption is pretty important to them.)
We used to claim outcomes
Rewind a few years and the headline number was different in kind, not just size. GitHub’s flagship claim was that developers completed tasks 55% faster with Copilot. Say what you like about that study (plenty did), but it was an outcome claim. Bold, falsifiable, about value. If it was wrong, you could show it was wrong.
The 2026 claims can’t fail. That’s the genius of them; “75% of our code is AI-written” could be true, and will keep going up, regardless of whether anything got better (faster delivery, fewer incidents, happier customers, etc). A volume number can only ever disappoint you if adoption stalls, and adoption is the one thing most of us agree is real. 📈
So the claims got bigger and started saying less. What happened in between?
The bit nobody puts on a billboard
The outcome evidence got complicated, that’s what happened.
... continue reading