AI Slows You Down—Or does it?
As reported in a recent study, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METR, developers using AI were, on average, 19% slower than those not using it. What's more, this observed result was contrary to how the developers felt about AI tools, both during and after the study. In short, developers felt 20% more productive but were actually 19% less so.
Take that, vibe coders! Robots 0, humans 1.
Or...?
I want to unpack two parts here. One is the discrepancy between experienced versus actual productivity, the other a larger view on what such results mean.
Motion is better than idleness
Once coming back from a weekend trip, Google Maps showed a lengthy slowdown on the typical highway route. It cheerfully told me that this was, still, the best route, despite a 30-minute slowdown. With tired kids in the backseats, I did not want to stuck in a jam, though, so I opted for a route that took 45 minutes longer but allowed me to keep moving. My hypothesis is that, with an AI tool at your fingertips, you're never at a standstill. Who cares that you have to prompt and re-prompt it to get to your desired outcome if at least it feels like code is being written.
The takeaway here: Feels matter—we want to keep our tool-users happy—but in the end, it must be cross-checked against real data.
Adoption and the initial drop
You do things a certain way, you get good at it, and eventually you reach a peak. To then ascend to another, even higher peak, you first have to come down, at least partway, from your current peak. This effect is near-universal. Whether it's your golf swing, tennis serve, chess opening repertoire, programming style/language, any change that will ultimately make you better results in an initial dip. Given how early we still are with AI coding tools, I'm not surprised that we observe initially decreased performance. As the tools and our knowledge of how to use them improve, we should come out ahead. (The study, incidentally, shows one outlier who was vastly more efficient when using AI. Maybe that individual was a bit ahead of the general learning curve.)
The takeaway here: Whatever change you introduce, with the hope of making things better, will initially make things worse. Prepare yourself and your team for it and have clear expectations about how to assess progress.
