The AI Coding Disconnect

How is it that there's such a mix of opinions on whether AI actually helps with coding or not? Some teams claim to go much faster, consistently and safely. Others say it's a dumpster fire. Who's right?

We've actually seen tremendous differences in software team velocity even before AI. The Accelerate State of DevOps Report, published annually by DORA, brings the receipts: The teams classified as elite vastly outperform low-performing teams in both speed and quality. The relevant metrics include deployment frequency, change lead time, change failure rate, time to restore service, and overall reliability; elite teams can be hundreds of times better on these metrics.

The individuals in the low-performing teams might be perfectly intelligent and capable, but stuck in a low-performing environment because "that's the way the company does things." Either way, the measurements are real.

Considering this staggering discrepancy, even without AI in the picture, is it any surprise that we see such a wide spectrum of reported experiences of AI? Consider a team that deploys rarely, has a great change failure rate and a glacial time to recover. Now throw AI into that mix, and it's only going to make things worse. Contrast with a team that deploys many dozen times a day in small incremental changes, with the testing infrastructure in place to make sure no change breaks anything and, if it happens anyway, can restore the system to a functioning state in matter of seconds. Throw AI into that mix and they'll go even faster.

Before you introduce high-octane fuel to a system, make sure that system is a well-oiled engine and not a dumpster fire.