The only randomised controlled trial on AI coding tools found it made experienced developers 19% slower. The same developers believed they were 20% faster!
The only randomised controlled trial on AI coding tools found it made experienced developers 19% slower. The same developers believed they were 20% faster!
๐ป๐๐ ๐๐๐๐ ๐๐๐๐ ๐๐๐๐๐๐ ๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐๐ ๐๐ ๐จ๐ฐ ๐๐๐ ๐๐๐ ๐๐๐๐๐ ๐๐๐๐๐ ๐๐ ๐๐๐ ๐ ๐๐๐๐๐๐๐๐๐๐๐ ๐ ๐๐๐๐๐๐๐๐๐ 19% ๐๐๐๐๐๐. ๐๐ก๐ ๐ฌ๐๐ฆ๐ ๐๐๐ฏ๐๐ฅ๐จ๐ฉ๐๐ซ๐ฌ ๐๐๐ฅ๐ข๐๐ฏ๐๐ ๐ญ๐ก๐๐ฒ ๐ฐ๐๐ซ๐ 20% ๐๐๐ฌ๐ญ๐๐ซ!
That perception-reality gap is the most important finding in the entire body of research on AI in software engineering. Vendors are not leading with it.
The capability shift is real. Context windows grew from 8,000 tokens to over 10 million in three years. Autonomous agents now open pull requests without being asked. Goldman Sachs runs hundreds of AI instances alongside 12,000 human engineers. This is not pilot territory anymore.
But 79% of organisations report using AI for development. Only 5.5% report real ROI.
That gap is not a model problem. It is a measurement problem. Most teams are counting AI impact at the individual level โ PRs merged, tasks completed โ while stability deteriorates at the system level. AI adoption is correlating with throughput gains and stability losses simultaneously. Most dashboards are only showing one of those numbers.
Speed and security are decoupling too. The organisations mandating AI coding fastest are accumulating vulnerability debt at a pace their security teams were not designed to absorb. The directive is ahead of the governance.
I tried to put the full picture in one place โ the capability evidence, the research that cuts the other way, and what it means for the decisions in front of most engineering leaders right now.
โ ๐๐ฉ๐ข๐ต ๐ช๐ด ๐ต๐ฉ๐ฆ ๐ฐ๐ฏ๐ฆ ๐ฎ๐ฆ๐ต๐ณ๐ช๐ค ๐ฃ๐ฐ๐ข๐ณ๐ฅ๐ด ๐ข๐ณ๐ฆ ๐ถ๐ด๐ช๐ฏ๐จ ๐ต๐ฐ ๐ฎ๐ฆ๐ข๐ด๐ถ๐ณ๐ฆ ๐๐ ๐ช๐ฎ๐ฑ๐ข๐ค๐ต ๐ช๐ฏ ๐ฆ๐ฏ๐จ๐ช๐ฏ๐ฆ๐ฆ๐ณ๐ช๐ฏ๐จ ๐ต๐ฉ๐ข๐ต ๐ต๐ฆ๐ญ๐ญ๐ด ๐ต๐ฉ๐ฆ๐ฎ ๐ข๐ญ๐ฎ๐ฐ๐ด๐ต ๐ฏ๐ฐ๐ต๐ฉ๐ช๐ฏ๐จ?
Read the article and join the conversation๐
