A landmark randomized controlled trial found experienced developers using AI tools were 19% slower, despite predicting a 24% speed boost — and they still refused to code without AI afterwards.

Create a landscape editorial hero image for this Studio Global article: What does recent research reveal about the productivity, code quality, and industry dependency effects of AI coding tools, including METR's. Article summary: Here is a synthesis of the recent research on all four fronts.. Topic tags: general, general web, user generated. Reference image context from search candidates: Reference image 1: visual subject "Our early 2025 study found the use of AI causes tasks to take 19% longer, with a confidence interval between +2% and +39%. For the subset of the" source context "We are Changing our Developer Productivity Experiment Design - METR" Reference image 2: visual subject "Three questions conceptualizing increase in value produced due to access to AI tools around March 2026, with estimates for March 2025 and March" source context "Measuring the Self-Reported Impact of Early-20
The promise of AI coding tools has been intoxicating: type a comment, watch a function appear, ship faster. But a wave of rigorous research from mid-2025 through 2026 has complicated that narrative significantly. Rather than a straightforward productivity multiplier, the data reveals a tool that slows experienced developers down, produces measurably buggier code, and creates a dependency that persists even when the numbers don't add up.
In July 2025, the nonprofit research organization METR published the most sobering finding in AI developer tooling. A randomized controlled trial assigned 16 experienced open-source developers to 246 real-world tasks, randomly allowing access to AI coding tools (Cursor Pro and Claude 3.5/3.7 Sonnet) for some tasks and not for others.
Before the study, those same developers predicted AI would make them 24% faster. The measured reality was the opposite: developers using AI tools took 19% longer to complete tasks than those working without assistance (95% confidence interval: +2% to +39%).
The slowdown wasn't from a lack of effort. Developers spent the extra time reviewing AI output, fixing errors, steering the model toward correct solutions, and waiting on code generation. Crucially, the gap between perception and reality survived the experiment itself. After experiencing the measured slowdown, developers still estimated they had been 20% faster — a 43-point gap between what the clock showed and what their brains believed.
METR later revisited the study design in early 2026, adjusting for task heterogeneity. The revised analysis found a modest 6% sample-wide speedup, but with extreme variation: some developers gained up to 25% on certain tasks, while others remained net slower. The core conclusion held: AI's benefit is highly task-dependent, and self-reported speed is not a reliable metric.
If time-to-completion numbers are noisy, code quality data is clearer. CodeRabbit's landmark "State of AI vs Human Code Generation" report analyzed 470 real-world GitHub pull requests — 320 AI-coauthored and 150 human-only — across open-source projects.
The headline is stark: AI-generated pull requests contained ~1.7x more issues on average than human-written code (10.83 issues per PR vs. 6.45). The quality deficit wasn't limited to style or formatting. It was concentrated in areas that cause real incidents:
CodeRabbit's analysis also identified a "heavier review tail" for AI-authored code, meaning human reviewers spent disproportionately more time finding and diagnosing problems in AI-generated changes. As the report's authors put it, humans and AI make the same kinds of mistakes — AI just makes many of them more often and at a larger scale.
The pattern aligns with CodeRabbit's broader observation that 2025 was defined by AI speed, but 2026 must become the year of AI quality. Postmortems and operational incidents increasingly traced back to subtle logic errors, configuration oversights, and design misunderstandings introduced by AI assistants.
The quality deficit translates directly into financial waste. Developer productivity platform Entelligence.AI aggregated data from 2,444 companies and produced a breakdown that has reverberated through engineering circles:
| Where the dollar goes | Cost per $1 of AI token spend |
|---|---|
| Fixing AI-introduced bugs | $0.44 |
| Rework | $0.27 |
| Review friction | $0.11 |
| Actual value reaching users | $0.18 |
In other words, 82 cents of every dollar spent on AI tokens goes to bugs, rework, and review overhead. Only 18 cents delivers user-facing value. The cost isn't theoretical. Uber exhausted its entire 2026 AI coding budget within four months and recorded zero measurable productivity gain. An unnamed Uber executive stated bluntly that the link between AI spend and product improvement "doesn't exist yet."
A complementary study from Stanford and MIT found that AI agents fixing code bugs can burn over a million tokens per task — approximately 1,000 times the token consumption of standard code Q&A tasks. The economics suggest that for many organizations, the downstream costs of AI adoption are currently eating the promised productivity gains.
Perhaps the most psychologically striking finding is that developers who experience this data still refuse to work without AI. Multiple outlets have reported that participants in the METR study resisted returning to unaided coding even after being shown their own slowdown figures. This has been described as an "AI dependency paradox" — once developers become accustomed to AI assistance, they lose confidence in their unaided ability, even when the tool is demonstrably slowing them down.
As one developer put it, AI "handles the boring parts — boilerplate, syntax, the stuff that feels like work but isn't where the actual difficulty lives." The tool makes coding feel faster even when the stopwatch says otherwise, because the friction shifts from writing initial drafts to conducting meticulous reviews.
Across METR's controlled trials, CodeRabbit's pull request analysis, and Entelligence.AI's enterprise data, a consistent set of recommendations has emerged:
The emerging evidence doesn't suggest that AI coding tools are useless. In specific contexts — onboarding unfamiliar codebases, generating boilerplate, and tasks where developers predicted AI would help substantially — measurable speedups do appear. But across the broader population of experienced developers working on their own mature codebases, the net effect in mid-2025 through 2026 has been slower deliveries, more defects, and a dependency that resists the data.
Studio Global AI
Use this topic as a starting point for a fresh source-backed answer, then compare citations before you share it.
A landmark randomized controlled trial found experienced developers using AI tools were 19% slower, despite predicting a 24% speed boost — and they still refused to code without AI afterwards.