AI Causes '19% Adverse Effect' on Developer Productivity

In the first half of 2025, contrary to rosy market forecasts that the latest AI development tools would improve experienced open-source developer productivity, actual results showed shocking productivity slowdown of an average 19%. This experiment is not a simple incident -- it raises important questions about the fundamental relationship between AI and skilled labor. The METR study: conducted by the US METR (Model Evaluation and Threat Research) team from February to June 2025. Recruited 16 open-source software developers with 10+ year average experience -- designed to resolve 246 actual issues in large open-source projects they regularly contributed to (average 23,000 stars, 1.1M line codebases). Experimental design: half of issues allowed AI tools (Claude 3.5+ series, Gemini 1.5+, OpenAI o1 series), other half prohibited -- controlled randomized trial. Key finding: AI-enabled tasks took 19% LONGER on average than non-AI tasks, contradicting pre-experiment developer estimates that AI would save 24% of time. Why AI slowed experts down: (1) Context loading cost -- experienced developers on familiar codebases have internalized context; AI requires explicit context provision through prompts, which takes time; (2) Output validation -- expert developers must validate AI suggestions rather than use them directly, adding a review step that does not exist in non-AI workflows; (3) Mental model disruption -- experienced developers have efficient problem-solving patterns; incorporating AI suggestions requires switching to a more explicit reasoning mode that disrupts automated expertise; (4) AI error correction -- AI makes plausible-sounding errors that require debugging, sometimes harder to diagnose than not having AI suggestions. The study implication: AI productivity benefits may be domain-specific and experience-specific -- AI tools may dramatically improve productivity for early-career developers on unfamiliar codebases while providing minimal or negative benefit for expert developers on familiar codebases.