METR.ORG
AI CAPABILITIES

Study shows developers using AI tools took 19% longer to complete their tasks

SUMMARY

A study observed the effects of early 2025 AI tools on the productivity of experienced open-source developers. Contrary to expectations, using AI tools slowed down experienced developers by 19% when completing tasks. Developers thought AI would save them time (predicting a 24% speedup before the study and 20% after), but instead, tasks took longer when AI was used. Experts in machine learning and economics also overestimated AI’s benefits, predicting it would speed things up.

The study recruited 16 developers(averaging 10 years of coding experience and 5 years with their specific projects) who completed 246 tasks on well-known open-source projects, comparing their performance with and without AI assistance.

Key factors contributing to the slowdown were identified, issues like AI code cleanup and learning curves, the study emphasized the need for realistic evaluations of AI capabilities. Future studies will track productivity trends and better understand AI's impact on development and research acceleration.


▶︎ Click here for more breaking news