I use AI assistants every day. They write first drafts of code, explain unfamiliar codebases and run the tedious parts of testing. I’ve also watched them make confident mistakes that would have gone live if I hadn’t checked.
Both experiences are common, and the research reflects the same split. Here’s what the studies actually found, as opposed to what the marketing and the backlash say.
Almost everyone uses it, and few fully trust it
| Finding | Source |
|---|---|
| 84% of developers use or plan to use AI tools | Stack Overflow Developer Survey 2025 |
| 47% use them every day (51% of professionals) | Stack Overflow 2025 |
| 33% trust AI output to be accurate; 46% distrust it | Stack Overflow 2025 |
| 66% say their biggest frustration is answers that are “almost right, but not quite” | Stack Overflow 2025 |
| 45% say debugging AI-generated code takes more time | Stack Overflow 2025 |
| 90% use AI at work, and more than 80% believe it made them more productive | Google DORA report 2025 |
| 30% have little or no trust in AI-generated code | Google DORA 2025 |
“Almost right, but not quite” is the most useful phrase in this whole field. Code that’s obviously wrong is easy to catch. Code that’s almost right compiles, passes a quick look, and fails later.
Speed: we feel faster, but the data is less clear
The most quoted study comes from METR, a research group. In early 2025 they gave 16 experienced open-source developers 246 real tasks from their own projects, and randomly allowed or banned AI tools on each one.
- Before starting, the developers expected AI to make them 24% faster.
- With AI, they were measured as 19% slower.
- Afterwards, they still believed AI had made them about 20% faster.
That gap between how fast people felt and how fast they were is the real finding.
The tools have moved on since then, and METR said so in February 2026: “Previously, we found a 20% slowdown. That finding is now outdated. Speedups now seem likely, but changes in developer behavior make our new results unreliable.” Part of the problem was that many developers now refused to take part, because they didn’t want to work without AI.
Google’s DORA research tells a similar story at team level. In 2025, unlike the year before, it found AI adoption linked to higher delivery throughput and product performance, but still linked to lower delivery stability. In their words, “AI doesn’t fix a team; it amplifies what’s already there.”
My reading: AI makes generating code faster. Whether it makes the whole job faster depends on how much time you then spend checking, fixing and reworking what it produced.
Quality: more code, less tidying up
GitClear analysed 211 million changed lines of code from 2020 to 2024, from private companies and large open-source projects. As AI assistants spread:
- Copied code went up. 2024 was the first year in which copy-pasted lines outnumbered lines that were moved, and moved lines are the usual sign of refactoring.
- Duplicated blocks rose sharply. Blocks of five or more lines repeating nearby code became several times more common.
The cause is easy to see in practice. An assistant will happily write a new function that does what an existing one already does, because it can’t see, or isn’t asked to look at, the rest of the codebase. Every duplicate is something to maintain twice.
Security: less secure, and more confident about it
Researchers at Stanford (Perry, Srivastava, Kumar and Boneh, ACM CCS 2023) asked participants to complete security-related programming tasks, with or without an AI assistant. Those with the assistant:
- wrote significantly less secure code, with the clearest differences in encryption and SQL injection tasks;
- were more likely to believe their code was secure.
The participants who did best with AI were the ones who trusted it less, and rephrased and refined their requests. The study used an older model, and today’s tools are better. But the combination of more mistakes and more confidence is a pattern worth remembering.
An example from this blog
The benchmarks on this blog were built with an AI assistant’s help, and it made two mistakes that I only caught by checking the results:
- In the MySQL benchmark, the test data generator used a simple random-number formula whose values were correlated. No product came out both “blue” and “in stock”, so one query returned nothing and looked impossibly fast.
- In the headless WordPress article, an early draft blamed WordPress for a delay that was partly just the distance to the server.
Both looked plausible. Both would have been published as “data”. Neither was the AI being careless in an obvious way. They were exactly the kind of “almost right” the surveys describe.
Where AI helps, and where it hurts
| Good use | Risky use |
|---|---|
| Boilerplate, configuration and repetitive code | Security-sensitive code: authentication, payments, encryption, SQL |
| Explaining unfamiliar code, error messages and APIs | Architecture decisions it can’t see the full context for |
| Writing tests for code you understand | Code you don’t understand and can’t review |
| Converting between formats, languages or library versions | Changing large codebases without tests to catch regressions |
| A second opinion in a code review | Replacing code review |
How I work with AI
- I review AI code the way I’d review a junior developer’s, line by line, before it goes anywhere near production.
- Tests come first, or at least alongside. An assistant with tests to run is far more reliable than one without.
- Security-sensitive code gets extra scrutiny, no matter who or what wrote it.
- I measure instead of trusting the feeling of speed. METR’s developers felt 20% faster while being slower.
- I give it context. Pointing it at the existing helpers and conventions avoids the duplication problem.
- When the output looks too good, I check it twice. Impossibly fast benchmark results are usually wrong benchmarks.
AI hasn’t made experience less valuable. It’s made it more valuable, because someone has to tell “almost right” from right.