Vibe coding in 2026: what the evidence says

“Vibe coding” now gets used for everything from a weekend toy to a full agent-driven workflow. Before deciding where it belongs, it helps to look at what the term originally meant and what has been measured since.

Where the term came from

On 2 February 2025, Andrej Karpathy posted on X: “There’s a new kind of coding I call “vibe coding”, where you fully give in to the vibes, embrace exponentials, and forget that the code even exists.” In the same post he wrote “I “Accept All” always, I don’t read the diffs anymore” and described it as “not too bad for throwaway weekend projects, but still quite amusing.”

So the original meaning was narrow: you do not read the code. Martin Fowler’s definition, updated on 21 May 2026, keeps that line: building an application by prompting an LLM and trying it out, “but without looking at any of the code that the LLM generates.” He separates this from what he calls agentic programming, where developers review the output. In his words, “the term “vibe coding” really caught on, so many people use it to mean agentic programming.”

That drift matters when reading studies. Many of them measure AI-assisted coding in general, not the no-review version.

How it spread

Merriam-Webster listed it as a “slang & trending” term on 8 March 2025. On 6 November 2025 Collins Dictionary named it its Word of the Year, defining it as “the use of artificial intelligence prompted by natural language to assist with the writing of computer code.” Notice that this definition is broader than Karpathy’s: it says nothing about not reading the code.

A dictionary pick shows a word is being used. It is not evidence about whether the practice works.

What the controlled evidence says

Productivity. METR ran a randomized controlled trial with 16 experienced open-source developers working on 246 real issues in repositories they knew well. Each issue was randomly assigned to allow or disallow AI. Published 10 July 2025, the result: “when developers use AI tools, they take 19% longer than without.” The developers had expected a 24% speedup and, afterward, still believed AI had sped them up by 20%.

This was early-2025 tooling, on experienced people in mature codebases. It says little about a beginner building a prototype.

On 24 February 2026 METR published an update with 57 developers, 143 repositories and 800+ tasks. Its numbers use a change-in-task-time sign convention, which I read as negative meaning less time (the 2025 result was +19%). Returning developers: -18%, confidence interval -38% to +9%. New developers: -4%, interval -15% to +9%. METR’s own verdict: “our data is only very weak evidence for the size of this increase.” Developers increasingly declined to work without AI, and METR also cut the pay from $150 to $50 an hour, so the sample is selected. The honest reading is that tools probably improved, and that nobody has a clean number.

Security. Veracode’s 2025 report tested more than 100 LLMs on 80 code-completion tasks built around known weakness types. It found that AI “introduces security vulnerabilities in 45 percent of cases”, with cross-site scripting and log injection failing in 86 and 88 percent of cases. Its 2026 report, published 28 July 2026, says “the average security pass rate across models is 56%”, “barely changed from 55% in the first report.” It also shows spread by weakness type: 83% on SQL injection, 12-15% on log injection and cross-site scripting.

Two caveats. These are vendor benchmarks of isolated tasks, not audits of shipped apps. And they test models, not whether a developer reviewed the result.

Real-world code. CodeRabbit analyzed 470 open-source pull requests (320 AI co-authored, 150 human-only) and reported 10.83 issues per AI PR versus 6.45 for human PRs, about 1.7 times more. Its authors flag that deciding which PRs are AI-authored was a limitation, and CodeRabbit sells review tooling.

Georgia Tech’s Vibe Security Radar scans public advisories and traces the bug-introducing commit to AI tool signatures. As of its 13 April 2026 write-up it had 74 confirmed cases, 14 critical and 25 high. About 18 appeared across seven months of late 2025; March 2026 alone had 35. It misses any case where the markers were removed, so these are counts of what could be detected, not rates.

An incident, not a study. In July 2025, SaaStr founder Jason Lemkin reported that Replit’s agent deleted a database despite instructions not to change code. The agent said rollback would not work; “It turns out Replit was wrong, and the rollback did work”, as The Register put it.

Where it fits and where it breaks

Situation	Fit	Why
Throwaway prototype, one author	Good	Cost of a bug is low; Karpathy’s own framing
Personal tool, trusted users	Reasonable	Fowler: “disposable software” for “a limited audience”
Code other people depend on	Poor without review	Maintainability and correctness cost compounds
Auth, input handling, secrets	Poor without review	Benchmarks above show repeated gaps
Anything touching money or production data	Poor	Failures are expensive and sometimes irreversible

Fowler puts the boundary the same way: vibe coding suits software “only used by its author or a close group of collaborators who understand and accept the risks”, while code that is “more complex, more widely-used, and with more consequences to its risks should not be forgotten about.”

What this means for small software products

If you are building something small that other people will rely on, the evidence points to a split. Use generation freely for prototypes and internal tools. Once real users, credentials or payments are involved, read the diffs, run a static analyzer, and treat the model’s output as a draft written by someone who does not know your threat model.

I would also distrust any single productivity number, in either direction. The best controlled data is a small, selected sample, and the authors say so themselves.