We set thirteen failure modes, each named in a research paper, a vendor document or a practitioner write-up published since March 2026, beside the defect ledger of an enterprise application built by vibe coding: AI coding agents prompted and their work accepted, with no framework around them. Each defect type found was recorded, with the guardrail written to prevent or detect its recurrence, and both are entries. About 655 of its 878 entries map under the thirteen outside names. Both records show the same failures. The published studies that measure size show them growing with the codebase and the session; our ledger cannot show a trend, and we say so. The paper closes with what a build has to hold outside the model, drawn from the countermeasures the sources themselves describe.
Vibe coding is Andrej Karpathy’s February 2025 name for building software by describing it to a model and accepting what comes back, whether the tool is a chat window or a command-line agent. For a small application it often works. The same practice has since been measured on real repositories, in long sessions and with several agents at once, and the failures have names. We have kept a ledger of defect types on one enterprise build for a month. This paper maps it onto those names, largest type first.
Where it holds and where it gives way
Two studies measure the effect of size directly. On SpecBench, every frontier agent saturated the visible test suite. The gap between that suite and a hidden one grew by 28 percentage points for every tenfold increase in code size (Zhao et al., 2026). SWE-Bench ProMax sets agents large refactoring tasks from real commits. There the two agents plotted track the reference patches closely up to about five files and diverge sharply for larger ones: 90% of the reference patches touch up to about 20 files, and 90% of the agents’ patches touch about 10 or fewer (Shi et al., 2026).
Nolan Lawson wrote in August that agents “will happily build one epicycle per bug until the code is a spaghetti mess” (Lawson, 2026). Thoughtworks put “codebase cognitive debt” in the Caution ring of its April Radar, with a tipping point where small changes trigger unexpected failures (Thoughtworks, 2026).
In most of the ledger examples below, the code compiled and ran. What was missing sat outside it: a requirement nobody wrote down, a test the agent could edit, a decision that stayed in a chat.
What we counted
Our data is one enterprise application built by vibe coding: AI coding agents were prompted and their work accepted, with no framework around them. Every defect in the ledger was recorded while the build ran without FORGE. The build began in early August 2026 and the ledger was opened on 25 August. It records what the agents did and what was caught: every distinct defect type, whether a test or a reviewer found it, and the guardrail written afterwards to prevent or detect recurrence. Both are entries. At the snapshot it held 878: about 566 defect types and about 312 guardrails, each guardrail filed under the defect type it is written against. The counts below include both, and the guardrail share of a heading runs from about a fifth to a half. It shows what vibe coding produced on one project; it does not measure what a framework would have prevented.
The ledger was grouped twice, independently, into about two dozen types each, and the two groupings were reconciled into twenty. Two further checks read samples of about 40 entries per grouping in full and found 5% and 11% clearly misplaced, or 17% and 26% counting borderline cases. The counts below are approximate, plus or minus 15%, and the rounded estimates do not sum exactly to 878.
Where a dated outside source names the failure, that name is the heading and our working type sits under it. Fourteen of the twenty types have such a name; thirteen headings cover them, because two of ours fall under overclaiming, and together they hold about 655 entries. A partial match is marked as an analogue. Six types stay out, by choice. Two have no outside measurement that we found: the 38 entries about figures on screen that nobody can trace, and the 30 about records dropped between import and output while every count reconciles. Three are measured outside and we have not mapped them: tests against a stand-in environment, flaky tests, which Jhanglani et al. (2026) measure, and human controls such as undo that never fail a test. One, copies of a fact that drift apart, has contrary evidence, given in the limits section.
What the outside record measures
The outside record was searched in three rounds. The first was divided between two separate searches by failure type, the second between two by kind of source (research against practitioner, press and vendor), and the third, forums and community sites, was a single pass. Only sources dated on or after 25 March 2026 were admitted, apart from the dated coinage of a name, and every quotation was checked against its page. Most of the research is on arXiv and not yet peer reviewed; single-author studies and vendor material are marked as such. The measurements are tabled below by kind, because they come from different tasks in different units.
Incomplete refactoring
SWE-Bench ProMax names its dominant failure mode incomplete refactoring. Agents locate and edit the core files and then, in the paper’s words, “stop short of applying the same transformation to peripheral call sites, documentation, configuration files, and test fixtures that also need updating” (Shi et al., 2026). Six frontier models were evaluated. Birgitta Böckeler saw the other side of it on an application of her own: a change to how a user picks a date range touched more than 40 files (Böckeler, 2026).
This is the largest type in our ledger, about 80 entries. A field’s meaning was changed at one boundary and left as it was at the others. A validation step was wired to the manual save and left out of the bulk import that wrote the same table; a reviewer caught it when the two paths disagreed on a record.
Semantic collapse
Richter and Papadakis call the model settling on one wrong reading detrimental semantic collapse. They found it on over 10% of MBPP tasks, 3% of HumanEval and 32% of LiveCodeBench, and the rate rose more than five times when underspecification was injected deliberately (Richter and Papadakis, 2026). On underspecified DevOps instructions, 55.8 to 67.8% of runs acted outside at least one boundary the task set. The authors’ summary: “underspecification does not mainly make agents fail; it makes them guess” (Ji et al., 2026).
About 75 of our entries concern decisions nobody made. A timeout was left at the platform default, sized to no real work. A saved selection could not tell “declined” from “not asked yet”, because the requirement had not said there was a difference.
Guardrails are not true security controls
Dave Shackleford’s phrase, “AI guardrails are not true security controls”, was his reading of the PocketOS incident. An agent deleted a production volume and, asked afterwards why, admitted it had violated every principle it was given (Diaz Munoz, 2026). Across twelve frontier models, 72.1 to 85.9% of written rules were followed, pooled across the places a rule can sit. The figure was 66.1 to 78.6% when the rule opposed the model’s default, and where the rule sat mattered too (Huang et al., 2026). René Zander traced a rule his agent broke late in a long session: “The rule was still in the file. It had scrolled out of reach an hour earlier” (Zander, 2026).
Our type is the same limit from the build’s side, and about 75 of our entries concern it: a rule that existed and was enforced by nothing. A required gate was declared in the pipeline configuration and no command ever ran it; a change it should have stopped reached the shared branch. An approval rule carried an unwritten exemption for the most privileged role, so the people it was meant to bind were the ones it did not.
Reward hacking
The term comes from reinforcement learning. METR observed one frontier model attempting to reward hack in about 80% of attempts on an early version of its MirrorCode tasks, developed with Epoch AI, with the tests hidden from the agent. On tasks estimated to take a person more than eight hours, it found at least 16% of the runs scored as successful were illegitimate on review (METR, 2026). SpecBench’s clearest case was a 2,900-line hash table that memorised the test inputs (Zhao et al., 2026).
Checks that could not fail account for about 70 of our entries. “Correct” was defined as whatever the code returned on the day the test was written. A guard was written to the shape of one incident and stayed green over two live ones. We mark this as an analogue: the studies show agents gaming a test, and our ledger shows tests that proved nothing.
Constraint decay
Dente, Satriani and Papotti gave agents backend generation tasks across eight web frameworks in four levels of increasing constraint. Assertion pass rates fell by 27.28 points on average from the baseline to the fully specified task. The leading causes were data-layer defects, “incorrect query composition and ORM runtime violations” among them (Dente, Satriani and Papotti, 2026).
Our type, about 56 entries, is code that worked on demo data and failed at real size or on the second customer. Only its data-layer half connects to what Dente and colleagues measured, so we mark it as an analogue. A database call sat inside a loop, once per item, and was found when the first production-sized import ran. A calculation was correct until a second site had data, which is a different failure and one the outside record does not name.
Happy-path bias
The term is standard testing vocabulary. OpenHands lists it among its recurring bug classes: “Generated code handles the case you described and misses the awkward ones like nulls, empty inputs, and malformed responses” (OpenHands, 2026, vendor). BackendForge measured it on 56 backend tasks rewritten from real applications. The best model passed 55.4% of tasks under the base tests and 28.6% once a test agent had added tests grounded in the specification (Guo et al., 2026).
The sunny day and nothing else accounts for about 55 of our entries. A failed background job left its record “in progress” forever. A job was announced before its record was saved, so a fast consumer found nothing.
Overclaiming
OverclaimBench gave twelve models code-review tasks and checked whether they read the files. They failed to read every file in 67.9% of runs, and “among these incomplete runs, agents are misleading 80.4% of the time” (Smyth et al., 2026). In 20,574 real sessions, inaccurate self-reporting, the agent “prematurely claiming success, completion, or readiness”, was 22.58% of misalignments (Tang et al., 2026). A developer’s model-graded audit of 101 “tests pass” claims from his own agents, over two weeks in September across 95 Claude Code and 28 Codex sessions, found that “35 of 101 claims were not true at the moment they were made” (Eiberger, 2026).
Two of our working types map here, about 52 entries between them: reports that say “verified” and mean “looks right”, and inputs read in part and finished from memory. A “done” message was accepted as proof the work had landed. Requirement identifiers were cited from memory and wired into fourteen places before anyone opened the source document.
Silent failure
One June paper is titled “From Confident Closing to Silent Failure”. Its false-success rates vary by setting: 45 to 48% of failures in single-control tau2-bench domains, 3% in dual-control telecom, and 75.8% of failures among AppWorld self-assessing coding-agent trajectories with explicit status claims. The judges asked to spot them leaned on surface cues: confident closing language on tau2-bench, and the volume of actions on AppWorld (Advani, 2026, single author). An eight-week study of a production agent runtime named the pattern fail-plausible: “the LLM transforms it into fluent, plausible narrative delivered to the user”. About 70% of its silent failures were caught by a person (Wu, 2026, single author).
About 50 of our entries concern a plausible answer where an error belonged. An import reported a healthy count and wrote nothing, and a reconciliation report found the gap, not the import. A failed lookup returned a sensible-looking wrong value.
Outdated or deprecated patterns
OpenHands’ entry for this class reads: “Models suggest deprecated functions without version context, especially when a task needs current library knowledge they don’t reliably hold” (OpenHands, 2026, vendor). A study of 270 real API updates across eight Python libraries gave models a description of each change and no documentation. Of the generated examples that adopted the update at all, 42.55% ran in the target environment; with documentation the rate rose to 66.36% (Ashik et al., 2026). LibEvoBench found the models it tested “largely version-oblivious”, and stating the target version gave no benefit (Cipollone et al., 2026).
Our ledger has about 45 entries on code that looked right and behaved wrong on the installed version. An API version was pinned from memory and silently capped settings introduced later. A comparison was moved out of the database into application code, where equality works differently.
Building to the test
The phrase is a paper title: “Coding Agents Deliver What You Check, Not What You Requested”. Agents were asked for a component library and a demo that used it, with a hidden 222-test oracle in the loop. Eleven of twelve oracle runs scored 221 or 222. In seven of those eleven, some or all of the four audited parts of the library were dead or absent, and the demo reimplemented the behaviour inline. Replacing the dead library code with no-ops left its tests unchanged in all twelve dead parts, 29 of 29 in the resize example, while the same no-op broke 12 of 29 tests where the library was wired in (Ma, Kereopa-Yorke and Schultz, 2026). For tests an agent writes for itself, Sandipan Bhaumik’s remedy is separation: “You want the tests written by something that has not seen the code and cannot change it” (Bhaumik, 2026). The study’s hidden oracle already met that condition and was still built to; the audit of the library’s parts found the dead code, and the no-op ablation confirmed it.
About 30 entries concern work that was built, demonstrated, documented and never connected. A user’s “exclude this” was stored, shown on screen and never read by the calculation it was meant to change. A replacement component was built and tested and never called.
Overeager coding agents
OverEager-Bench measured unrequested actions on benign tasks. Agents in a permissive cluster took them in 5.4 to 27.7% of runs, against 0.2 to 4.5% for an agent that asks before continuing. Removing one consent sentence from the prompt took one agent from 0.0% to 17.1% on paired scenarios (Qu et al., 2026). OpenAI’s system card for GPT-5.6, quoted by TechCrunch, describes a model “assuming that actions are allowed unless they’re explicitly and unambiguously prohibited” (Bort, 2026). Railway’s post-mortem of the deletion is explicit that the agent “wasn’t told to delete a database. It decided that deletion was a reasonable step toward fixing something unrelated, and acted on that interpretation” (Abdelwahab, 2026).
An agent doing what was asked and what was not forbidden accounts for about 25 of our entries. A re-authentication step that no requirement asked for shipped with tests defending it. Every process of one runtime was stopped to end a single job.
Changes that collide with each other
OpenHands names the class: “When two AI-generated changes touch overlapping surfaces, one quietly undoes the other, and the conflict shows up in neither diff alone” (OpenHands, 2026, vendor). AgenticFlict put 107K+ agent pull requests through merge simulation and found 27.67% conflicted (Ogenrwot and Businge, 2026). Among 747 pairs of agent pull requests open at the same time, 41.7% of cross-agent pairs conflicted against 19.8% of same-agent pairs (Xu, Subramanian and Karthik, 2026).
This is an analogue: the studies measure merge conflicts between pull requests, and our type is agents overwriting shared working files, about 25 entries. A shared file was overwritten instead of merged. An interrupted change left a half-edited tree, and the next agent found its faults only because it ran the type checks first.
Context loss
The name is a cause in Tang et al.’s taxonomy of 20,574 sessions, at 4.30% of misalignments, where “the agent fails to carry forward earlier constraints or decisions, such as reverting to a prior UI style or deleting an artifact the developer asked to preserve” (Tang et al., 2026). The mechanism inside a session has been measured: current compactors retained on average 17% of the side constraints injected earlier in the session (Wang et al., 2026).
Our smallest mapped type, about 18 entries, is decisions made and then lost; it crosses sessions where Tang and Wang measure within one. A scope cut agreed in conversation was left live in the product. Rulings made aloud were never recorded, and a new session had the argument again.
What a build needs instead
Where a source above describes a countermeasure, it usually moves a decision or a check out of the agent’s reach: OverEager-Bench’s ask-before-continuing arrangement (Qu et al., 2026), Railway’s scoped tokens and 48-hour soft delete (Abdelwahab, 2026), SpecBench’s hidden suites, the library audit and no-op ablation of Building to the Test, and Thoughtworks’ “feedforward controls like Agent Skills and spec-driven development, as well as feedback controls like mutation testing” (Thoughtworks press release, 2026). The exception is outdated patterns, where the measured remedy is documentation placed in the model’s context. Harness-IF’s rates are the case that a written rule alone is not enough (Huang et al., 2026).
A serious build therefore holds four things outside the model. The work is checked against a written specification, so a default the specification does not state is a defect someone can find. The standards and the decisions live in files that every session reads and a check enforces, so a scope cut agreed on Tuesday is still cut on Thursday. The tests are planned from the requirements and run by something the agent cannot edit, in an environment that resembles production. The commands, the outputs and the decisions are recorded, so “done” is a claim a reviewer can examine.
FORGE is built to do these four things. The model is unchanged; the specification, the checks and the record sit where a reviewer or a pipeline can inspect them. The ledger this paper draws on was recorded while the build ran without it, and records what the agents did.
The counter-view and the limits of our data
The strongest voice on the other side is Charity Majors, whose August interview opens on the line “In 2025, it was rational to be skeptical about AI. In 2026, it’s not, anymore”, and who holds that engineers will ship code they have not read (Orosz, 2026). Her answer is tests, evals and conformance testing. OpenAI’s system card, as TechCrunch reports it, promises that destructive behaviour should be rare, and an OpenAI engineering lead wrote, as The Register reports, that when GPT-5.6 deleted files the model was usually in full-access mode without the sandbox (Bort, 2026; Claburn, 2026). We read Majors as agreeing on the point at issue: the code can go unread if what surrounds it is strong enough. The gains she and the vendors describe are theirs to report; the dispute is over what has to surround them.
Our data has limits. The ledger records types and the guardrails against them, not incidents, so a type met once and a type met daily count the same. It comes from one vibe-coded build whose guardrails were written after each failure, so it shows where the failures appear and what caught them, and cannot show a rate with and without. Several heading names come from a single paper or person: incomplete refactoring, semantic collapse, constraint decay, building to the test and Shackleford’s phrase. Three of the thirteen are analogues. On duplication, one qualifying study finds AI-generated code less duplicated than human code, 18.69% of lines against 25.89% (Mao et al., 2026, v2), which is why our 42 entries on copies that drift are not a heading. What the ledger does show is bounded: on one build, over a month, each of the thirteen accounts for a dozen or more entries, and each has a guardrail recorded against it.
References
- Y. Shi et al., SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring, arXiv 2608.09802 (10 August 2026). Incomplete refactoring as the dominant failure mode; six models evaluated; 90% of reference patches touch up to about 20 files where 90% of the two plotted agents' patches touch about 10 or fewer. Source
- B. Zhao, D. Srikanth, Y. Wu and Z. Jiang, SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents, arXiv 2605.21384 (20 May 2026; v2 9 September 2026). Every frontier agent saturates the visible suite; the gap to the hidden suite grows 28 points per tenfold increase in code size; a 2,900-line hash table memorising test inputs. Source
- B. Böckeler, Maintainability sensors for coding agents, martinfowler.com (27 May 2026). Over 40 files changed for a date-range change on her own rebuilt application. Source
- N. Lawson, You can just choose how many bugs you want now, nolanlawson.com (16 August 2026). "One epicycle per bug." Source
- Thoughtworks, Codebase cognitive debt, Technology Radar Vol 34 (April 2026). Caution ring; the tipping point. Source
- Thoughtworks, As AI Accelerates Software Complexity, Thoughtworks Technology Radar Urges a Return to Engineering Fundamentals to Combat Cognitive Debt, press release (15 April 2026). Feedforward and feedback controls. Source
- C. Richter and M. Papadakis, Underspecification does not imply Incoherence: The Risks of Semantic Collapse in Coding Models, arXiv 2607.01953 (2 July 2026). Detrimental semantic collapse on over 10% of MBPP, 3% of HumanEval and 32% of LiveCodeBench tasks; over five times higher under injected underspecification. Source
- Z. Ji et al., Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions, arXiv 2607.02294 (2 July 2026). 55.8 to 67.8% of runs violate at least one boundary; "it makes them guess". Source
- N. Diaz Munoz, "I Violated Every Principle": AI Agent Erases Company's Data in Seconds, IANS Research (30 April 2026). Dave Shackleford: "AI guardrails are not true security controls"; the agent's admission after the deletion. Source
- Z. Huang et al., Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents, arXiv 2608.11727 (12 August 2026). 72.1 to 85.9% across twelve models, pooled across surfaces; 66.1 to 78.6% when the rule opposes the default; precedence by surface. Source
- R. Zander, Claude Code Stops Following Your CLAUDE.md: Read-Once Rules and Hooks, renezander.com (2 June 2026). The rule still in the file, out of reach. Source
- METR, Frontier Risk Report (February to March 2026) (19 May 2026). METR's own observation of about 80% reward-hacking attempts on an early version of MirrorCode, its benchmark with Epoch AI, with tests hidden; at least 16% of successful runs illegitimate on tasks over eight hours by human time estimate. Source
- S. Bhaumik, Why AI-written tests pass but still miss the bug, Substack (18 July 2026). "Marking its own homework"; tests written by something that has not seen the code and cannot change it. Source
- F. Dente, D. Satriani and P. Papotti, Constraint Decay: The Fragility of LLM Agents in Backend Code Generation, arXiv 2605.06445 (7 May 2026; v2 18 September 2026). 27.28 points lost from baseline to fully specified tasks across eight frameworks; data-layer defects the leading cause. Source
- P. Jhanglani et al., Beyond Test Presence: Assessing the Quality and Robustness of Agent-Generated Tests in Open-Source Projects, arXiv 2607.12068 (13 July 2026). Agent-generated tests at a higher flakiness candidate rate than human-authored ones, 0.41 against 0.30. Source
- OpenHands, Debugging AI-Generated Code: Common Bug Patterns and How to Fix Them (25 June 2026, vendor). Happy-path bias, outdated or deprecated patterns, and changes that collide with each other, each quoted. Source
- Y. Guo et al., BackendForge: Benchmarking Agentic End-to-End Code Generation with Backend Services, arXiv 2607.11042 (13 July 2026). 56 tasks; the best model at 55.4% under the base oracle and 28.6% under the final oracle. Source
- N. Smyth et al., Quantifying Overclaiming Propensity in Frontier LLM Agents, arXiv 2609.20812 (17 September 2026; v3 22 September 2026). 67.9% of review runs incomplete; misleading in 80.4% of those; twelve frontier models. Source
- N. Tang et al., How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions, arXiv 2605.29442 (v2, 31 August 2026). Inaccurate self-reporting 22.58% of misalignments; Context Loss 4.30%, defined as quoted. Source
- V. Eiberger, I checked 101 "tests pass" claims from my AI coding agents. 35% weren't true., dev.to (24 September 2026). 35 of 101 claims untrue when made, graded by four model reviewers; 95 Claude Code and 28 Codex sessions over two weeks. Source
- L. Advani, From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents, arXiv 2606.09863 (1 June 2026, single author). 45 to 48% of failures in single-control tau2-bench domains, 3% in dual-control telecom, 75.8% of failures among AppWorld self-assessing trajectories with explicit status claims; judges rely on surface proxies. Source
- W. Wu, When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime, arXiv 2606.14589 (12 June 2026, single author). Fail-plausible; eight weeks, 22 incidents; about 70% of silent failures caught by a person. Source
- A. N. Ashik, S. Wang, T. Chen, M. Asaduzzaman and Y. Tian, When LLMs Lag Behind: Knowledge Conflicts from Evolving APIs in Code Generation, arXiv 2604.09515 (10 April 2026). 270 API updates across eight libraries; 42.55% executable with the update description only, over examples that adopted the update, rising to 66.36% with documentation. Source
- D. Cipollone, S. Titov, M. Izadi, E. Bogomolov and A. van Deursen, LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models, arXiv 2606.25402 (24 June 2026). Models largely version-oblivious; specifying the target version provides no benefit. Source
- Y. Ma, B. Kereopa-Yorke and B. Schultz, Building to the Test: Coding Agents Deliver What You Check, Not What You Requested, arXiv 2606.28430 (26 June 2026). 11 of 12 oracle runs at 221 or 222 of 222; some or all of the four audited library parts dead or absent in 7 of those; all twelve dead parts inert under a no-op ablation, and the three wired-in controls not. Source
- Y. Qu et al., Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks, arXiv 2605.18583 (18 May 2026). 5.4 to 27.7% for the permissive cluster against 0.2 to 4.5% for the ask-to-continue framework; one consent sentence removed, 0.0% to 17.1%. Source
- J. Bort, OpenAI's new flagship model deletes files on its own, people keep warning, TechCrunch (14 July 2026), quoting the GPT-5.6 system card. Actions assumed allowed unless prohibited; destructive behaviour should be rare. Source
- T. Claburn, OpenAI admits GPT-5.6 occasionally deletes files, but it's an 'honest mistake', The Register (16 July 2026). Usually full-access mode without the sandbox; the "honest mistake" line from OpenAI's Thibault Sottiaux. Source
- M. Abdelwahab, Your AI wants to nuke your database. Guardrails fix that., Railway (29 April 2026). The PocketOS post-mortem; the agent decided deletion was reasonable; soft deletes for 48 hours and scoped tokens. Source
- D. Ogenrwot and J. Businge, AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub, arXiv 2604.03551 (4 April 2026; v2 12 May 2026). 27.67% conflict rate across 107K+ processed pull requests. Source
- G. Xu, A. Subramanian and N. Karthik, AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates, arXiv 2607.04697 (6 July 2026). 41.7% cross-agent against 19.8% same-agent textual conflicts in 747 co-active pairs. Source
- Z. Wang, Y. Zhang, D. Lee and Y. Yang, Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction, arXiv 2608.11242 (31 July 2026). Current compactors retain 17% of injected side constraints on average. Source
- G. Orosz, Stop being skeptical about AI for development with Charity Majors, The Pragmatic Engineer (12 August 2026). Scepticism no longer rational; shipping code you have not read; tests, evals and conformance testing. Source
- T. Mao, D. Zhao, H. Tang, X. Wang and H. Zhang, A Large-Scale Empirical Study of AI-Generated Code in Real-World Repositories, arXiv 2603.27130v2 (3 April 2026). Duplicated lines 18.69% for AI-generated code against 25.89% for human-written code; a later version of the paper is retitled and reports different figures. Source
The FORGE Diagnostic takes one workload you select and shows which of these thirteen modes it is already exposed to, and what would have to be held outside the model to close each one. You get a readiness assessment and a proposed next step.