How coding agents fill in the standards nobody gave them

The decisions an agent makes for you

The clearest measurement concerns dependencies. Spracklen and colleagues, at USENIX Security 2025, generated 576,000 code samples from sixteen models. Across 2.23 million package recommendations collected through code analysis and follow-up queries, 440,445, or 19.7%, named packages that did not exist on PyPI or npm. The average rate was 5.2% for the commercial models tested and 21.7% for the open-source ones, and the study counted 205,474 distinct invented names.

The invented names were not random. In a repetition experiment on a sample of Python prompts, the researchers re-ran each prompt that had produced a hallucinated package ten times, and 43% of those packages appeared in all ten runs. A name that recurs is a name an attacker can register in advance. The authors describe package hallucination as a new form of package confusion attack for that reason.

Dependencies are the part of this problem that has been counted. The engineering concern extends to choices the study did not measure, such as module boundaries and secret handling. A model given no rule for either still has to produce one. The figure below shows the three places such a decision can be settled, and what each one holds.

Where the standard can live

OpenAI published an account of building a product with its Codex agents (Lopopolo, 2026). It states the constraint from the agent’s side: “anything it can’t access in-context while running effectively doesn’t exist.” The team’s response was to make the repository the record of how the software is built. It encoded what it called golden principles, such as validating data at boundaries. It enforced its architecture with custom linters and structural tests that ran in continuous integration. The post’s summary is that “the discipline shows up more in the scaffolding rather than the code.”

The coding tools now read such a record by design. Claude Code reads a CLAUDE.md file at the start of every session. Codex “reads AGENTS.md files before doing any work”, combines a global file with project files, and rebuilds that chain on every run. GitHub analysed more than 2,500 of these files and described what the effective ones share. They put the build and test commands early, name the exact stack and versions, and show one real code sample. They also draw boundaries in three tiers: always do, ask first, never do. Its examples of the last tier include committing secrets or API keys and editing vendor directories. One example rule says an agent may write tests but should never remove a test because it is failing.

Instructions are advisory, checks are not

Anthropic’s guidance for Claude Code draws the line that matters for an enterprise. “Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens.” The advice is to use hooks, scripts that run at fixed points in the agent’s work, “for actions that must happen every time with zero exceptions.” The same guidance warns that when an instruction file grows too long, “important rules get lost in the noise.” It recommends treating the file like code: review it when things go wrong, prune it, and test whether a change alters the agent’s behaviour.

The practical result is a split. Preferences and conventions go in the instruction file, where they guide the agent, and whether it followed them still has to be checked. Rules that must hold go into checks: the approved dependency list, the boundary between two services, the ban on secrets in source code. A local hook gives early feedback. A required check in the pipeline decides the merge, and it blocks a change only when its result gates acceptance and the author cannot bypass or edit it.

The problem this addresses was described before coding agents existed. Sculley and colleagues at Google wrote about machine learning systems in 2015. They warned that “it is dangerous to think of these quick wins as coming for free.” They listed how such systems erode: boundaries blur, components become entangled, and configuration accumulates without review. A system whose builder settles its own structure on every run can erode the same way. A gate at the point of merge can stop the erosion it is written to detect before the change lands.

What this means for teams building with agents

The published practice adds up to a small number of steps:

• Write down the decisions the agent would otherwise make. Record the approved architecture, the permitted libraries, the security controls and the boundaries the agent may not cross. A rule that exists only in an architect’s head is invisible to the agent.
• Keep it in the repository, where every session reads it. Layer it so that an organisation’s rules sit under a project’s rules.
• Keep the instruction file short. Automate the rules that can be checked reliably, and make them required for merge. Give requirements that need judgement an explicit review step.
• Check every dependency against a registry and an approved list before merge. The package research shows that this is a guess an attacker can plan for.

An agent given no standard settles each question itself, and may settle it differently next time. Written rules guide it. Required checks hold the standard as far as those checks reach, so the reach of the checks is a design decision in its own right.