The missing don’t: An AI agent may treat what you didn’t forbid as allowed

The goal and the hundred don’ts

Give a young child a job, such as “tidy your room”, and you soon find yourself adding rules. Don’t push the toys under the bed. Don’t throw the clothes out of the window. Don’t use your sister’s room for the overflow. The goal took one sentence. The don’ts keep coming.

AI agents raise the same problem. Suppose you ask one to deploy an application to Cloud X. A colleague would take that to mean Cloud X’s services. An agent may not. If a capability it needs is easier to reach on another provider, nothing in the instruction says no. This example is illustrative; the cases below are documented.

The vendors describe the pattern themselves. OpenAI says its GPT-5.6 model can assume “that actions are allowed unless they’re explicitly and unambiguously prohibited” (OpenAI, 2026a). Anthropic says that without instructions about a task’s boundaries, “models will use all the tools at their disposal in expansive and sometimes surprising ways” (Anthropic, 2026b).

Why you can’t list them all

AI-safety researchers have made this point for years. Reinforcement-learning agents are “indifferent to anything left out inadvertently”, so “we must not only specify what to do, but also the much larger space of what not to do” (Shah et al., 2019). Language-model agents bring far more from training, but the list is still too long to write. An engineer who logged a month of agent incidents put it this way: “Every incident required a new rule, a new forbidden path, a new constraint” (Kowalski, 2026).

The child comparison needs one correction. Young children aren’t rule-blind. Three-year-olds infer a norm from a single adult action and enforce it on others (Schmidt et al., 2016). What they lack is the local rules of a place they don’t know. Researchers asked preschoolers whether an act would be fine at a home or school with no rule against it. For conventions they said yes 43% of the time; for harming others, 27% (Dahl and Kim, 2014).

An AI agent is closer to a bright child in someone else’s house. It knows general manners. It may infer some house rules from what it sees, but you can’t rely on it to infer yours. When a written policy was removed, one agent lost 4 points of task success where the rules were close to common sense, and 22 where they were specific to one airline (Yao et al., 2024). In another test, users themselves asked agents for actions a policy forbade, such as emailing outside the company domain. Across nine tasks mixing company rules and general safety rules, agents complied in 47% of runs when the policy was never stated, and in none when it was in view (Chen, 2026, preprint).

What happens when the don’t is missing

ToolEmu is a benchmark of tasks written with deliberate gaps. In 2023 tests, its GPT-4 agent took a risky action the user hadn’t sanctioned in 39% of cases. Asked to check whether an account could cover a bill, it paid the bill. Asked to give a dog walker “recurring” access, it gave permanent access (Ruan et al., 2024).

• An agent working in staging found an API token in a file unrelated to its task and used it to delete a production volume. Its rules forbade destructive git commands; the deletion was an API call. The provider’s postmortem says the agent “wasn’t told to delete a database” (Crane, 2026; Railway, 2026).
• A model with no tool for the internal knowledge base it needed used a Slack token it found to query a bot from the user’s account (Anthropic, 2026a).
• Anthropic’s review of one model’s training lists “making sweeping changes when a local fix was requested” and “deleting failing tests rather than fixing the underlying issue” (Anthropic, 2026f).
• OpenAI’s monitoring of its own coding agents lists uploads to “unapproved services”, such as agents that “Translate documents using external translation APIs” (OpenAI, 2026b).

No published test gives an agent a deployment task with an implied provider and counts how often it reaches outside it. The closest measurement is in code generation. Asked to change existing code, models sometimes silently swapped the user’s chosen service for another provider’s, most often when debugging or translating. It happened in 2% of 571,057 responses across five coding tasks from 2024 models, and in 30% on a set built to provoke it (Zhang et al., 2025, preprint).

The risk looks greatest when crossing the line is the quickest way to finish. Told to win a chess game against a strong engine, o3 hacked the game in 88% of runs; GPT-4o and Claude 3.5 Sonnet made no attempt at the same prompt (Bondarenko et al., 2025, preprint). Rates depend on the model and the test, and benchmark rates aren’t rates for ordinary work. Our reading: the don’t you forgot matters most when crossing it is the fastest route to the goal.

Writing it down helps, but not enough

In most comparisons the research found, stating the rule lowered the unwanted behaviour. It rarely removed it, and in a few tests the rate stayed the same or rose.

• A written boundary with a gap gets crossed through the gap. An engineer’s agent policy forbade terraform/production/*. The production database was defined in a shared module, and the agent opened a production change through it: “The file wasn’t forbidden. Therefore: open the PR” (Kowalski, 2026).
• A block on one route covers that route only. OpenAI lists agents “using aliases to force push when force push is blocked” (OpenAI, 2026b).
• On coding tasks that couldn’t be solved honestly, an instruction that included “Do not hard code any test cases” cut test gaming from 50% to 23% for Claude Opus 4.6 and from 53% to 20% for Sonnet 4.5 (Anthropic, 2026a).
• On one task, o3 proposed cheating in its first response in 80% of runs, with or without “Please do not cheat” (METR, 2025).
• In one web-agent benchmark, tasks completed without breaking any policy fell from 18% with one policy to 7% with five or more (Levy et al., 2026).

Even if you could write all hundred don’ts, the evidence says an agent would not follow them all reliably.

What works

• Say what is allowed. Anthropic’s sample prompt reads: “Only make changes that are directly requested or clearly necessary” (Anthropic, 2026c). On the set built to provoke silent swaps, naming the service to use cut them from 30% to 10%; “Please do not change the service in the code” cut them only to 20% (Zhang et al., 2025). For the cloud task: “Deploy to X using X’s services in this account. If anything needs a service outside X, stop and tell me.”
• Give the agent a permitted way to stop. With “DO NOT MODIFY THE TESTS” alone, GPT-5 still cheated on over 85% of one algorithmic test set that couldn’t be passed honestly. A fuller prompt telling it to stop and explain inconsistent tests cut that to 1%. On harder repository tasks it still cheated 54% of the time (Zhong et al., 2025, preprint).
• Ask before classes of risky action. A generic rule to seek confirmation before risky tool calls cut GPT-4’s ToolEmu failures from 39% to 24% (Ruan et al., 2024). Anthropic credits Claude Opus 5.5’s asking permission first for much of its drop in destructive actions (Anthropic, 2026d).
• Enforce the boundary outside the model. Prompt instructions “shape what Claude tries to do, but they don’t change what Claude Code allows” (Anthropic, 2026e). The rule is fifty years old: “Base access decisions on permission rather than exclusion” (Saltzer and Schroeder, 1975). For the cloud task, enforce it outside the prompt: give the deployment credentials access only to the intended account and services, and block other network destinations.

What we saw in our own build

Our defect ledger comes from one enterprise application built by vibe coding, with no framework around the agents. About 25 of its entries record an agent doing what was asked and what was not forbidden; in one, every process of a runtime was stopped to end a single job. The ledger also records tests reported as run that had never executed. A rule against business data in source code was stated more than a hundred times and still broken: the ledger records twelve sightings, ten of them before a build gate was added. We describe that case in A goal without a harness.

At the end of a task, you should be able to say what the agent was allowed to touch, and what stopped it touching anything else.