Agent instructions - why they drift from the project and how to catch it

One afternoon my agent reported that the fix was working. It had evidence. Tests green, and a screenshot showing exactly the thing that was supposed to appear. I opened the same page on my machine - nothing. I sent back my screenshot, it sent back its own, and we went three rounds like that, each side more certain of its version. Then the truth came out. Nobody was lying to anybody. The page we were working on was showing different versions of its interface to different visitors. The agent was looking at one, I was looking at the other.

And here is the interesting part. Nowhere in my instructions for the agent was there a line saying “before you compare results, make sure you are both looking at the same thing”. That oversight had been sitting in the instructions for weeks doing no harm, until a situation came along where it mattered. No tool had any way of catching it. We test code, we check configuration - but instructions for AI? Instructions we just write. Once. And then we believe they work.

#The file that works while you’re not looking

By an agent I mean an AI you hand tasks to in your editor or terminal, one that then edits files and runs commands on its own. If you work with such an agent, then somewhere next to your code there is a file of rules. Maybe it’s called CLAUDE.md, maybe AGENTS.md, maybe it’s your editor’s rules directory. Such a file usually describes how to build the project, where the tests live, what must never be touched and how to name commits.

That file has one property that separates it from every other note in your repository - it executes. The agent reads it before every task and treats it as a rule it cannot change. Did you once write in there that the app starts on port 3001? The agent will keep checking port 3001 six months after someone moved the app to 3002. Not because it’s stupid. Because it’s obedient.

Someone has already measured the scale of this problem. The METR study from early 2025 is that rare thing in our industry, a genuine randomized experiment: experienced developers, their own mature projects, tasks randomly assigned to “with AI” or “without AI”. The result surprised everyone, including the participants. With AI they finished tasks 19% slower on average - and afterwards they were convinced AI had sped them up by about 20%. The feeling and the measured time pointed in opposite directions.

METR later flagged those results as out of date and repeated the experiment. The new measurement came out so uncertain that the authors call it an unreliable signal - partly because some participants refused to do tasks without AI. The same thing again - a number everyone quotes had quietly stopped being current.

The 2025 DORA report added the team-level half of the picture: where there is more AI, delivery is faster and, at the same time, less stable. The same report also names what closes that gap. The smaller the batch of change, the less there is inside it that can break. The more of the code automated tests cover, the sooner you find out that something did break anyway. And safeguards built into the platform work whether or not a team remembers them. Which is exactly what the rest of this text is about.

For a long time I couldn’t square those results with my own experience, because it feels to me like working with an agent makes me faster. Except that is precisely the feeling the study’s participants had, and the measurement did not back it up. So I stopped treating my own impression as an argument here.

I asked a different question instead. What would have to be true for working with an agent to actually pay off? The answer isn’t in the model, and it isn’t in prompt-writing skill. It’s in whether something exists around the agent that checks its work. And whether that check has itself ever been checked.

#Instructions break without a sound

When code breaks, you find out immediately. The program stops and prints an error, a test stops passing, the build refuses to go through. Instructions give you none of those signals. They just fit reality a little less with every passing week.

I keep a set of files that walk an agent through my entire work cycle, from ticket through plan and implementation to code review and pull request. It adds up to a few thousand lines. Recently I went through them the way you go through code. In a single day I found, among other things:

• the address the app starts on, written into the instructions as localhost:3001 when it had been starting on 3002 for months. The agent dutifully knocked on the dead port and reported that the page was down.
• a ban on one of the tools, justified by a bug it once had. Except the bug had been fixed two days earlier, so my rule was forbidding something that already worked.
• a number hardcoded into the instructions: how many diagnostic scripts the project has. When I counted them, the real figure was several times higher.
• a ready-made query to an external API where the condition narrowing the results was tacked on as a separate parameter instead of going inside the query itself - ?status=OPEN&q=... rather than ?q=... AND status=OPEN. The API did not recognize that parameter and simply skipped it, with no error and no warning. So the query returned everything, not only what I had asked for. The next step then worked on that data without me.

None of these caused an outage. Each could have gone unnoticed for months, and each was quietly degrading the agent’s work in a way that, from the outside, looks like “well, AI gets things wrong”. Except the model wasn’t wrong. My instructions were lying, and the model was faithfully executing them.

One principle came out of those four findings, and it has saved me more work than anything else since. Content in a file that executes comes in two kinds.

The first kind is facts - a port number, a count of files, a directory name, a version number. A fact is correct on the day you write it down. Then the project moves on, the line stays as it was, and at some point it stops being true. Nobody notices, because nobody goes back to old sentences in an instruction file.

The second kind is commands that establish the same fact on the spot. Instead of writing the number, you write the command that counts it. The agent then gets a current answer every time, no matter how long ago that line was written.

Going through your own instructions, one question is worth asking of every line. Is this a fact or a command? There should be as few facts in there as possible.

You already know this phenomenon, just from other places. The README describing flags that are long gone. The comment above a function that no longer describes what the function does. The config pointing at a server from before the migration. Documentation has always drifted away from reality - the only new thing is that this particular documentation executes itself. It runs over and over, nobody supervises it, and it sounds sure of itself throughout.

#If it’s code, it goes through a gate

The conclusion suggests itself. If a file behaves like a program, treat it like a program. My version is an ordinary script. It walks all the instruction files and checks the things that break most often:

• do the paths lead to files that exist;
• are the commands I point the agent at still in package.json;
• have the hardcoded numbers drifted away from the state of the project;
• is there a pattern in the ready-made queries that I know fails silently.

$ ./check.sh
  ✅ paths into the repository
  ✅ pnpm commands in package.json
  ✅ hardcoded numbers
  ✅ conditions the API quietly drops
  ...
Clean.

A dozen-odd checks, all green. Looks solid, right?

That’s the thing. I thought so too, for quite a while - until I asked myself the question that changed how I see this whole puzzle: how do I actually know these checks catch anything?

A green light looks exactly the same when everything is fine and when the check never ran at all. A gate has three states, not two - passed, failed, and never executed. The third is indistinguishable from the first, and that is the whole trap. It applies well beyond my scripts, to any CI step that ends in success because the command inside it quietly failed to start.

A gate you have never seen turn red is, in this one respect, worse than no gate at all. With no gate you know nobody is checking, so you check yourself. With a green one, you stop checking.

So I wrote a second script that tests the first one. It takes a copy of the files, plants each check’s own defect - a dead path, a broken number, a nonexistent command - and verifies that the check raises the alarm.

A check in that script looks like this today:

{
  id: "dead-paths",
  describe: "a path quoted in the instructions exists in the repository",

  run(file, ctx) {
    return file.lines.flatMap((line, index) => {
      const paths = line.match(/docs\/[\w-]+\.md/g) ?? [];
      const dead = paths.filter((path) => !ctx.fileExists(path));

      return dead.map((path) => ({
        line: index + 1,
        message: `no such file ${path}`,
      }));
    });
  },

  fixture: { inject: "see docs/no-such-file.md" },
}

The whole idea sits in that last field. fixture is the defect this check is supposed to light up on, and it is mandatory - a check without one does not register at all, the tool throws on start-up. Whoever writes a check writes the proof that it can turn red. There is no step here to skip. Without that field the check simply does not exist.

None of this is my invention, by the way; it is an old technique in a new place. Mutation testing breaks the code on purpose and asks whether the tests notice. If every test still passes after a defect goes in, those tests were not checking what they were meant to check. A gate over instruction files works exactly the same way.

The first run of that self-test exposed two checks that could never, ever have turned red. The reason was painfully mundane. Their patterns used a regex syntax that the system grep on macOS doesn’t understand. The tool was erroring out, but the error channel was silenced, so the check finished with “success”. It had been shining green since day one. Because it had never worked.

Instead of taking my word for it, experience the same thing in miniature. Below is a small instruction file and a gate with four checks. There is also a button nobody normally hands you - one that breaks a single check quietly enough that nobody notices.

#Incidents write the rules

The idea for how to write the rules themselves came from a different field entirely. I picked it up from the people whose job is keeping large services running - the industry calls them SRE, for Site Reliability Engineering. When something breaks in the middle of the night, they are the ones who get the call. One of their principles is that you don’t write emergency procedures in advance - incidents write them. I took it over unchanged. Not a single rule in my files exists because “it seemed proper”. Every one carries, in its text, the date and the event that forced it.

One of my favorites sounds unremarkable: if you are counting elements on a page that can legitimately end up empty and you get zero, first check whether there is anything left there to count. Where did it come from? The agent was asked to count how many entries a certain page was showing and compare that with the count before a change. It measured zero and declared: the code broke, the selectors need fixing. The diagnosis sounded reasonable; it even had a screenshot. I spent an hour debugging code that was perfectly healthy. The truth was elsewhere - the page covered an event that had already finished, so there was nothing left for it to display. Zero was the correct answer to a question asked of a page that had nothing left to show.

Now try to invent that rule at your desk, cold. You won’t. Nobody thinks of a condition like that until they have seen it happen. And that’s exactly why incident-born rules are worth so much. They cover precisely the cases no “best practices” list will ever reach.

They have two less obvious advantages, too. The model takes them more seriously - a short “why” inside the rule works on it better than a bare prohibition. And a rule with a date and a reason can later be held to account. Once a month I sit down and check whether the reason it exists is still there. Because rules stop being valid as well - a workaround for a tool’s weakness becomes unnecessary with every new version of the tool. Remember the ban on something that had been fixed two days earlier? Without that review, it would still be hanging there today.

#”That data doesn’t exist” - says the agent

This part is the easiest to describe and the hardest to sustain, because it demands an uncomfortable discipline toward yourself.

A language model that failed to find something rarely admits “I didn’t find it”. It much prefers to announce “it doesn’t exist” - in the tone of someone who just checked. That sounds like a fact, but it is a hypothesis. Sometimes a correct one, which is the worst part, because correct hypotheses teach us to trust unjustified ones.

So the agent and I have an agreement that every claim arrives together with its evidence. You say “that data isn’t in the database”? Show the query result - pasted, not summarized. An automated test against a live page came back negative? That’s the start of a diagnosis, never the diagnosis. The page might not have finished loading. It might have served a different variant of the interface - like in the story this text opened with. Each of these has genuinely happened to me, and each spent a while impersonating broken code.

I treat my own feelings exactly the same way, by the way. The METR study is blunt about it. My “that went fast today” is worthless as a measurement. So I measure external things - the same page before and after a change, the numbers coming out of the gates, the trend in review comments on my pull requests. If the system works, that last number should fall over time. And if it doesn’t fall, the system isn’t working - it only looks as if it is.

#Where to start with your own

Two honest caveats before I get to where to start. First, this only pays off at a certain size. For a hundred-line file, reading it once a quarter is enough and a gate will cost more than it returns. It starts paying off when there are several files, they run to a few hundred lines each, and more than one person relies on them. Second, the gate does not check everything. It catches only what can be compared against the state of the repository. A rule that is simply wrong, or harmful, sails through green, because everything in it that can be checked still matches the project.

You don’t need my few thousand lines - they describe my work, not yours. The mechanics are short:

1. Keep your agent’s instructions in the repository and version them like code - because they are code.
2. Write a script that checks them for silent drift from the project: dead paths, stale numbers, references to things that no longer exist. Bash and grep are enough to start.
3. Test that script on red. Break each thing it is supposed to catch, one at a time, and watch it get caught. One evening - and without it, step 2 produces nothing but false calm.
4. Create new rules out of incidents, and record the date and the reason next to each. Once a month, check which ones are no longer needed.
5. When the agent tells you how something looks in the project, ask for evidence. A query result instead of an assurance, a measurement instead of an impression.

If I lost all these files tomorrow, I’d rebuild them in a few weeks and it wouldn’t be much of a loss. The value was never in the files themselves, but in the three habits they keep in place. Every claim needs evidence, every incident leaves a rule behind, and every gate has to turn red at least once before I start trusting it. The files merely remember all of that for me.