notes · · 14 min

You Don’t Know If Your AGENTS.md Helps. You Just Keep Adding Rules.

Claude Code, Codex and Antigravity in one workspace. New rules, lost decisions, and a server that sat idle while you slept.

On this page · 11 sections
  1. You write rules. Then you become their dispatcher
  2. I had my own logs counted
  3. I banned stopping. The system started an argument
  4. Yesterday Claude Code understood everything. Today you opened Codex
  5. A second opinion becomes another round of onboarding
  6. Codex can think brilliantly. The wait can be infuriating
  7. GPT-Sol thought it up. Opus built it. Antigravity read the documents
  8. “Go check the knowledge base” is my job too
  9. If I have to keep granting permission, it isn’t being remembered
  10. What I want from Confident
  11. Show me the result in the morning

TL;DR. You wrote the rules, talked through the plan and went to bed. In the morning you find that the agent did a small part of the work and stopped to ask whether it should start the next stage. You didn’t answer. You were asleep. The rules are still there, the task isn’t done, and the server sat idle. I want a system that keeps the decisions already made across sessions and across agents, uses what we have learned, and carries the assignment through to a result. I want to judge it by finished tasks, by cost, and by how often I have to step in.

You give an agent a task. You discuss the plan. You have a private harness, the plumbing that can launch different agents, hand work between them and set up code review. The servers are up, the data is ready. You go to sleep.

In the morning you open the conversation.

A small part of the task is done. Then comes the question: should I start processing the test data?

You say: yes, go. Now the processing needs several more hours.

Those hours could have run overnight. Instead the server stood still, the agent waited, and you woke up as the person who has to push the process along. Again.

Yes, you could say I never wrote a separate “start it” at the right spot. That is exactly what pisses me off. I delegated a task with a clear goal and discussed the path to it. It turns out I also had to guess every point where the agent would hand control back to me.

If getting a result means I have to list all those points in advance and then service them one by one, a good part of the execution is still on me.

You write rules. Then you become their dispatcher

After the next stop I can add a rule. Don’t ask about this again. Check the knowledge base first. If the step is part of the agreed plan, keep going.

Then an exception appears. Then a clarification of the exception. AGENTS.md gets more detailed, CLAUDE.md gets longer, another skill grows next to them.

And I feel calmer.

That is false confidence: the instruction got more detailed, and nothing checked whether the agent got better.

A while later the agent asks a question I have already answered. I explain it again. I point at the right record. I nudge it to the next step. Then I find it busy with work nobody asked for, and I write another instruction.

How do I know any of this makes it better?

I can see that the file changed. I can check that the rule is sensibly worded. That still doesn’t tell me what happens to the next task: will it be cheaper, more accurate, more independent?

I have dozens of projects. I don’t need a separate shift to babysit every running agent. I need to hand over a task, switch to other work and come back to a result.

It is easy to hide the owner’s constant involvement behind the pretty word “agentic”. But if the work regularly waits for my message to continue, I am still part of its main execution loop.

A new rule might help. It might make the agent do pointless work. It might collide with another instruction, or turn out to be useful in exactly one situation.

Meanwhile we keep changing models, tools, context and the tasks themselves. Then we see a good result and credit the latest tweak. Or we see a bad one and add the next tweak.

Evaluating AGENTS.md tests such files on coding tasks. In the third version the authors report no overall improvement in success rate and inference cost higher by more than 20% on average, even though the agents followed the instructions well. They also note that describing non-standard practices helps.

That is not a measurement of our sessions. But it hits hard at the comfortable belief that an instruction that was followed automatically makes the work better.

So I want to see what my rules actually do, not just be proud of how well I have learned to word them.

I had my own logs counted

So as not to rely on impressions alone, I asked for a pass over our local Claude Code logs. After excluding service messages, duplicates, sidechain sessions and selected private projects, the automated pass kept 3,433 of my messages from 10 August to 7 November 2026. In 461 of them, 13.4%, the script found profanity markers.

In 2,924 cases my message could be paired with the agent text right before it. The search flagged 316 of those endings, 10.8%, as wording that handed a decision back to me: waiting for a command, an approval or a choice. After those, a profanity marker showed up in my reply 20.3% of the time; after the rest, 14.1%.

This is a search over wording, not a manual rating of each question. It doesn’t prove that every such check-in was unnecessary, or that these check-ins caused the irritation. The data covers Claude Code only; there are no interactive Codex or Antigravity conversations in this set.

Method and aggregates are available without the raw conversation logs.

So the next step was to open one concrete episode: what exactly the agent left for me to decide, and why the work stopped.

I banned stopping. The system started an argument

I tried to keep the work going with more than instructions. In one Mitts development session a stop condition was in force: a check was supposed to send the agent back to work if the goal had not been reached yet.

In that session’s logs I found 27 separate firings of the check on the same condition. It rejected the finish, and the agent kept explaining why it needed a decision from the owner. Some of the repeats came almost back to back. That is 27 firings, not 27 different decisions.

Taking it apart showed that several different things had been glued into one stop. The agent was holding open a public release, options for working with the cache, and other technical changes. The release needed a separate decision. The technical options had to be weighed within the assignment, which was about speeding things up. The stop check, for its part, demanded proof that the result was as fast as it could get.

Instead of separating these questions, the system got stuck in a loop: the check sends the agent back to the goal, the agent sends the decision back to the human.

Here the instruction had already become an executable mechanism, and it still didn’t give me what I wanted. To untangle the argument, the owner has to come back and restore the meaning of the assignment for both halves of the system.

A mechanism that can tell the agent “you’re not done yet” is not enough for me. What I need is an understanding of what the agent can decide on its own, where my decision is genuinely required, and what result counts as good enough.

Yesterday Claude Code understood everything. Today you opened Codex

Say I work in one workspace through Claude Code for a long time. We have taken the system apart, rejected several approaches, made decisions. The agent writes notes, updates CLAUDE.md, saves memory.

That helps. While we are actively working together we share a line of reasoning: why we took this path, what we have already spent time on, what not to do.

Then several compactions go by, each one squeezing the history. Or I open Codex and Antigravity in the same folder because I need a second opinion.

Same files. Same repository. But the agent’s working state is different now.

In my practice it gets to the point where a new agent has to be shown separately where the relevant rules and records live. It goes to KYB, reads the knowledge base, studies the history. It rebuilds its picture of the system from scratch.

Even if it found AGENTS.md right away, reading that one file does not restore all the decisions we made in the previous session.

I have to explain which facts are already established, which options were tried, why we dropped them, and what exactly I want checked now.

The same folder on disk does not give me continuity of work between agents.

And I take little comfort in the fact that the information is stored somewhere in principle. If every new agent has to work out again which of it matters for the next decision, I pay for onboarding onto the project more than once.

A second opinion becomes another round of onboarding

I want a second agent precisely because the first one can be wrong. I want a different assessment, I want the weak spots probed, I want to catch what the previous agent missed.

But far too often, along with the second opinion, I get one more introduction to the task from scratch.

Take work on a complex legal case: the procedural history, the order of the filings, citations already checked, and the reasons behind earlier decisions. Material like that cannot be reduced to one line saying “we decided this”.

A new agent has to understand how it all connects. It starts reading the history again, collecting sources, asking questions. That costs time and tokens. And I am once again the person who has to remember what exactly we have already established.

Independent checking of citations can be absolutely necessary. I don’t want the second agent to take the first one’s conclusion on faith. I want it to see what has already been checked, which sources a conclusion rests on, where uncertainty remains, and which part needs an independent check.

That is a very different situation from an agent that simply never received the history of the decisions and has to rediscover the whole subject.

I need an independent assessment with the context preserved. For now, far too much of handing over that context depends on me.

Codex can think brilliantly. The wait can be infuriating

Let me say it straight: in my work Codex has lately been outrageously slow at times.

I use different working setups: Claude Code with Opus 5, previously Sonnet 4.5; Codex with GPT-Sol on ultra; and, separately, Antigravity.

In my tasks Opus and Sonnet 4.5 kept pinging me with questions. GPT-Sol on ultra makes me wait for a long time. I pay both for the waiting and for my own involvement; only the proportions change.

You start a heavy reasoning run on ultra, and you wait. A very long time. In interactive work it sometimes feels like forever. I won’t pretend that this is just pleasant thoughtfulness while I sit there waiting for the next useful action.

And yet the result can be excellent. I have had sessions where the agent worked for several days, barely pulling me in, and came back with a solution to a hard problem. That is real value: I work on other projects, and the work keeps moving.

So what annoys me is not the number of hours that went by. It’s the bad trade: a long wait, after which I still have to push again, re-supply context, or re-approve a decision already made.

And I really don’t want to run the same heavy mode for every following mechanical step once the decision has been found and just needs to be built quickly.

A model I am willing to wait a long time for to get a hard conclusion does not have to be my executor for every other stage.

GPT-Sol thought it up. Opus built it. Antigravity read the documents

I want to be able to put work together from different strengths without rebuilding the context at every handoff.

For example, a chain like this:

Stage Agent What has to be passed on
Hard analysis and choice of path GPT-Sol on ultra The decision, its grounds, the constraints and a concrete plan
Fast implementation Claude Code with Opus 5 on medium The changes and records of the work actually done
Analysis of the resulting documents Antigravity Extracted facts, discrepancies and links to the original passages
Review and the next decision GPT-Sol on ultra again A conclusion from the material, the reviews needed, and next actions

With Opus 5 on medium, carrying out a clear plan can go incredibly fast in my practice. For my tasks Antigravity analyzes documents so well that I want to use exactly that strength, and then return the material to GPT-Sol to make the decision.

This is my choice of agents, not a ranking.

The point of such a chain is not to draw four logos and connect them with arrows.

If the decisions already made, their grounds, the boundaries of the assignment and the definition of done are not passed along with the result, every handoff turns back into an introductory conversation.

A private harness can launch an agent. It can assign a review. It can return the result to the previous agent. But having those mechanisms does not by itself create a shared decision-making system.

“Go check the knowledge base” is my job too

We already have KYB: facts about services, decisions, change history, the reasons approaches failed. Agents can read it through MCP and the CLI.

It helps. But I want to be honest about the difference between having memory and how the agent behaves.

When I have to say “first look at how we did this before”, I am still the one choosing when to consult memory. When I point at the right record, I am helping pick the relevant experience. When I explain why an earlier conclusion no longer applies, I am again doing the work of applying that experience.

Later, with the Mitts 0.3.29 release, the gap showed up somewhere else: the version was out, the rest of the procedure existed, but the remaining cycle had to be raised separately and assigned to someone.

The knowledge base is a necessary part of our work. But after reading a record, a decision still has to be made in the current situation.

That is the level where I need continuity.

If I have to keep granting permission, it isn’t being remembered

I ask an agent to do something with a specific service. It already has access to the right server, it has our instructions, the operation is part of the assignment.

And it asks again whether I really want to do this.

Yes. That is why I gave you the task.

I don’t want to repeat the original assignment after a compaction or a change of agent.

Access, the current assignment and the constraints in force have to be considered together. If a new significant consequence appears, or the action goes beyond the agreed boundaries, the question makes sense. But repeating a decision already made at every handoff hands me back work I delegated.

What interests me is a system that can tell a new fork in the road from lost context.

Especially while I sleep. At that point even a simple question can stop useful work for the whole night.

What I want from Confident

The working name for the system I’m interested in is Confident.

I want to replace the confidence that comes from simply having rules with confidence in the next step, grounded in experience. That confidence has to be earned from earlier decisions, corrections and results, not handed over by a nice product name.

I want the system to learn from our experience: which action I allowed, what I corrected, why I dropped a path, and which result I finally accepted.

It needs to keep the link between the situation, the decision and the outcome. And pass it to the next agent regardless of whether I am working through Claude Code, Codex or Antigravity right now.

If I turned down extra tests for a narrow dependency bump, the lesson “don’t write tests” would be wrong. What mattered were the boundaries of the task and the consumer check that already existed. If I allowed an action on a particular service, the system has to remember which task and which circumstances that permission belonged to.

I want the agent, in a similar situation, to predict my decision more accurately instead of finding out my intentions from scratch every time.

Useful prediction is very practical: don’t propose a path that was already rejected, don’t ask me to repeat a permission that still stands, pull in the right experience, and run the next stage on time.

At the same time, my working style in one project must not become the preference of the whole team. The system’s confidence must not replace facts or widen its authority.

This kind of experience memory is already being studied. ExpeL and ReasoningBank look at accumulating and reusing agent experience. PRELUDE and CIPHER study preferences learned from user edits; their evaluation on emails and summarization with a simulated user does not prove anything for our process. ACE studies evolving playbooks and the loss of detail when context is rewritten.

Confident is a direction for now, not a proven solution to this problem.

Show me the result in the morning

My target is a task of several days, up to a week, with a clear result and an acceptable cost.

At a minimum you have to look at three things:

  • A finished result: was the original assignment done, and is it confirmed that the result works?
  • Cost: what did the models, tools and compute cost?
  • My involvement: how many reminders, repeated decisions, corrections and trips back into the context did it take?

A mode where the agent works on its own for a long time and a mode where I work next to the agent for a long time are worth completely different things to me.

When I leave a task overnight, I want to see progressed or finished work in the morning. If the work reasonably takes several days, let it take several days.

I don’t want to discover in the morning that the computation never even started because the agent was waiting for one more obvious message.

I need a result at an acceptable price, with an acceptable amount of my involvement.

The thickness of AGENTS.md is not part of that formula.

About this piece: the opening scene retells my own working scenario; the assessments of model speed and quality are personal experience. ultra and medium are the modes I use. I don’t yet have a controlled comparison of our instructions or overall statistics for these modes; a week of independent work is the target scenario, not a finished test.

Cite this article
Citation
Alexander Panasenko (2026-11-07). You Don’t Know If Your AGENTS.md Helps. You Just Keep Adding Rules. https://prod.codes/blog/agents-md-false-confidence/