An AI safety monitor needs a coverage receipt

What the review does, and does not, establish

On October 22, METR published an account of a three-week exercise conducted with Redwood Research inside OpenAI from July 21 to August 11. The work examined an internal agent-monitoring system as it existed on and before July 21. That date boundary is central to the story: this is a historical assessment, not a measurement of today's complete safeguards.

The evaluators reported gaps in coverage and ways monitoring could potentially be subverted. They estimated that omitted compaction events had left roughly a tenth of eligible internal Codex traffic unmonitored during an earlier period, with substantial uncertainty. OpenAI had independently identified that issue and fixed it during the exercise. METR says OpenAI subsequently stated that vulnerabilities affecting its production safeguards had been patched; the evaluators had not verified those claims.

Our interpretation is that the report gives ordinary builders a useful question to ask about any agent product: can you account for the actions outside the monitor's view? A reassuring dashboard is difficult to assess unless you know what could have appeared on it in the first place.

This article develops a proposed way to ask that question. It does not reproduce exploits, claim a current vulnerability, or treat every agent as equivalent to OpenAI's internal infrastructure. The useful connection is the measurement problem. If an assistant performs several actions before handing back control, a final answer alone cannot tell an owner whether the surrounding process was observed. A clear account of coverage is part of making human oversight meaningful.

Source: METR: Red-Teaming OpenAI’s Internal Agent Monitoring Systems ↗ · METR: Our approach to working with AI companies ↗ · NIST: AI RMF Core ↗

Start with a denominator you can explain

Imagine a hypothetical creator business using an agent to prepare newsletters. In a demonstration run, the assistant reads a source folder, drafts an email, updates a spreadsheet, and queues a send. The owner receives a green completion badge. What exactly did that badge measure?

A proposed coverage receipt would begin with the expected action types. Record the operations the workflow permits and which system records each one. Separate the assistant's own narrative from records produced by the service that performed the action. A claim that everything was monitored needs a definition of everything.

For a small experiment, you could make a paper inventory before running anything. List reading, drafting, file creation, approval, and sending as separate stages. Beside each, write where evidence would come from and whether a human can inspect it. If you do not know, leave the field unknown. Unknown is a useful finding because it gives you a question for the provider.

Avoid assigning a precise coverage percentage when your inventory is incomplete. Ten observed actions out of ten visible actions does not establish that ten actions were attempted. This is our measurement example, not a finding about a particular commercial service.

The practical payoff is a better conversation about trust. Instead of asking whether a provider has monitoring, ask which action classes are included, which are excluded, and how exceptions are reported. A modest, documented boundary is easier to reason about than an expansive promise without a denominator. It also makes comparisons fairer: two tools may use the same monitoring label for very different scopes.

Separate observation, interruption and permission

For evaluating a tool, it helps to draw three boxes. One concerns what an agent may do. Another concerns what its activity reveals. The third concerns what can stop or redirect it. These are our proposed review categories. Treating them separately prevents a strong feature in one box from being mistaken for a complete answer in the others.

In the newsletter example, an agent might be allowed to create a draft while lacking permission to send it. That permission boundary has a clear purpose even before anyone looks at a monitor. Observation would answer what draft was created and why it was flagged. Interruption would answer whether a pending action can be stopped before it takes effect.

Ask concrete questions. Does a human approval apply to one exact item or a broad class of future actions? Can the reviewer see the destination and content they are authorizing? What happens if the item changes after review? The answers depend on the product. Do not infer them from a button's reassuring name.

A useful review exercise is to write a hypothetical failure for each box. A permission might be broader than intended. A record might omit a stage. An alert might arrive after an irreversible action. Then write the evidence that would show the boundary works as expected in your chosen workflow.

This approach does not promise perfect safety. It gives you a way to identify which uncertainty matters for your use. For a draft-only personal experiment, a missing notification may be tolerable. For a workflow acting on other people's data or communicating externally, the same uncertainty deserves closer review before expanding access.

Make silent failures visible in a harmless rehearsal

A good monitoring question concerns the boring days when something fails to report. In a proposed rehearsal, ask the provider or your own development team how a failed monitor call appears. Is it clearly distinguished from a completed review that found no issue? Does the workflow pause, continue, or require a separate decision? This article does not establish how any particular service answers those questions.

You can practice the reasoning without probing anyone's infrastructure. Use a paper scenario or a sandbox you own with non-sensitive sample data. Write an expected event, an expected record, and an expected response. Then consider a missing record. The aim is to test your plan for dealing with absence, not to bypass a safeguard.

For the newsletter example, suppose a draft exists but the review record is missing. A reasonable proposed policy would keep sending unavailable until the owner resolves the missing review. That is a design choice for the example, not an instruction to change a live system without understanding its requirements.

Keep the rehearsal small enough to inspect manually. If you cannot explain the expected outcome of one draft, expanding to hundreds of autonomous actions will make the uncertainty harder to diagnose. Record what you expected, what you observed, and which question remains.

The most useful result may be discovering that your team has no agreed response to a missing record. That is an opportunity to clarify responsibility before it becomes an incident. A monitor becomes more useful when its own operational status is visible to someone able to act, rather than disappearing inside a single all-purpose success badge.

Independence has several parts

METR's separate October 14 explanation of its work with AI companies emphasizes access, evidence transparency, limitations, and financial and operational independence. These are distinct considerations. Deep access can make an assessment more informative, while publication constraints can still limit what outsiders can inspect.

For readers, a proposed review checklist has four questions. Who chose the scope? What could the evaluators inspect? What could they publish? Which follow-up claims did they verify? You do not need to assume bad faith to ask these questions. They describe how much weight a public conclusion can bear.

Imagine two hypothetical assessments. One has extensive internal access but publishes only a high-level account. Another has no internal access but releases a reproducible public test. Each could be valuable for a different question. Neither automatically answers every question about a system.

When sharing a finding, preserve the evaluator's uncertainty rather than compressing it into a stronger headline. If a reviewer examined a historical snapshot, keep the snapshot date close to the claim. If a company reports a fix, distinguish the company's statement from independent confirmation. If something was only considered possible, do not describe it as an observed real-world incident.

The ethical point is practical: precision helps people make decisions without either panic or false reassurance. It also gives companies and evaluators a clearer target for follow-up. A useful response can say what changed, what evidence supports that statement, and what another assessment would need to check.

This is a reading habit anyone can apply. Before reposting an alarming sentence, ask whether its time period, confidence, and scope survived the trip from the original report.

A small team can write a useful oversight agreement

NIST's AI Risk Management Framework organizes work around Govern, Map, Measure, and Manage. Its companion materials are voluntary guidance, not a certification conferred on a tool by mentioning the framework. For a small project, our recommendation is to translate that broad idea into a short agreement people can actually use.

Name the person responsible for the workflow. State what the agent is intended to do and which actions require approval. Identify the evidence you will inspect. Define how you will respond when a required check is unavailable. Set a review point for changes to the model, tools, or permissions. This is a proposed lightweight practice, not a claim that completing a page satisfies a particular compliance obligation.

Use plain examples. The agent may prepare a newsletter draft from approved public sources. It may not send messages until a named reviewer approves the exact draft and audience. If the review record is missing, the draft stays pending. The point is to make responsibility and boundaries legible to the person operating the system.

Include a way to reverse or contain mistakes where the workflow permits it. For example, retain the previous draft and a record of the approved version. Do not promise reversibility for actions that cannot be undone. That distinction should influence what you automate.

Finally, ask the intended reviewer to explain the agreement in their own words. If they cannot identify what they are approving or where to find evidence, the process needs revision. Human oversight should describe an achievable job, with enough information and time to do it, rather than simply assigning blame to the last person who clicked.

What a coverage receipt could look like

Here is a proposed final deliverable for your next agent experiment: a brief receipt alongside the output. It names the workflow and version, identifies the action types expected, points to the available records, lists missing evidence, and states whether a human approved the consequential step. It should use ordinary language rather than a wall of unexplained status codes.

For the hypothetical newsletter, the receipt might say a draft was created from an approved folder, the spreadsheet update was recorded, and sending remains pending. If a stage was not checked, say so directly. Do not substitute a confidence score for the missing evidence.

This receipt is our suggested design pattern, not a feature we tested or a capability the METR report promises. Its value is that someone else can challenge it. A reviewer can ask why a stage is absent, whether the record corresponds to the exact draft, or what changed since the last run.

Keep the receipt proportionate. A personal learning sandbox does not need enterprise paperwork. It does need an honest account of what happened. A consequential workflow may require much more than this starting point, including specialist review and stronger technical controls.

The central lesson of the new report is a question rather than a verdict about every current system. When someone says an agent is monitored, ask what activity that statement includes, when the claim was established, and how missing observations are handled. The answer should leave you with a traceable boundary. That is a stronger foundation for confidence than an unexplained green check, and a more constructive basis for improving the system when the boundary falls short.