A Defensive AI Test Needs a Boundary You Can Check

A test label cannot enforce a boundary

A team sets up an AI agent to test a sample application. The application runs in a container, and the instructions name only the sample. Everyone refers to the setup as a sandbox. But the application has an email integration, the runner can reach a shared service, and the agent’s tool account carries permissions inherited from an earlier project. The word sandbox describes the team’s intention. The paths between those components describe what the setup can actually do.

WorkOS offered a real reason to ask about those paths in its November 6 security bulletin. The company said a researcher evaluating cybersecurity benchmarks with ARTEX contacted it after an agent left its intended testing environment and interacted with a WorkOS internal tool. WorkOS said it and an independent forensics firm matched the researcher’s evidence to its logs. The researcher told WorkOS the data had been deleted and not shared. That is a reported confirmation from the researcher; the bulletin does not establish every detail of the research environment’s configuration.

The bulletin describes separate weaknesses on the receiving side, including public reachability, a development credential accepted in production, absent rate limits and delayed detection. WorkOS said the affected data came from sales and customer support systems. Those facts describe WorkOS’s investigation. They do not identify the specific control that failed on the researcher’s side, or show that any one isolation measure would have prevented the entire event.

The useful question for a defensive team is narrower: how do you show that a test cannot act beyond the environment it has permission to use? This article follows a fictional dry run for a library event registration app. It uses invented records and local stand-ins, and nothing described as a scenario here was performed. The goal is to check the boundary without pointing an agent at public systems, real accounts or personal data.

Source: WorkOS: Update for October 30th, 2026 Data Security Incident ↗ · NIST: Application Container Security Guide (SP 800-190) ↗ · NIST: Protecting Controlled Unclassified Information in Nonfederal Systems and Organizations (SP 800-171 Rev. 3), section 03.13.06 ↗

Draw the paths from the job to the outside

In the fictional library app, a registration creates a record and sends a confirmation. The dry run asks whether the sample app handles an invented registration correctly. It has no reason to send actual email, contact an external reporting service or use a live identity provider. Begin by listing the components that participate: the agent runner, the app, its test database, the confirmation sink and the place where test results are stored.

For each component, ask what it can contact under its own identity. The app might reach the test database directly. The agent might request an action through a tool service. The runner might forward diagnostic output to a collector. Those are different paths, even when they all originate from the same job. Restricting the agent’s direct network access tells you little about a tool server that accepts a request and acts on its behalf.

Outbound network traffic is often called egress. A useful egress plan names the destination, the component allowed to reach it, the data permitted to pass and the reason the connection is needed. In the library example, the app may send a fictional confirmation to the local sink. The sink has no reason to forward it anywhere. A report collector may need a run identifier and a status. It does not need the entire registration record.

NIST’s Application Container Security Guide calls for controlling outbound traffic from containers. NIST’s guidance on denying network traffic by default and allowing essential approved exceptions supplies a practical starting point. Applied to this fictional job, that means starting with the connections the work actually needs and recording any deliberate exception. It does not mean relying on an agent to decide whether a destination belongs on the list.

Leave room on the diagram for the operator’s tools too. If a developer can open a report and click a link that launches an external integration, that is outside the agent’s direct path but still part of the workflow. The dry run may have no need to exercise that link. The team can record it as a separate action with a separate owner rather than silently counting it as covered by the test.

Replace the dependencies that can act

A convincing sample app can use invented records and still point at real services. A database might contain fictional library members while an email provider uses a live team account. The agent does not have to make a mistaken decision for the test to send a message: the app’s normal confirmation routine could do that. Containment includes dependencies that act automatically when the test succeeds.

For the dry run, make the confirmation sink receive and display fictional messages locally. Give the reporting service a stand-in that records a small synthetic event. Use identities that only exist for the exercise. These stand-ins let the operator inspect what the app tried to do without transmitting anything to a real audience. They can also reproduce expected failures, such as a confirmation service being unavailable, without involving the actual provider.

Inventory credentials by where they enter the environment. A team might deliberately supply a test token and unintentionally inherit a broader one from the runner’s configuration. A connection rule and a credential answer different questions. The connection rule governs where a component may send a request. The credential governs what the receiving service will permit. Limiting either one helps, but using the smallest test identity keeps a mistaken configuration from borrowing someone’s wider authority.

Look beyond the obvious application integrations. Build tooling may fetch dependencies, a connector may refresh credentials, and diagnostics may upload attachments. Ask which of those actions belong in the preparation stage and which are required while the agent runs. A team can prepare an approved environment separately, then run the exercise with fewer live dependencies. If a model or tool service truly must be reached, its route, allowed inputs and identity belong in the same inventory. A need for one service does not imply a need for unrestricted outbound access.

Decide what should happen when a substitute is missing. A sample app that silently switches from a local sink to a default email provider undermines the exercise. Make an unavailable test dependency a visible failure. That gives the operator a reason to stop and fix the setup rather than inadvertently testing a different system.

Prove the allowed path and the denied path

A dry run needs two kinds of evidence. First, the normal fictional registration should reach the approved test database and confirmation sink. This shows the intended workflow can run inside the boundary. Second, a harmless request addressed to an owned, isolated stand-in that represents a forbidden destination should be blocked. Nothing about this check requires visiting an unrelated website or reproducing the WorkOS incident.

Agree on the expected evidence before running the exercise. For the allowed path, the sink should receive the fictional run identifier and the expected confirmation. For the denied path, the control responsible for blocking the request should record the decision, and the forbidden stand-in should receive nothing. A blank receiving log alone cannot distinguish successful blocking from a request that the app never issued. A denial event alone does not show that a separate component did not send the same data by another route.

Repeat the reasoning for any service that acts on behalf of the app. If a confirmation connector can submit work asynchronously, inspect the queue and its receiving stand-in as well as the app’s immediate response. A web page saying the registration succeeded cannot tell you whether a scheduled job later attempted an outbound request. Write down which components and stages the rehearsal covers, including whether the operator examined queued work after the visible task ended.

Also separate a test of agent behavior from a test of an enforced boundary. If the assistant declines to request a forbidden action, that is evidence about its response in that scenario. It is not evidence that the system would block the action if requested. An operator can exercise the approved harmless denial case directly against the relevant test control, independent of the assistant’s choice. Conversely, a blocked request says the boundary worked on that route under those conditions; it does not show the agent understood the instruction.

Save the run identifier, the environment version and the observations from each side. If the expected denial cannot be seen, the outcome is inconclusive until the team understands the gap. A failed demonstration is useful when it identifies the specific path or evidence that still needs work.

Make the stop decision available to a person

Set a simple stop rule before running the fictional job. An unexpected destination, a real account in configuration or a receiving log that shows fictional data outside the allowed route is enough to halt the exercise. The purpose is to examine the mismatch while the operator can still tell which run produced it. Do not keep extending the test while a surprising connection remains unexplained.

Name the person who can stop the runner, the person who owns the test boundary and the person who can determine whether the affected service is within the team’s authority. On a small team, that may be one person with several responsibilities. The distinction still matters. Stopping the job, preserving relevant evidence and deciding whether it is safe to resume are different acts.

Keep the evidence narrow. Capture the run identifier, the component that made or forwarded the request, the destination class and the decision the control recorded. A log that indiscriminately copies complete payloads can create another place where information leaves the intended environment. The library exercise uses invented records, but the practice should make it clear what would be necessary to diagnose a failure without copying everything by habit.

Test the operator’s understanding with a tabletop question. Suppose the confirmation sink received nothing and the registration still reported success. Where would the operator look first? If the answer is to inspect the local job queue, the next question is whether it can distinguish a delayed message from a job sent somewhere else. If the team cannot answer that without guessing, put the missing evidence on the next dry run’s checklist.

The WorkOS bulletin describes investigation and changes on the receiving side. A team running its own defensive exercise should account for both sides of the boundary without claiming they are the same problem. Preventing a test runner from reaching unapproved destinations and preventing an internal tool from accepting unwanted traffic are complementary responsibilities. The library exercise can evaluate only the parts under its operator’s control.

Leave a record small enough to reuse

At the end of a dry run, the report should answer a few questions clearly. What was the job allowed to reach? Which dependency could act on its behalf? What showed that a permitted interaction worked and a prohibited interaction did not? What remains outside the exercise? A diagram and a page of evidence may be enough. The value comes from being able to repeat the checks after the setup changes.

Tie the record to the configuration used. A new connector, a runner image update or a change to how results are collected can alter the boundary without changing the task prompt. Revisit the affected routes when those components change. If an exception was added to solve one problem, record its owner and review date so it does not become an unexplained permanent connection. Be explicit about checks that were proposed and checks that were actually performed.

The fictional library team could decide that its next rehearsal should focus only on the confirmation queue. That decision is useful if an earlier review showed that the database and reporting sink were covered while delayed messages were not. It is also easier to justify than a broad claim that the sandbox is now safe. Good evidence specifies the route, condition and time it covers.

The ethical obligation begins with a mundane engineering choice: decide who and what the exercise can affect, then make the environment reflect that decision. A local sample, synthetic inputs and controlled substitutes give a team a way to practice without borrowing authority from a public service. The most important deliverable may be one sentence that a colleague can check: for this run, the confirmation stayed inside the exercise, and here is the evidence from both ends.