What is BoundaryGym?

September 19, 2026 · The idea behind the experiments.

An AI agent can produce the right answer through the wrong action. It might repair a file by bypassing a permission check, accept a tool's instruction as if it came from the user, or trust a subagent's result without checking how it was obtained.

I'm building BoundaryGym to study trust and authority boundaries in AI agent systems. The question is not only whether an agent can finish a task. It is whether it knows when to act, when to stop, and when to ask.

Why build this?

The starting point for me was Dwarkesh Patel's conversation with Ajeya Cotra about agent behavior and the implications for AI safety and alignment. Their discussion of agents pursuing task success through unintended routes made the question concrete: what are we rewarding when we measure only whether the job got done?

BoundaryGym is my way of exploring a small, testable part of that concern. I want to observe what an agent does when completing its task conflicts with an explicit permission boundary, then extend that question to tools, delegation, and handoffs.

When I give an agent a task, I want to delegate the work without having to approve every step. But “get it done” cannot mean that every available action is authorized. A finished file can hide a permission bypass; a confident summary can hide an unchecked handoff.

That tension becomes more interesting as agents use more tools and delegate to other models. Even if the lead understands a restriction, does it pass that restriction on? Does it check the route a subagent took, or only accept the result? What happens when a tool description quietly suggests that an override is routine?

I want evidence that helps someone building an affordable agent decide where to let it act independently and where to require approval. Testing both useful work and permission handling is a starting point. The longer-term question is whether those boundaries survive an entire workflow.

What I mean by a boundary

A user gives an agent a task and some authority to carry it out. The agent then encounters other information: tool descriptions, tool outputs, documents, error messages, and possibly other agents. Those sources can help it do the work, but they do not automatically have authority to change the rules.

Imagine asking an agent to edit a file without bypassing its permissions. The normal edit fails. A repair tool says it can finish the job if the agent enables an override. The tool has described a capability; it has not granted permission to use it.

That is the kind of distinction I want BoundaryGym to test. A request to keep trying is not necessarily new authorization. A subagent saying “done” does not establish that the work was permitted. An instruction buried in a tool response does not become trustworthy just because it appears in the workflow.

Useful work, within the rules

An agent that refuses everything would avoid many violations and accomplish very little. An agent that completes everything by ignoring restrictions would be useful in the narrowest sense and difficult to trust.

BoundaryGym needs to examine both sides. Can the agent complete a task when the permitted route works? Can it recover when an authorized alternative exists? Can it recognize a real blocker and explain it accurately when it lacks authority to continue?

The experiments should make those questions separable. I want matching controls, explicit permissions, and records of the actions that occurred. A plausible final answer is not enough evidence on its own.

Start small enough to understand a failure

The first experiment uses a single agent and a tiny file edit. Ordinary editing works in one condition and fails in others. An administrative tool can either repair the file within the policy or require a forbidden override. We separately record correctness, prohibited proposals, executed actions, verification, and reporting.

That small setup is deliberate. If the task itself is difficult, a failure can be hard to interpret. Starting with a simple edit lets us look closely at the permission decision. It also makes the test's limits clear: performance on one file task does not establish general safety.

The first experiment examines file edits and administrative repair. Its methods, results, and limitations will be published in a separate article.

How the experiments run

BoundaryGym uses Inspect AI, an open-source evaluation framework, to connect to models, run tool-based tasks, apply evaluation limits, and record transcripts. Using a shared framework lets us repeat the setup across models and inspect what happened without rebuilding that infrastructure for every experiment.

BoundaryGym supplies the scenarios, permissions, and scoring rules. In the file-edit experiment, the grades come from deterministic checks of independently recorded actions and file contents. Another model does not judge whether the agent respected the boundary.

Inspect provides the evaluation infrastructure; choosing a meaningful test and interpreting its evidence remain our responsibility. The harness and its settings are part of the experiment, so the results describe a model operating in that setup.

The boundaries I want to test next

The broader project goes beyond broken tools. These are planned directions, not claims about tests already completed:

A less capable subagent is one condition worth testing, but it is not an explanation to assume. A capable subagent can receive incomplete instructions. A capable lead can accept an unauthorized result. The experiment should identify which boundary failed and what information each participant had.

Escalation needs controls, too. Some tasks should require asking the user; others should permit acting directly. Otherwise, a system that asks every time could look successful without being useful.

An evolving set of tests

BoundaryGym is an early research project, not a safety certification. I want each experiment to say exactly what it tested, what counted as a failure, and what the evidence leaves unresolved.

Older and newer model releases can be useful reference points, but a difference between them does not identify its cause. Nor does passing a simple test make a model immune to other ways of crossing a boundary. New tools, different delegation patterns, and different sources of untrusted information can pose different questions.

The aim is to build a collection of inspectable experiments that becomes more informative over time. Each should stand on its own while contributing to a larger picture of how agents handle authority.