AI / Tool use / Design brief
Before an agent acts, test the boundary.
A small evaluation brief for tool use: scope, meaningful approval, untrusted context, and uncertain outcomes.
A proposed experiment with six synthetic cases. No model comparison or measured safety result is claimed.
A useful answer is not the same as a permitted action.
An assistant can produce a convincing plan and still act on the wrong repository, grant the wrong permission, or repeat a change whose result it does not understand. For a tool-using workflow, I want the evaluation to include those boundaries—not just the quality of the final answer.
The starting point here is deliberately small: a fictional repository-access workflow with no network connection and no real credentials. The cases describe desired behaviour before a runner or model is chosen. That makes the assumptions available for someone else to challenge.
Write the contract before choosing the model.
Each case needs a request, an allowed scope, and an expected decision. A missing repository name should lead to clarification, not a guessed target. A request to prepare a change is not permission to execute it. If the target or permission changes, earlier approval should not silently carry over.
The expected decision alone is not enough. A fluent message saying “I will ask first” is a failure if the tool trace already contains the write. Evaluate the observable action and the explanation together.
- Keep read-only tasks read-only.
- Bind approval to an exact operation, resource, recipient, and permission.
- Treat retrieved documents as data, not as a new source of authority.
- Distinguish “not allowed”, “not specified”, and “outcome unknown”.
Include the case where nobody knows what happened.
A timeout does not prove that a state-changing operation failed. Automatically repeating it can create a duplicate action or make the eventual result harder to explain. One fixture therefore starts after a mocked write loses its response.
The proposed behaviour is to stop and report uncertainty, then use a safe status check or reconciliation step. The test should verify that no second write occurs just because the first response was missing. This is as much a tool-contract problem as a prompting problem.
The next step is a mocked-tool runner.
The downloadable fixture set contains six scenarios and no executable commands. A first runner could provide deterministic read and write stubs, capture a structured trace, and compare that trace with the expected behaviour. It should not connect to a real organisation or modify access.
There are no results yet. Before comparing models, the useful questions are whether two reviewers interpret a case the same way, whether the expected decision is too restrictive, and whether the trace captures an unwanted action. Those are concrete places to collaborate.
References & further reading
Have a counterexample or a better approach? That is a good reason to talk.
Discuss this noteExplore the open experiment