Engineering notes
The five layers of agent reliability
Conversation, policies, tools, outcomes and regressions, a practical model for deciding what to evaluate and in what order.
Most evaluation setups test one layer well and ignore the rest. The gaps between layers are where production incidents live.
Layer 1, Conversation
Did the agent understand the request, stay coherent across turns and communicate clearly? This is the layer LLM judges handle well.
Layer 2, Policies
Were business rules followed? ALLOW, REQUIRE and DENY rules evaluated mechanically against the transcript and trace, with the violated rule named.
Layer 3, Tools
Did the required actions execute, in the right order, with correct arguments, and did any silently fail?
Layer 4, Outcomes
The layer we cannot finish
This is the one ActionProof does not yet complete. Confirming an account really ended up in the right state needs a ledger independent of the agent, and we do not have one. Every run reports that check as not_verified rather than passing it, which is the honest version of a gap we would rather not have.
Was the customer's problem actually resolved, and is the resulting system state correct? Resolved-correctly, not just resolved.
Layer 5, Regressions
Does this version still handle everything previous versions handled? The layer that protects the work done in layers one to four.
What to take from this
- Each layer catches failures the others cannot see.
- Run deterministic layers before model-graded ones.
- Regressions are a layer, not an afterthought.
See it on your own agent
Connect an endpoint and run twenty scenarios. The first run usually finds something.
Start testingKeep reading
- Claimed, not executed
The most expensive agent failure is a confident sentence about something that never happened, and a transcript cannot see it.
- Not verified is not a pass
A test suite that quietly passes the checks it could not perform is worse than no suite at all.