Engineering notes
Policies as code: making business rules testable
Refund windows and discount ceilings should be typed rules evaluated mechanically, not paragraphs buried in a system prompt.
If your refund policy only exists inside a prompt, you cannot test it, version it or prove it was followed.
Three rule types cover most of it
- REQUIRE, refund_order must be called before claiming a refund.
- DENY, never offer a discount above 20% without approval.
- ALLOW, escalation is permitted for damaged goods at any order value.
Version the rules with the business
Policies change. Record which policy version each run was judged against, so a historical failure still makes sense six months later.
Explainability comes for free
A rule engine names the rule, the turn and the missing action. Nobody has to interpret a score to know what to fix.
Reuse across agents
One policy set can govern chat, email and future voice agents. Consistency across channels is a compliance requirement, not a nicety.
What to take from this
- Extract rules from prompts into typed, versioned definitions.
- Mechanical evaluation gives explainable violations.
- Share one policy set across channels.
See it on your own agent
Connect an endpoint and run twenty scenarios. The first run usually finds something.
Start testingKeep reading
- Claimed, not executed
The most expensive agent failure is a confident sentence about something that never happened, and a transcript cannot see it.
- Not verified is not a pass
A test suite that quietly passes the checks it could not perform is worse than no suite at all.