All notes

Engineering notes

Policies as code: making business rules testable

Refund windows and discount ceilings should be typed rules evaluated mechanically, not paragraphs buried in a system prompt.

22 July 20267 min read

If your refund policy only exists inside a prompt, you cannot test it, version it or prove it was followed.

Three rule types cover most of it

  • REQUIRE, refund_order must be called before claiming a refund.
  • DENY, never offer a discount above 20% without approval.
  • ALLOW, escalation is permitted for damaged goods at any order value.

Version the rules with the business

Policies change. Record which policy version each run was judged against, so a historical failure still makes sense six months later.

Explainability comes for free

A rule engine names the rule, the turn and the missing action. Nobody has to interpret a score to know what to fix.

Reuse across agents

One policy set can govern chat, email and future voice agents. Consistency across channels is a compliance requirement, not a nicety.

What to take from this

  • Extract rules from prompts into typed, versioned definitions.
  • Mechanical evaluation gives explainable violations.
  • Share one policy set across channels.

See it on your own agent

Connect an endpoint and run twenty scenarios. The first run usually finds something.

Start testing

Keep reading

  • Claimed, not executed

    The most expensive agent failure is a confident sentence about something that never happened, and a transcript cannot see it.

  • Not verified is not a pass

    A test suite that quietly passes the checks it could not perform is worse than no suite at all.