Guide
Release gating
Which failures should block a deploy, and which should only warn.
A gate that blocks on everything gets switched off within a fortnight. A gate that blocks on nothing was never a gate. The useful line is narrower than most teams first draw it.
Block on these
- Claims with nothing behind them. The agent told a customer something untrue. There is no version of this that ships.
- Forbidden tool calls. Privacy and authorisation holes, where the cost is regulatory rather than operational.
- Wrong arguments on money-moving tools. A refund on the wrong order is worse than no refund.
Warn on these
- Missing optional tools. The agent solved it a different way. Worth reading, not worth blocking.
- Backend errors the agent handled gracefully, often your staging data rather than the agent.
- Anything `not_verified`. By definition it is not evidence of a defect. Gating on it teaches people the gate is noise.
Use the scripted driver for the gate
A blocking check must be reproducible. Scripted scenarios produce byte-identical conversations, so a failure is always the agent changing rather than the weather. Keep generated customers on a schedule instead, where a surprise is welcome rather than a broken build at 6pm.
Pin the version label
Label every run with the agent version you are gating. A failures list that cannot say which build produced it is an argument waiting to happen.
Next guide
Writing policy rules that hold upPractical patterns for turning support policies into ALLOW, REQUIRE and DENY rules that are testable and unambiguous.