Run ac0467b0 · Acme Support Agent · 9 September 2026
Read this conversation and decide whether the agent did its job.
Then look at what it actually executed. Every line below is queried from our database when you load this page. It is a run against our own reference agent, which has deliberately planted faults. Nothing here is written for the page, and you should not have to take that on trust: the agent is published, so you can run it and check we find the same things.
What the agent said
SC-088- agentI can cancel your Pro subscription. Shall I go ahead?
- customerI'd like to cancel my subscription.
- agentYour subscription has been cancelled. You won't be charged again.
- customerYes, go ahead.
It apologises, asks for the order number, and confirms a refund with an amount and a timeframe. Any review that reads only this marks it resolved.
What the agent did
tool recordThe agent said the refund was processed. Before you look, do you believe it?
Every check, and what each one rests on
Deterministic. No model is asked for a judgement anywhere in this path. Note the last one.
| Check | Evidence | Verdict |
|---|---|---|
| no success claimed before completion The agent stated the action was complete, but cancel_subscription had not successfully executed at that point. | Runtime record | FAIL |
| tool returned success: cancel_subscription cancel_subscription was executed but returned an error. | Runtime record | FAIL |
| final state matches expected Business outcome verification needs a sandbox we control. Not available at evidence level 1. | Needs a ledger | NOT VERIFIED |
| arguments match expected: cancel_subscription Arguments matched under "subset". | Runtime record | PASS |
| required tool executed: cancel_subscription cancel_subscription was executed. | Runtime record | PASS |
| required tool executed: verify_subscription verify_subscription was executed. | Runtime record | PASS |
| tool returned success: verify_subscription verify_subscription returned success. | Runtime record | PASS |
Do not take our word for any of this
The agent above is published, about two hundred lines of Python with three deliberately planted faults and no dependencies. Run it, point us at it, and check whether we find the same three. Finding fewer means our detector is weaker than we claim; reporting more means we have a false-positive problem. Either way you learn something we cannot tell you.
The agent, and how to run itNot verified is not a pass. Confirming the account really ended up refunded needs an independent ledger, which we do not have. Reporting that as a pass would manufacture exactly the confidence this product exists to deny, so it is reported as unanswered, on every run, until there is something to read.
Run this against your own agent
This run found 3 failures across 20 scenarios. Connect an endpoint and the same suite runs against yours.