Your first suite

Before you start

• A non-production agent endpoint. Runs execute real tool calls.
• A credential for it, if it needs one.
• The path in its response where the tool record lives. This is the part people have to go and look up, do it first.

Connect it

Agents → Add agent. Fill in the endpoint and authentication, then the two mappings. Press Test connection before saving. It sends one real message and tells you three things: whether the endpoint answered, how long it took, and whether the tools path resolved.

Run the starter suite

Your workspace already has twenty scenarios. Run tests queues one item each; a worker picks them up and the live view fills in as results land. You do not need to keep the tab open.

Read the result properly

Resist the pass rate. It is the least informative number on the page. Go to Failures and open one.

1. The check that failed, named precisely. A missing tool call is a different problem from wrong arguments, which is different again from a claim with nothing behind it.
2. Expected against actual, as a diff you can act on.
3. The transcript, so you can see the conversation that produced it.
4. The tool record, so you can see what the agent actually did while saying that.

A first run against a real agent typically surfaces two or three genuine problems and one scenario whose expectations do not match your business rules. Fix the scenario, then fix the agent.