Guide
Connecting your agent endpoint
Authentication, request and response mapping, conversation state and tool trace export, the setup details that decide test quality.
Ninety percent of setup problems are one of four things. Here they are in order of frequency.
Authentication
Use a scoped key for testing with access to a sandbox dataset. Never point a suite at production orders, scenarios will attempt real refunds.
Request and response mapping
Tell the runner where the user message goes and where the agent reply, conversation ID and tool trace come back. Nested payloads are fine; ambiguous ones are not.
Conversation state
Multi-turn testing needs a stable conversation identifier. If your API is stateless, the runner must resend history, declare which mode you are in.
Exporting the tool trace
This is the step teams skip, and without it tool validation cannot run. Export call name, arguments, result status and timestamp for every call.
Verify with a probe run
One scenario, read end to end. Confirm the trace is populated and the arguments look right before scaling to a full suite.
What to take from this
- Always test against sandbox data.
- No exported trace means no tool validation.
- Probe with one scenario before running 128.
Next guide
How to read a version comparisonFixed failures, new regressions and behaviour changes, what each section means and which numbers to distrust.