$299.00
Simulation PM
Stress-test your support agent with 100 AI customers
For teams running an AI support or onboarding agent that needs to handle real customers reliably. I'll send 100 different persona-guided customer conversations to your actual test agent, including ambiguities, policy traps, unusual requests and escalation situations, and evaluate its responses against your rules. You receive the full transcripts, a scored failure breakdown and concrete fixes to test next. This tests the running agent, not just a generated list of prompts.
What you get
- 100 executed test conversations with your live test agent, across one customer-service or onboarding workflow.
- Up to 3 customer turns per conversation (maximum 300 simulated customer messages), with separate fresh sessions wherever supported.
- Scenario mix: routine requests, missing details, ambiguous intent, false assumptions, policy limits, difficult edge cases, refusals and handoff/escalation.
- Defined expectations: a task objective and policy-based success checks for each conversation.
- The actual transcripts: simulated customer messages, agent responses and conversation IDs, with no invented answers substituted for real outputs.
- A scored test matrix: pass, fail, or needs human review; error categories; and key examples with supporting transcript excerpts. Automated grading is used, with a human review of material failures and questionable assessments.
- Failure report: recurring patterns, the most consequential weaknesses, and up to 5 prioritized changes or regression tests.
- Reusable scenario file (CSV/JSON) for future runs, plus one clarification round.
Test and evaluation model usage is included. Scores measure performance on these simulated cases, not a certification of production reliability.
Output format
PDF evaluation report, CSV/JSON scenarios and scores, conversation transcripts, regression checklist
What to include
- A publicly accessible, automation-friendly test chat URL for one agent, with a way to start a fresh conversation. No login, CAPTCHA, email or phone verification.
- Permission to send up to 300 synthetic customer messages and to reset test conversations.
- One principal customer support or onboarding workflow to assess.
- Current product facts, policies and escalation rules (up to 5,000 words), including actions the agent must never take.
- Any must-test customer situations or known failure examples.
- A safe test environment: no real payments, refunds, tickets to real customers, outbound messages or irreversible actions. Disable real-world side effects for this test.
Confirm that the test endpoint works and permits automated interactions before purchase. Work starts after usable access and rules are supplied.
Expected completion
5 business days
Not included
Custom API, SDK or website integration; installing your agent; inaccessible or authenticated-only agents; more than one agent or workflow; unlimited dialogue length; real customer interactions; security/red-team certification; fixing prompts, tools or code; implementation of recommendations; or a retest after changes. If a test chat cannot start fresh sessions, contact me before ordering.