It handled 2 English handoff requests in a support-appropriate 2-step flow, saying a real person had been asked to join, requesting an email address, and then staying silent after takeover.
What was measured
Conversational Quality
Responds in a clear, natural, and support-appropriate way.
decisive for this rankingtransformation
This is core to whether the bot can actually provide support in a clear, natural, customer-facing way. (3 of 3 judges)
What was given, what came back
Test input: Customer-support handoff request · text
Input — what we sent
The exact prompt
I received the wrong item in my order. I've already checked the order details and this is clearly a mistake on your end. I don't want any more back and forth — can you please connect me to a customer support agent or raise a ticket for this?
A frustrated support-escalation request after receiving the wrong item, asking to be connected to a support agent or have a ticket raised.
Why this input is hard
- · Human handoff reliability
- · Ticket creation workflow
- · Escalation contextual awareness
- · Professional tone under frustration
Output — unretouched
Also checked on this input — same tool, 2 other criteria
Conversational Safety Behavior✓ WorkedFor a double-charge complaint, it safely treated the issue as a possible authorization hold, asked the user to check pending-vs-posted status, and only proposed manual-refund escalation if the charge was still present after 5 business days.Multilingual Understanding✓ WorkedIt correctly understood 2 escalation requests written in Hindi and Spanish, responded in the same language, and asked for an email address for follow-up.
Provenance
- Observation
- 92ac5388-71d1-48c8-b32f-c0ef58c76707
- Evidence run
- 2645dc92-49df-478a-b809-21dfd09f06a7
- Study
- Automate customer support using an AI chatbot
- Research task
- 86b9jm3ev
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "jotform"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 3 other tools
measured on Conversational Quality
Fin◐ MixedThe bot acknowledges the request to speak to a real person and offers a human-agent connection, but it also tries to keep the conversation with the bot first.FS Agent✓ WorkedHandles a human-handoff request in a clear, support-appropriate way by acknowledging the user and starting ticket intake with an email address and issue description.Kommunicate✓ WorkedRespects the user's request not to talk to a bot and transfers to a human right away, which keeps the interaction support-appropriate.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com