Testing voice agents that quote prices: a QA playbook for retail and sales voice AI - Roark

Something quietly shifted this summer. Voice agents built on OpenAI's GPT-Realtime aren't just answering questions anymore, they're holding real sales conversations, and the retailers deploying them are getting the numbers they were promised. Japanese home-appliance chain Yamada Denki ran a public campaign where roughly 30,000 shoppers talked to a 24/7 multilingual voice agent trained on their sales associates' expertise, and 92% of post-conversation surveys came back positive. The average enterprise now runs 13 agents in production, up from 5 in early 2025, per Salesforce's Agentic Enterprise Index. Voice is where a lot of that growth is happening.

That growth comes with a specific new failure mode: an agent that misquotes a price, invents a discount, or agrees to a policy that doesn't exist. On a chat surface, a bad quote is a screenshot. On a phone call, it's a verbal commitment, and regulators and small-claims tribunals have already made clear that "the chatbot said it" is not a defense. This post is about how to test agents that quote prices before they cost you money, and how to catch the ones that quote wrong once they're live.

When "the agent said it" becomes the company's problem

The precedent every voice-AI product lead should know is Moffatt v. Air Canada. A grieving passenger asked Air Canada's website chatbot about bereavement fares. The bot confidently described a retroactive-refund policy that didn't exist, and the airline refused to honor it. The British Columbia Civil Resolution Tribunal ruled in February 2024 that Air Canada was liable, treating the error as negligent misrepresentation and ordering the airline to pay the fare difference. Air Canada's defense was that the chatbot was "a separate legal entity" responsible for its own actions. The tribunal called that submission "remarkable" and rejected it flatly, on the reasoning that the chatbot was still just part of Air Canada's website and the airline was responsible for all the information on that site.

That reasoning doesn't narrow when you swap chat for voice. If anything, it widens: a voice quote lands faster, sticks harder, and is harder to walk back in the moment. The caller hangs up with a number in their head. The recording is either your defense or the plaintiff's exhibit.

Layer on top the fact that, since 2 August 2026, Article 50 of the EU AI Act requires that voice agents interacting with people in the EU disclose that they are AI, with penalties reaching €15 million or 3% of global turnover. That doesn't limit price-quote liability, but it does mean the caller now expects to be talking to a machine, and courts will treat the transcript accordingly.

What "wrong price" actually means on a call

Wrong is not just a made-up number. On a live sales or support call, the failure modes are:

Each of these leaves an audio recording that says the same word to the caller: "your price is X." Each of them is testable if you know what "right" looks like.

Define correct before you test for wrong

The most common failure I see is teams jumping to "we need a hallucination check" without a written source of truth. You cannot detect a wrong quote without a machine-readable version of the right quote.

For a sales agent, that usually means three artifacts, kept in one place and versioned:

  1. A price catalog with every SKU, tier, currency, and effective date range.
  2. A promotions engine that enumerates every discount, its eligibility rules, and its expiry.
  3. A policy document for anything the agent is allowed to speak to (refund windows, warranty, cancellation, delivery, escalation thresholds).

If it isn't in those three, the agent shouldn't be quoting it. If the agent does quote it, that's a test failure, not a rounding error. Getting this right at the artifact level is the biggest quality lever most teams have and the one they most often skip.

The test plan before launch

Once you have a source of truth, simulation is straightforward. You put your agent on a real phone line, and you dial it with a synthetic caller that pushes on every place a wrong quote can appear.

The scenario library that pays for itself:

Personas matter as much as scenarios. A skeptical enterprise buyer, a first-time consumer, an angry return-seeker, and a bilingual code-switcher will pull different failure modes out of the same agent. Roark dials agents over real telephony with personas that vary voice, language, accent, speech pace, emotional register, and background noise, across 45 languages and accents. You want your scenarios crossed with those axes, not run against a single default caller reading a script.

Old-way sampling versus scoring every quote against the catalog

Every scenario needs an explicit expected answer: the exact price, tier, term, and any conditions. If your harness only checks "did the agent respond fluently," you will ship an agent that fluently misquotes.

Metrics that catch price errors

Overall pass/fail is not enough. For agents that quote, four families of metric earn their place on the dashboard:

That last one matters because a lot of the eval tooling on the market grades a transcript and stops. Voice agents don't read scripts, they speak them, and the quality of the call is not just what was said but how. Roark scores calls with audio-native models that grade pronunciation, emotion, vocal stress, pace and pauses, and interruptions on top of transcript-level checks. A quote delivered at the wrong pace with a hedged tone is a quality signal even when the number happens to be right, and it's exactly the kind of signal that gives you a chance to catch a wobble before a customer files a complaint.

Score every production call, not a sample

Sampling is where price-quote testing usually breaks in production. You cannot QA one call in a hundred and then argue in front of a tribunal that the other 99 were fine. Every call your agent handles that involves a price, a promotion, or a policy statement needs to be scored against the source of truth, automatically.

That means:

Illustrative pre-launch simulation of a price-quote suite

That last point is worth pausing on. GPT-Realtime is now generally available with new voices, remote MCP servers, and SIP calling built in, per OpenAI's own announcement. Model swaps and tool-graph changes are going to be normal, not exceptional, for the next couple of years. If you don't have every production call automatically graded against your source of truth, you will not know when the swap silently changed how your agent quotes prices, until the third or fourth caller says something on Twitter or in a tribunal.

Turn a failing quote into a regression test

The last piece is the loop. When a live call misquotes a price, you don't want to fix the prompt and hope. You want the exact call, with the exact caller audio, dialed again against the fixed agent before you redeploy.

That's what production call replay is for: capture the real call, replay it against the updated agent logic, and confirm the fix holds. Do it every time a wrong quote surfaces, and the second occurrence never happens. Do it as part of CI on any prompt change, model swap, or catalog update via the Roark SDK, and you stop shipping the same class of error twice.

A price-quote regression suite that grows every time production embarrasses you is boring, which is the point. Boring compliance is the goal.

The short version

The agents that will earn their place on the sales floor over the next year are the ones whose builders treat every quote as a test case. The rest will be tomorrow's tribunal decisions.

Written by

Daniel Gauci Mizzi · Co-founder & CTO @ Roark

Building Roark — the quality platform that simulates, monitors, and auto-improves voice and chat agents.