Skip to main content

Evaluation

Voice Agents support the platform's evaluation framework, but evaluations operate on conversation transcripts rather than on input/output pairs.

What evaluations run on​

For voice agents, evaluators are run on:

  • The full conversation transcript of a call, or
  • Per-turn text within the transcript.

The standard eval criteria editor is unchanged — you build the same evaluators you would for a text agent. What changes is the input data: voice eval input is always a transcript, never a structured payload.

What's hidden for voice​

  • Dataset selection is not applicable for voice agents and is hidden in the eval configuration UI. Evals run on past run-history transcripts, not curated datasets.

What's available​

  • Run evaluators against a single voice run from Run history.
  • Run evaluators in scheduled evaluation runs over a sample of recent transcripts (using the cron-based scheduled evaluation framework — see Evaluation).
  • Aggregate eval results across runs and versions.

Trace-level evaluation​

Trace-level evaluation (running evaluators against individual spans within a run) is not yet supported for voice agents. Today's voice evals are at the call / transcript granularity.

Tying eval results back to call quality​

Voice evals are most useful when paired with call recordings and the audio-synced trace view. For each evaluated run, open the run from Run history to listen to the recording while reading the eval verdict — this is often where transcription errors, mis-VAD'd turns, or fallback events become obvious.