Execution modes

A voice agent runs on one of three execution paths. You pick the mode under Mode Selection in the configuration screen.

Sync

Sync mode executes the agent synchronously — no queue involvement.

• Best for: low-latency interactions, conversational tools, anything where Time-to-First-Speech (TTFS) matters most.
• Pipeline: standard STT → LLM → TTS.
• Tools attached to a Sync-mode voice agent must be configured as Sync so they fit inside the latency budget. See Tools.

Async

Async mode routes calls to a dedicated voice priority queue.

• Best for: tool-heavy or slower workflows where waiting a beat for results is acceptable.
• Pipeline: standard STT → LLM → TTS, but with queued execution between steps.
• The voice priority queue is isolated from batch workloads — it has its own consumer concurrency and is monitored independently (queue depth, queue wait time, throughput).
• If the voice queue is unavailable, the system falls back gracefully to the shared queue.

Realtime API (OpenAI)

Realtime API mode executes through the OpenAI Realtime API instead of the standard STT → LLM → TTS pipeline.

• Best for: very low-latency, tool-light agents that benefit from OpenAI's native multimodal model.
• When Realtime API is selected, the configuration screen changes:
  • LLM selector shows only OpenAI Realtime API models (e.g., gpt-realtime-1.5). No provider dropdown.
  • Transcriber shows only Realtime-API-supported transcribers (e.g., gpt-4o-mini-transcribe). No provider dropdown, no language selection.
  • TTS shows only a voice selection (e.g., a dropdown of voices like "Marin"). No provider, no BYOK toggles — TTS is handled natively by the Realtime API.
• The Voice Activity Detection panel shows Realtime-API-specific turn-detection parameters: Threshold (0.00–1.00), Prefix padding (ms), Silence duration (ms), Idle timeout (toggle). Only the Normal detection mode is exposed; semantic turn detection and disabling turn detection are not surfaced.
• Base instructions uses the same field as the other modes — there is no separate "System instructions" field for Realtime API mode.
• Tools are attached using the same tool selection UI as other modes; tools map to the Realtime API's Functions concept.

Billing for Realtime API mode

OpenAI's Realtime API has its own pricing model (token + audio second). On the SimplAI side, the standard per-second voice metering still applies. If you BYOK the OpenAI key, your OpenAI account is charged for the Realtime usage directly; if you use the SimplAI key, that's bundled. See Pricing for the breakdown.

How to pick

• Need the fastest possible TTFS, with simple agents? → Realtime API.
• Need full control over STT/TTS providers and voices, with low latency? → Sync with carefully chosen STT/TTS.
• Doing heavier orchestration where queueing is acceptable? → Async.

You can change modes later, but switching modes will reset some mode-specific configuration (e.g., switching to Realtime API hides the standalone provider selectors).