What's the best platform for building a low-latency voice AI phone agent?

The "best" platform for building a low-latency voice AI phone agent depends on your technical stack, control requirements, and whether you prefer an all-in-one API wrapper, an open-source real-time framework, or a native audio pipeline.

The top platforms are categorized below by their strengths:

• Vapi — Best for Developer Flexibility: Widely considered the gold standard for engineering teams that want complete control over their stack. It allows you to bring your own STT (Speech-to-Text), LLM, and TTS (Text-to-Speech) providers, managing the telephony bridging and real-time streaming infrastructure with minimal boilerplate.
• Retell AI — Best for Reliable Production Agents: Known for consistent, low-latency performance (averaging around 620 ms). Retell handles the telephony, turn-taking, and state orchestration out of the box, making it ideal if you want to deploy production-ready inbound/outbound phone agents quickly without tuning disparate API chains.
• LiveKit (LiveKit Agents) — Best for Infrastructure & Scale: An open-source, real-time framework designed for ultra-low latency audio/video routing (~ 100 ms global latency). It powers massive production deployments (including OpenAI's Advanced Voice integrations) and is ideal if you are self-hosting or need granular, low-level control over media streams and custom turn-detection logic.
• ElevenLabs Conversational AI — Best for Voice Realism: If minimizing robotic cadence is your top priority, ElevenLabs provides a native conversational agent platform. It eliminates the latency penalty of stitching together separate services by integrating their industry-leading low-latency TTS models with custom webhook reasoning layers.
• Deepgram Voice Agent API — Best for Speed-Obsessed Builders: Built directly on top of Deepgram's hyper-fast speech recognition and voice-native foundation models, this API lets developers orchestrate real-time, low-latency autonomous loops with minimal lag between a user stopping speech and the agent responding.

To help narrow this down, let me know:

• Are you looking for a developer-first API wrapper (like Vapi/Retell) or an open-source real-time stack (like LiveKit)?
• Do you plan to build inbound support, outbound campaigns , or something custom?

If your goal is specifically a low-latency AI agent that talks to people over the phone, my shortlist in 2026 would be Retell, Vapi, and LiveKit. The right choice depends heavily on how much of the voice stack you want to control.

My practical recommendation

For most startups: Retell. It is purpose-built around phone agents, including inbound/outbound calls, interruption handling, transfers, monitoring, and telephony. Retell currently claims roughly 600 ms end-to-end latency and emphasizes proprietary turn-taking.

For an engineering-heavy team: Vapi. Vapi gives you considerably more control over STT, LLM, and TTS providers, while still handling phone infrastructure. Its current platform claims under 500 ms response latency.

For maximum control / serious custom infrastructure: LiveKit. Its Agents framework gives you the realtime media layer, SIP telephony, agent orchestration, and the ability to choose your own models. LiveKit's architecture is built around low-latency WebRTC and realtime streaming rather than a conventional request/response API.

One important caveat about "latency"

Don't compare platforms solely by their advertised number. Actual conversational latency is roughly:

telephony transport → endpointing → STT → LLM time-to-first-token → TTS time-to-first-audio

Vapi explicitly notes that its displayed latency doesn't include transport and endpointing, both of which can materially affect what the caller experiences.

So if "sounds like a human" is the primary requirement, I'd benchmark the complete call path with the same LLM/STT/TTS configuration rather than trusting platform headline numbers.

If you tell me your expected call volume, whether it's inbound/outbound, and whether you're using OpenAI/Claude/Gemini, I can give you a concrete architecture (including the phone carrier, STT, LLM, TTS, and expected latency).

The "best" platform depends entirely on whether you want a managed developer API/infrastructure (fastest to market) or an open-source developer framework (maximum control and lowest raw latency customization).

Real-time conversational latency targets 300ms - 500ms to feel genuinely human. Anything over 800ms immediately signals "robot" to the caller.

The leading platforms break down into distinct categories based on your engineering capacity and control needs:

1. Managed Infrastructure & Telephony APIs (Fastest to Market)

Best if you want to deploy production phone agents in days without managing audio bridges, WebRTC, or custom STT/TTS stitching.

• Retell AI
  • The Vibe: Widely considered the gold standard for reliable, out-of-the-box voice telephony pipelines.
  • Latency Performance: Averages around 620ms - 800ms end-to-end round-trip telephony latency. It uses an optimized proprietary turn-taking orchestration model instead of loosely stitching public APIs together.
  • Pros: Excellent visual conversation flow builder, built-in concurrency scaling, robust handling of interruptions, and simple SIP/telephony bridging (Twilio/Telnyx).
  • Cons: Limited native integrations outside of webhooks/Model Context Protocol (MCP).
• Bland AI
  • The Vibe: Built heavily for enterprise scale, complex outbound batch campaigns, and deep customization.
  • Latency Performance: Competitively fast (450ms - 650ms), though heavily dependent on the chosen LLM and persona settings.
  • Pros: Incredible depth for QA, post-call analytics, and complex branching conversation paths.
  • Cons: Steeper learning curve than Retell and higher baseline infrastructure or number costs.
• ElevenLabs (Conversational AI / ElevenAgents)
  • The Vibe: Unmatched voice naturalness. If your primary requirement is that the caller literally cannot tell they are talking to an AI, ElevenLabs wins outright.
  • Latency Performance: Powered by ultra-fast TTS models (like Flash v2.5 targeting sub-100ms voice synthesis), though full-conversation telephony round-trips land around 400ms - 800ms depending on your reasoning layer.
  • Pros: Best-in-class emotional resonance, accents, and hyper-realistic human inflections.

2. Open-Source Developer Frameworks (Maximum Control & Lowest Latency)

Best if you are a developer writing custom code, want to avoid per-minute managed markup fees, and need sub-400ms optimization.

• LiveKit (LiveKit Agents)
  • The Vibe: Built on WebRTC rather than standard WebSockets, making it natively optimized for real-time media streaming and ultra-low latency.
  • Latency Performance: Easily pushes under 400ms when paired with fast STT/TTS (like Deepgram or Cartesia) and tuned Voice Activity Detection (VAD).
  • Pros: Peerless real-time audio/video transport layer; handles messy interruptions smoothly.
• Pipecat
  • The Vibe: An open-source, highly pluggable pipeline framework designed specifically for real-time voice and video agents (frequently paired with infrastructure like Daily.co).
  • Latency Performance: Highly competitive sub-400ms potential. Features advanced LLM-based "Smart Turn Detection" which reduces instances of the AI accidentally talking over the user compared to basic silence thresholds.
  • Pros: Incredible flexibility to hot-swap different LLMs, STT, and TTS providers on the fly.

Summary Recommendation

• Choose Retell AI if you want a managed platform with a visual builder that gets you live on a phone number with minimal engineering overhead.
• Choose ElevenLabs if voice realism and emotional tone are your absolute top priorities.
• Choose LiveKit or Pipecat if you have engineering bandwidth, want open-source freedom, and need to squeeze every millisecond of latency out of the audio pipeline.

To help narrow this down, tell me:

• Do you prefer a managed API/no-code builder or an open-source code framework?
• Are you building primarily for inbound support or outbound calling campaigns?

If your priority is a genuinely low-latency phone agent, my default pick in 2026 is Retell AI, with Vapi a very close second.

My ranking

🥇 Retell AI — my choice for most teams

Retell is particularly good if the agent is primarily a telephone agent, rather than a general voice application. Recent comparisons consistently put it around the ~600–800 ms range out of the box, with strong turn-taking and telephony features.

I'd choose it for things like:

• Receptionists
• Appointment scheduling
• Lead qualification
• Inbound customer support
• Outbound sales/calling
• Call transfers and IVR navigation

The important distinction is that perceived latency isn't just model latency. Endpointing, interruption handling, STT, LLM TTFT, TTS time-to-first-audio, and telephony network latency all contribute.

🥈 Vapi — better if you're an engineering-heavy team

Vapi is what I'd pick if you want to control the individual pieces of the stack.

You can mix and match STT, LLM, and TTS providers rather than being locked into one pipeline. Vapi explicitly supports providers such as OpenAI, Anthropic, Google, Deepgram and ElevenLabs.

It also advertises sub-600-ms response times with appropriate configurations.

So I'd summarize the choice as:

Retell = fastest path to a great phone agent. Vapi = greatest control over how that phone agent works.

🧑‍💻 LiveKit — if you're building infrastructure, not just an agent

LiveKit becomes very interesting if you're willing to own more of the architecture.

You can optimize the entire realtime media pipeline yourself and build around your preferred STT/LLM/TTS providers. The tradeoff is considerably more engineering and operational responsibility.

One thing I'd avoid

I wouldn't start with Twilio → your server → STT → LLM → TTS → Twilio unless you have a specific reason to own the entire pipeline.

Every additional network hop and buffering layer can make a phone conversation feel sluggish. For a conversational agent, I'd optimize for time-to-first-audio after the caller stops speaking, not simply the LLM's token latency.

My recommendation

If you told me:

"I need to launch a production phone agent, and I want it to feel as close to a human as possible." I'd start with Retell.

If you told me:

"I have engineers and want to squeeze every millisecond out of the system and choose my own models." I'd start with Vapi, or go one level lower with LiveKit.

If you tell me your use case (inbound/outbound), expected call volume, and whether you need SMS/CRM/calendar integration, I can give you a concrete stack—e.g. telephony + STT + LLM + TTS + platform—optimized specifically for sub-second response time.

For a low-latency voice AI phone agent, the “best” platform depends on whether you optimize for speed to launch, absolute latency, control, or scale. There is no single winner, but these are the strongest choices:

App Stack Builder The Prompt Bench## My recommendation by scenario

1. Building a serious product where latency matters most

Pick LiveKit Agents.

A typical low-latency architecture:

Phone/SIP
   ↓
LiveKit (real-time audio)
   ↓
Voice activity detection + turn detection
   ↓
Realtime LLM
   ↓
Streaming TTS
   ↓
Caller

Why:

• WebRTC-based audio pipeline
• You control where your agent runs
• Easier to optimize geography, model choice, caching, and streaming
• Good fit for multi-agent systems and custom workflows

The downside: you own more engineering: orchestration, monitoring, evaluation, and call logic.

2. You want the fastest path to a production phone agent

Pick Retell AI.

Good for:

• customer support
• appointment scheduling
• sales qualification
• inbound call handling

It has many voice-agent primitives already handled:

• interruption/barge-in
• turn-taking
• call analytics
• telephony integrations

Xenolve Evalgent### 3. You are a developer who wants flexibility without building everything

Pick Vapi.

Strong points:

• swap LLMs easily
• choose different TTS/STT providers
• integrate tools/functions
• iterate quickly

It is often the best “startup default” because you can get a good agent running quickly while keeping architectural flexibility.

For the lowest possible latency, focus on the stack—not just the platform

The biggest latency killers are usually:

1. Turn detection

• Avoid waiting for long silence.
• Use good endpointing/VAD.

2. Speech pipeline

• Streaming STT → streaming LLM → streaming TTS.
• Avoid “record whole sentence → transcribe → think → speak.”

3. Model choice

• A slightly weaker but faster model often feels better in conversation.

4. Geography

• Put telephony, agent workers, and inference close together.

5. TTS startup time

• First audio byte matters more than total generation time.

A stack I would build today

For a high-quality commercial phone agent:

Option A (fastest to market):

• Telephony: Twilio or Telnyx
• Agent layer: Vapi or Retell
• STT: Deepgram
• LLM: a fast streaming model
• TTS: ElevenLabs / Cartesia / similar

Option B (best long-term architecture):

• Telephony: SIP/Telnyx/Twilio
• Media: LiveKit
• Agent runtime: LiveKit Agents
• Realtime model or streaming STT+LLM+TTS
• Your own observability/evals

For most teams, I’d start with Vapi or Retell, then move toward LiveKit if latency, cost, or customization becomes a bottleneck.

If you tell me your use case (inbound support, outbound sales, receptionist, healthcare, collections, etc.) and expected call volume, I can recommend a more specific stack.

Choosing the "best" platform for a low-latency voice AI phone agent depends on how much of the stack you want to own versus how fast you want to deploy. End-to-end conversational latency (typically averaging 500ms–800ms for a full round-trip of Speech-to-Text → LLM → Text-to-Speech over telephony) relies heavily on whether you choose a managed API or self-host infrastructure.

The top platforms are categorized below by how they fit different engineering and product requirements:

1. Managed & Phone-Native APIs (Fastest Time-to-Market)

• Retell AI: Widely regarded as one of the best for production-grade phone agents due to its highly consistent turn-taking model and optimized audio streaming (averaging ~600ms latency). It handles telephony, SIP trunking, and voice orchestration smoothly, making it a favorite for high-volume inbound/outbound calls, appointment setting, and deep CRM integrations.
• Vapi: Extremely developer-friendly and flexible. Vapi allows you to easily swap out STT, LLM, and TTS providers on the fly while managing the complex telephony and WebSocket infrastructure for you. Latency is competitive (sub-600ms), and it offers great extensibility if you want to orchestrate custom logic or tool calling without building the audio pipelines from scratch.
• ElevenLabs Conversational AI: If absolute voice naturalness is your highest priority, ElevenLabs handles the telephony bridge, transcription, and synthesis natively. Their ultra-low latency TTS models set the industry benchmark, making it ideal if you need callers to genuinely believe they are speaking with a human. You wire your reasoning layer (like Claude or GPT-4o) via webhooks.

2. Infrastructure & Open-Source Frameworks (Maximum Control & Lowest Raw Latency)

• LiveKit: If you have engineering resources and need granular, packet-level control over WebRTC, SIP, and real-time audio routing, LiveKit is the gold standard infrastructure. By hosting your own media servers globally, you can drastically reduce jitter and trim milliseconds off your pipeline—though it requires a much heavier engineering lift than a managed API.
• Pipecat (by Daily): An exceptional open-source Python framework built specifically for voice and video AI pipelines. It lets you compose agents using modular blocks for STT/TTS/LLMs. Like LiveKit, it puts the responsibility of infrastructure management on you, but it gives you absolute transparency and optimization over the entire execution loop.

Summary Recommendation

• Choose Retell AI or Vapi if you want to deploy a reliable, low-latency phone agent in days with minimal infrastructure overhead.
• Choose LiveKit or Pipecat if you are scaling to millions of minutes, have a dedicated real-time media engineering team, and need absolute control over data privacy and latency optimization.

To help narrow down the right choice, tell me:

• Are you looking for a managed API (fast setup) or an open-source framework (full control)?
• Will this be primarily inbound customer support or outbound campaigns?
• What is your expected call volume?

Choosing the "best" platform for a low-latency voice AI phone agent depends on your engineering bandwidth and how much infrastructure control you want versus a managed plug-and-play setup.

Low-latency voice AI relies on minimizing round-trip time (RTT) across five layers: transport, Speech-to-Text (STT), LLM reasoning, Text-to-Speech (TTS), and endpointing (knowing when the user stops speaking).

The top platforms on the market are categorized below by their strengths:

1. Best for Developers Seeking Ultimate Control & Lowest Latency: LiveKit (with Agents Framework)

• What it is: An open-source real-time transport and WebRTC infrastructure layer designed specifically for audio/video.
• Why it’s low latency: LiveKit uses WebRTC with UDP transport, bypassing the massive overhead of standard telephony or HTTP streaming bottlenecks. Their native LiveKit Agents framework lets you orchestrate STT, LLMs, and TTS directly at the edge, supporting both traditional pipelined approaches and end-to-end voice models (like OpenAI's Realtime API).
• Best for: Engineering teams that want sub-500ms response times, robust barge-in (interruption) handling, and self-hosted or deeply customized infrastructure.

2. Best Developer-First Managed Platforms: Vapi or Retell AI

• What they are: Developer-focused API platforms that abstract away the heavy lifting of telephony (SIP/trunking), STT, and TTS orchestration.
• Why they’re low latency: Both Vapi and Retell have optimized pipelines that typically achieve round-trip latencies between 500ms and 750ms out-of-the-box. They allow you to swap in ultra-fast models (like Deepgram Nova/Flux for transcription, Cerebras/Groq or GPT-4o-mini for fast reasoning, and ElevenLabs Flash for TTS).
• Vapi vs. Retell: Vapi often wins slightly in raw benchmark customizability and speed tuning, while Retell provides exceptional out-of-the-box consistency for telephony state-management and turn-taking.
• Best for: Developers who want to deploy production-ready phone agents via API in days rather than building WebRTC/SIP architectures from scratch.

3. Best for Infrastructure-Level Speech & Models: Deepgram (Voice Agent API)

• What it is: A foundational speech AI provider that recently released dedicated real-time Voice Agent APIs.
• Why it’s low latency: Instead of making separate round-trips for STT, LLM, and TTS across different third-party vendors, Deepgram integrates the loop tighter at the infrastructure level, heavily cutting down on transit lag.
• Best for: Technical teams building their own orchestration who want raw speech-to-speech primitives optimized for speed.

4. Best No-Code / Low-Code Solution: Synthflow AI

• What it is: A visual, graph-based platform for building voice agents.
• Why it’s low latency: It handles the complex backend orchestration for you, offering optimized edge routing, though total latency typically hovers on the higher end (600ms–900ms+) compared to raw code implementations.
• Best for: Product managers or agencies who need to spin up structured, template-driven customer service calls without heavy coding.

Summary Recommendation

• Choose LiveKit if you have backend/WebRTC engineers and need the absolute lowest, hyper-optimized, interruption-friendly latency.
• Choose Vapi or Retell AI if you want a fast, reliable developer API with flexible pricing and great out-of-the-box telephony integration.

To help narrow down the best fit, tell me:

• Are you looking for a developer-first API or a low-code/no-code builder?
• What is your expected call volume and telephony requirement (e.g., Twilio, SIP trunking)?

If low latency is the #1 requirement for a phone-based voice agent, my shortlist in 2026 is:

My recommendation: Vapi

For most startups, I'd start with Vapi. It gives you a realtime orchestration layer while letting you swap STT, LLM, and TTS providers independently. Vapi specifically supports phone calls and advertises sub-500ms responses, although its own documentation correctly notes that actual caller-perceived latency also includes endpointing and network/transport time.

A particularly good low-latency stack would be something like:

PSTN/SIP → Vapi → streaming STT → fast LLM → streaming TTS

The key is not simply picking a platform with a good advertised latency number. Endpointing + model TTFT + TTS time-to-first-audio + telephony routing dominate the experience.

When I'd choose LiveKit instead

If you're building a serious voice product where you care about squeezing every ~100ms out of the system, I'd seriously consider LiveKit Agents.

LiveKit gives you the realtime media infrastructure itself, supports SIP/PSTN telephony, lets you choose your STT/LLM/TTS providers, and can be self-hosted.

That's more engineering work, but it gives you substantially more control over:

• geographic routing
• SIP/telephony topology
• streaming behavior
• turn detection
• model selection
• audio pipeline
• scaling
• observability

LiveKit estimates a well-optimized production voice pipeline around 700ms–1.2s end-to-end, with ~1 second being a practical target.

One important caveat

Don't compare vendor claims like "500ms latency" and "600ms latency" at face value. They may measure different portions of the pipeline. An independent 2026 phone-call benchmark, for example, measured complete caller-to-first-agent-audio latency rather than just server-side model latency and found materially higher numbers across platforms.

So my ranking would be:

1. 🥇 Vapi — best balance of latency, flexibility, and development speed
2. 🥈 LiveKit — best if latency/control are more important than engineering simplicity
3. 🥉 Retell — best if you want a highly polished phone-agent product with less infrastructure work
4. Custom Twilio/Telnyx + realtime stack — potentially best performance, but only worth it at sufficient scale/technical sophistication

If you tell me your expected calls/minute, inbound vs. outbound, and whether you're using OpenAI/Claude/Gemini, I can recommend the exact telephony + STT + LLM + TTS stack I'd use to target <700ms perceived latency.

PSTN/SIP → Vapi → streaming STT → fast LLM → streaming TTS

The key is not simply picking a platform with a good advertised latency number. Endpointing + model TTFT + TTS time-to-first-audio + telephony routing dominate the experience.

When I'd choose LiveKit instead

If you're building a serious voice product where you care about squeezing every ~100ms out of the system, I'd seriously consider LiveKit Agents.

LiveKit gives you the realtime media infrastructure itself, supports SIP/PSTN telephony, lets you choose your STT/LLM/TTS providers, and can be self-hosted.

Choosing the "best" platform for a low-latency voice AI phone agent depends entirely on whether you want to write code for ultimate control or use a managed/no-code service to ship fast . Real-time phone conversations break down if end-to-end latency exceeds 1.5 to 2 seconds.

1. Developer-First Open-Source Frameworks (Best Control & Lowest Latency)

If you have engineering resources and want to avoid vendor lock-in, open-source orchestration frameworks combined with WebRTC or optimized transports achieve the lowest latencies (often under 1.6 seconds).

• • LiveKit Agents: Built on top of a powerful WebRTC media server and global edge network. It treats AI agents like participants in a room, making it exceptionally fast for handling audio streaming, multi-modal inputs, and instantaneous interruptions.

• •   - **Pipecat:** A Python-centric framework focused on rapid voice agent prototyping and pipeline orchestration. It gives you modular control over swapping STT (Speech-to-Text), LLM, and TTS (Text-to-Speech) providers and integrates cleanly with telephony providers like Twilio or Daily.[](https://www.dograh.com/feeds/blog/pipecat-livekit-agents) [[1]](https://www.dograh.com/feeds/blog/pipecat-livekit-agents)[[2]](https://www.youtube.com/watch?v=ES3HhoYCtIc&t=179)[[3]](https://medium.com/@mahadise0011/top-voice-ai-agent-frameworks-in-2026-a-complete-guide-for-developers-4349d49dbd2b)
    

• •   - **TEN Framework:** An open-source, graph-based runtime built specifically for real-time, interruptible conversational AI and telephony routing.[](https://www.youtube.com/watch?v=ES3HhoYCtIc&t=179) [[1]](https://www.youtube.com/watch?v=ES3HhoYCtIc&t=179)
    

2. Managed Developer Platforms (Best Balance of Speed & Infrastructure)

If you don't want to self-host WebRTC media servers or stitch together individual STT/LLM/TTS APIs manually, dedicated managed platforms handle the heavy lifting while maintaining competitive latencies.

• •   - **Retell AI:** Widely favored by developers for phone agents. It abstracts telephony (SIP/Twilio), provides low-latency streaming pipelines (averaging ~1.6s), and includes built-in function/tool calling for real-time actions like booking or checking databases.[](https://www.youtube.com/watch?v=bpvKtZVsqpk) [[1]](https://www.youtube.com/watch?v=bpvKtZVsqpk)[[2]](https://www.youtube.com/watch?v=BpqgtEk9Zyc&t=29)
    

• •   - **Synthflow AI:** Offers a streamlined infrastructure and graph-based orchestration specifically for automated phone workflows, appointment setting, and customer support.
    

• •   - **ElevenLabs Agents Platform:** Best-in-class voice naturalness and expressiveness with integrated turn-taking optimizations, though its end-to-end phone infrastructure is newer compared to dedicated telecom players.
    

Key Architectural Rule for Low Latency

No matter which platform you pick, your end-to-end latency is bound by the slowest link in your chain:

To help narrow down the ideal choice, let me know:

• Are you looking for a fully managed API or an open-source framework to self-host?
• What is your anticipated call volume?

If your priority is a low-latency AI agent that actually handles phone calls in production, my pick today is Retell AI.

My ranking

🥇 Retell AI — my default choice

Retell is particularly strong because it isn't just a collection of STT → LLM → TTS components. It has its own orchestration and turn-taking layer designed around phone conversations. Retell reports roughly 600 ms latency, with interruption handling, phone/SIP support, transfers, outbound campaigns, and call analytics built in.

That's important because perceived latency isn't just model latency. A phone agent has transport + endpointing + STT + LLM first-token latency + TTS time-to-first-audio. Vapi's documentation, for example, explicitly breaks latency into those components.

I'd choose Retell if you're building something like:

• Appointment booking
• Lead qualification
• Inbound customer support
• Restaurant/service-business answering
• Outbound sales or reminders
• Agents that need warm transfer to humans
• Anything where "sounds like a real person" matters more than owning every layer

🥈 Vapi — if you're an engineer who wants control

Vapi is probably the better choice if you want to swap models/providers aggressively and construct your own stack. It gives you control over STT, LLM, voice, tools, telephony, etc., while handling much of the realtime infrastructure. Vapi advertises responses in under 500 ms and provides extensive developer/API control.

The tradeoff is that getting your particular configuration to feel excellent can require more tuning.

🥉 LiveKit — if you're building the infrastructure

I'd use LiveKit if the goal is "we want to own the realtime voice architecture", rather than simply "we need an AI phone agent."

It's an open-source realtime framework with Python/Node support, SIP telephony, WebRTC transport, multiple model providers, and the ability to build complex multi-agent workflows.

That's extremely powerful, but you're taking on more engineering responsibility.

The important distinction

If you're asking "What gets me the fastest, most natural phone agent?":

Retell → Vapi → LiveKit

If you're asking "What gives my engineering team the most control over latency and architecture?":

LiveKit → Vapi → Retell

And if you're asking "Which one would I start a startup on?", I'd start with Retell or Vapi, not build the realtime stack myself.

One caveat: don't choose based purely on vendors' advertised latency numbers. Independent measurements can produce substantially different results depending on the exact STT/LLM/TTS/telephony configuration and how latency is measured.

If you tell me your use case (inbound/outbound, expected call volume, language, and whether you're using OpenAI/Claude/etc.), I can give you a concrete stack—e.g. Retell + Claude + ElevenLabs + Twilio—optimized for <1-second perceived response time.