Archived article. This article describes an earlier version of the Ollalink platform. Translation, realtime speech-to-speech and streaming text-to-speech described here are not offered today, and the voices and setup it mentions have changed. For what the API offers now — speech-to-text, live captions, text-to-speech, text tools and on-demand models — see the current docs.

1. The Realtime Voice Bottleneck

Building conversational voice agents that feel natural requires total round-trip latency under 500ms. If a user speaks, pauses, and must wait 1.2 seconds before the agent replies, conversational flow breaks down. The human brain interprets delays exceeding 600ms as an awkward pause or network interruption.

In a classic pipeline (Speech-to-Text → LLM Token Generation → Text-to-Speech), each stage introduces latency:

  • ASR Endpointing & Transcription: 150ms–300ms
  • LLM Time-To-First-Token (TTFT): 80ms–180ms
  • Neural TTS Synthesis & Streaming: 350ms–700ms (in conventional REST setups)

The primary bottleneck is almost always the TTS synthesis stage when handled through monolithic API gateways. Gateways buffering audio frames before responding will destroy interactive voice latency.

Key Architectural Rule: Never route heavy raw audio synthesis through the same reverse proxy queues that manage text completions or RAG vector lookups.

2. Two-Tier Service Separation: Translation vs. Speech GPU

To solve this fundamental physical constraint, Ollalink separates its infrastructure into two specialized planes:

Layer Host & Protocol Authentication Workloads
Translation https://ai.ollalink.com X-NH-GPU-Key: <key> Technical text translation
Speech GPU Layer gpu-*.ollalink.com X-NH-GPU-Key: <key> Batch, file and realtime STT, 41-Voice TTS, Voice Cloning, Live Captions

By isolating speech on dedicated GPU hardware, audio generation never queues behind other workloads.

3. Audio Streaming Mechanics: Raw PCM over a WebSocket

On the streaming text-to-speech route, synthesis starts as soon as text arrives. Rather than waiting for the complete sentence to finish rendering, audio is streamed out as raw 48 kHz 16-bit mono PCM binary frames, framed by speak.started and speak.done events. 

# Streaming text-to-speech: wss://sound-stream.ollalink.com/v1/tts/speak
# 1. session.configure with your GPU API key, language and voice
# 2. send {"type": "speak", "text": "..."}; receive binary 48 kHz PCM frames until speak.done
# Full protocol: https://ollalink.com/docs/realtime-speech/

For a whole file at once, POST https://gpu-tts.ollalink.com/v1/audio/speech returns a 24 kHz WAV.

4. Zero-Hallucination Technical Normalization

A common failure mode in off-the-shelf TTS engines is mispronouncing networking acronyms. For example, standard models often pronounce TCP/IP as "TCP slash IP", or read 192.168.1.1 with erratic cadence.

Ollalink's normalizer introduces deterministic rule passes before the acoustic model:

  • TCP/IP is normalized to "TCP IP".
  • IP addresses (e.g. 10.0.0.1) are parsed into clear octet groups with controlled pauses.
  • Port numbers (e.g. port 443) are read the way engineers say them: "port four forty three".

5. Conclusion & Next Steps

By isolating compute domains, enforcing deterministic text normalization, and streaming raw PCM straight from the GPU, Ollalink keeps first audio fast for realtime voice.

Ready to test real voice quality? Try our live Voice Quality Survey or check out the Getting Started Documentation.