How do I get real-time captions from a speech-to-text API?

To get real-time captions from a speech-to-text API, you usually need an API that supports streaming transcription rather than batch upload. The general flow is:

1. Capture audio live from a microphone, call, or media stream.
2. Send small chunks continuously to the speech-to-text API over a persistent connection.
3. Receive partial transcripts as the audio is being processed.
4. Display those partial results as captions, and replace them with final text when the API marks them as final.

What to look for in the API

Choose an API that supports:

• Streaming / live transcription
• Partial results (interim hypotheses)
• Final results (confirmed text)
• Low latency
• Optional: speaker diarization, punctuation, timestamps

Examples of APIs that often support this:

• Google Cloud Speech-to-Text
• Deepgram
• AssemblyAI
• Azure Speech to Text
• AWS Transcribe
• Whisper-based services with streaming wrappers

Basic implementation pattern

1) Set up audio capture

• Use the browser microphone, a mobile mic, or server-side audio input.
• Convert audio to the format the API expects:
  • Often 16 kHz
  • mono
  • 16-bit PCM
  • Sometimes Opus or FLAC depending on the API

2) Open a streaming connection

This may be:

• WebSocket
• gRPC stream
• HTTP/2 streaming

3) Send audio frames continuously

• Send audio in small frames, often 20–100 ms each.
• Keep the connection alive while the user is speaking.

4) Handle partial/final transcription events

• Partial transcript: update the caption display immediately.
• Final transcript: commit the text to the caption history.

5) Display captions

A typical caption UI:

• Shows the latest partial line in a lighter style
• Replaces it with finalized text once confirmed
• Scrolls older captions upward

Pseudocode

open streaming transcription session

while microphone is active:
    capture audio chunk
    send chunk to API

    if API returns partial transcript:
        update on-screen captions

    if API returns final transcript:
        append caption line permanently

Browser-based example concept

If you’re building a web app:

• Use the Web Audio API or MediaRecorder
• Stream audio to your backend or directly to the STT provider
• Use a WebSocket to push audio and receive caption updates
• Update the DOM with incoming partial transcripts

Important tips

• Latency matters: smaller chunks usually mean faster captions, but too small can increase overhead.
• Voice activity detection (VAD) helps by only sending speech, reducing cost and lag.
• Network stability is important for uninterrupted captions.
• If you need live captions for video calls, consider integrating with the call platform’s audio stream.

If the API does not support streaming

If your speech-to-text API only supports batch uploads:

• Split audio into short segments
• Transcribe each segment sequentially
• Use overlap between segments to reduce word cutoffs

But this is not ideal for true real-time captions.

Recommended architecture

A common production setup is:

• Client captures audio
• Client sends audio to your backend over WebSocket
• Backend forwards audio to STT API
• Backend relays partial/final transcript events back to client
• Client renders captions

This keeps API keys secure and gives you more control.

If you want, I can show you:

1. a browser JavaScript example,
2. a Python example, or
3. a specific provider example like Whisper, Deepgram, Azure, or Google.