Realtime Text to Speech

Overview

Realtime TTS models convert text into streaming audio. Unlike standard TTS that processes a full prompt and returns an audio file, realtime TTS streams AI-generated speech as a continuous PCM audio stream over a WebSocket connection — in real time. The text prompt is submitted via POST /Run, not over the WebSocket. The WebSocket carries only task events (task_info, task_stream_ready, etc.), binary audio frames, and control messages (task_session_end).

No microphone is required. The flow is one-directional: text goes in, audio comes out.

The flow is:

1. Run the realtime TTS model via POST /Run with your text prompt in the parameters
2. Connect to the WebSocket and send task_info with your socketaccesstoken
3. Wait for task_stream_ready — the model has loaded and is generating audio
4. Receive AI audio as binary frames and play them
5. End the session with task_session_end or wait for the stream to finish naturally

How It Differs from Realtime Voice Conversation

Run Parameters

Each realtime TTS model has its own set of parameters. Use POST /Tool/Detail to discover the exact parameters for a specific model. See Model Parameters for parameter types.

Browse available realtime TTS models on the models page (filter by Realtime TTS category). Common parameters typically include the input text and an optional reference audio clip for voice cloning. Example run for MOSS-TTS-Realtime:

curl -X POST "https://api.wiro.ai/v1/Run/openmoss/moss-tts-realtime" \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "text": "Hello, this is a realtime text-to-speech demo."
  }'

Connection & Registration

After running the task, connect to the WebSocket and register with task_info:

var ws = new WebSocket("wss://socket.wiro.ai/v1");

ws.onopen = function() {
  ws.send(JSON.stringify({
    type: "task_info",
    tasktoken: "YOUR_SOCKET_ACCESS_TOKEN"
  }));
};

Realtime Events

During a realtime TTS session, you'll receive these WebSocket events:

Event Sequence

A typical TTS session produces events in this order:

task_stream_ready     ← model is ready, audio chunks start arriving
[binary frames]       ← PCM audio data (many frames)
task_stream_end       ← audio generation complete for this segment
task_cost             ← cost for this segment
task_end              ← model process exiting
task_postprocess_end  ← safe to close WebSocket

Audio Format

Audio flows in one direction only: server → client. The client does not send any audio.

Binary Frame Format

Every binary WebSocket frame from the server is structured as:

[tasktoken]|[PCM audio data]

The pipe character | (0x7C) separates the token from the raw audio bytes. To extract the audio:

1. Find the first | byte in the binary frame
2. Everything after it is raw PCM Int16 audio data
3. Convert Int16 samples to your playback format (e.g., Float32 for Web Audio API)

Receiving AI Audio

AI speech arrives as binary WebSocket frames in PCM Int16 24 kHz format. To play them:

1. Check if the incoming message is binary (a Blob in JavaScript, bytes in Python) before attempting JSON parse
2. Find the pipe | separator and extract audio data after it
3. Convert Int16 → Float32 and create an AudioBuffer
4. Schedule gapless playback using AudioBufferSourceNode to avoid clicks between chunks

Gapless Playback

Audio arrives in many small chunks. To play them seamlessly:

• Track a nextPlayTime variable initialized to 0
• For each chunk, schedule it at max(audioContext.currentTime, nextPlayTime)
• Advance nextPlayTime by the chunk's duration
• This ensures chunks play back-to-back with no gaps or overlaps

Ending a Session

To gracefully end a realtime TTS session, send task_session_end:

{
  "type": "task_session_end",
  "tasktoken": "YOUR_SOCKET_ACCESS_TOKEN"
}

After sending this, the server will finish any in-progress generation, send final cost events, and then emit task_postprocess_end. Wait for task_postprocess_end before closing the WebSocket.

For TTS sessions, the stream often ends naturally when the model finishes generating audio for the provided text. In this case, you'll receive task_stream_end followed by task_end without needing to send task_session_end. However, it's good practice to send it explicitly for a clean shutdown, especially if you want to stop playback early.