Realtime Speech to Text

Overview

Realtime speech-to-text models convert streaming audio into text transcripts as the user speaks. Unlike Realtime Voice Conversation which produces two-way audio, this mode is audio in → text out only. There is no AI audio playback — the server returns transcript strings over the WebSocket.

The flow is:

1. Run the realtime STT model via POST /Run to get a socketaccesstoken
2. Connect to the WebSocket and send task_info with your token
3. Wait for task_stream_ready — the model is ready to receive audio
4. Stream microphone audio as binary frames (client → server)
5. Receive transcript text as task_output messages with TRANSCRIPT_USER: prefix
6. End the session with task_session_end

How It Differs from Realtime Voice Conversation

Run Parameters

Each realtime STT model has its own set of parameters. Use POST /Tool/Detail to discover the exact parameters for a specific model. See Model Parameters for parameter types.

Browse available realtime STT models on the models page (filter by Realtime STT category). Common parameters typically include a transcription delay setting that trades speed for accuracy; some models also take a language. Example run for Voxtral Mini 4B Realtime:

curl -X POST "https://api.wiro.ai/v1/Run/mistralai/voxtral-mini-4b-realtime-2602" \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "transcriptionDelay": "balanced"
  }'

Connection & Registration

After running the task, connect to the WebSocket and register with task_info:

var ws = new WebSocket("wss://socket.wiro.ai/v1");

ws.onopen = function() {
  ws.send(JSON.stringify({
    type: "task_info",
    tasktoken: "YOUR_SOCKET_ACCESS_TOKEN"
  }));
};

Realtime Events

During a realtime speech-to-text session, you'll receive these WebSocket events:

Audio Format

Audio flows in one direction only: client → server.

Binary Frame Format

Every binary WebSocket frame sent from the client is structured as:

[tasktoken]|[PCM audio data]

The pipe character | (0x7C) separates the token from the raw audio bytes. There are no binary frames from the server — all responses are JSON text.

Sending Microphone Audio

Capture microphone audio at 24 kHz using the Web Audio API with an AudioWorklet. Convert Float32 samples to Int16, prepend your task token, and send as a binary frame.

Key steps:

1. Request microphone with getUserMedia (enable echo cancellation and noise suppression)
2. Create an AudioContext at 24,000 Hz sample rate
3. Use an AudioWorklet to buffer and convert samples to Int16
4. Send each chunk as tasktoken|pcm_data binary frame
5. Continue sending until you end the session

Transcripts

Transcripts arrive as task_output messages with the TRANSCRIPT_USER: prefix. Each message contains a segment of transcribed speech:

{
  "type": "task_output",
  "message": "TRANSCRIPT_USER:What's the weather like today?"
}

{
  "type": "task_output",
  "message": "TRANSCRIPT_USER:I need to book a flight to New York."
}

Progressive Results

Transcripts arrive progressively as words and phrases are recognized — not just at segment boundaries. The model streams TRANSCRIPT_USER: messages word-by-word as speech is detected, so the client can display live, incremental results. To build a full transcript, concatenate all received messages:

var fullTranscript = [];

// Inside your message handler
if (msg.type === 'task_output' &&
    typeof msg.message === 'string' &&
    msg.message.startsWith('TRANSCRIPT_USER:')) {
  var segment = msg.message.substring(16);
  fullTranscript.push(segment);
  console.log('Segment:', segment);
  console.log('Full:', fullTranscript.join(' '));
}

Ending a Session

To gracefully end a realtime session, send task_session_end:

{
  "type": "task_session_end",
  "tasktoken": "YOUR_SOCKET_ACCESS_TOKEN"
}

After sending this, the server processes any remaining buffered audio, sends final transcript and cost events, and then emits task_postprocess_end. Wait for task_postprocess_end before closing the WebSocket.