openai / gpt-realtime-2.1-mini

Overview

GPT-Realtime 2.1 Mini is an OpenAI realtime model built for speech-first assistants. It can take live audio and respond with spoken audio, plus an aligned transcript you can show on screen. (developers.openai.com)

It runs in low-latency realtime sessions over WebRTC, WebSocket, or SIP. It also accepts text input, and it can use tools for voice-agent workflows. (developers.openai.com)

This model is a distilled reasoning variant tuned for faster voice interactions. OpenAI lists a 128,000 token context window and up to 32,000 output tokens. (developers.openai.com)

What you can build

• A hands-free voice assistant for support, scheduling, and FAQs, with spoken replies and on-screen captions. (platform.openai.com)
• A voice agent that calls your internal functions, then explains results out loud. (developers.openai.com)
• A realtime meeting companion that listens, transcribes asynchronously, and summarizes. (developers.openai.com)
• A multimodal assistant that can take a reference image as context while you speak. (developers.openai.com)
• A telephony-style assistant using VoIP-friendly audio formats and server VAD. (developers.openai.com)

Inputs

• The assistant voice you want for spoken responses. Built-in options documented for Realtime include Alloy, Ash, Ballad, Coral, Echo, Sage, Shimmer, Verse, Marin, and Cedar. Choose once per session if you need consistent output. (developers.openai.com)
• A system instruction message that defines the assistant’s role, tone, and speaking style. Keep it short and specific for steadier behavior. (developers.openai.com)
• A reasoning effort setting to control how much deliberation the model uses before it answers. Higher effort can help with harder tasks but can feel less “snappy” in voice chats. (developers.openai.com)
• An optional live transcription choice for your microphone audio. The model consumes audio directly, so this transcript is guidance and may not match exactly what the model heard. (developers.openai.com)
• Your microphone audio stream format. PCM is supported and is fixed at 24 kHz. G.711 μ-law and G.711 A-law are also supported for telephony-style pipelines. (developers.openai.com)
• Your desired response audio format. Choose PCM for highest fidelity, or choose a G.711 format for VoIP pipelines. (developers.openai.com)
• Voice activity detection (VAD) sensitivity and the silence duration that ends a user turn. Lower sensitivity triggers on quieter speech. Shorter silence makes the model answer sooner. (platform.openai.com)

Outputs

The model returns a realtime assistant message that can include:

• Spoken audio, delivered as base64-encoded audio bytes in the output format you selected. (platform.openai.com)
• A text transcript aligned to the spoken audio. When the output modality is audio, the transcript is included with the content. (platform.openai.com)
• Optional text-only responses when you disable audio output for the session. Realtime sessions can be configured to respond in text or in audio, but not both at the same time. (developers.openai.com)

Recommended settings

• Use PCM for both input and output when you can. OpenAI documents PCM as 24 kHz only in Realtime. (developers.openai.com)
• Start with Marin or Cedar if you want the best built-in voice quality. (developers.openai.com)
• Use a moderate VAD threshold and about half a second of silence for turn taking. Tune up the threshold in noisy rooms. (platform.openai.com)
• Keep response speed at normal unless you have a strong product reason. The supported speed range is 0.25× to 1.5×. (developers.openai.com)
• Use low reasoning effort for most voice chat. Increase it only when tool calls or complex tasks fail. (developers.openai.com)

Limitations

• Video isn’t supported by this model. (developers.openai.com)
• Image input is supported, but only as input. OpenAI’s Realtime schema documents image bytes as base64 data URIs, with PNG and JPEG support. (developers.openai.com)
• Voice selection can’t be changed after the session has produced audio at least once. Plan voice choice before the first assistant reply. (developers.openai.com)
• PCM audio in Realtime is limited to 24 kHz. If your pipeline requires other PCM rates, you must resample outside the model. (developers.openai.com)
• Built-in turn detection can cut users off if you set silence too short. It can also lag if you set it too long. (platform.openai.com)
• Transcription for input audio is asynchronous and “best effort.” Treat it as a UI aid, not a ground truth record. (developers.openai.com)
• Structured outputs aren’t supported on this model, so don’t rely on strict JSON formatting. (developers.openai.com)
• Low-quality audio hurts results. Background noise, clipping, or inconsistent gain will reduce VAD accuracy and response quality. (developers.openai.com)

Safety & compliance

Follow OpenAI’s Usage Policies for voice and realtime applications. Use moderation and policy checks where your product needs them. (platform.openai.com)

If you use custom voices, OpenAI’s voice APIs require an audio sample and a consent recording. Custom voices are limited to eligible customers. (platform.openai.com)

If you need provenance workflows, OpenAI provides an API to check whether an image or audio file contains known OpenAI provenance signals. (developers.openai.com)