openai / gpt-realtime-translate

Overview

GPT Realtime Translate is a streaming speech-to-speech translation model made by OpenAI. It listens to live spoken audio, detects the source language automatically, and speaks the translation while the speaker is still talking. It also streams text transcripts so you can show live captions. This helps you add interpretation to calls, events, and media without building a multi-step pipeline.

What you can build

• Live interpretation for 1:many broadcasts like webinars, lectures, earnings calls, and livestreams
• 1:1 bilingual phone or video calls where each person hears the other in their chosen language
• Multilingual customer support lines that translate callers in real time
• Live translated audio tracks for training videos and product demos
• Live captions in a target language for accessibility and cross-language viewing

Inputs

• A target output language selection. The model translates into 13 supported target languages: English, Spanish, Portuguese, French, German, Italian, Russian, Chinese, Japanese, Korean, Hindi, Indonesian, and Vietnamese.
• A live audio stream from a microphone or another real-time audio source. The model is designed for continuous audio, including natural pauses.
• An optional microphone sample rate choice. Use 24 kHz for best quality. Use 8 kHz for phone-quality audio.

Outputs

• A live stream of translated speech audio. The service emits audio in short chunks designed for real-time playback.
• A live stream of translated text transcript updates. You can render these as captions as they arrive.
• If you enable input transcription in your setup, you can also receive source-language transcript updates alongside the translated output.

Recommended settings

• Use 24 kHz audio when you control the capture path. It’s the recommended rate for this model.
• If your source is telephony audio, capture and send 8 kHz audio to match the call path.
• Keep sending audio continuously, including silence between phrases. This helps the model keep timing and context.
• If your app mutes the original audio, provide an option to mix or duck the original audio. This reduces confusion during mixed-language speech.

Limitations

• The model is built for translation, not general chat. It isn’t meant to follow free-form instructions.
• It doesn’t support custom prompts, glossaries, or pronunciation guides. Test proper nouns, brand names, and domain terms directly.
• It doesn’t support fixed voice selection. The translated voice adapts to the source speaker’s tone and style.
• Mixed-language speech can create gaps. If the speaker talks in the same language as the chosen output, the model may stay quiet for that segment.
• Low-quality inputs reduce accuracy. Expect problems with heavy noise, clipped audio, strong echo, overlapping speakers, or inconsistent volume.
• This model doesn’t support image or video inputs.

Safety & compliance

• Follow OpenAI’s usage policies for harmful content, deception, and spam.
• Tell end users they are hearing AI-generated translated speech and captions when it isn’t obvious.
• Don’t use the model to impersonate people, mislead listeners, or hide that translation is machine-generated.
• Build your own guardrails for sensitive settings like healthcare, legal advice, finance, or emergency response. Human review is often required.