PersonaPlex Realtime: Real-time Speech-to-Speech for Live Voice Agents
PersonaPlex Realtime targets live speech-to-speech workflows. The model runs as a streaming WebSocket service that transforms incoming audio into produced speech with very low latency. This post explains what the model does, how to integrate it, and production trade-offs.
What the model does
PersonaPlex Realtime converts an input audio stream to an output audio stream. The service accepts short prompts and voice controls while processing live audio. It supports common audio formats and several built-in voice presets.
Key features
- Streaming interface (WebSocket) for low-latency voice agents.
- Accepts inputAudio, prompt, and voice selection.
- Controls for text randomness and audio sampling: tempText, topkText, topkAudio.
- Support for common formats: wav, mp3, m4a, ogg, opus, webm.
- Multiple voice presets (natural and variety voices) for quick style changes.
How it integrates
Integration requires a client that opens a WebSocket to the model service and streams short audio frames. The client sends a brief text prompt and voice params. The model returns generated audio in a streaming form or as chunked audio segments.
Typical parameters
| Parameter | Purpose |
|---|---|
| inputAudio | Reference audio or live stream for voice cloning or conversion. |
| prompt | Short instruction or context to guide the output speech. |
| voice | Pick a voice preset (natural or variety family). |
| tempText | Control text randomness for generated speech content. |
| topkAudio / topkText | Sampling limits for audio and text generation. |
Common use cases
- Live customer support agents that translate or rewrite spoken content on the fly.
- Interactive NPCs in games where speech must respond in real time.
- Voice conversion for live dubbing or local language switching.
- Assistive voice interfaces that require immediate feedback.
Production considerations
- Latency and bandwidth determine responsiveness. WebSocket stability matters more than raw throughput.
- Audio quality depends on the input reference. Short, clean clips give better clones.
- Privacy and consent matter when cloning or transforming real voices. Include explicit user consent in the flow.
- Handling of edge cases such as crosstalk, overlapping speech, and noisy channels requires preprocessing and guard rails.
Developer checklist
- Set up a secure WebSocket client that sends frames and receives audio chunks.
- Stream short prompts when a new context starts (no long blocking prompts).
- Test with different voice presets and tempText values to find the right balance for the product.
- Monitor for audio artifacts and add fallbacks to recorded TTS when streaming fails.
