fishaudio / s2-pro

## Overview
S2-Pro is a 4B parameter text-to-speech model with Dual-Autoregressive architecture designed for fast inference and high-quality audio generation. It supports ~50 languages with automatic detection and offers voice cloning capabilities through reference audio. The model enables fine-grained prosody control via natural language emotion tags.
## What you can build
- Multilingual voice assistants
- Automated audio narration systems
- Custom voice cloning applications
- Multi-speaker dialogue generation
- Emotionally expressive speech synthesis
## Inputs
- **Prompt**: Text to synthesize into speech; use <|speaker:N|> tags for multi-speaker and emotion tags like (excited) or [laugh] for prosody control.
- **Reference Audio**: Reference audio file for voice cloning (10-30s recommended)
- **Reference Text**: Exact transcription of the reference audio file; must match the spoken content for accurate voice cloning.
## Outputs
- High-fidelity synthesized speech audio (WAV format)
- Voice cloning results matching reference speaker identity
## Multi-Speaker Control
Use <|speaker:N|> tags to control multiple voices in a single generation:
- Example: <|speaker:0|>Hello, I'm the first speaker. <|speaker:1|>And I'm the second speaker.
- Speaker IDs (0, 1, 2...) map to voices defined in reference audio/text
- Without reference audio, model assigns random internal voices
- No hard limit on speaker count; processed in batches of 5
- Reference text must include matching speaker tags for voice mapping
## Emotion & Prosody Control
Control speech style with inline natural language tags. The model uses free-form descriptions - experiment with custom tags.
- **Basic Emotions**: (angry) (sad) (excited) (surprised) (satisfied) (delighted) (scared) (worried) (nervous) (joyful)
- **Advanced Emotions**: (sarcastic) (melancholic) (confident) (anxious) (frustrated) (impatient) (furious) (disdainful)
- **Tone/Style**: (whispering) (shouting) (in a hurry tone) (soft tone) (dramatic voice) (speaking slowly)
- **Sound Effects**: [laugh] [chuckle] [sigh] [cry] [gasp] [clears throat] [yawns]
- Example: <|speaker:0|>(excited)Wow, this is amazing! [laugh] I can't believe it worked!
## Recommended settings
- Use default values for standard outputs
- Adjust temperature (0.1-1.5) for creativity vs consistency
- Adjust top_p (0.5-0.95) for sampling diversity
- Set chunk_length based on text complexity (200-500)
- Use seed for reproducible outputs
## Limitations
- Voice cloning accuracy depends on quality and length of reference audio (10-30s ideal)
- Multi-speaker requires matching speaker IDs between reference_text and prompt
- Emotion tag effectiveness varies; not all descriptions work equally well
- ~50 languages supported with automatic detection; no language parameter needed
- Reference audio and prompt text can be in different languages
## Safety & compliance
- Ensure all generated content complies with applicable regulations
- Respect copyright and usage rights of reference audio files
- Use responsibly to avoid misuse in deceptive applications