Replica Preview 0.6B
Replica Preview is the first text-to-speech (TTS) model from TokenAI and the beginning of the Replica model family. It is a non-autoregressive, diffusion-based multilingual TTS system built on top of OmniVoice, with a focus on natural, expressive speech and non-verbal sounds (breaths, sighs, laughter, fillers).
Replica Preview 0.6B is part of the Replica Preview model family by TokenAI. Learn more about the family: https://tokenai.llc/projects/replica-preview
Model Details
| Property | Value |
|---|---|
| Model type | Text-to-Speech (TTS), non-autoregressive diffusion |
| Base model | OmniVoice (k2-fsa/OmniVoice) |
| LLM | Qwen/Qwen3-0.6B-Base |
| Audio codec | HiggsAudioV2Tokenizer: 9 codebooks x 1024, hop 960, 16 kHz in / 24 kHz out |
| Audio heads | 8 heads with weights [8, 8, 6, 6, 4, 4, 2, 2] |
| Parameters | 612.6M (LLM 595.8M + audio embeddings 8.4M + audio heads 8.4M) |
| Model size | 2.45 GB (safetensors, float32) + 0.77 GB codec = 3.22 GB total |
| Sample rate | 24 kHz output |
| Languages | 646 (inherited from OmniVoice); optimized for Egyptian Arabic (arz), English, French, German, Spanish |
| License | Apache 2.0 |
Capabilities
- Zero-shot voice cloning: clone any speaker from a short reference audio (unlimited speakers)
- Voice Design: gender x age x pitch x style x accent
- Auto voice: automatic voice selection
- Non-verbal sounds:
[inhale],[exhale],[sigh],[laughter],[laugh], and fillers (ูู ู ู,ุขู,ah,oh,Hmmm,phew) - Number normalization: Arabic digits are converted to Egyptian words (e.g., 5000 -> ุฎู ุณุฉ ุขูุงู)
- Streaming: sentence-chunked streaming with RTF 0.153 (~6.5x realtime)
- Multilingual: 646 languages inherited, optimized for Egyptian Arabic + code-switching
Performance
Evaluated via round-trip CER (generate speech, transcribe with Whisper large-v3, compare to input):
| Metric | Value |
|---|---|
| Egyptian Arabic CER | 0.066 - 0.086 |
| Conversational CER | 0.0937 |
| English CER | 0.041 |
| Code-switch CER | 0.2488 - 0.270 (weakest area) |
| Numbers CER | 0.00 - 0.26 |
| Fail rate (CER > 0.2) | 0.03 |
| RTF (streaming) | 0.153 (~6.5x realtime) |
| TTFA (time-to-first-audio) | ~0.65s |
| DNSMOS ovrl_mos | 3.35 (Arabic) / 3.47 (English) |
| MFCC speaker similarity | 0.9985 (Arabic) / 0.9972 (English) |
| Voice Design fidelity | 0.999 |
Latency
| Scenario | Value |
|---|---|
| First-chunk latency (streaming) | ~0.65s |
| Full sentence (~30s audio) | ~2-4s wall time |
| RTF (streaming, sentence-chunked) | 0.153 |
| RTF (batched, num_step=8) | ~0.01 (very fast) |
Memory
| Scenario | VRAM |
|---|---|
| Inference (bf16) | ~2.5-3 GB |
Usage
Voice cloning (recommended for production)
from omnivoice import OmniVoice
import soundfile as sf
model = OmniVoice.from_pretrained("tokenaii/Replica-Preview", device_map="cuda:0", dtype="bfloat16")
prompt = model.create_voice_clone_prompt(ref_audio="reference.wav", ref_text="transcript of reference")
audio = model.generate(text="ู
ุฑุญุจุงุ ุฃูุง ุฑูุจูููุง.", language_id="arz", voice_clone_prompt=prompt, num_step=32)
sf.write("output.wav", audio[0], 24000)
Voice Design
audio = model.generate(text="Hello, I am Replica.", language_id="en",
instruct="female, young adult, american accent", num_step=32)
Non-verbal sounds
Include markers in the text: [inhale], [exhale], [sigh], [laughter], [laugh], and fillers like ูู
ู
ู
, ุขู, Hmmm, phew.
Number normalization
Arabic digits are automatically converted to Egyptian word forms. Example: "5000" -> "ุฎู ุณุฉ ุขูุงู".
Audio Samples
Audio samples are not hosted or displayed in this repository. To hear the official audio samples โ and to see the full technical details and all features of this model โ visit the official Replica Preview 0.6B page on TokenAI's website: https://tokenai.llc/projects/models/replica-preview-0.6b
Limitations
- Code-switching (Arabic-English mixing) is the weakest area (CER 0.25)
- Number pronunciation occasionally varies ("ุฎู ุณ ุชูุงู" vs "ุฎู ุณุฉ ุขูุงู")
- Requires the Higgs Audio V2 tokenizer (included in
audio_tokenizer/)
About TokenAI
TokenAI is a non-profit AI startup founded on May 10, 2025 by Assem Sabry. TokenAI builds advanced AI models, research, datasets, and developer tools. We focus on language, multimodal, speech, cybersecurity, and Arabic-native intelligence. Our mission is to make advanced intelligence accessible and useful for everyone.
Website: https://assem.one
Contact TokenAI
For inquiries, partnerships, or support, contact TokenAI at info@tokenai.llc.
Citation
@misc{replica_preview_2026,
title={Replica Preview 0.6B: Egyptian Arabic TTS with Code-Switching},
author={TokenAI},
year={2026},
url={https://huggingface.co/tokenaii/Replica-Preview-0.6B}
}
License
Apache 2.0. See LICENSE for details.
- Downloads last month
- 22