mehdi-hf/pocket-tts-farsi

Pocket TTS — Farsi (Persian)

A 100M-parameter Persian text-to-speech model that runs on a CPU at several times real time, with voice cloning from a short audio prompt. Trained with kyutai-labs/pocket-tts on 497 hours of public-domain (CC0) Persian speech.

This is the 6-layer student, distilled from a 24-layer teacher trained from scratch. It outperforms its own teacher on every measure below.

Usage

The easy way: farsi_tts.py

This repo includes a self-contained script with the settings below already built in, safe chunking for long text, automatic recovery when a generation fails to stop, and the exact Persian text normalization the model was trained on. Download farsi_tts.py and normalize_fa.py (they must stay in the same folder) and run:

uv run farsi_tts.py --text "سلام، حال شما چطور است؟"
uv run farsi_tts.py --text-file article.txt --out article.wav
uv run farsi_tts.py --list-voices

uv run builds an isolated environment on first use — no manual pip install. See farsi_tts.py --help for every option (voice selection, chunk size, pause lengths).

Directly with pocket-tts

# example_voice.wav is included in this repo
uvx pocket-tts generate --config hf://mehdi-hf/pocket-tts-farsi/farsi.yaml \
    --voice example_voice.wav \
    --text "سلام، حال شما چطور است؟" \
    --temperature 0.3 --eos-threshold -2 --frames-after-eos 0

Prompts must be 16-bit WAV unless you add soundfile: uvx --with soundfile pocket-tts generate ... for FLAC, MP3 and the rest.

From Python:

from pocket_tts import TTSModel

model = TTSModel.load_model(config="hf://mehdi-hf/pocket-tts-farsi/farsi.yaml", temp=0.3)
state = model.get_state_for_audio_prompt("your_voice.wav")
audio = model.generate_audio(state, "سلام، حال شما چطور است؟", frames_after_eos=0)

Recommended settings. Classifier-free guidance is baked in by distillation — there is no --cfg flag on this model's generate command, and none is needed. --frames-after-eos 0 trims a trailing inhale the model learned from its training data. --temperature 0.3 and --eos-threshold -2 were chosen by listening. Text longer than one short sentence should be chunked — see farsi_tts.py above, or split it yourself at sentence/clause punctuation into pieces under ~18 tokens; the raw CLI has no chunking of its own and can run past its length cap on long, unpunctuated input.

Voice prompts

This is where the model's main limitation shows. Speaker similarity is 0.948 on voices resembling its training data but 0.728 on unfamiliar speakers, and on an unfamiliar voice its stability degrades as generation runs longer — it can fail to stop, producing a capped, garbled clip.

What works:

• Start from example_voice.wav (included). It is the closest match to the training distribution and gives the best results by a clear margin.
• Clean audio, one speaker, no music. Noisy prompts measurably increase repetition failures.
• Keep prompts short — around 2-5 seconds. Training capped voice prompts at 5 s, so longer ones are out of distribution.
• Chunk text into ~18-token pieces with an unfamiliar voice. Longer chunks are where "maximum generation length reached without EOS" appears: measured on held-out speakers, chunks of 21+ tokens ran past EOS deterministically (no seed recovered them), while 9-16 token chunks were clean.

Persian voices similar in character to broadcast narration clone best. Strongly-accented, very high-pitched, and children's voices are under-served by the training corpus.

Long text must be chunked. The average training utterance is 3.8 seconds (~11 tokens), so paragraphs are out of distribution. Split at Persian punctuation into chunks under ~18 tokens and join them with ~0.15 s of silence. farsi_tts.py does this for you; the splitter it uses is also in the training repo as training/farsi/synthesize.py.

Results

Scored with whisper-large-v3 (language="fa"). floor is that ASR's own WER on the real recordings — the measurement floor. Persian ASR is far from perfect, so compare against the floor, not against zero.

Clean set (hand-verified transcripts, studio narrator), 100 items:

model	WER	floor	WER/floor	speaker sim	UTMOS
this model (6L)	0.174	0.134	1.30×	0.948	2.89
teacher (24L)	0.315	0.134	2.35×	0.933	2.58

Held-out speakers (unseen voices, subtitle transcripts), 50 items. loops counts items that collapsed into repetition:

model	WER	loops
this model (6L)	1.62	9/50
teacher (24L)	2.53	15/50

Whisper disagrees with those subtitle transcripts 48% of the time on the real recordings, so roughly half of that apparent error is not the model's.

Training data

All CC0, ~497 hours after filtering:

source	hours	transcripts
Mana-TTS	~60	hand-verified
Filimo ASR	245	subtitle-derived
YouTube ASR	~297	auto-generated subtitles

Word alignments from m3hrdadfi/wav2vec2-large-xlsr-persian-v3 (99.96% of utterances aligned). Text normalized to a single Persian spelling — Arabic letter forms folded to Persian, harakat stripped, ZWNJ preserved, all digit systems converted to words. Tokenizer: sentencepiece BPE, vocab 4000.

Teacher: 24 layers, 316M params, 400k steps at effective batch 64. Student: 6 layers, 200k steps of depth distillation with guidance baked in. Both on 8×H100.

Limitations

Ezafe. Persian does not write the linking -e between a noun and its modifier, so the model must infer it from context and often does not: حملات برون‌مرزی gets a short pause where the ezafe belongs instead of being read as hamalāt-e borun-marzi. Writing the kasre in the input does not help — the text normalizer strips harakat. Fixing this properly requires a phoneme front-end with ezafe restoration.

Repetition loops. The model can fail to emit end-of-speech and run to its length cap, filling the tail with repetition — roughly 18% of held-out items with noisy voice prompts, and 9% on a 300-item held-out set with clean ones.

farsi_tts.py mitigates this rather than fixing it: it detects a generation that hit the cap, retries once (sampling is stochastic, so marginal cases recover), and otherwise splits the text and regenerates the halves, which resolved every deterministically-stuck case tested. Using the raw CLI or the Python API directly, you get no such recovery — keep chunks short. A real fix needs retraining on longer utterances.

Short-form bias. Trained on 3.8-second utterances on average; chunk long text rather than sending it whole.

Domain. Mostly film dialogue, podcasts and magazine narration. Under-served: regional accents, children's voices, and (from Common Voice's absence here) crowd-sourced read speech.

Licence and attribution

MIT, matching the pocket-tts code. All training data is CC0. If you use this, credit is appreciated but not required.

Built with pocket-tts by Kyutai; the model architecture follows the CALM paper.