Text-to-Speech
Chatterbox
voice-cloning
multilingual
hausa
yoruba
igbo
nigerian-english
african-languages
Instructions to use PlotweaverAI/naija-tts with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Chatterbox
How to use PlotweaverAI/naija-tts with Chatterbox:
# pip install chatterbox-tts import torchaudio as ta from chatterbox.tts import ChatterboxTTS model = ChatterboxTTS.from_pretrained(device="cuda") text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill." wav = model.generate(text) ta.save("test-1.wav", wav, model.sr) # If you want to synthesize with a different voice, specify the audio prompt AUDIO_PROMPT_PATH="YOUR_FILE.wav" wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH) ta.save("test-2.wav", wav, model.sr) - Notebooks
- Google Colab
- Kaggle
Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: [ha, yo, ig, en]
|
| 3 |
+
license: mit
|
| 4 |
+
pipeline_tag: text-to-speech
|
| 5 |
+
tags: [text-to-speech, voice-cloning, chatterbox, multilingual, hausa, yoruba, igbo, nigerian-english, african-languages]
|
| 6 |
+
base_model: ResembleAI/chatterbox
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# Naija TTS — multilingual Hausa / Yoruba / Igbo / Nigerian English
|
| 10 |
+
|
| 11 |
+
One Chatterbox model covering four Nigerian languages, with zero-shot voice
|
| 12 |
+
cloning. LoRA fine-tune of Chatterbox Multilingual (MIT), with the grapheme
|
| 13 |
+
vocabulary extended for the characters these languages need.
|
| 14 |
+
|
| 15 |
+
Base Chatterbox ships 23 languages, of which Swahili is the only African one.
|
| 16 |
+
This model adds Hausa, Yoruba, Igbo and Nigerian English — the last under its
|
| 17 |
+
own `en-ng` tag so base English is left intact.
|
| 18 |
+
|
| 19 |
+
## Evaluation
|
| 20 |
+
|
| 21 |
+
| Language | n | CER | CER floor | WER | Speaker SIM | RTF |
|
| 22 |
+
|---|---|---|---|---|---|---|
|
| 23 |
+
| Hausa (`ha`) | 40 | 8.83 % | 4.87 % | 32.35 % | n/a | 0.680 |
|
| 24 |
+
| Yoruba (`yo`) | 40 | 45.02 % | 12.13 % | 77.73 % | n/a | 1.065 |
|
| 25 |
+
| Igbo (`ig`) | 40 | 14.87 % | 8.41 % | 42.57 % | n/a | 0.840 |
|
| 26 |
+
| Nigerian English (`en-ng`) | 40 | 12.27 % | 13.09 % | 31.44 % | n/a | 0.932 |
|
| 27 |
+
|
| 28 |
+
**Read each CER against its own floor**, not against zero. The floor is what
|
| 29 |
+
MMS-ASR scores on genuine recordings of that language — its own error rate.
|
| 30 |
+
A CER at the floor means the synthesized speech is as intelligible as real
|
| 31 |
+
speech. The floors differ substantially between languages; a raw CER comparison
|
| 32 |
+
across languages is meaningless.
|
| 33 |
+
|
| 34 |
+
## Usage
|
| 35 |
+
|
| 36 |
+
```python
|
| 37 |
+
import torchaudio as ta
|
| 38 |
+
import chatterbox.mtl_tts as M
|
| 39 |
+
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
|
| 40 |
+
|
| 41 |
+
for code, name in [("ha","Hausa"), ("yo","Yoruba"), ("ig","Igbo"),
|
| 42 |
+
("en-ng","Nigerian English")]:
|
| 43 |
+
M.SUPPORTED_LANGUAGES[code] = name
|
| 44 |
+
|
| 45 |
+
model = ChatterboxMultilingualTTS.from_local("path/to/repo", device="cuda")
|
| 46 |
+
wav = model.generate("Sannu da zuwa.", language_id="ha",
|
| 47 |
+
audio_prompt_path="reference.wav")
|
| 48 |
+
ta.save("out.wav", wav, model.sr)
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
Normalize text to Unicode NFC and keep the languages' special characters as
|
| 52 |
+
written — for Yoruba this includes the combining tone marks.
|
| 53 |
+
|
| 54 |
+
## Limitations
|
| 55 |
+
|
| 56 |
+
- **Hausa tone** is not modelled: Boko orthography marks neither tone nor vowel
|
| 57 |
+
length, so a character-input model has no signal for it. Yoruba and Igbo write
|
| 58 |
+
tone with diacritics and are not subject to this.
|
| 59 |
+
- Long utterances can truncate (Chatterbox forces EOS on long tails).
|
| 60 |
+
- The corpus was capped per language for balance, so no language sees its full
|
| 61 |
+
available data.
|
| 62 |
+
- Outputs carry Resemble's Perth watermark, inherited from the base model.
|
| 63 |
+
|
| 64 |
+
## Licence
|
| 65 |
+
MIT, from base Chatterbox.
|