Add model card with val loss table and usage example
Browse files
README.md
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: openbmb/VoxCPM2
|
| 3 |
+
language:
|
| 4 |
+
- dag
|
| 5 |
+
language_bcp47:
|
| 6 |
+
- dag-gh
|
| 7 |
+
tags:
|
| 8 |
+
- text-to-speech
|
| 9 |
+
- full-finetune
|
| 10 |
+
- dagbani
|
| 11 |
+
- farmerline
|
| 12 |
+
- african-languages
|
| 13 |
+
license: apache-2.0
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# VoxCPM2 Dagbani β Full SFT
|
| 17 |
+
|
| 18 |
+
Full fine-tune of [VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) on Dagbani speech data. Trained by [FarmerlineML](https://huggingface.co/FarmerlineML) for the darli AI agricultural voice assistant.
|
| 19 |
+
|
| 20 |
+
## Training Details
|
| 21 |
+
|
| 22 |
+
| Parameter | Value |
|
| 23 |
+
|---|---|
|
| 24 |
+
| Base model | openbmb/VoxCPM2 (2B) |
|
| 25 |
+
| Method | Full SFT (all parameters) |
|
| 26 |
+
| Learning rate | 1e-5 |
|
| 27 |
+
| Batch size | 1 (grad accum 16, effective batch 16) |
|
| 28 |
+
| Sample rate | 16kHz (AudioVAE encoder input) |
|
| 29 |
+
| Final step | 3062 |
|
| 30 |
+
|
| 31 |
+
**Validation Loss:**
|
| 32 |
+
|
| 33 |
+
| Step | loss/total | loss/diff | loss/stop |
|
| 34 |
+
|------|-----------|-----------|----------|
|
| 35 |
+
| 0 | 1.107430 | 0.857099 | 0.166888 |
|
| 36 |
+
| 500 | 0.868330 | 0.819919 | 0.032275 |
|
| 37 |
+
| 1000 | 0.864477 | 0.812996 | 0.034321 |
|
| 38 |
+
| 1500 | 0.855272 | 0.806172 | 0.032734 |
|
| 39 |
+
| 2000 | 0.870591 | 0.798133 | 0.048305 |
|
| 40 |
+
| 2500 | 0.865124 | 0.800481 | 0.043095 |
|
| 41 |
+
| 3000 | 0.851482 | 0.798783 | 0.035133 |
|
| 42 |
+
|
| 43 |
+
**Datasets:**
|
| 44 |
+
- [ghananlpcommunity/navigation-corpus-dagbani-speech](https://huggingface.co/datasets/ghananlpcommunity/navigation-corpus-dagbani-speech)
|
| 45 |
+
- [FarmerlineML/DagbaniTTS2025_dataset](https://huggingface.co/datasets/FarmerlineML/DagbaniTTS2025_dataset)
|
| 46 |
+
|
| 47 |
+
## Usage
|
| 48 |
+
|
| 49 |
+
```python
|
| 50 |
+
from voxcpm import VoxCPM
|
| 51 |
+
import soundfile as sf
|
| 52 |
+
import numpy as np
|
| 53 |
+
|
| 54 |
+
model = VoxCPM.from_pretrained(
|
| 55 |
+
"FarmerlineML/voxcpm2-dagbani-sft",
|
| 56 |
+
load_denoiser=False,
|
| 57 |
+
)
|
| 58 |
+
|
| 59 |
+
def trim_audio(wav, sr, silence_thresh=0.01, max_silence_secs=2.0):
|
| 60 |
+
abs_wav = np.abs(wav)
|
| 61 |
+
window = int(0.05 * sr)
|
| 62 |
+
n_wins = len(abs_wav) // window
|
| 63 |
+
max_sil = int(max_silence_secs / 0.05)
|
| 64 |
+
silence_count, cut_sample = 0, len(wav)
|
| 65 |
+
for w in range(n_wins):
|
| 66 |
+
chunk = abs_wav[w * window:(w + 1) * window]
|
| 67 |
+
if chunk.max() < silence_thresh:
|
| 68 |
+
silence_count += 1
|
| 69 |
+
if silence_count >= max_sil:
|
| 70 |
+
cut_sample = (w - max_sil + 1) * window
|
| 71 |
+
break
|
| 72 |
+
else:
|
| 73 |
+
silence_count = 0
|
| 74 |
+
return wav[:min(cut_sample + int(0.1 * sr), len(wav))]
|
| 75 |
+
|
| 76 |
+
wav = model.generate(
|
| 77 |
+
text="a nyΙla Dagbanli lΙri yubu daluu",
|
| 78 |
+
reference_wav_path="your_dagbani_speaker.wav",
|
| 79 |
+
cfg_value=2.0,
|
| 80 |
+
inference_timesteps=15,
|
| 81 |
+
retry_badcase=False,
|
| 82 |
+
max_len=max(50, len(text) * 4),
|
| 83 |
+
)
|
| 84 |
+
wav = trim_audio(wav, 48000)
|
| 85 |
+
sf.write("output.wav", wav, 48000)
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
## Repo Structure
|
| 89 |
+
|
| 90 |
+
```
|
| 91 |
+
βββ model.safetensors # Model weights (~9.2GB)
|
| 92 |
+
βββ audiovae.pth # AudioVAE decoder
|
| 93 |
+
βββ config.json # Model architecture config
|
| 94 |
+
βββ tokenizer.json # Tokenizer
|
| 95 |
+
βββ training/
|
| 96 |
+
β βββ train.log # Full training log
|
| 97 |
+
β βββ val_loss_summary.txt # Validation losses per checkpoint
|
| 98 |
+
β βββ training_state.json # Final training state
|
| 99 |
+
βββ tensorboard/ # TensorBoard event files
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
## Notes
|
| 103 |
+
|
| 104 |
+
- Reference audio is required at inference for voice identity anchoring
|
| 105 |
+
- Use `max_len=max(50, len(text) * 4)` to prevent hallucination after sentence end
|
| 106 |
+
- A post-generation 2-second silence trim is strongly recommended
|