harrrshall commited on
Commit
04c2b2c
·
verified ·
1 Parent(s): 91ea8c6

research-style comparison table, exact params, drop UTMOS

Browse files
Files changed (2) hide show
  1. README.md +10 -11
  2. comparison.png +2 -2
README.md CHANGED
@@ -36,14 +36,14 @@ general speaker capacity is what makes <100M reachable for code-switch speech.
36
 
37
  ![comparison](comparison.png)
38
 
39
- | Model | Params | Accent ↑ | UTMOS ↑ | SECS ↑ | Tail ↓ |
40
- |---|---|---|---|---|---|
41
- | 400M (original fine-tune) | 441M | 0.861 | 3.32 | 0.855 | 4.9% |
42
- | 200M (distilled + RL) | 265M | 0.831 | 3.22 | 0.860 | 6.7% |
43
- | **sub-100M (this, staged + RFT)** | **90M** | **0.820** | **3.26** | **0.851** | **4.4%** |
44
- | Kokoro-82M (reference) | 82M | 0.886\* | 4.38\* | N/A\*\* | n/a |
45
-
46
- \* Kokoro's accent (English-word recall) and UTMOS (English-MOS) are favoured by its English-primary design.
47
  \*\* Kokoro uses its own single voice (no target-voice cloning) and is not code-switch tuned, so SECS does
48
  not apply. The point of this row: a generic 82M TTS does not deliver fixed-voice Hindi-English code-switch;
49
  this 90M model does, at the 265M teacher's quality.
@@ -53,12 +53,11 @@ this 90M model does, at the 265M teacher's quality.
53
  | axis | this 90M | 265M teacher | delta | 95% CI |
54
  |---|---|---|---|---|
55
  | code-switch accent | 0.820 | 0.831 | -0.011 | [-0.038, +0.016] |
56
- | naturalness (UTMOS) | 3.259 | 3.224 | +0.034 | [-0.021, +0.089] |
57
  | voice fidelity (SECS) | 0.851 | 0.860 | -0.009 | [-0.014, -0.003] |
58
  | runaway-tail rate | 4.4% | 6.7% | | |
59
 
60
- Accent is statistically even (delta -0.011); UTMOS and SECS pass non-inferiority; the failure tail is lower
61
- than the teacher's. A 3x larger model at the same task quality.
62
 
63
  ## How it was built
64
 
 
36
 
37
  ![comparison](comparison.png)
38
 
39
+ | Model | Params | Accent ↑ | SECS ↑ | Tail ↓ |
40
+ |---|---|---|---|---|
41
+ | XTTS-Hinglish-443M | 443M | 0.861 | 0.855 | 4.9% |
42
+ | XTTS-Hinglish-265M | 265M | 0.831 | 0.860 | 6.7% |
43
+ | **XTTS-Hinglish-90M (this)** | **89.96M** | **0.820** | **0.851** | **4.4%** |
44
+ | Kokoro-82M | 82M | 0.886\* | n/a\*\* | n/a |
45
+
46
+ \* Kokoro's accent (English-word recall) is favoured by its English-primary design.
47
  \*\* Kokoro uses its own single voice (no target-voice cloning) and is not code-switch tuned, so SECS does
48
  not apply. The point of this row: a generic 82M TTS does not deliver fixed-voice Hindi-English code-switch;
49
  this 90M model does, at the 265M teacher's quality.
 
53
  | axis | this 90M | 265M teacher | delta | 95% CI |
54
  |---|---|---|---|---|
55
  | code-switch accent | 0.820 | 0.831 | -0.011 | [-0.038, +0.016] |
 
56
  | voice fidelity (SECS) | 0.851 | 0.860 | -0.009 | [-0.014, -0.003] |
57
  | runaway-tail rate | 4.4% | 6.7% | | |
58
 
59
+ Accent is statistically even (delta -0.011) and SECS passes non-inferiority, with a lower failure tail than
60
+ the teacher. A 3x smaller model at the same code-switch quality.
61
 
62
  ## How it was built
63
 
comparison.png CHANGED

Git LFS Details

  • SHA256: 7b8d108b8aa0e3c9f650f39ed8cea3e5214be910d6cf43dde79f211819bb02bf
  • Pointer size: 131 Bytes
  • Size of remote file: 128 kB

Git LFS Details

  • SHA256: 909bfda01e0fbe5effa875f507c60f178eeb8ea03d2bb400073d1789d6dd1956
  • Pointer size: 131 Bytes
  • Size of remote file: 141 kB