Luigi commited on
Commit
316ce5d
Β·
verified Β·
1 Parent(s): cb85050

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +38 -5
README.md CHANGED
@@ -42,11 +42,17 @@ correctly β€” phone numbers, emails, addresses, prices, dates, temperatures, %,
42
 
43
  Pick **v2.1** for multiple voices; pick **v1** when the budget is CPU-only and tight.
44
 
45
- > **Also: `v2_mbistft_16k/` β€” PrimeTTS v2, the peak-fidelity single Xinran voice.** Same architecture as v2.1
46
- > (one speaker) but trained on the *full* clean 29k Xinran corpus, so its Xinran is the most intelligible in
47
- > the family β€” **CER 0.027** vs v2.1's 0.059 β€” at the **same** on-device RTF (0.52 @4thr). Use v2 when you only
48
- > need Xinran and want the cleanest possible output; use v2.1 when you want a choice of voice.
49
- > (`v3_4.6M/` and the top-level `*.onnx` are legacy 24 kHz variants, kept for provenance.)
 
 
 
 
 
 
50
 
51
  > πŸ”Š **Live demo:** https://huggingface.co/spaces/Luigi/PrimeTTS-vs-Inflect-Nano-v1 β€” pick a model, pick a voice, type text.
52
 
@@ -204,6 +210,33 @@ diverse code-mix**, and **the teacher** (a student's language is only as good as
204
 
205
  ---
206
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
207
  ## Credits & licenses
208
 
209
  - **v2.1 architecture:** [MB-iSTFT-VITS](https://github.com/MasayaKawamura/MB-iSTFT-VITS) (Kawamura et al., Apache-2.0) Β·
 
42
 
43
  Pick **v2.1** for multiple voices; pick **v1** when the budget is CPU-only and tight.
44
 
45
+ **The full family** (all MB-iSTFT-VITS except v1; all 16 kHz; single Xinran voice unless noted):
46
+
47
+ | model | folder | params (deploy) | CER | use when |
48
+ |---|---|---|---|---|
49
+ | **v2** | `v2_mbistft_16k/` | 34.7M (17.5M) | **0.027** | you want the cleanest single Xinran voice |
50
+ | **v2.1** | `v21_mbistft_16k/` | 37.9M (~18M) | 0.059 | you want a choice of 3 voices |
51
+ | **V2 Lite** | `v2lite_mbistft_16k/` | 24.8M (17.5M) | 0.041 | a lighter, still-good single voice for tighter GPU budgets |
52
+ | **v1** | `v1b_16k/`,`v1b_8k/` | ~5M | 0.11–0.15 | pure-CPU, real-time on a Jetson Nano |
53
+
54
+ (`v3_4.6M/` and the top-level `*.onnx` are legacy 24 kHz variants, kept for provenance.) V2 Lite uses the exact
55
+ same ONNX I/O + frontend as v2 β€” it's a drop-in, smaller replacement.
56
 
57
  > πŸ”Š **Live demo:** https://huggingface.co/spaces/Luigi/PrimeTTS-vs-Inflect-Nano-v1 β€” pick a model, pick a voice, type text.
58
 
 
210
 
211
  ---
212
 
213
+ ## Findings & lessons (what building tiny on-device zh/en TTS actually taught us)
214
+
215
+ Transferable lessons from taking this from a babbling 5M model to a shippable family. Full analysis in
216
+ [`docs/zh-en-tts-arch-survey-2026.md`](./docs) and [`docs/streaming-arch-design.md`](./docs).
217
+
218
+ - **A tiny model's quality is bounded by its *inputs*, not its parameter count.** Held-out Mandarin CER fell
219
+ **0.88 β†’ 0.06 at a fixed ~5M** purely from **phone-level forced alignment** + broad character coverage β€”
220
+ no architecture change. Sub-syllable (not character) boundaries are the difference between intelligible
221
+ speech and fluent babble. **Gate on resynth CER, not on how balanced the duration histogram looks.**
222
+ - **CER-gate the *teacher* audio, never voice-similarity alone.** Our first multi-speaker attempt trained on
223
+ teacher clips filtered only for the right *voice*; four of the "voices" were speakers that can't actually
224
+ pronounce Mandarin (teacher CER 0.45–0.79), and the student faithfully learned garbled speech. Filtering on
225
+ intelligibility (teacher X-ASR CER < 0.05) fixed it.
226
+ - **Deterministic (FastSpeech-class) models mean-regress prosody; distributional (VITS/flow) models don't.**
227
+ This is the wall that caps a tiny deterministic model at "intelligible but flat" β€” and why the flagship is
228
+ a VITS, not a bigger FastSpeech.
229
+ - **On a launch-bound GPU (Maxwell sm_53, no CUDA-graph replay), RTF is set by kernel *count*, not FLOPs.** A
230
+ smaller VITS is a smaller download but **not faster** (~0.42 RTF floor regardless of params). The lever for
231
+ *speed* is an architecture with fewer, larger kernels (flow-matching + Vocos measured ~0.18) β€” a different
232
+ axis from *size*.
233
+ - **On an ARMv8.0 CPU (Cortex-A57): fp32 is the fast format.** No int8 dot-product and no fp16 arithmetic, so
234
+ int8 either breaks the voice (static) or runs *slower* than fp32 (dynamic), fp16 casts to fp32, and XNNPACK
235
+ β‰ˆ MLAS. The only CPU speed lever is a smaller/faster architecture β€” quantization is a *download-size* option.
236
+ - **"Lighter" and "faster" are different goals.** VITS deploy size is dominated by flow + decoder + encoder,
237
+ which don't shrink with `hidden_channels`; below ~17M deploy, quality craters. **V2 Lite (17.5M) is the
238
+ practical quality floor** for this arch β€” there is no free "smaller *and* still good."
239
+
240
  ## Credits & licenses
241
 
242
  - **v2.1 architecture:** [MB-iSTFT-VITS](https://github.com/MasayaKawamura/MB-iSTFT-VITS) (Kawamura et al., Apache-2.0) Β·