waxal-benchmarking/voxcpm2-all28-clean-full

WAXAL VoxCPM2 — 28-Language African TTS (Clean Data)

A full fine-tune of VoxCPM2 (2B params, tokenizer-free, diffusion-AR TTS with voice cloning) on 28 African languages, produced by the WAXAL project at LyngualLabs.

This is a research checkpoint. It is not production-polished — see Known Limitations below before using it for anything beyond experimentation.

This model supersedes voxcpm2-multilang28-full, which was trained on the raw pooled corpus. This one is trained on a cleaned version of the same 28-language corpus (see Data Cleaning below) — the largest of three sibling models (Top-5 / Top-10 / all-28) trained from the same cleaned data and methodology, to compare whether language count or data quality matters more at this scale. The public demo Space now runs this checkpoint.

Languages

VoxCPM2 has no built-in mechanism for specifying language at inference — the base model infers pronunciation purely from text orthography. We use a dedicated <lang:XX> tag per language, registered as a real token in the tokenizer and resized into the text embedding table (mean-initialized, then learned during fine-tuning). The tag is mandatory — always prepend it to the text you want synthesized, e.g.:

<lang:yo> Ẹ káàárọ̀! Báwo ni oorun rí lálẹ́ àná?

Language	Tag	Hours after cleaning	Hours after resampling	Retention after cleaning
Swahili	<lang:sw>	922.8	508.4	96.4%
Kinyarwanda	<lang:rw>	910.5	505.0	96.1%
Yoruba	<lang:yo>	732.3	452.8	74.0%
Igbo	<lang:ig>	620.2	416.8	69.6%
Hausa	<lang:ha>	534.2	386.8	82.0%
Dholuo	<lang:luo>	165.1	215.0	95.4%
Malagasy	<lang:mlg>	160.2	211.8	95.5%
Kikuyu	<lang:kik>	155.1	208.4	85.1%
Amharic	<lang:am>	154.5	208.0	86.6%
Oromo	<lang:om>	151.1	205.7	85.7%
Wolaytta	<lang:wal>	141.1	198.8	78.1%
Sidama	<lang:sid>	140.4	198.3	79.6%
Tigrinya	<lang:tig>	114.4	179.0	68.5%
Kalenjin	<lang:kal>	109.6	175.2	77.1%
Ikposo	<lang:ikp>	97.5	165.2	98.0%
Somali	<lang:som>	94.1	162.3	98.0%
Luganda	<lang:lug>	87.4	156.5	97.1%
Shona	<lang:sna>	75.0	144.9	96.5%
Dagaare	<lang:dagr>	69.1	139.1	67.9%
Ewe	<lang:ewe>	67.2	137.2	67.3%
Akan	<lang:aka>	65.7	135.6	91.0%
Wolof	<lang:wol>	62.5	132.3	67.4%
Dagbani	<lang:dagb>	57.2	126.5	61.5%
Nyankole	<lang:nya>	43.1	109.9	94.2%
Maasai	<lang:maa>	42.7	109.3	57.9%
Masaaba	<lang:mas>	36.0	100.4	90.4%
Lusoga	<lang:lus>	34.4	98.2	95.7%
Acholi	<lang:ach>	21.3	77.3	80.1%

Source data: the AfroNet speech collection (Professor/<language>-speech-data), after the Phase 1 cleaning pass described below.

Data Cleaning

The earlier voxcpm2-multilang28-full trained directly on the pooled raw corpus. This model and its Top-5 / Top-10 siblings are trained on data that first passed through a dedicated cleaning pipeline, applied per language before any resampling:

1. Clip/peak check — reject clips with a sustained near-full-scale amplitude (likely clipped/distorted audio).
2. Silence trim — trim leading/trailing silence before any downstream check, so later checks evaluate the audio that's actually used for training.
3. Duration-vs-text-length ratio check — flag clips whose trimmed duration is a statistical outlier relative to their text length for that language. This caught a real upstream data bug: a small number of rows had their text field corrupted with concatenated Common Voice metadata (worst case, 225,940 characters for a single 9-second clip in the Luganda corpus).
4. ASR-CER gate — transcribe the trimmed audio with a small ASR model (omniASR_CTC_1B) and compare against the label text (CER threshold 0.3, diacritic/tone-mark-insensitive comparison so correct transcriptions aren't penalized for missing a tone mark the ASR doesn't produce). For two languages (Somali, Ikposo) the CER gate was disabled entirely after confirming via a diacritic-fix test that their high drop rates (70-80%) reflected ASR-model weakness on those languages, not actual bad data — for those two, only the clip/duration checks apply.

Retention ranges from 57.9% (Maasai) to 98.0% (Somali, Ikposo) across the 28 languages — see the table above for the full breakdown. The lowest-retention languages (Maasai, Dagbani, Tigrinya, Wolof, Ewe, Dagaare) were judged to have plausible, not clearly artifactual, elevated CER and were left on the standard gate rather than special-cased further.

Training

• Base: VoxCPM2_langtagged28 — openbmb/VoxCPM2 with 28 <lang:XX> tokens added and the text embedding table resized (mean-initialized rows) before any fine-tuning.
• Mode: full fine-tuning (all parameters trainable except the frozen AudioVAE codec).
• Data balancing: sqrt-temperature resampling (weight ∝ √hours, renormalized to preserve total volume) across the 28 languages, same methodology as the other models in this collection.
• Scale: 2,424,223 training rows (clips) after resampling, 8,065 held-out validation rows.
• Steps: 37,000 steps (~1.01 epochs over the resampled training set), batch size 3 × grad-accum 11 × 2 GPUs (effective batch 66).
• Hardware: 2× NVIDIA H200, via torchrun DDP.
• Learning rate: 1e-5, 100 warmup steps.
• Final validation loss: total 0.903 (diff 0.884, stop 0.019) at step 36,999.

Usage

from huggingface_hub import snapshot_download
from voxcpm import VoxCPM
import soundfile as sf

model_dir = snapshot_download("waxal-benchmarking/voxcpm2-all28-clean-full")
model = VoxCPM.from_pretrained(model_dir, load_denoiser=False, optimize=False)

wav = model.generate(
    text="<lang:sna> Mangwanani! Warara sei?",
    cfg_value=2.0,
    inference_timesteps=22,
)
sf.write("out.wav", wav, model.tts_model.sample_rate)

optimize=False is required — torch.compile (the default optimization path) conflicts with some inference environments (e.g. HF ZeroGPU Spaces); disabling it costs some speed but avoids crashes.

Voice cloning works the same way as base VoxCPM2 — pass prompt_wav_path and prompt_text (the literal transcript of what's spoken in the reference clip) alongside the tagged text.

Try it live: WAXAL VoxCPM Multilingual Demo Space (now running this checkpoint).

Evaluation

Pending. The automated evaluation pipeline used for the original voxcpm2-multilang28-full (ASR-based CER against ground-truth text, a real-recording CER noise floor per language, and DNSMOS naturalness) has not yet been run against this checkpoint. Those earlier numbers are not applicable here — that model was trained on uncleaned data and is not a valid proxy for this model's quality. This section will be updated once Stage A/B/C evaluation has run, alongside a comparison against the Top-5 and Top-10 sibling models.

Known Limitations

• No evaluation results yet (see above) — training loss converging normally is not the same as verified downstream speech quality.
• Retention varies considerably by language (57.9%–98.0%) — the lowest-retention languages (Maasai, Dagbani, Tigrinya, Wolof, Ewe, Dagaare) effectively trained on noticeably less data relative to their raw corpus size than the rest.
• Voice-design and cross-lingual flexibility are likely reduced, consistent with what we've seen in other full-fine-tune checkpoints in this collection — full-FT on read-speech-only data tends to degrade the base model's text-prompted style/voice-design control and non-trained-language generation.
• Non-terminating generation was an observed failure mode in voxcpm2-multilang28-full's evaluation (worst for Malagasy, Sidama, Somali); whether it recurs here is one of the things the pending evaluation will check.

Related

• waxal-benchmarking/voxcpm2-top5-full and voxcpm2-top10-full — smaller sibling models, same base-tagging and cleaning methodology, trained on subsets of this model's languages.
• waxal-benchmarking/voxcpm2-multilang28-full — the predecessor, trained on the uncleaned pooled corpus, same 28 languages.
• AfroNet speech collection — the source training data.