# Shruti Marathi: how every model in the results table was run (30 Oct - 2 Nov 2026) Same 1,200 clips for every model: 200 per set, the first clips of 1-29 s in a fixed seeded order (SPRING-INX: a fixed set of 200 code-mixed clips). Same scorer: Unicode NFC, lowercase, punctuation to spaces, word error rate with a 1,000-resample bootstrap 95% interval, plus a script-blind romanised view and English recall. | Set | What it is | Licence | References | |---|---|---|---| | FLEURS mr_in test (Google) | read Wikipedia sentences | CC-BY-4.0 | Devanagari | | Kathbath Marathi valid (AI4Bharat) | read sentences over the phone; the public valid split stands in for the test split | CC-BY-4.0 | Devanagari | | DiarBench Marathi (Sarvam, AI4Bharat) | real conversations, single-speaker turns with at least one English word | CC-BY-4.0 | English in Latin; and native | | IndicVoices Marathi valid (AI4Bharat) | read, extempore and conversational | CC-BY-4.0 | English in Latin (from the dataset's own English tags); and native | | SPRING-INX R2 Marathi test (IIT Madras) | phone conversations | CC-BY-4.0 | as written (English in Latin) | | Vaani Marathi test (ARTPARK-IISc) | image-prompted spontaneous speech | CC-BY-4.0 | English in Latin; and native | | Model | How it ran | |---|---| | Shruti Marathi | the shipped `shruti-marathi.cact` in the Cactus needle engine on a Mac (Apple silicon), 5-beam, language detected | | Whisper tiny, base | transformers fp16 on a GPU, beam 5, Marathi set (size: whisper.cpp q8_0 files) | | Dolphin base | sherpa-onnx int8 CTC on a Mac; the export picks its own language | | marathi-conformer-33m-moe | the published fp32 checkpoint on a Mac CPU, greedy CTC; the published code lacks its MoE layer class, so we rebuilt it from the checkpoint | | IndicConformer Marathi | the OpenVoiceOS int8 ONNX export (AI4Bharat's 120M per-language model), onnx-asr greedy CTC on a CPU | Whisper runs away under plain beam search without temperature fallback (output longer than twice the reference on many clips), which inflates its numbers; Whisper's own Marathi fine-tune (whisper-small-marathi, 967 MB) still scores 55.1 on FLEURS and 54.0 on Kathbath in the same harness.