MLX support
#10
by evewashere - opened
Can we expect MLX support?
just build it.
There's a community MLX conversion you can use now: mlx-community/VibeVoice-ASR-4bit (5.7 GB vs 17.3 GB for the original), run with mlx-audio. We ran it with mlx-audio 0.3.0 on an M2 16 GB and compared it with the original bf16 weights on the same recordings, one run each:
| Recording | Mac time | Mac peak memory | WER original / Mac |
|---|---|---|---|
| AMI meeting, 1 min | 1 min 28 s (RTF 1.47) | 7.7 GB | 0.301 / 0.277 |
| AMI meeting, 5 min | 5 min 18 s (RTF 1.06) | 9.6 GB | 0.229 / 0.230 |
| Press conference, 10 min | 7 min 9 s (RTF 0.71) | 11.4 GB | 2% of words differed from the original |
Transcripts were nearly the same as the original. Speaker separation wobbled on the 1-min clip (4 speakers merged into 2) but was right on the 5-min window. We measured with other apps open and 8–10 GB of swap in use, so a Mac with more memory should do better.
Details: https://github.com/MindrLabs/speech-to-text-vibevoice#a-16-gb-mac-runs-it-in-4-bit