Phonon-2 for LiteRT

This repository is a conversion of FermionResearch/Phonon-2 (English speech-to-text) to LiteRT, made with litert-torch 0.9.4 and ai-edge-quantizer 0.10.0 for LiteRT 2.3.0: on a Galaxy S26 its float16 encoders on the NPU transcribe 109× faster than real time with the runner's two-thread pipelining, 39× on the GPU and its int8 encoders 45× on the CPU without it (655 LibriSpeech test-clean utterances, one after another, measured 2026-11-08). On all of LibriSpeech test-clean the word error rate on the Galaxy S26's NPU is 1.728 %, against 1.736 % for our float32 PyTorch run of the same windows. Speed still being tuned (updates follow).

FermionResearch/Phonon-2 keeps the architecture of NVIDIA's parakeet-tdt-0.6b-v3 and packs its weights into a 164 MB download: the linear weights of the encoder are re-trained to five values per output row ({0, ±lo, ±hi}), and the other large weights are 6-bit tables. These LiteRT files do not keep that packing. The large weights are expanded to float16 (GPU, NPU) or int8 (CPU), so the files are larger: 1.95 GB for the CPU set, 3.68 GB for the GPU and NPU set. What they add is the phone's NPU and GPU through LiteRT. For the 164 MB form, Fermion Research publishes its own Core ML and Android CPU builds (links on its card).

The graphs are built by the same code as litert-community/parakeet-ultra's, and the mel files are the same bytes.

Files

file	bytes	what	backend
phonon2_encoder_10s_i8.tflite	616,306,528	encoder for windows up to 10 s; int8 weights, one scale per output channel, in the linear layers and the two 1×1 convolutions of the subsampling, with int8 activations computed at run time; the depthwise convolutions and the 24 relative-position tables stay float32	CPU
phonon2_encoder_20s_i8.tflite	640,882,528	the same, up to 20 s	CPU
phonon2_encoder_30s_i8.tflite	665,458,528	the same, up to 30 s	CPU
phonon2_encoder_10s_f16.tflite	1,194,680,752	encoder for windows up to 10 s; float16 weights in the linear and convolution layers, LayerNorm inputs rescaled so that float16 arithmetic does not overflow	GPU, NPU
phonon2_encoder_20s_f16.tflite	1,219,256,752	the same, up to 20 s	GPU, NPU
phonon2_encoder_30s_f16.tflite	1,243,832,752	the same, up to 30 s	GPU, NPU
phonon2_mel_10s.tflite	1,608,068	log-mel front end, 10 s of 16 kHz audio (no weights: the same bytes as litert-community/parakeet-ultra's and litert-community/parakeet-redux's)	CPU
phonon2_mel_20s.tflite	2,252,068	the same, 20 s	CPU
phonon2_mel_30s.tflite	2,896,076	the same, 30 s	CPU
phonon2_decoder_i8.tflite	17,866,768	TDT prediction network (predict) and joint network (joint), int8 weights	CPU
tokenizer.json	1,159,960	the tokenizer of nvidia/parakeet-tdt-0.6b-v3 (Phonon-2 keeps it)	host
ultra_asr_runner_p.cc, build_android.sh		a command-line runner for Android (LiteRT C API) and its build script: parakeet-ultra's files, unchanged (the runner reads the model files from its arguments)
tdt_text.py		turns the runner's token ids into text (parakeet-ultra's file, unchanged)	host

Each encoder file holds the whole encoder; the three window lengths only save the work of padding short audio. Take the i8 set for the CPU (1.95 GB with the mel and decoder files) or the f16 set for the GPU or the NPU (3.68 GB with them).

How it works

audio (16 kHz mono) → 30 s windows → mel front end → encoder → greedy TDT decoding → text

• Windows. The audio is cut into consecutive windows of 479,840 samples (30 s); the last one is shorter. A window under 320 samples (two mel frames) is dropped and has no text. Each window is transcribed on its own, and the window texts are joined with one space. Long audio is cut at fixed 30 s points, not at pauses, so a word that crosses a cut can be split, lost or repeated.
• Buckets. Each window goes to the smallest graph pair that holds it: up to 160,000 samples → the 10 s mel and encoder, up to 320,000 → 20 s, else 30 s. The shorter graphs only skip the work on padding: in float32 they gave the same tokens as the 30 s graphs on all 83 windows of our 82-utterance test subset.
• Mel front end (phonon2_mel_*, inputs audio [1, samples] float32 and n_valid [1] int32): pre-emphasis, STFT (512 points, 400-sample Hann window, hop 160), 128 Slaney mel bins, log and per-window normalization over the valid frames, as transformers' ParakeetFeatureExtractor. Outputs features [1, 128, T], mel_mask [1, T] and frame_mask [1, T / 8] (T = 1,000 / 2,000 / 3,000).
• Encoder (phonon2_encoder_*, inputs features and mel_mask as the mel graph gives them): output enc [1, T / 8, 640]; rows past the valid frames are undefined.
• Decoding (on the host, phonon2_decoder_i8): predict (token [1, 1] int32, h and c [2, 1, 640]) → dec_out [1, 640] and the new h, c; joint (one enc row [1, 640], dec_out) → 8,198 logits = 8,193 token logits (8192 = blank) + 5 duration logits (0–4 frames). Start from blank with zero state; at each step take the arg-max token and duration; a blank with duration 0 moves one frame; a non-blank token is emitted and fed to predict; ten tokens on one frame move one frame; token 3 ends the window; stop at the last valid frame (the sum of frame_mask) or after 3,750 steps.
• Pipelining (the Android runner's --pipeline 1). The runner decodes window k on its main thread while a second thread runs the mel front end and the encoder of the next windows (up to two ahead), also across files; --pipeline-depth sets how many (2 by default). Each window's decoding still starts from blank with zero state, so the tokens are the ones --pipeline 0 (one thread, window after window) gives: the same tokens, frames and durations on all 658 windows of our 655-utterance runs on the Galaxy S26's NPU. With a single window there is nothing to overlap: the gain comes from runs of several windows or files.
• Text. Decode each window's token ids with tokenizer.json, special tokens skipped and repeated ids kept (Tokenizer.decode(ids, skip_special_tokens=True)). Id 3 (<|endoftext|>) is not flagged special in tokenizer.json, so a window that ended on it would print the literal token; no window did in our runs.

Use it

1. Download

pip install -U huggingface_hub
hf download litert-community/Phonon-2 --local-dir Phonon-2
cd Phonon-2

2. Your audio

Both runners below read 16 kHz mono 16-bit WAV, and the Android runner reads a folder with an index file. The block below writes those WAVs and the index from any format ffmpeg reads; replace sample1.wav sample2.wav with your files.

mkdir -p audio && : > audio/index.tsv && i=0
for f in sample1.wav sample2.wav; do
  id=$(printf %03d $i)
  ffmpeg -loglevel error -y -i "$f" -ac 1 -ar 16000 -c:a pcm_s16le "audio/$id.wav"
  printf '%s\t%s\t%s.wav\n' "$id" "$(basename "$f")" "$id" >> audio/index.tsv
  i=$((i + 1))
done

The accuracy tables below score LibriSpeech's 16-bit files and VoxPopuli's files as published (32-bit float containers holding 16-bit values), whose samples this block keeps; a 32-bit float source with finer values is rounded to 16-bit PCM here, which can change a transcript by a token.

3a. On a desktop (Python, CPU)

Save this as transcribe.py in the Phonon-2 folder:

import sys
import time
import wave

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from tokenizers import Tokenizer

BUCKETS = (10, 20, 30)  # seconds: each 30 s window goes to the smallest graph that holds it
mel = {b: Interpreter(f"phonon2_mel_{b}s.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
enc = {b: Interpreter(f"phonon2_encoder_{b}s_i8.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
dec = Interpreter("phonon2_decoder_i8.tflite", num_threads=1)
predict, joint = dec.get_signature_runner("predict"), dec.get_signature_runner("joint")
tokenizer = Tokenizer.from_file("tokenizer.json")
BLANK, EOS, WINDOW = 8192, 3, 479840


def read_wav(path):  # 16 kHz mono 16-bit PCM
    with wave.open(path) as w:
        assert (w.getframerate(), w.getnchannels(), w.getsampwidth()) == (16000, 1, 2), path
        return np.frombuffer(w.readframes(w.getnframes()), "<i2").astype(np.float32) / 32768


def transcribe_window(x):
    n = len(x)
    b = next(b for b in BUCKETS if n <= b * 16000)
    audio = np.zeros((1, b * 16000), np.float32)
    audio[0, :n] = x
    m = mel[b](audio=audio, n_valid=np.array([n], np.int32))
    e = enc[b](features=m["features"], mel_mask=m["mel_mask"])["enc"][0]
    frames = int(m["frame_mask"].sum())
    state = predict(token=np.array([[BLANK]], np.int32), h=np.zeros((2, 1, 640), np.float32), c=np.zeros((2, 1, 640), np.float32))
    ids, t, steps, run = [], 0, 0, 0
    while t < frames and steps < 3750:
        steps += 1
        logits = joint(enc_frame=e[t:t + 1], dec_out=state["dec_out"])["logits"][0]
        token, d = int(np.argmax(logits[:8193])), int(np.argmax(logits[8193:]))
        if token == BLANK and d == 0:
            d = 1
        if token != BLANK:
            ids.append(token)
            if token == EOS:
                break
            state = predict(token=np.array([[token]], np.int32), h=state["h_out"], c=state["c_out"])
        if d == 0:
            run += 1
            if run >= 10:
                d = 1
        if d > 0:
            run = 0
        t += d
    return tokenizer.decode(ids, skip_special_tokens=True)


for path in sys.argv[1:]:
    x = read_wav(path)
    t0 = time.perf_counter()
    texts = [transcribe_window(x[s:s + WINDOW]) for s in range(0, len(x), WINDOW) if len(x[s:s + WINDOW]) >= 320]
    print(f"{path}\t{len(x) / 16000:.1f} s audio\t{time.perf_counter() - t0:.2f} s\t" + " ".join(t for t in texts if t))

pip install ai-edge-litert==2.3.0 tokenizers numpy
python transcribe.py audio/*.wav

3b. On an Android phone (native runner)

Build the runner with the Android NDK r29 (we built it on macOS; the path below is Android Studio's default there). The script downloads the LiteRT 2.3.0 C headers (litert_cc_sdk.zip from LiteRT's GitHub release) and the runtime libraries (Google's Maven), and checks their sha256.

NDK=$HOME/Library/Android/sdk/ndk/29.0.13113456 bash build_android.sh

Copy the runner, the CPU files and your audio to the phone, run it, and turn its output into text. --threads 8 is what we ran on the Galaxy S26; set it for your phone (the Pixel 11 Pro has 7 CPU cores, and its row in the performance table used 5):

adb shell mkdir -p /data/local/tmp/Phonon-2
adb push ultra_asr_runner_p libs/libLiteRt.so phonon2_mel_*.tflite phonon2_encoder_*_i8.tflite phonon2_decoder_i8.tflite audio /data/local/tmp/Phonon-2/
adb shell chmod 755 /data/local/tmp/Phonon-2/ultra_asr_runner_p

adb shell 'cd /data/local/tmp/Phonon-2 && LD_LIBRARY_PATH=. ./ultra_asr_runner_p --accel cpu --libdir . \
  --model phonon2_encoder_30s_i8.tflite --mel phonon2_mel_30s.tflite \
  --model10 phonon2_encoder_10s_i8.tflite --mel10 phonon2_mel_10s.tflite \
  --model20 phonon2_encoder_20s_i8.tflite --mel20 phonon2_mel_20s.tflite \
  --dec phonon2_decoder_i8.tflite --threads 8 --mel-threads 2 --dec-threads 1 --pipeline 0 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

pip install tokenizers
python tdt_text.py tokenizer.json out.jsonl

Each line of out.jsonl that starts with {"kind":"utt" holds one file's token ids (tok), their encoder frame (frame = window index × 375 + the frame within the window) and duration (dur), and the stage times in ms. The burst_summary line gives rtf_wall, the run's wall clock from the first window's start to the last window's end over the audio time (the model load and the warm-up file excluded), and rtf_burst, the summed stage times over the audio time, which counts the overlapped time twice under --pipeline 1. The CPU run prints GPU accelerator could not be loaded and registered on stderr: the GPU library is not pushed for it, and the run goes on.

NPU (Galaxy S26, Qualcomm Hexagon). The runner compiles each f16 encoder for the Hexagon NPU on the phone the first time (LiteRT's Qualcomm compiler plugin, on-device JIT) and keeps the result in --cache-dir. It needs LiteRT's Qualcomm plugin libraries (libLiteRtDispatch_Qualcomm.so and libLiteRtCompilerPlugin_Qualcomm.so, from the litert-npu-runtime-qualcomm package that build_android.sh fetches) and the QNN libraries of the Qualcomm AI Runtime SDK (QAIRT) 2.50.0.260828, which you download from Qualcomm; copy these eight files flat into a folder qnn/: from lib/aarch64-android/, libQnnHtp.so, libQnnHtpPrepare.so, libQnnHtpV81Stub.so, libQnnHtpV81CalculatorStub.so, libQnnSystem.so, libQnnIr.so and libQnnSaver.so; from lib/hexagon-v81/unsigned/, libQnnHtpV81Skel.so. --window-samples is the window each graph holds: 479,840 samples for the 30 s graph, 320,000 and 160,000 for the others. Compile one encoder per process (one compile alone takes the runner to 4.8–6.3 GB, see Notes): the loop stops at the first encoder that does not compile, and the last command runs all three from the cache only when all three compiled. Run this block before the GPU block, with other apps closed:

adb push libs/libLiteRtDispatch_Qualcomm.so libs/libLiteRtCompilerPlugin_Qualcomm.so qnn/. phonon2_encoder_*_f16.tflite /data/local/tmp/Phonon-2/
NPU_ENV='LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH="/data/local/tmp/Phonon-2;/vendor/dsp/cdsp;/vendor/lib/rfsa/adsp;/system/lib/rfsa/adsp;/dsp"'
compiled=0
for b in 30 20 10; do
  adb shell "cd /data/local/tmp/Phonon-2 && $NPU_ENV ./ultra_asr_runner_p --accel npu+cpu --libdir . --cache-dir cache \
    --model phonon2_encoder_${b}s_f16.tflite --mel phonon2_mel_${b}s.tflite --window-samples $((b * 16000 - (b / 30) * 160)) \
    --dec phonon2_decoder_i8.tflite --fixtures audio --count 1 --rounds 0 --dump none --out out_compile" > compile_$b.log &&
    grep -q '^done wall_ms=' compile_$b.log || { echo "the ${b} s encoder did not compile (see compile_$b.log): stopped" >&2; break; }
  compiled=$((compiled + 1))
done
[ "$compiled" = 3 ] && adb shell "cd /data/local/tmp/Phonon-2 && $NPU_ENV ./ultra_asr_runner_p --accel npu+cpu --libdir . --cache-dir cache \
  --model phonon2_encoder_30s_f16.tflite --mel phonon2_mel_30s.tflite \
  --model10 phonon2_encoder_10s_f16.tflite --mel10 phonon2_mel_10s.tflite \
  --model20 phonon2_encoder_20s_f16.tflite --mel20 phonon2_mel_20s.tflite \
  --dec phonon2_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 1 \
  --fixtures audio --count \$(grep -c . audio/index.tsv) --rounds 0 --dump none --out out" > out.jsonl

GPU. Push libs/libLiteRtClGlAccelerator.so and the f16 encoders, and run the CPU command with --accel gpu --gpu-precision fp16 and the phonon2_encoder_*_f16.tflite files, without --threads (the encoder no longer runs on the CPU):

adb push libs/libLiteRtClGlAccelerator.so phonon2_encoder_*_f16.tflite /data/local/tmp/Phonon-2/
adb shell 'cd /data/local/tmp/Phonon-2 && LD_LIBRARY_PATH=. ./ultra_asr_runner_p --accel gpu --gpu-precision fp16 --libdir . \
  --model phonon2_encoder_30s_f16.tflite --mel phonon2_mel_30s.tflite \
  --model10 phonon2_encoder_10s_f16.tflite --mel10 phonon2_mel_10s.tflite \
  --model20 phonon2_encoder_20s_f16.tflite --mel20 phonon2_mel_20s.tflite \
  --dec phonon2_decoder_i8.tflite --mel-threads 2 --dec-threads 1 --pipeline 0 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

To remove everything from the phone: adb shell rm -rf /data/local/tmp/Phonon-2.

Accuracy

Word error rate (%) of the NPU form on a Galaxy S26 (the f16 encoders on the NPU, the int8 decoder on the CPU), against the PyTorch reference. The reference is transformers 5.20.2 ParakeetForTDT in float32 with the weights read out of Phonon-2's container (every key loaded, none missing or unexpected), the same 30 s windows with a valid-length mask, and the same greedy TDT decoding. Every utterance of each set. The first pair of WER columns is scored as the Open ASR Leaderboard scores (its English normalizer, compound merging, kaldialign), the (jiwer) pair is plain corpus WER (jiwer) on the normalized text, without the compound merging.

set	utterances	WER, Galaxy S26 NPU	WER, PyTorch reference	WER (jiwer), Galaxy S26 NPU	WER (jiwer), PyTorch reference
LibriSpeech test-clean	2,620	1.728	1.736	2.151	2.153
LibriSpeech test-other	2,939	3.921	3.938	4.355	4.364
VoxPopuli voxpopuli_cleaned_aa (human-corrected references)	628	2.452	2.458	2.677	2.671
VoxPopuli voxpopuli (references from parliamentary records)	1,842	6.880	6.905	7.166	7.191

Sets and revision: hf-audio/open-asr-leaderboard @ de79e57a.

Against Phonon-2's card. Run on whole utterances instead of 30 s windows, our PyTorch reference gives 1.726 on LibriSpeech test-clean, 3.916 on test-other and 2.458 on voxpopuli_cleaned_aa, where Phonon-2's card lists 1.72, 3.92 and 2.46 (2.46 under the name VoxPopuli; on the voxpopuli split the same run gives 6.896, and the leaderboard's own script notes that split's references contain spans that were not spoken). The 628 clips of voxpopuli_cleaned_aa are a subset of the 1,842 with the same audio bytes; their references differ on 626.

Every backend on the 655-utterance LibriSpeech subset of the performance table, same scoring (reference 1.769 / 2.173):

phone, backend	WER	WER (jiwer)
Galaxy S26, NPU	1.784	2.211
Galaxy S26, GPU	1.799	2.218
Galaxy S26, CPU	1.792	2.211
Pixel 11 Pro, CPU	1.784	2.158

Performance

phone	backend	RTF	× real time	peak memory
Galaxy S26	NPU (Hexagon, f16, --pipeline 1)	0.009210	109×	4.80 GB (4,688,840 kB)
Galaxy S26	GPU (Adreno, f16, fp16 arithmetic, --pipeline 0)	0.025732	39×	7.37 GB (7,193,436 kB)
Galaxy S26	CPU (i8, 8 threads, --pipeline 0)	0.022280	45×	4.51 GB (4,406,376 kB)
Pixel 11 Pro	CPU (i8, 5 threads, --pipeline 0)	0.087680	11×	4.50 GB (4,389,672 kB)

• RTF = rtf_wall: the wall clock of a run from the first window's start to the last window's end over the audio time, the model load and one warm-up utterance excluded, for 655 LibriSpeech test-clean utterances (4,875.8 s of audio) run one after another in one process with ultra_asr_runner_p and the settings of "Use it" (plus a per-window dump of the outputs for scoring; the mel graphs at 2 threads, the decoder at 1). The windows went 504 / 131 / 23 to the 10 / 20 / 30 s graphs. The NPU row is the mean of two runs, the other rows one run each.
• --pipeline 1 against --pipeline 0 on the NPU (RTF): 0.009210 against 0.011520 (one session, in the order 0, 1, 1, 0). The other rows have only run with --pipeline 0 so far.
• State of the phone. The runs did not start from the same state. On the Galaxy S26's NPU the first --pipeline 0 run (thermal status 0 at its start) took 0.011097 and the last (status 2) 0.011942. The Galaxy S26's CPU run started at status 0 with no CPU frequency cap and ended with the CPU clocks capped at 1.79 / 1.75 GHz. Its GPU run came last in its session: status 2, the GPU's clock ceiling at 646 MHz at the start and 500 MHz at the end (1,300 MHz at rest), the CPU clocks capped at 1.79 / 1.75 GHz at the start. The Pixel 11 Pro's CPU run started at status 1 and ended with the CPU clocks capped at 1.92 / 1.80 / 2.00 GHz.
• Encoder, mel and decoder times per call follow in an update.
• Peak memory: the runner's VmHWM during the 655-utterance run (GB = kB × 1024 / 10⁹; of the NPU row's two runs, the larger).
• Galaxy S26 (SM-S942Q, Android 16) and Pixel 11 Pro (Tensor G6, Android 17); LiteRT 2.3.0 from Google's Maven; on the NPU the QNN libraries of QAIRT 2.50.0.260828 for Hexagon v81; 2026-11-08.

Notes

• First NPU run. On a Galaxy S26 the on-device compile took 78–84 s for the 30 s and 20 s encoders and 176–209 s for the 10 s encoder (one encoder per process), and each compiled encoder takes 1.2 GB in the cache folder; a later run loads each of the three from the cache in 1.0–3.2 s. While one encoder compiles, the runner's peak memory is 4.8–6.3 GB, and the phone's available memory went down to 1.28 GB. Start the first NPU run fresh, with other apps closed.
• GPU and fp16. In Phonon-2's encoder the squared deviation inside the first LayerNorm reaches 59,451,440 over the 91 windows of our range check, beyond float16's largest value 65,504. The f16 encoders scale each of the 120 LayerNorm inputs by 2^-k (k = 3–8 per norm) and its epsilon by 4^-k, which leaves the float32 result bit-identical and keeps every square in range (1,014.5 at most over the same windows); the GPU (float16 arithmetic) and the NPU run these files.
• int8 decoder. The decoder's weights are int8, one scale per output channel, with int8 activations computed at run time (dynamic range), as in the i8 encoders. Against the float32 decoder (Mac CPU, float32 encoders) it gives the same token sequence on 82 of the 83 windows of our 82-utterance test subset.
• int8 encoders. The i8 encoders round the five-value weights to int8 per output channel (each matrix's median relative error is 0.17 % at the median of the 240 matrices; 0.444 % of the small values round to zero), and their WER on a Mac CPU stays within 0.023 points of the reference on the four sets above. Their depthwise convolutions stay float32 so that XNNPACK runs them.
• Converted from the weights Phonon-2's fermion_container.py format holds, loaded into transformers 5.20.2's ParakeetForTDT (a five-value matrix becomes float32 with the same values), rewritten as plain PyTorch graphs (masks as float arithmetic, the LSTM step as explicit gates) and exported with litert-torch 0.9.4; the steps and scripts are in hf-to-litertlm (REPRODUCE.md, Phonon-2), run as printed from a clean checkout on 2026-11-08 with every published .tflite file coming out byte for byte the same.

License

The model files are under CC-BY-4.0, as FermionResearch/Phonon-2 and NVIDIA's parakeet-tdt-0.6b-v3 are. Weights © Fermion Research, derived from Parakeet TDT 0.6B v3, © NVIDIA Corporation. Changes made here: the weights were expanded to float16 and int8 as described above, and the graphs were rewritten for LiteRT.

Phonon-2's NOTICE says its training audio includes CHiME-6, distributed under CC-BY-SA-4.0 (https://chimechallenge.github.io/chime6/), and SPGISpeech, used under Kensho's public terms (https://datasets.kensho.com/datasets/spgispeech). Read that NOTICE before you redistribute: https://huggingface.co/FermionResearch/Phonon-2/blob/main/NOTICE.

The runner and the scripts (ultra_asr_runner_p.cc, build_android.sh, tdt_text.py) are under the Apache License 2.0.