Download INTEGRATION.md from spybyscript/nemotron-speech-streaming-en-0.6b-aimodel: direct link, hf CLI and curl.
- Browser
- Download file 5.07 kB
-
https://huggingface.co/spybyscript/nemotron-speech-streaming-en-0.6b-aimodel/resolve/main/INTEGRATION.md
- Command line
-
hf download hf://spybyscript/nemotron-speech-streaming-en-0.6b-aimodel/INTEGRATION.md
-
curl -L -o INTEGRATION.md https://huggingface.co/spybyscript/nemotron-speech-streaming-en-0.6b-aimodel/resolve/main/INTEGRATION.md
English Nemotron Core AI streaming contract
The model is a component bundle with a host RNN-T decoder. It does not turn arbitrary PCM directly into a string in one graph invocation. Use CoreAISpeechRunner.swift and SwiftTranscribe.swift for the tested executable macOS example.
Source and identity
- Original checkpoint:
nvidia/nemotron-speech-streaming-en-0.6batebe59e5a817142986528bbbee5dba8db7b38ed50. - Source safetensors SHA256:
bddd8a7300826efd19cf7e01f1c7db8402bed6786fc4c7739632894f69c71473. - Adapter:
nemotron-en-coreai-560-v1. - Vocabulary size 1025; blank ID 1024; left attention context 70 frames.
- No language prompt. This is the English model, not Nemotron 3.5.
Pin a Hugging Face revision and the bundle manifest hash for deployment. Each variant's manifest.json records every asset file's size and SHA256, named inputs, types, shapes and output order. Preserve each .aimodel directory, including its main.mlirb, main.hash and metadata.json files.
Pipeline
- Input is finite Float32 mono PCM at 16 kHz. Resample/downmix on the host. Do not apply the emotion model's RMS normalization to ASR audio.
- The frontend emits 128-bin log-mel features. It uses a 512-point DFT, 400-sample non-periodic Hann window, 160-sample hop, pre-emphasis 0.97, Slaney mel filters and
log(mel + 2^-24). - The initial invocation consumes 49 mel frames; subsequent invocations consume 56. The subsampler emits 7 encoder frames per chunk. Lookahead is 6 encoder frames; this recipe is the 560 ms profile. It is not the upstream 80 ms profile.
- For mel offset
m, frontend PCM begins atm*160 - 257and hasmel_frames*160 + 353samples. Samples outside the recording are zero.valid_frames=min(mel_frames, max(0, total_samples//160-m)).valid_samples=min(window_samples-1, max(0,total_samples-begin-1)). Live calls wait until their full window is available; end-of-stream flush pads the final window. - Initialize three subsampling caches and each of 24 blocks' key, value and convolution caches to zero using the manifest shapes. Pass
valid_cache=min(chunk_index*7,70)as Int32. Replace caches by named outputs after each call. These are explicit tensors, not hidden framework state. - Project the final encoder output to
[1,7,640]. - Initialize the two-layer LSTM by evaluating the blank token once with zero hidden/cell
[2,1,640]. Keep its prediction and returned state. - For each encoder frame, evaluate joint logits with the current predictor output. Blank advances to the next frame without changing predictor state. A nonblank token is appended and fed to the predictor with its retained state. Guard at 10 symbols per frame and report guard hits. Do not collapse repeated token IDs: this is RNN-T, not CTC decoding.
- Decode with the supplied BPE/Metaspace tokenizer. Reset every cache, predictor state and PCM history for each new utterance.
Frontend arithmetic and I/O are FP32. Network floating-point inputs/outputs and caches are FP16. Token and valid-cache inputs are Int32. A weight-compressed variant, if present, still uses floating-point activations; its name does not claim integer execution.
Native integration
The following pattern is valid for Xcode 27's Core AI API. Obtain shapes, scalar types and named outputs from the bundle manifest. Keep model/function objects resident, and allocate reusable tensor buffers for the eventual app.
import CoreAI
import Foundation
let model = try await AIModel(contentsOf: componentURL)
guard let function = try model.loadFunction(named: "main") else {
fatalError("Missing model function")
}
var outputs = try await function.run(inputs: namedInputArrays)
let prediction = outputs.remove("prediction")?.ndArray
SwiftProbe.swift is a compiled and executed predictor example. It is not a complete Swift transcriber. The complete SpeechRunner implements the decoder, PCM windowing and tokenizer with reusable graph buffers; its transcripts and resets were validated separately. Preserve names and state transitions when adapting it for an app.
Compilation and device validation
Install Apple's Metal Toolchain if coreai-build is unavailable:
xcodebuild -downloadComponent MetalToolchain
xcrun coreai-build compile fp16/block_00.aimodel --platform iOS --output compiled/
.aimodel assets specialize on the destination device. AOT .aimodelc outputs are architecture-specific and still need device specialization. This release distributes source assets, not the local compiler cache. The release's compilation report records what was checked. Compilation success is not proof of iPhone execution or Neural Engine placement.
Before selecting an iPhone default, measure cold load and first inference separately from warm inference, peak process/GPU memory, transcription error, partial/final latency, thermal behavior and energy during concurrent audio and emotion analysis. The Python harness copies graph outputs through host arrays; it establishes correctness, not the fastest possible Swift implementation.