# English Nemotron Core AI streaming contract The model is a component bundle with a host RNN-T decoder. It does not turn arbitrary PCM directly into a string in one graph invocation. Use `CoreAISpeechRunner.swift` and `SwiftTranscribe.swift` for the tested executable macOS example. ## Source and identity - Original checkpoint: `nvidia/nemotron-speech-streaming-en-0.6b` at `ebe59e5a817142986528bbbee5dba8db7b38ed50`. - Source safetensors SHA256: `bddd8a7300826efd19cf7e01f1c7db8402bed6786fc4c7739632894f69c71473`. - Adapter: `nemotron-en-coreai-560-v1`. - Vocabulary size 1025; blank ID 1024; left attention context 70 frames. - No language prompt. This is the English model, not Nemotron 3.5. Pin a Hugging Face revision and the bundle manifest hash for deployment. Each variant's `manifest.json` records every asset file's size and SHA256, named inputs, types, shapes and output order. Preserve each `.aimodel` directory, including its `main.mlirb`, `main.hash` and `metadata.json` files. ## Pipeline 1. Input is finite Float32 mono PCM at 16 kHz. Resample/downmix on the host. Do not apply the emotion model's RMS normalization to ASR audio. 2. The frontend emits 128-bin log-mel features. It uses a 512-point DFT, 400-sample non-periodic Hann window, 160-sample hop, pre-emphasis 0.97, Slaney mel filters and `log(mel + 2^-24)`. 3. The initial invocation consumes 49 mel frames; subsequent invocations consume 56. The subsampler emits 7 encoder frames per chunk. Lookahead is 6 encoder frames; this recipe is the 560 ms profile. It is not the upstream 80 ms profile. 4. For mel offset `m`, frontend PCM begins at `m*160 - 257` and has `mel_frames*160 + 353` samples. Samples outside the recording are zero. `valid_frames=min(mel_frames, max(0, total_samples//160-m))`. `valid_samples=min(window_samples-1, max(0,total_samples-begin-1))`. Live calls wait until their full window is available; end-of-stream flush pads the final window. 5. Initialize three subsampling caches and each of 24 blocks' key, value and convolution caches to zero using the manifest shapes. Pass `valid_cache=min(chunk_index*7,70)` as Int32. Replace caches by named outputs after each call. These are explicit tensors, not hidden framework state. 6. Project the final encoder output to `[1,7,640]`. 7. Initialize the two-layer LSTM by evaluating the blank token once with zero hidden/cell `[2,1,640]`. Keep its prediction and returned state. 8. For each encoder frame, evaluate joint logits with the current predictor output. Blank advances to the next frame without changing predictor state. A nonblank token is appended and fed to the predictor with its retained state. Guard at 10 symbols per frame and report guard hits. Do not collapse repeated token IDs: this is RNN-T, not CTC decoding. 9. Decode with the supplied BPE/Metaspace tokenizer. Reset every cache, predictor state and PCM history for each new utterance. Frontend arithmetic and I/O are FP32. Network floating-point inputs/outputs and caches are FP16. Token and valid-cache inputs are Int32. A weight-compressed variant, if present, still uses floating-point activations; its name does not claim integer execution. ## Native integration The following pattern is valid for Xcode 27's Core AI API. Obtain shapes, scalar types and named outputs from the bundle manifest. Keep model/function objects resident, and allocate reusable tensor buffers for the eventual app. ```swift import CoreAI import Foundation let model = try await AIModel(contentsOf: componentURL) guard let function = try model.loadFunction(named: "main") else { fatalError("Missing model function") } var outputs = try await function.run(inputs: namedInputArrays) let prediction = outputs.remove("prediction")?.ndArray ``` `SwiftProbe.swift` is a compiled and executed predictor example. It is not a complete Swift transcriber. The complete `SpeechRunner` implements the decoder, PCM windowing and tokenizer with reusable graph buffers; its transcripts and resets were validated separately. Preserve names and state transitions when adapting it for an app. ## Compilation and device validation Install Apple's Metal Toolchain if `coreai-build` is unavailable: ```sh xcodebuild -downloadComponent MetalToolchain xcrun coreai-build compile fp16/block_00.aimodel --platform iOS --output compiled/ ``` `.aimodel` assets specialize on the destination device. AOT `.aimodelc` outputs are architecture-specific and still need device specialization. This release distributes source assets, not the local compiler cache. The release's compilation report records what was checked. Compilation success is not proof of iPhone execution or Neural Engine placement. Before selecting an iPhone default, measure cold load and first inference separately from warm inference, peak process/GPU memory, transcription error, partial/final latency, thermal behavior and energy during concurrent audio and emotion analysis. The Python harness copies graph outputs through host arrays; it establishes correctness, not the fastest possible Swift implementation.