Alpamayo 1.5 10B · MLX 8-bit

The middle option. Half the memory of bf16.

What this is

The vision-language model of nvidia/Alpamayo-1.5-10B converted to MLX and quantized to 8 bits, paired with the original diffusion expert. It runs on Apple-silicon Macs.

vlm/       the VLM, in MLX format — 8-bit affine, group size 64, 14 GB on disk
expert/    the diffusion expert, unchanged from upstream (PyTorch)
mlx/       the conversion tool and the bridge that joins the two

torch alone cannot hold Alpamayo 1.5 on a 64 GB Mac. The bridge runs the VLM under MLX and hands its KV cache to the PyTorch expert, so only the expert needs MPS memory.

Usage

This is not a drop-in from_pretrained model. It needs the bridge in mlx/mlx_bridge.py together with the patched upstream code: drivetechodyssey-tech/alpamayo1.5, branch apple-silicon.

from mlx_bridge import MlxBridge

model = MlxBridge("drivetechodyssey/Alpamayo-1.5-10B-mlx-8bit")          # or a local path to this folder
out = model.generate(inputs)         # trajectory samples + reasoning text

Speed and memory

Measured on an M5 Max (64 GB).

condition time peak memory
Alpamayo 1.5, 4-bit bridge 3 real driving frames about 2 s 14.1 GB
Alpamayo 1.5, 4-bit bridge official clip, 6 samples 4.03 s —
Alpamayo 2 Super, 4-bit bridge official clip, 1 sample 14.22 s 34.0 GB

Quality: measured once, not ranked

One official clip, 3,086 tokens, 4 cameras, 6 trajectory samples, diffusion noise pinned on both sides so the comparison is like for like:

time minADE
torch, full model 13.51 s 1.210 m
bridge, bf16 7.20 s 0.848 m
bridge, 4-bit 4.03 s 0.674 m

All three produced the same reasoning text.

Do not read the minADE column as a ranking. With one clip and one seed the spread is dominated by which of the six samples happened to land closest to the ground truth. The defensible claim is that 4-bit did not visibly degrade on this clip — not that it beats bf16. Ranking these would take several clips across several seeds, which has not been done.

8-bit was not in that comparison.

Precision changes decoding, not prefill. Prefill runs at 2,160 / 1,993 / 2,046 tok/s for bf16 / 8-bit / 4-bit — the same within noise. Generation is 32 / 59 / 98 tok/s. The bridge's advantage over torch is the framework, not the quantization: torch on MPS prefills at 323 tok/s, about a sixth of MLX.

Notes

  • The expert is unchanged. Only the VLM was converted.
  • mx.random.seed() in MLX returns a first value that ignores the sampling distribution. The bridge discards it; without that, the first token of every plan is effectively a coin flip.
  • Requires the Apple-silicon patches to the upstream inference code (MPS autocast, SDPA, device-name handling). They live in the fork above; upstream is NVlabs/alpamayo1.5.

Origin and license

Derived from nvidia/Alpamayo-1.5-10B by NVIDIA, licensed under OpenMDW-1.1. The upstream licence is included as LICENSE / ORIGINAL_LICENSE, and the upstream model card as ORIGINAL_README.md, as that licence requires.

The conversion adds no new training data and no new weights — it is a numeric format change of the published parameters.

Downloads last month

-

Downloads are not tracked for this model. How to track
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Video Preview
loading

Model tree for drivetechodyssey/Alpamayo-1.5-10B-mlx-8bit

Quantized
(3)
this model