4-bit DSpark Drafters for Inkling-2-Mini

4-bit MLX quantizations (178.9 MB each, down 3.55x from the 635.8 MB BF16 mtp.safetensors inside thinkingmachines/Inkling-2-Mini) of Inkling-2-Mini's 3-layer DSpark MTP block-8 speculative drafter, for use with the LookAtMyQuant Apple Silicon MLX runtime beside 3.9GB, 4.7GB, 5.4GB, 8.9GB, 11.2GB, or 12.8GB.

The drafter reads 4 target hidden taps (logical visits 5, 23, 41, and 48) and proposes adaptive linear blocks (6 <-> 8, or opt-in 8-node causal trees and N-gram prompt lookup) verified in one batched pass β€” supporting lossless greedy, exact unbiased temp > 0 speculative sampling (1.38x over serial at temp=0.6, top_p=0.9, top_k=40), and opt-in tight-margin greedy (+31–36%). Live on Apple M5 Max with mtp_4bit.safetensors:

  • 3.9GB (1.18 bpp, SQYFR v3 8 GB iPhone): 75.8 tok/s serial -> 113.0 tok/s median (117.3 math, 108.5 code, 3.54 GiB active / 4.57 GiB decode peak).
  • 4.7GB (1.45 bpp, SQYFR v3 12 GB iPhone + DSpark): 69.2 tok/s serial -> 129.0 tok/s math (3.25 tok/check, 94.7 code, 4.35 GiB active / 5.35 GiB decode peak).
  • 5.4GB (1.65 bpp, SQYFR v2): 1,807 tok/s T=512 prefill, 75.3 tok/s serial -> 124.4 tok/s median (142.9 math, 126.5 reasoning, 5.33 GB active).
  • 8.9GB (2.72 bpp): 63.5 tok/s serial -> 106.0 tok/s median (125.8 reasoning, 123.8 math; mtp_4bit_8p9gb_v2.safetensors lifts held-out tpc_bl8 3.26 -> 3.53).
  • 11.2GB (3.43 bpp): 2,032 tok/s T=512 prefill, 62.4 tok/s serial -> 103.7 tok/s median (111.7 math).
  • 12.8GB (3.92 bpp): 2,016 tok/s T=512 prefill, 60.5 tok/s serial -> 112.7 tok/s median (119.2 math 1.97x, 106.2 code).

Run it

pip install git+https://github.com/ShinMegamiBoson/lookatmyquant.git

lamq generate shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-4.7GB \
    --drafter shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-drafter \
    -p "What is 17 times 23? Show the steps." --stats

Files

  • mtp_4bit.safetensors (default): 4-bit (group_size = 64) MLX-quantized stock DSpark weights (norms, fuse_weight, mask_embed, and hidden_out kept in native precision)
  • mtp_4bit_5p4gb_v2.safetensors: 4-bit DSpark drafter distilled onto 5.4GB
  • mtp_4bit_8p9gb_v2.safetensors: 4-bit DSpark drafter distilled onto 8.9GB
  • config.json: model & mtp_config metadata
  • manifest.json: drafter format descriptor (inkling2-dspark-4bit-v1)
Downloads last month
460
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-drafter

Quantized
(172)
this model