GLM-5.2-NVFP4-dspark

A DSpark speculative-decoding head for nvidia/GLM-5.2-NVFP4, trained with the Speculators library.

DSpark is a semi-autoregressive drafter: it proposes a whole block of tokens in a single forward pass, with a lightweight sequential head injecting intra-block dependencies.

Training

Our training recipe differs from the original DSpark recipe. We use a number of modifications, among them a per-layer fusion mixture over the tapped target layers and an additional deep-layer tap, together with an extended on-policy training schedule. The resulting head uses a 16-token draft block.

Usage

This head requires a vLLM patch — it will not load on stock vLLM. The recipe modifications above have no counterpart upstream, and vLLM's weight loader is strict, so an unpatched server fails with ValueError: There is no module or parameter named 'fusion_layer_logits'. See PATCH.md for the full explanation, install paths and tuning knobs.

Apply the patch to vLLM 0.25.0 (or 0.25.1), then serve:

# 1. get the head + patch
hf download makora-ai/GLM-5.2-NVFP4-dspark --local-dir glm52-dspark

# 2. patch vLLM (see PATCH.md for Docker / source-checkout variants)
bash glm52-dspark/apply_patch.sh       # prints "PATCHES OK (12/12)"

# 3. serve
DSPARK_DRAFT_EAGER=0 vllm serve nvidia/GLM-5.2-NVFP4 \
  --speculative-config '{"method":"dspark","model":"makora-ai/GLM-5.2-NVFP4-dspark","num_speculative_tokens":15}' \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --kv-cache-dtype fp8_e4m3 \
  --reasoning-parser glm45 \
  --compilation-config '{"cudagraph_capture_sizes":[16,32,64,128,256,512]}'

The flags above are not garnish: num_speculative_tokens, the cudagraph capture sizes and DSPARK_DRAFT_EAGER all have to match this head's 16-token block, and getting any of them wrong costs most of the speedup while leaving a server that looks healthy. PATCH.md explains each one.

Files

File	Purpose
model.safetensors, config.json, config.py	the draft head
vllm-v0.25.0-dspark-glm52.patch	required vLLM patch (53 hunks, 12 files)
apply_patch.sh	applies the patch to an installed vLLM and verifies it
PATCH.md	why the patch is needed, install paths, tuning knobs, caveats

Validation

Accepted length on the held-out validation split, by training epoch:

epoch	accepted length
1	3.697
2	4.321
3	4.600
4	4.743
5	4.788
6	4.825
7	4.903
8	4.911
9	4.915

After deploying, check the accepted length reported in vLLM's spec_decode metrics — it should land near 4.9. A value close to 1.0 means the drafter is not contributing, and points at the patch or the serving config rather than the weights.

Limitations

• Trained and evaluated against nvidia/GLM-5.2-NVFP4. Other quantisations of GLM-5.2 are untested and may accept at a lower rate.
• Validated on B200 at TP=4; other hardware and TP sizes are unvalidated.
• Requires patched vLLM 0.25.0 or 0.25.1 — see PATCH.md.
• The checkpoint ships a confidence head. Confidence-scheduled verification is not implemented in stock vLLM; the patch adds it as an opt-in (DSPARK_CONF_THRESHOLD, off by default). Without the patch those weights are unused.