cfontes/GLM-5.3-Flash-DFlash2-TP4-Spark

GLM-5.3-Flash DFlash2 — TP4 DGX Spark Champion Drafter (v5)

Fine-tuned DFlash2 drafter for zai-org/GLM-5.3-Flash, tuned and served on a 4× NVIDIA DGX Spark cluster (TP4, RoCE) with vLLM speculative decoding (DFlash2, K=7).

What this is

The drafter weights = the published incoai/GLM-5.3-Flash-DFlash2 checkpoint with our Phase-0 rank-64 fc-merged adapter folded in (W + (W @ (B @ A)) / 64, trained on 446k tokens of TP4 captured hidden states). Config unchanged.

Honest note (proxy-eval trap): the adapter improved teacher-forced top-1 (0.57 → 0.64) but produced no served acceptance gain (3.01–3.30 vs stock 3.28; statistically tied across 3 rechecks). We publish it because it is the exact champion artifact we validated — not because the fine-tune is proven to help.

Benchmark (temp=0, llm-inference-bench, 30s cells, DGX Spark ×4 TP4)

Metric	stock drafter	this (v2 → v5)
decode c1 ctx0	42.2	40.9–43.2 (median 42.0)
decode c2	—	62.6
decode c4	84.7	94.2 (with MNBT 7168)
coding peak	71.5	73.8 median / 75.8 max
8k prefill	695	2,052 tok/s (MNBT 7168 fix)

Serving config (the "v5" champion)

vLLM TP4 ×4 nodes, --enforce-eager (PIECEWISE CUDA graphs regressed −10–20% — falsified, don't retry), --max-num-batched-tokens 7168, spec-config {"method":"dflash","model":"<this repo>","num_speculative_tokens":7}, KV fp8_e4m3, block 2304, max-model-len 262144, MoE Marlin. Full launcher: launch-glm53-dflash2-tp4-v5.sh. Launch order: workers first (ranks 1–3), head last. Boot ~15–20 min.

Ops notes: GB10 stuck-DVFS clocks are only fixed by power-cycle — gate every bench with a matmul-rate check, never idle clocks.

Files

• model.safetensors, config.json — merged drafter (2,342,169,824 B, md5 4a4fa6163e3f across all 4 nodes)
• launch-glm53-dflash2-tp4-v5.sh — champion launcher
• results.tsv — autoresearch ledger (v5 baseline + round-1 single-knob results, all discards)
• glm53-v5-*.json — champion bench evidence; glm53-v4-*.json — CUDA-graphs falsification data

License

GLM-5.3-Flash model license applies (inherited from base model). DFlash2 drafter architecture: see citation below.

Lineage & credit

We did not do this alone, and this release deliberately says so:

• Base model — zai-org/GLM-5.3-Flash by Z.ai / Zhipu AI
• DFlash method — Chen, Liang & Liu (2026), DFlash: Block Diffusion for Flash Speculative Decoding, arXiv:2602.06036 (Z Lab)
• Vendor drafter checkpoint — incoai/GLM-5.3-Flash-DFlash2 (starting weights)
• Build recipe & MNBT-7168 default — MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks by Mia (@MiaAI_lab). Our TP4 build is derived from her 2×Spark setup; the 8k-prefill cliff fix (chunk clamp 2048→7168) came directly from her tested production default. TRITON_ATTN acceptance-collapse warning also hers.
• Kernel wins we're adopting next — Reederey (@Reederey): EXL3 fat-expert MoE kernel (+40%), long-prefill (+20% at 100–300K)
• Autoresearch loop — adapted from Andrej Karpathy's autoresearch (LLM-driven experiment loop)
• Prior art we checked and ruled out — @runsonai's vLLM long-prefill fairness-cap writeup (led us to the MNBT clamp), @ciprianveg's TP8 sparse-indexer split (GLM-5.2/TP8-specific)

Original work on top (all on our 4-node cluster): TP4 drafter capture/training harness, GB10 DVFS ops fixes, benchmark evidence (JSONs), v5 config tuning, CUDA-graphs falsification, autoresearch round-1 results.