license: mit
base_model: deepseek-ai/DeepSeek-V4-Flash-0731
tags:
- gguf
- rocmfpx
- strix-halo
- mixed-precision
- quantization
DeepSeek-V4-Flash-0731 — ROCmFPX mixed precision for Strix Halo
File: ds4-0731-gu106down-v3.gguf — 98.29 GB, single file, no sidecars. 2.766 bits per weight, computed all-in: whole file (header and embedded codebooks included) over the artifact's 284.33B parameters. The published reference for this checkpoint is 2.88 bpw at the same 82/92 score.
A mixed-precision GGUF of DeepSeek-V4-Flash-0731 (660B MoE) sized for the AMD Strix Halo 96 GiB single-allocation window: it loads unsplit on the iGPU of a 128 GB Strix Halo box (Ryzen AI MAX+ 395 / Radeon 8060S) with stock parameters, leaving headroom for a speculative-decode draft. All codebooks are embedded in the GGUF as KV — download one file and serve.
Quality
| suite | score |
|---|---|
| COMPSEC-17 (think budget 15488) | 17/17 |
| full ds4-eval 92 (COMPSEC + AIME2025 + GPQA-Diamond + SuperGPQA) | 82/92 |
At 2.766 bpw overall — ~4% fewer bits per weight than the 2.88-bpw published reference that scores the same 82/92.
82/92 equals the published reference artifact for this checkpoint — with one
important difference: this model's calibration holds out all 92 eval items
(the reference's calibration set contained 75 of them). Scores were produced with
the reference's own grader (ds4-eval --regrade-trace, self-check 92/92) at the
matched thinking budget, serving on dflash with fused decode off.
Serving configs (measured, Radeon 8060S iGPU, 128 GB)
Two supported configurations — pick per workload. The speed lever is
--ds4-expert-top-k 4 (4 instead of the model-default 6 routed experts per
token) plus fused decode; it costs measured quality, stated below, never
hidden:
| config | flags | decode | COMPSEC-17 |
|---|---|---|---|
| quality (default) | model-default top-k 6, fused decode off | 18.1 tok/s | 17/17 |
| throughput | --ds4-expert-top-k 4 --ds4-fused-decode |
22.3 tok/s (+23%) | 16/17 |
The 82/92 full-eval score was measured at the quality config. The DSpark speculative drafter does NOT currently help this artifact (accept rate 0.51–0.67 vs 0.837 on the reference it was calibrated for — net slower); draft–target co-calibration for mix-quantized targets is an open item.
Bit allocation
Every tensor's format is chosen by a measured allocator (layer-output damage under real MoE routing, one knapsack over the whole file under the 98.4e9-byte ceiling) — nothing is hand-picked. The result:
| family | format | bpw |
|---|---|---|
| routed gate/up experts (all 43 layers) | Q2_1_ROCMFP2_MIX (106) — learned per-expert codebooks | 2.50 |
| routed down experts, 15 layers | Q2_1_ROCMFP2_MIX (106) | 2.50 |
| routed down experts, 28 layers | Q3_1_ROCMFP3_MIX (105) — adaptive | 3.50 |
| attention / dense / shared expert | Q4_0_ROCMFP4_FAST (101) | 4.25 |
| norms / router / scaffolding | F32 / 101 passthrough | — |
The artifact is gated before release: a registered golden probe (COMPSEC ≥16 bar, positive-control validated) must pass at the byte ceiling.
Serving
Needs a dflash build with qtype-106 down-expert (gumix surface=2) registration
and the mix-qtype monolithic-load fallback — branch feat/qtype106-down-surface
of GeometricAGI/lucebox-hub.
dflash_server ds4-0731-gu106down-v3.gguf --target-device hip:0 --max-ctx 8192
On Strix Halo use stock settings (auto-UMA on). If a discrete GPU is present,
mask it (HIP_VISIBLE_DEVICES=<iGPU index>) or pick the iGPU explicitly. A
healthy load prints registered 101 qtype-106 and registered 28 qtype-105,
then serves with fused decode off.
The repo also carries ds4-0731-gu106-v1.gguf (+ its .gumix.bin
sidecar), an earlier 102.32 GB revision kept for reproducibility — superseded
by v3 on size, quality, and deployability.