DeanoCalver commited on
Commit
66838ee
·
verified ·
1 Parent(s): 37c8d3a

card: publish both serving configs with measured quality/speed tradeoff

Browse files
Files changed (1) hide show
  1. README.md +16 -4
README.md CHANGED
@@ -27,10 +27,22 @@ important difference: **this model's calibration holds out all 92 eval items**
27
  the reference's own grader (`ds4-eval --regrade-trace`, self-check 92/92) at the
28
  matched thinking budget, serving on dflash with fused decode off.
29
 
30
- ## Speed (measured, Radeon 8060S iGPU, 128 GB)
31
-
32
- - **18.1 tok/s** median decode (plain autoregressive, no draft), 1200-token
33
- generations, coherent long-form output; prefill ~1.1 s on short prompts.
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  ## Bit allocation
36
 
 
27
  the reference's own grader (`ds4-eval --regrade-trace`, self-check 92/92) at the
28
  matched thinking budget, serving on dflash with fused decode off.
29
 
30
+ ## Serving configs (measured, Radeon 8060S iGPU, 128 GB)
31
+
32
+ Two supported configurations pick per workload. The speed lever is
33
+ `--ds4-expert-top-k 4` (4 instead of the model-default 6 routed experts per
34
+ token) plus fused decode; it costs measured quality, stated below, never
35
+ hidden:
36
+
37
+ | config | flags | decode | COMPSEC-17 |
38
+ |---|---|---|---|
39
+ | **quality** (default) | model-default top-k 6, fused decode off | 18.1 tok/s | **17/17** |
40
+ | **throughput** | `--ds4-expert-top-k 4 --ds4-fused-decode` | **22.3 tok/s** (+23%) | 16/17 |
41
+
42
+ The 82/92 full-eval score was measured at the quality config. The DSpark
43
+ speculative drafter does NOT currently help this artifact (accept rate
44
+ 0.51–0.67 vs 0.837 on the reference it was calibrated for — net slower);
45
+ draft–target co-calibration for mix-quantized targets is an open item.
46
 
47
  ## Bit allocation
48