Qwen3.8-27B-heretic-dflash ⚡

Qwen3.8-27B-heretic-dflash is a high-performance DFlash speculative drafter trained on multi-layer hidden representations from Qwen 3.8 27B Heretic models (including heretic-org/Qwen3.8-27B-heretic-ara and quantized variants like MXFP4).

By predicting future token blocks in parallel using single forward-pass target hidden representations, this drafter accelerates autoregressive generation on Apple Silicon (MLX) and NVIDIA GPUs by 2.6× to 2.8× with exact target model output fidelity.

---

🚀 Live Speculative Benchmarks (Apple Silicon MLX)

Evaluated against Qwen3.8-27B-heretic-ara-mxfp4 on real-world prompts with standard temperature:

Task / Domain	Prompt Scenario	Speculative Acceptance ($k$)	Acceptance Rate (%)	Decode Speed	End-to-End Speedup
Async Python Coding	Concurrent rate-limited web scraper with retry backoff & error handling	3.01 tokens / round	82.05%	21.9 tok/s	2.58× faster (vs ~8.5 tok/s baseline)
Math & Reasoning	Recursive sequence formula derivation & proof ($\sum_{n=1}^{100} rac{1}{a_n}$)	4.34 tokens / round	80.50%	24.0 tok/s	2.82× faster (vs ~8.5 tok/s baseline)

• Draft Acceptance Rate: Over 81% of all proposed draft tokens are verified and accepted.
• First-Token Acceptance Rate ($k \ge 1$): >95%, eliminating stalling during token generation.

---

📈 Multi-Round Continuous Training Progress

The model is iteratively trained on multi-domain instruction sequences (coding feedback, mathematical reasoning, system architecture, and multi-turn chat) using pure bfloat16 extracted hidden states:

Checkpoint Stage	Cumulative Sequences	Cumulative Tokens	Held-Out Val Loss	Block Accuracy ($k=15$)	Status
Untrained Base	0	0	7.3247	7.7%	Initial z-lab weights
Round 1 – 2	6,000	12.4M	3.4830	19.3%	Baseline Convergence
Round 3 – 5	15,000	31.0M	3.8210	14.8%	Cross-Domain Diversity
Round 6 – 8	24,000	49.5M	3.6890	16.2%	Reasoning & Proofs
Round 9 – 10	30,000	61.9M	3.3995	17.8%	Current Checkpoint (Live)
Round 11 – 17	51,000	~105M	Queued	Queued	Full 50k Step Schedule

---

🛠️ Architecture & Specifications

• Drafter Architecture: DFlash Dual-Hidden Non-Autoregressive Drafter
• Target Model: Qwen 3.8 27B Heretic / ARA
• Hidden State Feature Dimension: $5 imes 5,120 = 25,600$ (concatenated target layers in pure bfloat16)
• Context Length: 2,048 tokens + 16 block tokens
• Block Size ($N$): 16 future tokens per speculative step
• Storage Footprint: 3.46 GB safetensors

---

💻 How to Use with MLX / MLX-VLM

Launch mlx_vlm.server with --draft-model and --draft-kind dflash:

python3 -m mlx_vlm.server     --host 0.0.0.0     --port 8080     --model /path/to/Qwen3.8-27B-heretic-ara-mxfp4     --trust-remote-code     --kv-bits 4     --kv-quant-scheme uniform     --kv-group-size 64     --draft-model jfan/Qwen3.8-27B-heretic-dflash     --draft-kind dflash

Once running, send standard OpenAI-compatible /v1/chat/completions requests to enjoy accelerated 22–24+ tokens/second inference.