Qwen3.8-27B-oQ4e-mtp

This model was quantized using oQ (oMLX v0.5.7) mixed-precision quantization.

Base model: Qwen/Qwen3.8-27B

Chat template: froggeric/Qwen-Fixed-Chat-Templates

Quantization details

• Model type: qwen3_5
• Bits: 4
• Group size: 64
• Format: MLX safetensors
• MTP: Preserved (mtp_num_hidden_layers: 1)
• Calibration: oQ4e (enhanced, imatrix-based; oqe_code_multilingual)

Environment

• Hardware: M5 MacBook Air 32GB
• Inference Framework: oMLX v0.5.7
• Max Concurrent Requests: 4
• Settings:
  • Thinking: Disabled
  • TurboQuant KV Cache: Enabled (4-bit)
  • Lightning MTP: Enabled (key speed improvement)

Performance Benchmarks

Note: Results are for reference only and may vary depending on hardware, software configuration, and workload.

Single Request Results

Test	TTFT(ms)	TPOT(ms)	pp TPS	tg TPS	E2E(s)	Throughput	Peak Mem
pp1024/tg128	5219.3	53.59	196.2 tok/s	18.8 tok/s	12.054	95.6 tok/s	16.62 GB
pp4096/tg128	21245.1	68.21	192.8 tok/s	14.8 tok/s	29.957	141.0 tok/s	18.19 GB

Continuous Batching (pp1024 / tg128)

Batch	tg TPS	Speedup	pp TPS	pp TPS/req	TTFT(ms)	E2E(s)
1x	18.8 tok/s	1.00x	196.2 tok/s	196.2 tok/s	5219.3	12.054
2x	14.0 tok/s	0.74x	172.2 tok/s	86.1 tok/s	11894.9	30.238
4x	11.4 tok/s	0.61x	135.4 tok/s	33.9 tok/s	29810.5	75.007

Intelligence Benchmark

Note: Each benchmark round tests only 30 questions. Results are for reference only.

Benchmark	Accuracy	Correct	Total	Time(s)	Think
MMLU	76.7%	23	30	164.5	No
TRUTHFULQA	90.0%	27	30	67.6	No
GSM8K	96.7%	29	30	1128.3	No
MATHQA	46.7%	14	30	73.4	No
HUMANEVAL	93.3%	28	30	526.1	No