Instructions to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: llama cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: llama cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: ./llama-cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: ./build/bin/llama-cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Use Docker
docker model run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
- LM Studio
- Jan
- Ollama
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Ollama:
ollama run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
- Unsloth Desktop
- Docker Model Runner
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Docker Model Runner:
docker model run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
- Lemonade
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Run and chat with the model
lemonade run user.DeepSeek-V4-Flash-0731-ROCmFP3-MIX-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
card: whole-file bits-per-weight (2.766 vs reference 2.88 at equal score)
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ tags: [gguf, rocmfpx, strix-halo, mixed-precision, quantization]
|
|
| 6 |
|
| 7 |
# DeepSeek-V4-Flash-0731 — ROCmFPX mixed precision for Strix Halo
|
| 8 |
|
| 9 |
-
**File: `ds4-0731-gu106down-v3.gguf` — 98.29 GB, single file, no sidecars.**
|
| 10 |
|
| 11 |
A mixed-precision GGUF of DeepSeek-V4-Flash-0731 (660B MoE) sized for the AMD
|
| 12 |
Strix Halo **96 GiB single-allocation window**: it loads **unsplit on the iGPU**
|
|
@@ -21,6 +21,9 @@ embedded in the GGUF as KV — download one file and serve.
|
|
| 21 |
| COMPSEC-17 (think budget 15488) | **17/17** |
|
| 22 |
| full ds4-eval 92 (COMPSEC + AIME2025 + GPQA-Diamond + SuperGPQA) | **82/92** |
|
| 23 |
|
|
|
|
|
|
|
|
|
|
| 24 |
82/92 equals the published reference artifact for this checkpoint — with one
|
| 25 |
important difference: **this model's calibration holds out all 92 eval items**
|
| 26 |
(the reference's calibration set contained 75 of them). Scores were produced with
|
|
|
|
| 6 |
|
| 7 |
# DeepSeek-V4-Flash-0731 — ROCmFPX mixed precision for Strix Halo
|
| 8 |
|
| 9 |
+
**File: `ds4-0731-gu106down-v3.gguf` — 98.29 GB, single file, no sidecars. 2.766 bits per weight**, computed all-in: whole file (header and embedded codebooks included) over the artifact's 284.33B parameters. The published reference for this checkpoint is 2.88 bpw at the same 82/92 score.
|
| 10 |
|
| 11 |
A mixed-precision GGUF of DeepSeek-V4-Flash-0731 (660B MoE) sized for the AMD
|
| 12 |
Strix Halo **96 GiB single-allocation window**: it loads **unsplit on the iGPU**
|
|
|
|
| 21 |
| COMPSEC-17 (think budget 15488) | **17/17** |
|
| 22 |
| full ds4-eval 92 (COMPSEC + AIME2025 + GPQA-Diamond + SuperGPQA) | **82/92** |
|
| 23 |
|
| 24 |
+
At **2.766 bpw** overall — ~4% fewer bits per weight than the 2.88-bpw published
|
| 25 |
+
reference that scores the same 82/92.
|
| 26 |
+
|
| 27 |
82/92 equals the published reference artifact for this checkpoint — with one
|
| 28 |
important difference: **this model's calibration holds out all 92 eval items**
|
| 29 |
(the reference's calibration set contained 75 of them). Scores were produced with
|