Instructions to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: llama cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: llama cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: ./llama-cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX # Run inference directly in the terminal: ./build/bin/llama-cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Use Docker
docker model run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
- LM Studio
- Jan
- Ollama
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Ollama:
ollama run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
- Unsloth Desktop
- Docker Model Runner
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Docker Model Runner:
docker model run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
- Lemonade
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
Run and chat with the model
lemonade run user.DeepSeek-V4-Flash-0731-ROCmFP3-MIX-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
card: publish both serving configs with measured quality/speed tradeoff
Browse files
README.md
CHANGED
|
@@ -27,10 +27,22 @@ important difference: **this model's calibration holds out all 92 eval items**
|
|
| 27 |
the reference's own grader (`ds4-eval --regrade-trace`, self-check 92/92) at the
|
| 28 |
matched thinking budget, serving on dflash with fused decode off.
|
| 29 |
|
| 30 |
-
##
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
## Bit allocation
|
| 36 |
|
|
|
|
| 27 |
the reference's own grader (`ds4-eval --regrade-trace`, self-check 92/92) at the
|
| 28 |
matched thinking budget, serving on dflash with fused decode off.
|
| 29 |
|
| 30 |
+
## Serving configs (measured, Radeon 8060S iGPU, 128 GB)
|
| 31 |
+
|
| 32 |
+
Two supported configurations — pick per workload. The speed lever is
|
| 33 |
+
`--ds4-expert-top-k 4` (4 instead of the model-default 6 routed experts per
|
| 34 |
+
token) plus fused decode; it costs measured quality, stated below, never
|
| 35 |
+
hidden:
|
| 36 |
+
|
| 37 |
+
| config | flags | decode | COMPSEC-17 |
|
| 38 |
+
|---|---|---|---|
|
| 39 |
+
| **quality** (default) | model-default top-k 6, fused decode off | 18.1 tok/s | **17/17** |
|
| 40 |
+
| **throughput** | `--ds4-expert-top-k 4 --ds4-fused-decode` | **22.3 tok/s** (+23%) | 16/17 |
|
| 41 |
+
|
| 42 |
+
The 82/92 full-eval score was measured at the quality config. The DSpark
|
| 43 |
+
speculative drafter does NOT currently help this artifact (accept rate
|
| 44 |
+
0.51–0.67 vs 0.837 on the reference it was calibrated for — net slower);
|
| 45 |
+
draft–target co-calibration for mix-quantized targets is an open item.
|
| 46 |
|
| 47 |
## Bit allocation
|
| 48 |
|