nace-ai/drex-v2.0-Q4_K_M

Drex v2.0 Q4_K_M GGUF

A Q4_K_M GGUF of Drex v2.0, a decision model that takes a state and named typed questions and returns a probability for every option. The file is a single drex-v2.0-Q4_K_M.gguf, 21,170,954,464 bytes (21.2 GB), not sharded. It is quantized from the Q8_0 GGUF with the pointer head kept in F32.

Stock llama.cpp and Ollama cannot serve it: the decision endpoint needs the Nace forks or Kapitsa, see the three sections below.

Drex v2.0 · Drex repository · llama.cpp fork (branch drex-v2.0) · Kapitsa · Apple silicon page

Measured on an M5 Pro, 48 GB

Client-side wall time over localhost HTTP, one request at a time, 100 requests per cell, p95 is nearest-rank over successful timed requests.

Request	Kapitsa	llama.cpp	Ollama
Short support ticket, p50	112.7 ms	248.0 ms	281.6 ms
Short support ticket, p95	114.8 ms	253.2 ms	309.0 ms
First read of a 2K-token state	1610.8 ms	—	—
Question on a 2K state already read	47.8 ms	—	—

Answer parity is measured separately from the 2,710-question comparison above. In a separate check against llama.cpp on the same 4-bit GGUF (19 requests, 18 compared because the fork rejects one batch request), Kapitsa picks the same top answer on all 71 questions, with probabilities within 0.003 on average (max 0.033). The 4-bit build picks the same top answer as full precision on 96.97% of questions (2,628/2,710), with 80.42% accuracy on a 2,710-question Decision Index 0.3.1 subset against 80.51% at full precision (the full public Decision Index 0.3.1 score of Drex v2.0 is 61.25, see the model page).

Hardware

Needs a Mac with 36 GB or more (tested on 36 GB and 48 GB). The 21.2 GB download is the GGUF and the Kapitsa pack is 24.1 GB, so keep room for both; the weights sit on Metal with -ngl 99; -ngl 0 runs entirely on the CPU.

Download

hf is the Hugging Face CLI (pip install -U huggingface_hub).

hf download nace-ai/drex-v2.0-Q4_K_M drex-v2.0-Q4_K_M.gguf --local-dir weights/drex-v2.0-Q4_K_M

Run with llama.cpp

Build the drex-v2.0 branch of the llama.cpp fork as described on the model page and the Apple silicon page, then:

SYSTEMONE_CONTEXT=131072 llama.cpp/build/bin/llama-server \
  -m weights/drex-v2.0-Q4_K_M/drex-v2.0-Q4_K_M.gguf \
  --embedding --pooling none -np 2 -c 262144 -b 8192 -ub 8192 --no-repack -ngl 99 --port 8000

The server answers POST /v1/systemone; the request format is in the repository docs.

The tables above were measured with -b 4096 -ub 1024; this command raises the batch to 8192 because -b 4096 -ub 1024 rejects any request whose batch exceeds 1,024 tokens (81 of the 1,000 quality requests), while all 1,000 ran at this command. -c 262144 is the headroom SYSTEMONE_CONTEXT=131072 needs for a 131,072-token state. -np 2 shares the state across two questions.

Run with Kapitsa

Kapitsa v0.2.25 or later serves this file on Apple silicon. The commands use the GGUF from Download.

brew install nace-ai/tap/kapitsa
kapitsa import weights/drex-v2.0-Q4_K_M/drex-v2.0-Q4_K_M.gguf --name drex-v2.0-q4
kapitsa-drex-pointer weights/drex-v2.0-Q4_K_M/drex-v2.0-Q4_K_M.gguf \
  ~/.cache/kapitsa/models/drex-v2.0-q4/pointer.bin
kapitsa-systemone --model ~/.cache/kapitsa/models/drex-v2.0-q4 --context 65536 --cache-mb 2048

From a clone of the Drex repository, in another terminal:

curl --fail-with-body http://127.0.0.1:8000/v1/systemone \
  -H 'Content-Type: application/json' \
  --data-binary @examples/request.json

The server listens on 127.0.0.1:8000.

Run with Ollama

Use the Nace fork of Ollama as described on the model page and the Apple silicon page, with this Modelfile next to the GGUF:

FROM ./drex-v2.0-Q4_K_M.gguf
CAPABILITY decision
PARAMETER num_ctx 131072
PARAMETER num_gpu 99
PARAMETER parallel 2

Start the daemon with SYSTEMONE_CONTEXT=131072 set (without it any request over 16,384 tokens is rejected, whatever num_ctx says) and OLLAMA_LLAMA_SERVER pointing at the fork's llama-server-darwin, then ollama create drex-v2.0 -f Modelfile. num_gpu 99 keeps the 4-bit build on Metal on a 48 GB Mac; use PARAMETER num_gpu 0 for the CPU. The request needs "model": "drex-v2.0" added.

License

Nace.AI Open RAIL-M (modified) license, see LICENSE.