Instructions to use empero-ai/Quark-150M-128K-Zero with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use empero-ai/Quark-150M-128K-Zero with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="empero-ai/Quark-150M-128K-Zero", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("empero-ai/Quark-150M-128K-Zero", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use empero-ai/Quark-150M-128K-Zero with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "empero-ai/Quark-150M-128K-Zero" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "empero-ai/Quark-150M-128K-Zero", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/empero-ai/Quark-150M-128K-Zero
- SGLang
How to use empero-ai/Quark-150M-128K-Zero with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "empero-ai/Quark-150M-128K-Zero" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "empero-ai/Quark-150M-128K-Zero", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "empero-ai/Quark-150M-128K-Zero" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "empero-ai/Quark-150M-128K-Zero", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use empero-ai/Quark-150M-128K-Zero with Docker Model Runner:
docker model run hf.co/empero-ai/Quark-150M-128K-Zero
Quark-150M-128K-Zero
Quark-150M-128K-Base trained with reinforcement learning (GRPO) directly, with no supervised fine-tuning stage: a small chat model that can think in a private monologue and call tools.
Research artifact. This model is released for research on small language models: architecture, training dynamics, long context and reinforcement learning from a base model. It is not a product and has not been evaluated for production use. Its only safety training is a small set of refusal tasks during RL, and its safety has not been evaluated. Like any small language model, it makes factual and reasoning errors and can produce biased or harmful text. Do not use it for decisions that affect people, or deploy it without your own evaluation and safeguards.
| Parameters | 165.3M (140.0M without embeddings) |
| Layers | 16 (12 local, 4 global) |
| Context | 131,072 tokens |
| Training tokens | ~506B pretraining, then RL |
| Vocabulary | 32,772 (embedding table padded to 32,896) |
| Weights | float32 safetensors |
What this checkpoint is
This is the RL model. It was trained with GRPO starting from
Quark-150M-128K-Base, with no supervised fine-tuning stage ("zero", as in
RL-from-base). It answers in the chat format, can think first in a [MONOLOGUE] (it decides when, or you can ask it to
with enable_thinking=True), and can call tools.
The Quark-150M family
| Model | What it is |
|---|---|
| Quark-150M-32K-Raw | Pre-anneal checkpoint: end of the constant learning-rate phase (~472B tokens, trained at up to 4K context) |
| Quark-150M-32K-Base | Base model after the 32K-context anneal (~500B tokens) |
| Quark-150M-128K-Base | Base model extended to 128K context (+10B tokens) |
| Quark-150M-128K-Zero | Reinforcement learning (GRPO) straight from the 128K base, without supervised fine-tuning |
Architecture
- Decoder-only transformer hybrid, 16 layers, width 768, 12 attention heads (head size 64), tied input/output embeddings.
- Local layers (12): sliding-window attention over the last 256 tokens, with RoPE, QK-norm and a learned attention sink per head. A double-gated short causal convolution (kernel 4) runs in parallel, and the two are blended per channel.
- Global layers (4, every fourth layer): full causal grouped-query attention (12 query heads, 4 key/value heads) with RoPE and QK-norm.
- Gated value residuals: every attention after the first layer mixes its values with the first layer's values, through a per-token, per-head gate.
- Residual stream: depth-scaled residual updates with an RMSNorm before and after every sublayer (sandwich norm). The stream is kept in float32 even when the weights are in bfloat16.
- MLP: gated MLP with a squared-ReLU gate, hidden size 2048.
- Small inference state: only the 4 global layers keep a full key/value cache. Local layers keep 255 tokens of keys/values and a 3-token convolution state.
- Tokenizer: byte-level BPE with 32,768 tokens; every chat tag is a single token.
Training
- Algorithm: GRPO with group-relative advantages and a token-level loss, on-policy samples at temperature 1, no KL penalty and no reference model. Under 1,300 steps on 8ΓH100, stopped early for budget while its scores were still improving; the learning rate was annealed over the final 3%.
- Rewards: about 95% of reward groups were scored by programmatic checks: exact answers, format and instruction rules, unit tests, and file checks inside sandboxes. The rest were scored by LLM judges: chat quality, creative writing, second opinions on safety refusals, and rendering of SVG/HTML answers.
- Tasks: arithmetic and math word problems, commonsense, world-state tracking, theory of mind, perception, format and instruction following, system prompts, long-context retrieval, identity, values and safety, chat, creative writing, code, and tool use. Tool use covered simulated tools, live web search, and shell/file tasks in real sandboxes, written both in Quark's own tool format and in the formats of common coding-agent harnesses.
- No supervised fine-tuning: every token the model was trained on was its own sample, apart from short code-written hints (e.g. given solution steps on hard reasoning tasks, or the opening of a tool call).
Results after RL
These come from training rollouts and our in-house task suite, not from held-out benchmarks.
- Simple terminal tasks in 15 coding-agent harness formats (real commands in real sandboxes, training rollouts): 2.6% solved at the start of harness training, 64.5% over the last 100 steps of the run. Harness formats added late in the run started at 42% to 54% solved.
- Task suite (15 families, four samples per task at temperature 1, step 50 to the final step): well-formed replies 71% to 96%, replies cut off at the length limit 11.7% to 0.5%, correct answers 12.8% to 36.4%.
Usage
Requires transformers>=5.23 (tested with 5.23.0) and trust_remote_code=True (the modeling code ships
with the weights).
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "empero-ai/Quark-150M-128K-Zero"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda")
messages = [{"role": "user", "content": "Explain in two sentences why the sky is blue."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt",
return_dict=True).to("cuda")
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
- Thinking:
apply_chat_template(..., enable_thinking=True)opens a[MONOLOGUE]and the model thinks before it answers. Withoutadd_generation_prompt, the model chooses for itself whether to think, call a tool or answer. - Stopping: generation stops at
[/ASSISTANT],[/TOOL_CALL]or<|endoftext|>(set ingeneration_config.json). - Tools: pass
tools=[...], either Hugging Face/OpenAI-style function specs or{"name", "description", "parameters": {arg: {"type": ...}}}. When the model stops at[/TOOL_CALL], run the call, append{"role": "tool_result", "content": result}, and generate again. - Sampling: the model was trained on its own samples at temperature 1.0. Greedy decoding and lower temperatures also work.
Chat format
Turns are wrapped in tags, and each tag is a single special token:
<|endoftext|>[SYSTEM]
You are quark, a helpful assistant made by Empero AI
[/SYSTEM]
[USER]
What is 2 + 3?
[/USER]
[MONOLOGUE]
Two plus three is five.
[/MONOLOGUE]
[ASSISTANT]
5
[/ASSISTANT]
[SYSTEM] and [MONOLOGUE] (the model's private thinking) are optional. <|endoftext|> marks the start of a
document; the tokenizer and the chat template add it.
The 128K models also have tool-call tokens. Tools are offered in the system turn, and the template writes
them for you from tools=[...]. The model calls a tool with
[TOOL_CALL] {"name": ..., "arguments": {...}} [/TOOL_CALL]. The caller runs the tool and appends the result as a
[TOOL_RESULT] ... [/TOOL_RESULT] turn (role tool_result, or tool), then the model continues.
Sample completions
Generated with the code above in float32, one run per prompt with the settings shown: nothing was re-rolled or selected. Generation stops at <|endoftext|>, [/ASSISTANT] or [/TOOL_CALL], or at the token limit. Special tokens are shown as text; the prompt is everything up to where the model's output begins.
Chat (greedy, up to 200 new tokens)
Prompt:
[USER]
Hi! Who are you?
[/USER]
[ASSISTANT]
Output:
I am quark, and I am here to chat with you.
[/ASSISTANT]
Chat (greedy, up to 200 new tokens)
Prompt:
[USER]
Explain in two sentences why the sky is blue.
[/USER]
[ASSISTANT]
Output:
The sky looks blue because sunlight is made of all colors and when it hits the air, the blue light scatters in all directions more than the others.
[/ASSISTANT]
Chat (greedy, up to 200 new tokens)
Prompt:
[USER]
Write a haiku about autumn.
[/USER]
[ASSISTANT]
Output:
The leaves turn gold and red, falling from the trees like a gentle rain.
[/ASSISTANT]
Chat with thinking (enable_thinking=True) (greedy, up to 200 new tokens)
Prompt:
[USER]
A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?
[/USER]
[MONOLOGUE]
Output:
Say the ball costs $1.10 more than the bat.
So the bat costs $1.10 + $1.00 = $2.10.
Answer: $2.10.
[/MONOLOGUE]
[ASSISTANT]
$2.10
[/ASSISTANT]
Chat with a web-search tool offered (the model chooses how to continue) (greedy, up to 200 new tokens)
Prompt:
[SYSTEM]
You are quark, a helpful assistant made by Empero AI
[TOOLS]
You have the following tools available:
[
{"name": "web_search", "description": "Search the web.", "parameters": {"query": {"type": "string"}}}
]
To use a tool, write a tool call in this format:
[TOOL_CALL]
{"name": <tool name>, "arguments": {<argument>: <value>, ...}}
[/TOOL_CALL]
Example:
[TOOL_CALL]
{"name": "web_search", "arguments": {"query": "<query>"}}
[/TOOL_CALL]
The result comes back in a [TOOL_RESULT] block. Then call another tool, or answer in [ASSISTANT]. You may think in [MONOLOGUE] before or after a call.
Only call the tools listed above, with exactly their arguments. Never write a [TOOL_RESULT] yourself. If no tool is needed, answer directly.
[/TOOLS]
[/SYSTEM]
[USER]
Who won the most recent Nobel Prize in Literature?
[/USER]
Output:
[TOOL_CALL]
The answer is Literature, Nobel laureate Elena Ferrante.
[/ASSISTANT]
Limitations and intended use
- A ~165M-parameter model: little world knowledge, frequent factual and reasoning errors, and confident hallucinations.
- Trained mainly on English. Other languages appear in the pretraining data but are weaker.
- Little safety training (a small set of refusal tasks during RL) and no safety evaluation. Outputs can be wrong, biased or inappropriate.
- Released as-is for research. Expect rough edges, and evaluate it on your own task before relying on anything it says.
- Tool use is unreliable beyond simple tools: live web search did not work well during training, and the model sometimes writes an answer inside a tool call (see the samples).
- Long-context accuracy fell during RL (from 35% to 25% on our long-context tasks). For long inputs, compare it with Quark-150M-128K-Base on your task.
- Thinking is not always better: on some tasks (for example, questions about what other people know or believe) forcing a monologue lowers accuracy.
License
The weights and code are released under the Apache License 2.0.
Compute for this work was partly sponsored by lium.io.
- Downloads last month
- 422