README.md

---
license: other
license_name: tencent-hunyuan-community
library_name: mlx
base_model: tencent/Hy3
base_model_relation: quantized
pipeline_tag: text-generation
tags:
  - hunyuan
  - hy3
  - mixture-of-experts
  - mlx
  - apple-silicon
  - reasoning
  - tool-use
  - quantized
  - jang
  - osaurus
quantization_config:
  family: jang-affine-mixed
  profile: JANG_2K
  group_size: 128
  routed_avg_bits: 2.33
---

<p align="center"><img src="osaurus-x-banner.png" width="100%" alt="OsaurusAI"/></p>

# Hy3-JANG_2K

Quantized **[tencent/Hy3](https://huggingface.co/tencent/Hy3)** for Apple Silicon MLX / JANG runtimes — a 295B-total / 21B-active text MoE, packed to ~94 GiB. This is the **clean non-MTP** JANG_2K bundle (smallest 2K pack). For the variant that keeps Hy3's native Multi-Token-Prediction head, see `Hy3-JANG_2K-MTP`.

|  |  |
|---|---|
| Source | [tencent/Hy3](https://huggingface.co/tencent/Hy3) |
| License | `other` — inherits the upstream Tencent Hunyuan Community License |
| Architecture | `hy_v3` (`HYV3ForCausalLM`), text-only |
| Parameters | 295B total / 21B active per token |
| Format | JANG_2K (mixed-affine), routed experts avg **2.33-bit** |
| Bundle size | 101.40 GB (94.44 GiB), 22 shards, 2,876 tensor keys |
| MTP | none (`num_nextn_predict_layers = 0`) — MTP head not included |
| Context | 262,144 tokens |

## What this is

`Hy3-JANG_2K` is a JANG mixed-affine quantization of Tencent's Hy3 dense-MoE, targeting Apple Silicon runtimes (MLX / vMLX). The `2K` profile spends an extra bit on the routed `down_proj` (3-bit vs the 2-bit `gate`/`up`), which cleans up the sampling tail relative to a uniform 2-bit pack. This bundle drops the native MTP layer for the smallest footprint; use `Hy3-JANG_2K-MTP` if you want speculative decoding.

## Quantization (JANG_2K)

| Tensor family | Policy |
|---|---|
| Routed expert `gate_proj` / `up_proj` | affine **2-bit**, group size 128 |
| Routed expert `down_proj` | affine **3-bit**, group size 128 |
| Attention `q/k/v/o` | affine 8-bit |
| Shared expert | affine 8-bit |
| Dense layer-0 MLP | affine 8-bit |
| `embed_tokens` | affine 6-bit |
| `lm_head` | affine 8-bit |
| RMSNorms, router gate, expert bias | 16-bit passthrough |

Routed-expert effective average: **2.33 bit**. AWQ scaling is disabled for this bundle (measured negligible on Hy3).

## Architecture

Hy3 is a **text-only** dense-causal-GQA MoE — not MLA, not SSM, not sliding-window, not a VLM.

- 80 decoder layers, `hidden_size` 4096
- GQA: 64 attention heads / 8 KV heads, `head_dim` 128, QK-norm
- RoPE `default`, `rope_theta` 11,158,840, `max_position_embeddings` 262,144
- MoE: 192 routed experts, top-8, **sigmoid** router + expert-correction bias, `route_norm`, `router_scaling_factor` 2.826, 1 shared expert, `first_k_dense_replace` 1
- No MTP layer in this bundle (`num_nextn_predict_layers = 0`)
- `vocab_size` 120,832

## Reasoning & tool use

- **Reasoning**: `<think>…</think>` tags, `reasoning_effort` (`no_think` / `low` / `high`).
- **Tool calling**: Hunyuan / Tencent XML-style tags (`<tool_calls>`, `<tool_call>`, `<arg_key>`, `<arg_value>`).
- Hy3's tokenizer uses a **`:opensource` special-token dialect** (e.g. `<｜hy_eos:opensource｜>`, `<think:opensource>`); the bundled `chat_template.jinja` is the upstream template. A compatible runtime must resolve these variant-suffixed tokens at the token→text boundary.

## Runtime support

- **Converted and structurally verified** (index complete, 2,876 tensors / 22 shards, no MTP tensors).
- Runs on the **vMLX Python engine** with Hy3 support: JANG affine loader, GQA KV cache, `<think>` reasoning stream, and Hunyuan tool-call parsing.

Requires a Hy3-aware MLX/JANG runtime. Stock `mlx-lm` / `transformers` will not load the JANG mixed-affine layout as-is.

## Known limitations

- No published quality benchmark yet for this specific pack.
- Very loose sampling (`top_p` 1.0 + `temperature` 0.9) exposes more of the routed-expert tail; a mild `top_p ≤ 0.9` or `min_p` floor is recommended for long-form generation.

## 소개 (Korean)

이 번들은 Tencent의 **Hy3** (295B 총 파라미터 / 21B 활성 MoE, 텍스트 전용)를 Apple Silicon MLX / JANG 런타임용으로 양자화한 모델입니다. JANG_2K 프로파일은 라우팅 전문가의 `down_proj`를 3-bit로, `gate`/`up`을 2-bit로 양자화합니다(평균 2.33-bit). 이 번들은 MTP 헤드를 포함하지 않는 가장 작은 2K 팩이며, 스펙티브 디코딩이 필요하면 `Hy3-JANG_2K-MTP`를 사용하세요. Hy3의 GQA 어텐션과 MoE 라우팅, `:opensource` 특수 토큰 방식을 정확히 구현한 런타임에서만 사용해야 합니다.