___   _  ____             _
    |_  | |_| |  _ \ __ _  ___| | __
      | | | | | |_) / _` |/ __| |/ /
  /\__/ / | | |  _ < (_| | (__|   <
  \____/  |_| |_| \_\__,_|\___|_|\_\ 

  Java Intelligent Rack System with PyTorch models with ML script and chatbots
  JiRack LLM by CMS Manhattan with Models open source code

  Enjoy JiRack Micro-Agent Deployment architecture and JiRack Ternary Transformers architecture for models.
 
  Web services with OnnxRuntime Launch Inference images with Open AI and Ollama REST API support in docker hub.
  OnnxRuntime supports running models on many GPU cards and data centers
  Docker repo : https://hub.docker.com/u/cmsmanhattan

## PARTNERSHIP

- NVIDIA
- FISERV
- LAMBDA CLOUD

🔥 CMSManhattan - Frontier Ternary Neural Networks

Ternary (1.58-bit) language models for cheap, reliable tool calling, agents and robotics

🎯 Mission

Democratizing access to large language models through extreme efficiency. Training state-of-the-art LLMs with 1.58-bit (ternary) weights that run on accessible hardware.

We developed the JiRack Micro-Agent Deployment architecture to reduce inference cost for agentic systems.

---

Tool calling

• Internal test: single calls, parallel calls, answers from tool results, questions that need no tool, and asking for a missing argument (117 cases, greedy decoding).
• BFCL v3 run with evalscope in function-calling mode; totals are weighted by the number of cases, so they are not directly comparable with the official leaderboard averages.
• The ternary 8B is within 1.6 points of the ternary 16B on live requests at half the size.

Compression without losing quality

JiRack Ternary Precision 16B
  bf16 safetensors:      31.8 GB
  GGUF TQ2_0:             4.8 GB   (6.6x smaller, runs on CPU with llama.cpp)
  tool test:   95.7% -> 94.9%      (one case of 117)

JiRack Ternary Precision 8B
  bf16 safetensors:      16.2 GB
  GGUF TQ2_0:             2.6 GB   (6.3x smaller)
  tool test:   94.9% -> 95.7%      (one case of 117, in the other direction)

JiRack Precision 2B
  bf16 safetensors:       4.0 GB
  GGUF TQ2_0:            0.73 GB   (5.5x smaller)
  tool test:   93.2% -> 92.3%      (one case of 117)

Our GGUF export rounds every weight to exactly the ternary values the model was trained with (per-layer scale) before packing, so the packed file computes what the trained model computes.

What the models are trained on

• Ternary pretraining in phases: bf16 warm-up on C4 → gradual ternarization (lambda 0 → 1) → C4 + Wikipedia; the 36B adds a code phase (Python, C++, Java, JavaScript, C, TypeScript, Go; permissive licenses only).
• Fine-tuning: chat and reasoning, tool calling, real coding-agent trajectories (SWE-rebench, CC BY 4.0) and Docker-sandbox trajectories verified by hidden tests, robotics software in Python / C++ / Java, and "do not call / ask first" data so the model does not call tools when none fits.

---

📊 Benchmarks

All numbers are measured by us with greedy decoding (temperature 0); none are copied from base-model reports.

BFCL v3 by category — ternary models

evalscope 1.14.0 / bfcl-eval 2026.3.23, function-calling mode. Totals are weighted by the number of cases, as evalscope reports them; multi-turn categories were not run.

On standard calls the 8B matches the 16B; the 16B is ahead on real user requests and Java / JavaScript, while the 8B declines irrelevant requests more often.

Internal tool test by category (117 cases)

GGUF files differ from the reference models by one case in either direction.

Full precision vs ternary (16B, same fine-tuning recipe)

Ternary weights keep most of the tool-calling skill; the gap is largest on BFCL live (real user requests).

CPU speed (llama.cpp, GGUF TQ2_0, 2x Intel Xeon E5-2697 v2 from 2013, 48 threads, no GPU)

2B and 8B measured with llama-bench (512-token prompt, 128 generated tokens); the 16B figure is from an earlier run on the same machine. A worst case: this CPU lacks AVX2, which the TQ2_0 kernels use for speed. Modern CPUs and GPUs are next on the list.

Not measured yet

• GSM8K, HumanEval and MMLU for the 8B–36B models (scripts are ready, runs planned after fine-tuning).
• Precision 2B: MMLU 30.8% (57 subjects, cloze 0-shot, acc_norm) — a tool-calling specialist, not a knowledge base.

---

Solved issues

Benefits of JiRack Micro-Agent Architecture:

• Solves catastrophic forgetting during training by using small, specialized models for each domain, managed by a smart router
• Enables extremely cheap inference using ternary models
• Significantly reduces cloud inference costs while maintaining high performance
• In classical architecture, an expensive model has to search for MCP-agents every time, while JiRack uses a very small model and cheap router for agent tasks, saving big money right from the start
• Considered one of the best approaches for enterprise AI deployments

---

ROBOS HUMANOID Project

• https://github.com/cmsmanhattan/ROBOS

---

JiRack Coding Agent IDE

• It is an Agent Coding IDE for JiRack models that runs via Ollama on a home PC
• A good choice next to agent coding IDEs such as Cursor, Windsurf or Devin, but safer: it asks you to review and apply changes.
• Web site https://www.jirack.com
• Eclipse plugin site https://marketplace.eclipse.org/content/jirack-coding-agent
• JetBrains plugin site https://plugins.jetbrains.com/plugin/35073-jirack-agent
• VS Code plugin site https://marketplace.visualstudio.com/items?itemName=JiRackautomationCodingAgentIDE.jirack-agent

JiRack Coding Agent IDE files

• Eclipse Enterprise release version https://huggingface.co/spaces/CMSManhattan/README/resolve/main/Jirack_Enterprise_ide.zip
• Eclipse final release version https://huggingface.co/CMSManhattan/JiRackDeltaNet_27b/resolve/main/jirack_ide_final.zip
• Microsoft VS Code plugin https://huggingface.co/spaces/CMSManhattan/README/blob/main/jirack-agent-0.9.0.vsix
• JetBrains plugin https://huggingface.co/spaces/CMSManhattan/README/resolve/main/jirack-jetbrains-0.11.0.zip
• JiRack Desktop UI https://huggingface.co/spaces/CMSManhattan/README/resolve/main/windows_v11.zip

---

🏆 JiRackTernary Series

A family of efficient large language models based on BitNet architecture with ternary weights {-1, 0, 1}.

JiRack Ultra agentic & robotics model line for JiRack Micro-Agent Deployment architecture.

🌐 Public Models

🔒 Private Models (API / on request)

---

⚡ Key Innovations

Up to 7x Compression

Traditional LLaMA-3 70B:  ~140 GB (FP16)
JiRackTernary 70B:        ~25 GB  (1.58-bit)
Compression ratio:        7x smaller! 🔥

Measured on JiRack Ternary Precision 16B:
bf16 31.8 GB -> GGUF TQ2_0 4.8 GB (6.6x), tool-calling accuracy 95.7% -> 94.9%
Measured on JiRack Ternary Precision 8B:
bf16 16.2 GB -> GGUF TQ2_0 2.6 GB (6.3x), tool-calling accuracy 94.9% -> 95.7%

Accessible Training

• Ternary quantization-aware training with a smooth lambda ramp: the model starts from strong full-precision weights and learns to work with ternary weights step by step
• FSDP training on rented cloud GPUs (Lambda Cloud: 8x A100, H100, GH200) — no supercomputer clusters required
• Earlier experiments: layer-by-layer training of 70B on a single A100 80GB

Production-Ready Architecture

• Qwen2 / LLaMA-style decoders with BitLinear layers (ternary weights, 8-bit activations)
• GGUF export for llama.cpp (TQ2_0 / TQ1_0), CPU inference without a GPU
• OpenAI-compatible API with tool calling
• Optimized for inference speed and cost

---

🔬 Technical Highlights

Architecture Details

- Precision: 1.58-bit ternary weights {-1, 0, 1}, per-layer absmean scale, int8 activations
- Layers: BitLinear with straight-through estimator during training
- Normalization: RMSNorm

Training pipeline (JiRack Precision)

Pretraining
├── Phase 0: bf16 warm-up on C4
├── Phase 1: ternarization, lambda 0 -> 1, then LR anneal
├── Phase 2: ternary, C4 + Wikipedia
└── Phase 3: ternary, code (7 languages, permissive licenses)

Fine-tuning
├── Main SFT: chat, reasoning, tool calling
├── Agentic: SWE-rebench trajectories, Docker-sandbox trajectories verified by tests
└── Robotics: robotics software (Python / C++ / Java), BFCL-style tool data, no-call data

---

🎓 Research & Publications

Upcoming

📝 Technical Report (In Preparation)

• Ternary quantization-aware training at 8B–36B: what it costs and how to recover it
• Tool calling and agents with ternary models

📊 Benchmark Suite (see Benchmarks above for what is already measured)

• GSM8K, HumanEval, MMLU for 8B–36B
• BFCL multi-turn categories
• Inference speed on AVX2 CPUs and GPUs

🎤 Open Source Release

• Training code & documentation
• Reproducibility guidelines

---

🚀 Why This Matters

For Researchers

✅ Train large ternary models on rented cloud GPUs
✅ Reproducible recipe from pretraining to tool use
✅ New compression research directions

For Industry

✅ Tool-calling agents on CPU or a single small GPU
✅ Faster inference with ternary operations
✅ Lower hosting costs (6–7x smaller)

For Community

✅ Democratization of large language models
✅ Accessible AI for everyone
✅ Open research methodology

---

🗺️ Roadmap

• JiRack Ternary Precision 36B: finish pretraining, fine-tune for tools, agents and robotics
• JiRack Precision 72B and DeltaNet 70B fine-tuned for robotics
• Scaling the ternary recipe toward 405B parameters

---

🤝 Community

Get Involved

• ⭐ Star our models on HuggingFace
• 💬 Join discussions in model repos
• 🐛 Report issues or suggestions
• Email: grabko@cmsmanhattan.com
• Phone: +1 (516) 777-0945
• Location: New York, USA

Citation

@misc{jiracternary2026,
  author = {CMSManhattan (kgrabko)},
  title = {JiRack Precision: Ternary Language Models for Tool Calling, Agents and Robotics},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/CMSManhattan}
}

---

Making AI accessible, one ternary weight at a time. ✨

---

Last updated: 2026-10-26

• PHONE 516-777-0945

• Demo JiRack LLM and CMS Manhattan RAG System

• https://www.youtube.com/watch?v=M4Q8_Dr35Cc

• Download RAG System

• git clone https://grabko1@bitbucket.org/cmsmanhattan/rag.git

• https://bitbucket.org/cmsmanhattan/rag

• Deployment kit on Docker or Kubernetes with API Gateway and Service Discovery by Microservice Architecture https://www.youtube.com/watch?v=M4Q8_Dr35Cc

• Deployment script for RAG System: https://bitbucket.org/cmsmanhattan/rag

Trademark information : https://uspto.report/TM/90579072

• SERIAL NUMBER 90579072