Pre-trained models (LNQ)
Collection
15 items • Updated
Model Card
- Base model:
meta-llama/Meta-Llama-3-8B - Quantization method: LNQ
- Target bit-width: 2
- Backend kernel: Any-Precision-LLM kernel (
ap-gemv) - Calibration data: RedPajama (1024 sentences / 4096 tokens)
- Calibration objective: Next-token prediction
How to run
- Follow the instruction in https://github.com/snu-mllab/GuidedQuant.
References
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for jusjinuk/Meta-Llama-3-8B-2bit-LNQ
Base model
meta-llama/Meta-Llama-3-8B