Amazing !

#1
by Elsephire - opened

Very nice work! I started to work for compress in GGUF and make this work in my llama.cpp fork!

Thank you for this!

Small update: I've implemented GGUF conversion and model loading, but the model output is only ////.

I'm working on it , I still have a performance issue to resolve.

I'll be publishing the GGUF file and a link to the source code on GitHub of my llama.cpp fork.

Your work is very important for the accessibility of local LLMs. Don't let it drop , it's a great improvement!

TelperionAI org

Is the repo public? I'd love to take a look and see if I can help

Sign up or log in to comment