Hugging Face's logo
s3nh's picture
Open to Collab
88 17 284

s3nh PRO

s3nh
aifeifei799's profile picture zhangtao103239's profile picture hhy150's profile picture
·

AI & ML interests

Quantization, LLMs, Deep Learning for good. Follow me if you like my work. Patreon.com/s3nh

Recent Activity

reacted to fblgit's post with 👍 about 14 hours ago
Introducing `HarEmb - PII` a single-transformer-block distilled layer from OpenMed PII Privacy filter. Its a very tiny model that reaches comparable results at PII classification thru viterbi BIOES decoding, harnessing 98%~ the original model performance while being a tiny fraction of the base model. It doubles the performance tk/s, reduces the active params dramatically and the VRAM footprint. The evaluation & benchmarking is within the model repository and can be reproduced. I trained it with an RTX4090 without issues and it is compatible with OpenMed suite and a in-place replacement for openai privacy-filter model. https://huggingface.co/fblgit/haremb-privacy-filter-opennemo I'm looking for people who wants to co-author/contribute/endorse HarEmb research and the technical paper for the model. Contact xavi@juanako.ai
reacted to lbourdois's post with 🤗 about 14 hours ago
New blog post! An introduction to a little-known but highly effective model reduction method: 𝗧𝗿𝗶𝗺𝗺𝗶𝗻𝗴✂️ We show how to reduce model size (we went up to 87.24% reduction) while preserving its performance. We applied this technique to 16 different model families across several modalities to illustrate that it works on any architecture (as long as the embedding layer is the last one of the model) and on any modality involving text. From these 16 families, we generated over 𝟱,𝟱𝟬𝟬 𝗺𝗼𝗻𝗼𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 𝗶𝗻 𝟭𝟮𝟰 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀 🌍 Key takeaways from our experiments: 1️⃣ Trimming does not require a GPU. Our models were obtained on a CPU. 2️⃣ This method scales up to at least 4B parameters (we did not test beyond that). 3️⃣ Trimmed model is smaller than the original while preserving its performance. If you observe a slight performance drop, just fine-tuned to recover or even surpass the original performance. 4️⃣ For an equivalent compute budget, it is better to trim then fine-tune rather than fine-tuning the original model. Since the model is smaller, you can run more epochs/show more data and get in fine a better model than the original. 5️⃣ Trimming is a competitive alternative to distillation and quantization. E.g. we obtained our alternative to DistilBERT in 9 minutes on CPU vs. 90 hours of GPU for the latter. 6️⃣ Trimming could generate reasoning traces in the language of the trimmed model. This could be an alternative to generating traces in English and then translating them into the desired language. And many other things (such as how much data are needed, the impact of the database used, the order in which it should be done, etc.) are available in the blogpost! Blogpost: https://huggingface.co/blog/lbourdois/introduction-to-trimming Models: https://huggingface.co/spaces/alphaedge-ai/Trimming_models_search
reacted to appvoid's post with 👀 2 days ago
yikes! i missed the small model hackathon i guess i'll have to make sota for people to notice
View all activity

Organizations

ESPnet's profile picture Gradio-Blocks-Party's profile picture Lajonbot's profile picture The Waifu Research Department's profile picture AblateIt's profile picture Blog-explorers's profile picture BangumiBase's profile picture CyberHarem's profile picture HydraLM's profile picture GOAT.AI's profile picture That Time I got Reincarnated as a Hugging Face Organization's profile picture ZeroGPU Explorers's profile picture Social Post Explorers's profile picture Spinner-GPT-4's profile picture Hugging Face Discord Community's profile picture open/ acc's profile picture Smol Community's profile picture Blue Collard Hard Working Class's profile picture Hugging Science's profile picture