ECMWF's AI forecasting model is open source: now let's make it easy to run.
• 12
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
Banaxi-Tech
posted an update about 4 hours ago
Post
82
We're excited to release BananaMind 2 SLMoE, an experimental sequence-level mixture-of-experts model.
It uses only 8M parameters per message but has 25M total parameters, 13 experts (out of 64) are selected based on the message prefix and reused for the entire response.
We're testing with this sequence-level architecture to find out how big the capability loss actually is and how much of it can be fixed.
The long-term idea is that this could make very large sparse models usable on machines that can't fit them in RAM by putting the entire model (which is big) on disk and only loading the active parts into VRAM.
This architecture is still in research and shouldn't be used for production models.
We trained it on 60B tokens (of FineWeb-HQ, FineWeb-Edu, DCLM ,Cosmopedia v2, FineMath and NPSet-2) on 8 RTX Pro 6000s.
Check it out at BananaMind/BananaMind-2-SLMoE
Follow us for future models:
BananaMind
@vovaRL
@Banaxi-Tech
@DedeProGames
BananaMind 2 Pro in a few days. You've been waiting 22 days for it.
It uses only 8M parameters per message but has 25M total parameters, 13 experts (out of 64) are selected based on the message prefix and reused for the entire response.
We're testing with this sequence-level architecture to find out how big the capability loss actually is and how much of it can be fixed.
The long-term idea is that this could make very large sparse models usable on machines that can't fit them in RAM by putting the entire model (which is big) on disk and only loading the active parts into VRAM.
This architecture is still in research and shouldn't be used for production models.
We trained it on 60B tokens (of FineWeb-HQ, FineWeb-Edu, DCLM ,Cosmopedia v2, FineMath and NPSet-2) on 8 RTX Pro 6000s.
Check it out at BananaMind/BananaMind-2-SLMoE
Follow us for future models:
@vovaRL
@Banaxi-Tech
@DedeProGames
BananaMind 2 Pro in a few days. You've been waiting 22 days for it.
- 2 replies
Bc-AI
updated a
model about 15 hours ago
Bc-AI
published a
model about 16 hours ago
Banaxi-Tech
posted an update 1 day ago
Post
1875
We're excited to release BananaMind 2 Micro, our smallest model yet.
It fits a compact architecture in only 2.9M parameters achieving the highest parameter efficiency on BananaMind Base Bench against comparable models.
It achieves comparable performance to GPT S2 5M and GPT S 5M at almost half the size while beating CMA 1M Mini.
BananaMind 2 Micro achieved the #1 spot on the Open SLM Leaderboard for the sub 3M category (not added yet but it achieves #1)
For the training we used Muon + the XSA Refresh Gate with a 5e-2 lr for Muon and 4e-3 for the 1D weights.
Its score on our efficiency measure is 0.326 getting the first place with Syn 2.6M on the second place scoring 0.291 and GPT S 5M at 0.235*
Check it out at BananaMind/BananaMind-2-Micro and follow us at:
@vovaRL
@DedeProGames
@Banaxi-Tech
BananaMind
Our new releases aren't stopping 🚀 August 13-14 BananaMind 2 Pro
It fits a compact architecture in only 2.9M parameters achieving the highest parameter efficiency on BananaMind Base Bench against comparable models.
It achieves comparable performance to GPT S2 5M and GPT S 5M at almost half the size while beating CMA 1M Mini.
BananaMind 2 Micro achieved the #1 spot on the Open SLM Leaderboard for the sub 3M category (not added yet but it achieves #1)
For the training we used Muon + the XSA Refresh Gate with a 5e-2 lr for Muon and 4e-3 for the 1D weights.
Its score on our efficiency measure is 0.326 getting the first place with Syn 2.6M on the second place scoring 0.291 and GPT S 5M at 0.235*
Check it out at BananaMind/BananaMind-2-Micro and follow us at:
@vovaRL
@DedeProGames
@Banaxi-Tech
Our new releases aren't stopping 🚀 August 13-14 BananaMind 2 Pro
- 7 replies
EmmaScharfmann
updated a
dataset 1 day ago
Banaxi-Tech
posted an update 2 days ago
Post
2652
Today.
- 9 replies
Bc-AI
updated a
model 2 days ago
Bc-AI
published a
model 2 days ago
Bc-AI
updated a
Space 3 days ago
Bc-AI
updated a
model 3 days ago
Bc-AI
published 2
models 3 days ago
Banaxi-Tech
posted an update 3 days ago
Post
1920
We're exited to announce BananaMind OS, our OS specically for running BananaMind models!
Its able to run BananaMind 2 Nano at 4 bit on only 7-8MB of ram, the 2 bit on 6MB of ram and the 8 bit version on 14MB of RAM!
It runs on a 486 or newer!
Check out this video and image running BananaMind 2 Nano 4 Bit on 9
MB of RAM and a emulated 486 in QEMU at ~1TPS!
We asked it: "What is the first letter of the alphabet?"
The response is:
"The first letter of the alphabet is:
- A.
"
And if you're asking because of the video, yes I am a arch btw.
Comment and like this post for a GitHub link and comment for adding other models!
Its able to run BananaMind 2 Nano at 4 bit on only 7-8MB of ram, the 2 bit on 6MB of ram and the 8 bit version on 14MB of RAM!
It runs on a 486 or newer!
Check out this video and image running BananaMind 2 Nano 4 Bit on 9
MB of RAM and a emulated 486 in QEMU at ~1TPS!
We asked it: "What is the first letter of the alphabet?"
The response is:
"The first letter of the alphabet is:
- A.
"
And if you're asking because of the video, yes I am a arch btw.
Comment and like this post for a GitHub link and comment for adding other models!
- 16 replies
Bc-AI
updated 2
models 4 days ago
Bc-AI
published 2
models 4 days ago
Bc-AI
updated a
Space 4 days ago
Banaxi-Tech
posted an update 5 days ago
Post
1213
BananaMind 2 Pro got better!
Release on August 13-14
Follow our team for BananaMind 2 Ultra:
@vovaRL for training code
@Banaxi-Tech
@DedeProGames for providing compute
BananaMind
Want to become a previewer?
👇
BananaMind-Model-Previewers
Release on August 13-14
Follow our team for BananaMind 2 Ultra:
@vovaRL for training code
@Banaxi-Tech
@DedeProGames for providing compute
Want to become a previewer?
👇
- 3 replies
Bc-AI
updated a
Space 6 days ago