New update! We are currently training a few new models now! Our 3rd generation main LLM standard edition is in training right now. We are also training a new LLM line called Tiny Coder around 350~ish M params. Thanks to @Banaxi-Tech for inspiring the architecture with his Bananamind-2.1-unified test model. Thanks to our beta testers: @juiceb0xc0de@ProCreations@Sbui503@Fishtiks@MUK-IS-GOAT
We're announcing our BananaMind 2.1 model series! The models will include: - BananaMind 2.1 Nano: 10M parameters with 60B tokens. - BananaMind 2.1 Lite: 25M parameters with 40B tokens. - BananaMind 2.1 Flash: 50M parameters with 55B tokens. - BananaMind 2.1 Pro: 135M-145M parameters (still deciding) with 100B tokens. These model will use a multi tower architecture (like BananaMind/BananaMind-2.1-Unified) with some more architectural changes.
BananaMind 2.1 Pro will probrably use 2 no output towers, instead of one!
We're currently training some experimental models based on this architecture to see its scaling!