🔄 In a Training Loop

i64 Systems PRO

i64systems

2 4

·

https://bob-talk.org/

i64systems

AI & ML interests

deterministic, auditable local AI · integer/low-bit ML · memory systems

Recent Activity

repliedto Hoglet-33's post about 1 hour ago

Pebble 10M and Pebble 10M Chat are now released! Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens. Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities. You can find them here: - https://huggingface.co/basically-ai/Pebble-10M - https://huggingface.co/basically-ai/Pebble-10M-Chat We hope you enjoy using them. The rest of the Pebble family will be released soon. Follow for more: @Hoglet-33 https://huggingface.co/basically-ai

reacted to CodeSoft's post with 🔥 about 2 hours ago

Sorbet Mini Experimental is out! It's not the best model in the 5M parameter range, but it's going to be a really useful model to train on top of. This release was mostly to prove that the model actually works. I trained it on 150M tokens from TinyStories in about 12 minutes. Does anyone have any tips on training models in this size range? I want to make the full Sorbet Mini release as good as it can be.

posted an update about 2 hours ago

remat is no longer a one-model claim!!proved it on Qwen3-30B-A3B, K=32 of 128 experts resident, output task byte-identical to the full reference, zero bytes different *in bf16*🥰🥰 GPU comes next😈

View all activity

Organizations

None yet

i64systems 's collections 1