Article
A 165M model that out-remembers SmolLM2-360M: notes, not scale
fausap
• Fausto Saporito PRO
fausap
AI & ML interests
LLM esoteric architectures blended with cutting-edge mathematical frameworks
Recent Activity
liked a model about 2 hours ago
DavidAU/Qwen3.8-27B-Jumbo-Fusion-IQ-747-900-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF reacted to DedeProGames's post with 🚀 about 2 hours ago
🚀 Introducing Kiyo-230M-Instruct — my compact English & Portuguese chat model!
Built on Kiyo-230M-Preview, a base model I pretrained from scratch, Kiyo now has an instruction-tuned release for everyday conversations, rewriting, and lightweight assistant experiments.
🧠 229.7M parameters
🌍 English + Portuguese
💬 Native chat template for multi-turn conversations
📏 2,048-token context window
📦 459 MB of BF16 safetensors weights
🔓 Apache-2.0 license
For instruction tuning, I used 165,735 conversations from SmolTalk2, smol-smoltalk, UltraChat, and Dolci-Instruct-SFT. Training ran for two complete epochs with assistant-only loss, using a roughly 70% English / 30% Portuguese token mix.
The updated model card includes benchmarks against SmolLM2-135M-Instruct, BananaMind, and Supra2, plus example conversations, CPU measurements, training details, and a Transformers quickstart.
The results are mixed: Kiyo is competitive with these small models across the reported evaluations, while SmolLM2 and BananaMind score higher on the instruction-following benchmarks. Portuguese was a highlight in the small conversational sanity test.
This is an experimental 230M model, with plenty of room to improve reasoning, factual accuracy, and strict instruction following. I’m excited to share it with people exploring how much we can get from small, accessible models!
🤗 Model & full README:
https://huggingface.co/DedeProGames/Kiyo-230M-Instruct
💬 Try the chat demo:
https://huggingface.co/spaces/DedeProGames/kiyo-230m-chat
Try it in English or Portuguese — I’d love to see your examples, feedback, and experiments! reacted to DedeProGames's post with 🔥 about 2 hours ago
🚀 Introducing Kiyo-230M-Instruct — my compact English & Portuguese chat model!
Built on Kiyo-230M-Preview, a base model I pretrained from scratch, Kiyo now has an instruction-tuned release for everyday conversations, rewriting, and lightweight assistant experiments.
🧠 229.7M parameters
🌍 English + Portuguese
💬 Native chat template for multi-turn conversations
📏 2,048-token context window
📦 459 MB of BF16 safetensors weights
🔓 Apache-2.0 license
For instruction tuning, I used 165,735 conversations from SmolTalk2, smol-smoltalk, UltraChat, and Dolci-Instruct-SFT. Training ran for two complete epochs with assistant-only loss, using a roughly 70% English / 30% Portuguese token mix.
The updated model card includes benchmarks against SmolLM2-135M-Instruct, BananaMind, and Supra2, plus example conversations, CPU measurements, training details, and a Transformers quickstart.
The results are mixed: Kiyo is competitive with these small models across the reported evaluations, while SmolLM2 and BananaMind score higher on the instruction-following benchmarks. Portuguese was a highlight in the small conversational sanity test.
This is an experimental 230M model, with plenty of room to improve reasoning, factual accuracy, and strict instruction following. I’m excited to share it with people exploring how much we can get from small, accessible models!
🤗 Model & full README:
https://huggingface.co/DedeProGames/Kiyo-230M-Instruct
💬 Try the chat demo:
https://huggingface.co/spaces/DedeProGames/kiyo-230m-chat
Try it in English or Portuguese — I’d love to see your examples, feedback, and experiments!Organizations
None yet
published an article 3 days ago