arxiv:2609.37501
Dipankar Sarkar PRO
dipankarsarkar
AI & ML interests
Building the AI-native stack. Agents as infrastructure, safety as architecture, performance as plumbing. I publish the receipts: papers, datasets, demos.
Recent Activity
liked a model 32 minutes ago
precision-numerics/Boole-1.7B reacted to Zaptrapper's post with 🔥 33 minutes ago
I’ve published an article about Horizon RunMap, an MoE inference architecture I’m developing independently to explore more memory-efficient execution.
In my RTX 5070 benchmark, OLMoE-1B-7B-0924-Instruct reached 61.26 tokens/s during decode after the first token, with 5.99 GiB peak VRAM usage, keeping all compact experts on the GPU.
The architecture combines compact expert representations for both storage and execution, custom GPU kernels, and full or partial GPU residency.
The article explains how these pieces work together, with results across three models and an instruction-following evaluation using IFEval.
If you’re working on MoE inference or GPU memory efficiency, I’d welcome your thoughts.
Read the article: https://huggingface.co/blog/Zaptrapper/horizon-runmap-moe-inferenceOrganizations
arxiv:2609.33312
arxiv:2609.30074
arxiv:2608.14329
arxiv:2608.14771
arxiv:2607.02541
arxiv:2606.20128
arxiv:2606.27396
arxiv:2606.19616
arxiv:2402.04881
arxiv:2402.05318
arxiv:2401.00503
arxiv:2312.12654
arxiv:2311.18545
arxiv:2311.08234
arxiv:2311.00422
arxiv:2311.00551
arxiv:2106.08569
arxiv:2011.07229
arxiv:2011.06283
