Article
Atom2.7m: Representation-Level Specialization for Arithmetic-Aware Small Language Models
ucr-max
• • 6Maksymilian PRO
ucr-max
AI & ML interests
None yet
Recent Activity
updated a Space about 16 hours ago
UniversalComputingResearch/README repliedto their post about 16 hours ago
Published under Universal Computing Research: Atom2.7m, a 2.74M-parameter causal LM experiment with arithmetic-aware numeric representation.
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: https://huggingface.co/UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: https://huggingface.co/spaces/UniversalComputingResearch/Atom2.7-Demo posted an update about 16 hours ago
Published under Universal Computing Research: Atom2.7m, a 2.74M-parameter causal LM experiment with arithmetic-aware numeric representation.
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: https://huggingface.co/UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: https://huggingface.co/spaces/UniversalComputingResearch/Atom2.7-DemoOrganizations
published an article 1 day ago