Pretrained checkpoints for OLMo 20M-150M models used in experiments for our paper "Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting"
Ishaan Watts
WattsIshaan
AI & ML interests
Machine Learning, NLP
Recent Activity
updated a collection 3 days ago
OLMo Pretrained 20M-150M updated a collection 3 days ago
OLMo Pretrained 20M-150M updated a collection 3 days ago
OLMo Pretrained 20M-150MOrganizations
None yet
models 11
WattsIshaan/OLMo-150m-120B-sam-cosine
Updated • 3
WattsIshaan/OLMo-150m-120B-adamw-cosine
Updated • 3
WattsIshaan/OLMo-60m-192B-adamw-cosine
Updated • 3
WattsIshaan/OLMo-60m-192B-adamw-wsd-10percent
Updated • 1
WattsIshaan/OLMo-60m-192B-sam-cosine
Updated • 3
WattsIshaan/OLMo-20m-64B-adamw-cosine
Updated • 3
WattsIshaan/OLMo-20m-64B-sam-cosine
Updated • 3
WattsIshaan/OLMo-2-1B-Midtrain-50B-AdamW
Updated • 7
WattsIshaan/OLMo-2-1B-Midtrain-50B-SAM-rho1e-1
Updated • 5
WattsIshaan/OLMo-2-1B-Midtrain-50B-SAM-rho5e-2
Updated • 6
datasets 0
None public yet