27 days ago
-
v2 loader: VOCAB 8192 -> 12288 (#8)
- v2 loader: VOCAB 8192 -> 12288 (34da9d02c8b0ebe3ca89c5120e42bfa66746a6be) -
v2 config: vocab 12288, 56,902,144 params (#7)
- v2 config: vocab 12288, 56,902,144 params (563c2233c1a53b21af0ba89ba75e9d8f60e3614f) -
v2 card: 12288-vocab retrain, 56,902,144 params, val 1.3837, measured evals (#6)
- v2 card: 12288-vocab retrain, 56,902,144 params, val 1.3837, measured evals (e217b62bd6301533cb840d643442406a6e85aec4) -
v2: 12288-vocab BPE tokenizer (was 8192) (#5)
- v2: 12288-vocab BPE tokenizer (was 8192) (15b1964609cb001829169aa44501482753b4e438) -
v2: retrain with 12288-vocab BPE (was 8192), 56,902,144 params, best val 1.3837 (#4)
- v2: retrain with 12288-vocab BPE (was 8192), 56,902,144 params, best val 1.3837 (2e799a020378896a2c2680c5cb7a346f4e6d93dd)
29 days ago
-
Add weights (54,804,992 params, 99 tensors, float32)
-
Add BPE-8192 tokenizer
-
Add self-contained loader + TinyStoriesGPT class
-
Add architecture config (D512/L16/H8/FFN2048/SEQ512/VOCAB8192, weight-tied)
-
Add model card: 54.8M from-scratch TinyStories GPT, verified params + held-out PPL 5.24
-
initial commit
