Update card to v8 (current weights): 79,375 steps / 2.60B tokens (hits target), val 3.63. Honest eval: 40-sample sweep shows mean loop frac 0.219 (up from v7's 0.167) and 14/40 degenerate — more tokens did not buy coherence. Replaces the stale v7-only card that described the weights as 'well short of 2.6B'. (#15)
- Update card to v8 (current weights): 79,375 steps / 2.60B tokens (hits target), val 3.63. Honest eval: 40-sample sweep shows mean loop frac 0.219 (up from v7's 0.167) and 14/40 degenerate — more tokens did not buy coherence. Replaces the stale v7-only card that described the weights as 'well short of 2.6B'. (1cc0879f2495049ed479e73bde1800fe57440f15)
v8: continued training to 2.6B tokens (from 1.51B). Improved coherence on narrative prompts. Same architecture (10,052,864 params, 5L d320 5-head SwiGLU RoPE). (#14)
- v8: continued training to 2.6B tokens (from 1.51B). Improved coherence on narrative prompts. Same architecture (10,052,864 params, 5L d320 5-head SwiGLU RoPE). (adec7f2dee715ec4253caeceef3db40df34e3e23)
20 days ago
Update card to v7 checkpoint (step 30000, val 3.6892 / ppl 40.01)
Fix fabricated sample quote + overstatement: replace sample #1 (not in eval output) with real samples from eval_ldt10m_final.json; correct "coherent/fluency" to "grammatically structured but semantically incoherent" (#10)
- Fix fabricated sample quote + overstatement: replace sample #1 (not in eval output) with real samples from eval_ldt10m_final.json; correct "coherent/fluency" to "grammatically structured but semantically incoherent" (fabff55c405fa83f6ae256f74b119b87bac4ee1c)
