karpathy/climbmix-400b-shuffle
Preview • Updated • 47.6k • 76
datasets out there
some interesting datasets to use for language modeling
-
-
allenai/dolmino-mix-1124
Viewer • Updated • 170M • 16.7k • 105 -
XYZAILab/XYZ-Aquila-SFT
Viewer • Updated • 7k • 555 • 378 -
54rt1n/wikipedia-summary-dataset
Viewer • Updated • 5.32M • 98 • 4 -
pszemraj/simple_wikipedia
Viewer • Updated • 238k • 374 • 11 -
common-pile/youtube
Viewer • Updated • 1.13M • 495 • 13 -
srinivasbilla/self-instruct-base
Viewer • Updated • 82.6k • 40 • 5 -
agentlans/high-quality-english-sentences
Viewer • Updated • 1.71M • 1.1k • 40 -
agentlans/note-taking-v2
Viewer • Updated • 17.6k • 50 • 1 -
PleIAs/SYNTH
Viewer • Updated • 68M • 18.9k • 307 -
wikimedia/structured-wikipedia
Viewer • Updated • 10.5M • 10k • 408 -
OpenLeecher/lmsys_chat_1m_clean
Viewer • Updated • 273k • 323 • 89 -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 56.1M • 1.41k • 3 -
OrionLLM/OpenAgentInstruct
Viewer • Updated • 15.2k • 51 • 4 -
IFM/Code-Reasoning
Viewer • Updated • 452M • 19.6k • 102 -
openbmb/UltraData-Code
Viewer • Updated • 267M • 30.4k • 212