Preference datasets trl-lib/hh-rlhf-helpful-base Viewer • Updated Jan 8, 2025 • 46.2k • 77 • 3 trl-lib/lm-human-preferences-descriptiveness Viewer • Updated Jan 8, 2025 • 6.26k • 52 • 1 trl-lib/lm-human-preferences-sentiment Viewer • Updated Jan 8, 2025 • 6.26k • 84 trl-lib/rlaif-v Viewer • Updated Jan 8, 2025 • 83.1k • 125 • 3
Prompt-completion datasets trl-lib/tldr Viewer • Updated Jan 8, 2025 • 130k • 2.1k • 31 trl-lib/OpenMathReasoning Viewer • Updated Apr 26, 2025 • 3.2M • 1.14k
Unpaired preference datasets trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness Viewer • Updated Jan 8, 2025 • 16.6k • 89 • 4 trl-lib/kto-mix-14k Viewer • Updated Mar 25, 2024 • 15k • 198 • 9
trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness Viewer • Updated Jan 8, 2025 • 16.6k • 89 • 4
Online-DPO trl-lib/pythia-1b-deduped-tldr-online-dpo 1B • Updated Aug 2, 2024 • 1 trl-lib/pythia-1b-deduped-tldr-sft 1B • Updated Aug 2, 2024 • 55 trl-lib/pythia-6.9b-deduped-tldr-online-dpo 7B • Updated Aug 2, 2024 • 3 trl-lib/pythia-2.8b-deduped-tldr-sft Updated Aug 2, 2024 • 5
Stepwise supervision datasets trl-lib/math_shepherd Viewer • Updated Jan 8, 2025 • 445k • 8.67k • 12 trl-lib/prm800k Viewer • Updated Jan 8, 2025 • 41.2k • 452 • 4
Prompt-only datasets trl-lib/ultrafeedback-prompt Viewer • Updated Jan 8, 2025 • 39.8k • 104 • 9 trl-lib/DeepMath-103K Viewer • Updated Nov 14, 2025 • 103k • 3.84k • 11
Comparing DPO with IPO and KTO A collection of chat models to explore the differences between three alignment techniques: DPO, IPO, and KTO. teknium/OpenHermes-2.5-Mistral-7B Text Generation • 7B • Updated Feb 19, 2024 • 6.4k • • 901 Intel/orca_dpo_pairs Viewer • Updated Nov 29, 2023 • 12.9k • 1.84k • 322 trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.1-steps-200 Updated Dec 20, 2023 • 6 trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.2-steps-200 Updated Dec 20, 2023 • 7
teknium/OpenHermes-2.5-Mistral-7B Text Generation • 7B • Updated Feb 19, 2024 • 6.4k • • 901
Preference datasets trl-lib/hh-rlhf-helpful-base Viewer • Updated Jan 8, 2025 • 46.2k • 77 • 3 trl-lib/lm-human-preferences-descriptiveness Viewer • Updated Jan 8, 2025 • 6.26k • 52 • 1 trl-lib/lm-human-preferences-sentiment Viewer • Updated Jan 8, 2025 • 6.26k • 84 trl-lib/rlaif-v Viewer • Updated Jan 8, 2025 • 83.1k • 125 • 3
Stepwise supervision datasets trl-lib/math_shepherd Viewer • Updated Jan 8, 2025 • 445k • 8.67k • 12 trl-lib/prm800k Viewer • Updated Jan 8, 2025 • 41.2k • 452 • 4
Prompt-completion datasets trl-lib/tldr Viewer • Updated Jan 8, 2025 • 130k • 2.1k • 31 trl-lib/OpenMathReasoning Viewer • Updated Apr 26, 2025 • 3.2M • 1.14k
Prompt-only datasets trl-lib/ultrafeedback-prompt Viewer • Updated Jan 8, 2025 • 39.8k • 104 • 9 trl-lib/DeepMath-103K Viewer • Updated Nov 14, 2025 • 103k • 3.84k • 11
Unpaired preference datasets trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness Viewer • Updated Jan 8, 2025 • 16.6k • 89 • 4 trl-lib/kto-mix-14k Viewer • Updated Mar 25, 2024 • 15k • 198 • 9
trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness Viewer • Updated Jan 8, 2025 • 16.6k • 89 • 4
Comparing DPO with IPO and KTO A collection of chat models to explore the differences between three alignment techniques: DPO, IPO, and KTO. teknium/OpenHermes-2.5-Mistral-7B Text Generation • 7B • Updated Feb 19, 2024 • 6.4k • • 901 Intel/orca_dpo_pairs Viewer • Updated Nov 29, 2023 • 12.9k • 1.84k • 322 trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.1-steps-200 Updated Dec 20, 2023 • 6 trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.2-steps-200 Updated Dec 20, 2023 • 7
teknium/OpenHermes-2.5-Mistral-7B Text Generation • 7B • Updated Feb 19, 2024 • 6.4k • • 901
Online-DPO trl-lib/pythia-1b-deduped-tldr-online-dpo 1B • Updated Aug 2, 2024 • 1 trl-lib/pythia-1b-deduped-tldr-sft 1B • Updated Aug 2, 2024 • 55 trl-lib/pythia-6.9b-deduped-tldr-online-dpo 7B • Updated Aug 2, 2024 • 3 trl-lib/pythia-2.8b-deduped-tldr-sft Updated Aug 2, 2024 • 5
