sergiopaniego/huggingface-static-ab21bb-bucket
9.71 kB
Sergio Paniego PRO
sergiopaniego
AI & ML interests
None yet
Recent Activity
posted an update about 9 hours ago
Can you do RL over taste?
I've spent some time reproducing, in the open, Surya N's idea of training a model to paint with code. It's a coding model that learns to paint watercolours by writing JS code, trained with GRPO. I used TRL and OpenEnv for this, with the whole pipeline running on Hugging Face.
The interesting part is that the reward has no correct answer, unlike a math problem. In this case it's based on the artistic preferences of the person who builds the dataset.
Everything is published: the environment, the reference pool, the trained adapters, every painting of every run with the code that made it, and a write-up with all the decisions, including the ones that went wrong.
Blog post: https://huggingface.co/blog/train-to-paint-with-code updated a model about 11 hours ago
HuggingEnvs/watercolour-grpo-hps-led updated a model about 11 hours ago
HuggingEnvs/watercolour-grpo-judge-ledOrganizations
Posts 112
Post
58
Can you do RL over taste?
I've spent some time reproducing, in the open, Surya N's idea of training a model to paint with code. It's a coding model that learns to paint watercolours by writing JS code, trained with GRPO. I used TRL and OpenEnv for this, with the whole pipeline running on Hugging Face.
The interesting part is that the reward has no correct answer, unlike a math problem. In this case it's based on the artistic preferences of the person who builds the dataset.
Everything is published: the environment, the reference pool, the trained adapters, every painting of every run with the code that made it, and a write-up with all the decisions, including the ones that went wrong.
Blog post: https://huggingface.co/blog/train-to-paint-with-code
I've spent some time reproducing, in the open, Surya N's idea of training a model to paint with code. It's a coding model that learns to paint watercolours by writing JS code, trained with GRPO. I used TRL and OpenEnv for this, with the whole pipeline running on Hugging Face.
The interesting part is that the reward has no correct answer, unlike a math problem. In this case it's based on the artistic preferences of the person who builds the dataset.
Everything is published: the environment, the reference pool, the trained adapters, every painting of every run with the code that made it, and a write-up with all the decisions, including the ones that went wrong.
Blog post: https://huggingface.co/blog/train-to-paint-with-code
Articles 31
Article
4
Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps
spaces 189
pinned
Running on Zero
Agents
Featured
123
VLM Object Understanding
🦀
Explore object detection, visual grounding, keypoint Detecti
pinned
Runtime error
Agents
11
Qwen2-VL-7B
🔥
Ask questions about charts in images
pinned
Running on Zero
Agents
SmolVLM-trl-dpo-rlaif-v
🔥
Generate text from an image and question
pinned
Runtime error
Agents
SmolVLM-trl-sft-ChartQA
🔥
Ask questions about charts in images
Running
Huggingface Static Ab21bb
🎯
View and manage your data with the Trackio dashboard
Running
Huggingface Static F1eb57
🎯
View and monitor your project's metrics in real time
models 152
sergiopaniego/rl-envs-youtube-livestream-4-scripts
Updated
sergiopaniego/qwen3-1.7b-mbpp-grpo
Text Generation • 2B • Updated • 519
sergiopaniego/qwen3-1.7b-wordle-grpo
Text Generation • 2B • Updated • 451
sergiopaniego/Qwen3-4B-claude-code-local-grpo
Text Generation • 4B • Updated • 9
sergiopaniego/Qwen3-4B-claude-code-deepcoder-grpo
Text Generation • 4B • Updated • 11
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B-judged
Text Generation • 2B • Updated • 9
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B
Text Generation • 2B • Updated • 8
sergiopaniego/Qwen3-8B-opencode-deepcoder-grpo
Text Generation • 8B • Updated • 50 • 1
sergiopaniego/grpo-youtube-livestream-3-scripts
Reinforcement Learning • Updated • 2
sergiopaniego/qwen3-0.6b-mbpp-grpo-k2
Text Generation • 0.6B • Updated • 10
datasets 15
sergiopaniego/pelican-svg-drawings
Viewer • Updated • 169 • 653 • 1
sergiopaniego/math-sdpo-hints-plain
Viewer • Updated • 600 • 29
sergiopaniego/math-sdpo-hints
Viewer • Updated • 600 • 30
sergiopaniego/gsm8k-sdpo-plain
Viewer • Updated • 700 • 8
sergiopaniego/gsm8k-sdpo-hints
Viewer • Updated • 700 • 24
sergiopaniego/pi-mono-chat
Viewer • Updated • 886 • 39
sergiopaniego/requests-pr-diff
Viewer • Updated • 1 • 80
sergiopaniego/trl-r2e-test
Viewer • Updated • 1 • 13
sergiopaniego/chain-sum-rollouts
Viewer • Updated • 50 • 13
sergiopaniego/ttt-scripted-smoke
Viewer • Updated • 20 • 27