SolarRenderer: Few-Step Generative Reference Rendering

🌐 Project Page • 📄 Paper • 💻 Code • 🤗 Data

SolarRenderer turns a structure video (grey box, semantic map, skeletons, depth, body mesh, or a sketch it never saw in training) into a finished video, with an optional first frame and a prompt deciding the look. These are its LoRAs on MiniMax-H3 (Ref2VA transformer): a control LoRA and a four-step LoRA, each in a full-clip version and a block-causal streaming version. Together they render a clip in four steps.

Method

Files

File	What it is
solarrenderer_control_lora.safetensors	control LoRA, rank 384: the structure video enters H3's reference pathway, frame-aligned
solarrenderer_4step_lora.safetensors	four-step LoRA, rank 384: fine-tuned from DMAD by structure-guided distribution matching
solarrenderer_causal_control_lora.safetensors	causal control LoRA, rank 384: the control LoRA trained further with block-causal attention. Each block of up to 5 latent frames sees the prompt, the first frame, the structure of the last 6 blocks and the 5 blocks rendered before it, never a later block
solarrenderer_causal_4step_lora.safetensors	causal four-step LoRA, rank 384: the four-step LoRA post-trained, with the causal control LoRA frozen, on its own block-by-block rollouts by distribution matching with structure guidance (w = 1.5) toward the full-clip control model

Quick Start

git clone https://github.com/Enderfga/SolarRenderer && cd SolarRenderer
pip install -e . && pip install flash-attn==2.7.4.post1 --no-build-isolation
python scripts/prepare_h3.py --dir ./MiniMax-H3
hf download Enderfga/SolarRenderer --local-dir ./weights
python scripts/render.py --h3 ./MiniMax-H3 \
    --control weights/solarrenderer_control_lora.safetensors \
    --fewstep weights/solarrenderer_4step_lora.safetensors \
    --structure structure.mp4 --first first.png --prompt prompt.txt --out render.mp4

Output: 1344×768 (or 768×1344), 24 fps, 158 frames; about 3.5 minutes of sampling on one GPU. The prompt follows one six-part form: see the prompt guide.

Streaming. The two causal LoRAs render the video block by block, four steps per block; each block can be shown as soon as it is done. Same inputs, one flag:

python scripts/render.py --h3 ./MiniMax-H3 --causal \
    --control weights/solarrenderer_causal_control_lora.safetensors \
    --fewstep weights/solarrenderer_causal_4step_lora.safetensors \
    --structure structure.mp4 --first first.png --prompt prompt.txt --out render.mp4

Citation

@article{fang2026solarrenderer,
  title  = {SolarRenderer: Few-Step Generative Reference Rendering},
  author = {Fang, Guian and Qian, Shengju and Lu, Dongyue and
            Huang, Junchao and Jiang, Lutao and Wang, Xin and
            Song, Yiren and Jiang, Li and Shou, Mike Zheng},
  year   = {2026}
}

License

The weights are LoRA adapters on MiniMax-H3 and are distributed under the MiniMax H3 Community License Agreement (see LICENSE and NOTICE), including its territorial restrictions (it excludes the European Union, the United Kingdom, the Republic of Korea and the United States) and its Acceptable Use Policy. The four-step LoRAs descend from the public DMAD four-step LoRA and are released under the same license. The code is released under the Apache License 2.0.