H-Liu1997/Video-Motion-Graphs

Video Motion Graphs

Original weighted-pose diffusion interpolation checkpoint and example inputs for Video Motion Graphs.

finetune_weighted/checkpoint-8000/pytorch_model.bin is the original full model checkpoint, including reference UNet, denoising UNet, pose guider and 4-channel seed guider. It requires the matching architecture and base components documented in the code repository; it is not a standalone Diffusers pipeline.

The checkpoint uses 14 frames: three context frames at each end and eight generated middle frames. The supplied inference configuration uses 20 denoising steps and guidance scale 3.5.

Verified on one NVIDIA H100 80GB: real video interpolation, music-driven dance rearrangement, and three real training updates across checkpoint save and new-process resume. These short runs do not establish training convergence or reproduce the full paper benchmark.

Weights retain the CreativeML OpenRAIL-M terms of their Stable Diffusion base. Project source code and third-party notices are distributed separately in the code repository. Example media is provided to reproduce the demonstration and is not covered by the source-code license.

Checkpoint SHA256: 9cc0f0354c5ec4b33085196ba5786f845e0901fd1dc7159e57be14d972c371af.

Reproduced example

Starting with only the raw 3,051-frame example video and music, the release code rebuilt pose and segmentation, constructed a 762-node / 1,059-edge graph, and generated three new diffusion transitions. The resulting video is 18 seconds, 768×768, 30 fps with audio. This run completed in 12 minutes 2 seconds on one H100 80GB.