NT_checkpoint

Two models for a Franka cube-stacking task. A video model generates multi-view footage of the arm stacking three cubes; an inverse dynamics model reads video back and recovers the actions that would produce it.

folder	model	size	base
vgm_lora/	rank-32 LoRA, video generation	288 MB	nvidia/Cosmos-Predict2-14B-Sample-GR00T-Dreams-DROID
idm/	inverse dynamics, full fine-tune	2.4 GB	seonghyeonye/IDM_franka

Each folder has its own README with the details. Start there.

Quick orientation

vgm_lora/ — adapter weights only. Attach to the base Cosmos-Predict2 checkpoint with rank 32, alpha 32, targets q_proj,k_proj,v_proj,output_proj,mlp.layer1,mlp.layer2. Conditioned on one 768×432 four-view frame plus a prompt, it produces a 93-frame clip at 16 fps.

idm/ — load with IDM.from_pretrained(). One forward pass sees two frames, s and s+16, from three cameras and emits 16 actions. Its gripper output needs post-processing — see idm/README.md. Reading action[12] raw gives a vote tally, not a gripper state.

Scope

One task, one scene: three 5 cm cubes on a white table, a fixed three-camera rig, a Franka arm. Both models are specific to that setup.

Licence

Both are derivatives of NVIDIA-licensed weights and the upstream terms govern. Attribution to NVIDIA Cosmos-Predict2 and to the GR00T-Dreams IDM is required. The cosmos-predict2 repository code is Apache-2.0, which is separate from the weights licence.