SADL world models with shared BatchNorm
Checkpoints and datasets for SADL: On the Demand-Supply Geometry of Representation Collapse. Project page.
Checkpoints and training data for the SADL training and evaluation code. Models are trained on PushT, Cube, Reacher and TwoRoom with one BatchNorm population shared by all tasks and GPUs, for two objectives:
- INTACT: latent prediction and goal-conditioned action (planners: Direct, Guarded A);
- LeWM: latent prediction only (planner: CEM).
Contents
checkpoints/<model>/<training>/seed<seed>/<task>/
weights_epoch_1.pt model weights and BatchNorm statistics
config.yaml model configuration
action_stats.json action normalization of the task
training_config.json training configuration of the run
datasets/
pusht_expert_train.h5 cube_single_expert.h5 reacher.h5 tworoom.h5
SHA256SUMS
CHECKPOINTS_SHA256.txt
<model> is intact or lewm; <training> is four_task (one model trained on all four tasks; the four task
folders share every weight and normalization statistic and differ only in action dimension and action scaling) or
single_task (one model per task); seeds are 3072, 3073 and 3074. Every checkpoint is trained for one epoch.
JEPA glyph checkpoints
Training checkpoints for J, E, P, and A are available separately. Each letter is an epoch-1 INTACT-Lite model trained jointly on PushT, Cube, Reacher, and TwoRoom.
Usage
git clone https://github.com/zju3dv/SADL.git && cd SADL
pip install -r requirements.txt
hf download DavidSunok/SADL --local-dir hf
python evaluate.py --checkpoint hf/checkpoints/intact/four_task/seed3072/reacher --task reacher \
--data-dir hf/datasets --planner direct
python evaluate.py --checkpoint hf/checkpoints/lewm/single_task/seed3072/pusht --task pusht \
--data-dir hf/datasets --planner cem
Results
Success rate (%), mean +- SD over the three training seeds; each seed is evaluated with seeds 0, 1 and 42, 100 episodes each, on an NVIDIA RTX 4090 (goal 25 steps ahead, at most 50 actions, horizon 5, replanning every 5 actions).
| Model | Training | Planner | PushT | Cube | Reacher | TwoRoom | Mean |
|---|---|---|---|---|---|---|---|
| INTACT | four tasks | Direct | 85.7 +- 0.9 | 100.0 +- 0.0 | 93.4 +- 1.7 | 100.0 +- 0.0 | 94.8 +- 0.4 |
| INTACT | four tasks | Guarded A | 88.1 +- 0.8 | 99.6 +- 0.4 | 93.1 +- 0.7 | 99.8 +- 0.4 | 95.1 +- 0.3 |
| INTACT | single task | Direct | 85.1 +- 0.4 | 100.0 +- 0.0 | 94.9 +- 2.0 | 99.7 +- 0.6 | 94.9 +- 0.5 |
| INTACT | single task | Guarded A | 89.8 +- 0.2 | 99.6 +- 0.2 | 93.7 +- 3.2 | 99.3 +- 0.9 | 95.6 +- 0.9 |
| LeWM | four tasks | CEM | 64.7 +- 18.5 | 59.6 +- 1.3 | 46.7 +- 26.1 | 94.4 +- 1.0 | 66.3 +- 9.2 |
| LeWM | single task | CEM | 83.1 +- 1.6 | 58.9 +- 1.4 | 81.0 +- 4.3 | 91.3 +- 1.5 | 78.6 +- 1.9 |
Datasets
The HDF5 files hold expert demonstrations with uint8 RGB frames (224 x 224) and actions. PushT, Cube, Reacher and TwoRoom come from the LeWorldModel benchmark. The original dataset cards specify the MIT license: PushT, Cube, Reacher, and TwoRoom. The upstream copyright and permission notice is included in datasets/LeWM-MIT-LICENSE.txt.
License
The MIT license applies to the released weights. The upstream dataset license is linked above.