WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
Abstract
WorldMemBench evaluates world memory in video models by testing static and dynamic retention across repeated revisits and state transitions.
As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requires both preserving previously observed content and ensuring coherent world evolution over time. World memory is a critical capability to evaluate alongside visual fidelity and video duration. We introduce WorldMemBench, a benchmark for systematically evaluating world memory in video world models. Across 260 test cases, it assesses static memory through the retention of human identity and appearance, object appearance and geometry, and environmental content and geometry, and dynamic memory through event progression and state persistence. Using controlled camera departures and multi-revisit trajectories, the benchmark evaluates static content against fixed initial references and dynamic events against expected state transitions. It combines appearance and geometry metrics with event-specific assessments, and reports video quality and camera-following accuracy as complementary diagnostics. Evaluation of nine representative models shows that strong static retention does not imply reliable dynamic state evolution, while repeated revisits reveal memory degradation beyond the first return. Human studies support the alignment between our automatic evaluations and human judgments. WorldMemBench provides a systematic testbed for diagnosing memory failures and guiding the development of persistent, temporally coherent video worlds.
Community
Hi everyone! We are the authors of WorldMemBench. Here are the official resources:
- ๐ Paper: https://arxiv.org/abs/2610.19063
- ๐ GitHub: https://github.com/ZJU-REAL/WorldMemBench
- ๐ Website and leaderboard: https://zju-real.github.io/WorldMemBench/
- ๐ค Dataset: https://huggingface.co/datasets/rescuerz/WorldMemBench
- ๐ฌ Examples: https://huggingface.co/datasets/rescuerz/WorldMemBench-examples
- ๐ฆ Evaluation weights: https://huggingface.co/rescuerz/WorldMemBench-weights
Questions and feedback are welcome!
We introduce WorldMemBench, a benchmark for evaluating whether video world models remember what they have seen and sustain coherent changes when events move out of view. Across 260 cases and nine models, we find that strong static memory does not guarantee reliable dynamic evolution, and repeated revisits expose degradation that a single return can miss. By jointly assessing static memory, dynamic memory, visual quality, and camera following, WorldMemBench helps diagnose failures and evaluate strategies for building more persistent, temporally coherent video worlds.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- R2M-Bench: Evaluating Revisit Memory via Relative Consistency in Interactive Video World Models (2026)
- WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory (2026)
- World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models (2026)
- OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models (2026)
- WorldWeave: Growing Persistent Geometric Worlds for Video Generation (2026)
- LOCI: Spatial Linear Memory for Streaming World Models (2026)
- Honeycomb: Constant-Size Scene Memory Representation for Video World Models (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.19063 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 2
rescuerz/WorldMemBench
rescuerz/WorldMemBench-examples
Spaces citing this paper 0
No Space linking this paper