Video World Models · 2026-10-23
From Single-Actor Environments to Multi-Actor Event Worlds: Can LLMs Be Event World Models?
Juntong Ni, Zewen Liu, Bohan Wang, Lige Zhang, Wei Jin
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study “event worlds” where multiple actors evolve the system and only an event log is observed. They ask whether LLMs can predict future observations from the log plus the agent’s proposed actions. In Diplomacy and Polymarket, prediction improves when all actors’ recorded actions are given, but with only the agent’s action models struggle when other actors strongly affect outcomes, limiting rollouts and plan success; imagined experience can still help policy learning.
Why it matters
This paper frames multi-actor, partially observed systems as event worlds and builds benchmarks and evaluation protocols to test whether LLMs can serve as world models for planning and policy learning. The results isolate the impact of not knowing other actors’ actions and show how that gap propagates to multi-step rollouts and agent decision-making.
Method
- Define event worlds and event world models as predicting future observations from event logs and the agent’s proposed actions, with a comparison between Task S (agent action only) and Task J (joint action with other actors’ recorded actions).
- Build benchmarks from human Diplomacy game records and real Polymarket trade logs; Diplomacy uses a rules engine to verify outcomes, Polymarket uses real market observations for prediction checks.
- Evaluate single-step prediction, multi-step closed/open-loop rollouts, action-not-taken counterfactuals, and usefulness to agents in Diplomacy (planning consistency, choosing candidate actions, and imagined-game initialization).
Limitation
The authors state that “without them (Task S), an LLM also has to anticipate the other actors,” and that “the fine-tuned ones far less so once the other actors’ actions are given,” highlighting difficulty anticipating other actors’ actions as a central open problem.
Abstract (from arXiv)
World models help agents assess the consequences of their actions before acting. Many real systems, however, involve multiple actors who make their own decisions, can evolve even when the agent takes no action, and are only partially observed through event logs. We formalize these systems as event worlds and ask whether large language models can predict future observations from the log and the agent's proposed actions. We build complementary benchmarks from human Diplomacy games and real Polymarket trades. Our hierarchical framework compares prediction with and without other actors' actions, then tests multi-step rollouts, predictions for actions never taken, and usefulness to agents. Fine-tuning and more data improve prediction when all actors' recorded actions are given. With only the agent's action, models struggle when other actors strongly affect the outcome, and larger models and longer histories help little. This gap also limits multi-step prediction and agent decisions, with models overestimating plan success. Yet imagined experience and initialization from world models can still improve policy learning. Anticipating other actors' actions remains a central challenge for event world modeling.
Related papers
- AnyGoal: On World Model Generalization Beyond Expert Rollouts
- Anticipating Social Consequences with Distributional World Models
- Beyond Prediction: Steering VLM Agents with Retrospective World Modeling
- RoleWorld: Knowledge Transfer in World Modeling with Functional Role Abstractions
- Second Thoughts: Exploring and Planning through Comparative Imagination in World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-21 · Indexed by Pub-AI: 2026-10-23 UTC
From Single-Actor Environments to Multi-Actor Event Worlds: Can LLMs Be Event World Models?
Preprint posted: 2026-10-21 · Original record ↗
- arxiv
- 2610.30488
Explicit source versions (1)
- v1 · 2026-10-21
- Preprint available
Source date: 2026-10-21 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World models help agents assess the consequences of their actions before acting. Many real systems, however, involve multiple actors who make their own decisions, can evolve even when the agent takes no action, and are only partially observed through event logs. We formalize these systems as event worlds and ask whether large language models can predict future observations from the log and the agent's proposed actions. We build complementary benchmarks from human Diplomacy games and real Polymarket trades. Our hierarchical framework compares prediction with and without other actors' actions, then tests multi-step rollouts, predictions for actions never taken, and usefulness to agents. Fine-tuning and more data improve prediction when all actors' recorded actions are given. With only the agent's action, models struggle when other actors strongly affect the outcome, and larger models and longer histories help little. This gap also limits multi-step prediction and agent decisions, with models overestimating plan success. Yet imagined experience and initialization from world models can still improve policy learning. Anticipating other actors' actions remains a central challenge for event world modeling.