JEPA · 2026-10-01
World-as-Graph: Relational World Modeling Through Latent Space Graphs
Yaqi Yang, Shuo Huang, Yujin Huang, Fucai Ke, Jiatong Han, Xin Zheng
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
World-as-Graph (WaG) is an object-centric latent world model that adds relational inductive bias to JEPA-style predictive representation learning. It builds time-varying latent graphs from object slots, uses relation-aware masking policies, and maintains object-level dynamic memory via a temporal GNN transition encoder plus a memory-driven autoregressive future predictor for future latent state prediction.
Why it matters
The authors target two issues in object-centric JEPA-style predictors: (C1) object-object relations are often implicit, limiting explicit relational/temporal structure learning; (C2) object-centric predictors may lack explicit object-level dynamic memory. WaG explicitly constructs evolving latent graphs and uses memory transition to improve relational and long-horizon prediction.
Method
- Relation-aware structure induction: constructs time-varying latent KNN graphs over object slots using cosine similarity, then selects masked objects using relational centrality or temporal dynamics.
- Object-centric memory transition: maintains per-object memory states updated by a temporal GNN transition encoder (graph-guided message passing + MemGRU) and evolves them autoregressively with a memory-driven future predictor.
- Training uses two latent-space predictive losses: historical masked-state prediction and future-state prediction, both with MSE in latent space.
Abstract (from arXiv)
World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.
Related papers
- A Review Of Robotic World Models For Dynamic Environments Based On Factor And Scene Graphs
- Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- WorldGraph: Graph-Native World Modeling
- Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
World-as-Graph: Relational World Modeling Through Latent Space Graphs
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.38927
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.
