JEPA · 2026-10-01

World-as-Graph: Relational World Modeling Through Latent Space Graphs

Yaqi Yang, Shuo Huang, Yujin Huang, Fucai Ke, Jiatong Han, Xin Zheng

arXiv:2609.38927PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

World-as-Graph (WaG) is an object-centric latent world model that adds relational inductive bias to JEPA-style predictive representation learning. It builds time-varying latent graphs from object slots, uses relation-aware masking policies, and maintains object-level dynamic memory via a temporal GNN transition encoder plus a memory-driven autoregressive future predictor for future latent state prediction.

Why it matters

The authors target two issues in object-centric JEPA-style predictors: (C1) object-object relations are often implicit, limiting explicit relational/temporal structure learning; (C2) object-centric predictors may lack explicit object-level dynamic memory. WaG explicitly constructs evolving latent graphs and uses memory transition to improve relational and long-horizon prediction.

Method

  • Relation-aware structure induction: constructs time-varying latent KNN graphs over object slots using cosine similarity, then selects masked objects using relational centrality or temporal dynamics.
  • Object-centric memory transition: maintains per-object memory states updated by a temporal GNN transition encoder (graph-guided message passing + MemGRU) and evolves them autoregressively with a memory-driven future predictor.
  • Training uses two latent-space predictive losses: historical masked-state prediction and future-state prediction, both with MSE in latent space.
Abstract (from arXiv)

World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC

preprint

World-as-Graph: Relational World Modeling Through Latent Space Graphs

Yaqi Yang, Shuo Huang, Yujin Huang, Fucai Ke, Jiatong Han, Xin Zheng

Preprint posted: 2026-09-30 · Original record ↗

arxiv
2609.38927
Explicit source versions (1)
  • v1 · 2026-09-30
  1. Preprint available

    Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.