Latent Dynamics · 2026-11-05

Motion-guided Contrastive Regularization For Latent World Models

Yiwen Chen, Qiqi Hou, Jisoo Jeong, Yinhao Zhu, Huaizu Jiang, Amir Said

arXiv:2611.06901PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors identify “static collapse” in JEPA-style latent world models trained with distributional regularization (e.g., SIGReg): encoders may preserve episode-specific static background while suppressing manipulation-relevant state changes. They propose motion-guided contrastive regularization that matches predicted latent transitions to candidates from the same episode, using optical-flow similarity to create soft targets. They report large planning gains on OGBench-Cube-RB and LIBERO-Object versus LeWM.

Why it matters

Control requires latent representations that preserve action-conditioned state distinctions. The paper shows that distribution-only regularization can still fail under background variation, and that motion-guided contrastive objectives can mitigate this failure without pixel reconstruction or action-decoding losses.

Method

  • Identify static collapse for JEPA-style world models with SIGReg: representations can cluster by episode-specific static background, suppressing manipulation-relevant state changes.
  • Propose motion-guided contrastive regularization: contrast the predicted latent transition with transitions to candidate observations from the same episode.
  • Use optical-flow similarity to form soft contrastive targets, assigning higher target probability to candidates with similar motion (and thus similar transitions), and train jointly with latent prediction and SIGReg.

Limitation

The paper only explicitly reports planning success numbers for the listed benchmarks and does not state other limitations in the provided text.

Abstract (from arXiv)

JEPA-style world models learn action-conditioned latent dynamics without reconstructing observations, but distributional regularization alone does not ensure that their representations preserve information needed for control. We identify static collapse, a failure mode in which the encoder preserves episode-specific static content while suppressing manipulation-relevant state changes. Because static content is temporally predictable yet diverse across episodes, this failure can persist despite regularizers such as SIGReg. A controlled study on OGBench-Cube shows that introducing randomized backgrounds substantially degrades planning performance while leaving the underlying manipulation task unchanged. To address this problem, we propose motion-guided contrastive regularization. Our approach compares predicted latent transitions with transitions to candidate observations from the same episode, reducing the usefulness of static background identity for distinguishing action outcomes. Optical-flow similarities provide soft contrastive targets, allowing candidates with similar motion to receive similar supervision. Combined with latent prediction and SIGReg, our objective encourages representations to preserve within-episode state distinctions and similarities between related transitions. Our method improves planning success over LeWorldModel (LeWM) from 52% to 78% on OGBench-Cube with randomized backgrounds and from 14% to 54% on LIBERO-Object, while maintaining comparable performance on four other visual-control benchmarks.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-04 · Indexed by Pub-AI: 2026-11-05 UTC

preprint

Motion-guided Contrastive Regularization For Latent World Models

Yiwen Chen, Qiqi Hou, Jisoo Jeong, Yinhao Zhu, Huaizu Jiang, Amir Said

Preprint posted: 2026-11-04 · Original record ↗

arxiv
2611.06901
Explicit source versions (1)
  • v1 · 2026-11-04
  1. Preprint available

    Source date: 2026-11-04 · First detected: 2026-11-05 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

JEPA-style world models learn action-conditioned latent dynamics without reconstructing observations, but distributional regularization alone does not ensure that their representations preserve information needed for control. We identify static collapse, a failure mode in which the encoder preserves episode-specific static content while suppressing manipulation-relevant state changes. Because static content is temporally predictable yet diverse across episodes, this failure can persist despite regularizers such as SIGReg. A controlled study on OGBench-Cube shows that introducing randomized backgrounds substantially degrades planning performance while leaving the underlying manipulation task unchanged. To address this problem, we propose motion-guided contrastive regularization. Our approach compares predicted latent transitions with transitions to candidate observations from the same episode, reducing the usefulness of static background identity for distinguishing action outcomes. Optical-flow similarities provide soft contrastive targets, allowing candidates with similar motion to receive similar supervision. Combined with latent prediction and SIGReg, our objective encourages representations to preserve within-episode state distinctions and similarities between related transitions. Our method improves planning success over LeWorldModel (LeWM) from 52% to 78% on OGBench-Cube with randomized backgrounds and from 14% to 54% on LIBERO-Object, while maintaining comparable performance on four other visual-control benchmarks.