JEPA · 2026-10-09
SplitJEPA: Learning Invariant and Variant Latent Worlds without Reconstruction
Ruijin Hua, Zichuan Liu, Zhuokai Zhao, Yujia Zheng
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SplitJEPA is a reconstruction-free JEPA that learns an invariant/variant decomposition of a latent dynamical world directly in representation space. The authors prove that under stationary Gaussian predictive dynamics and a full-rank variation condition, SplitJEPA identifies invariant and variant subspaces (no observation decoder), up to independent within-block orthogonal transformations. Experiments on synthetic nonlinear systems and robotic manipulation support robustness and efficiency.
Why it matters
The paper targets a gap: JEPAs model latent dynamics without reconstruction, but prior results do not recover invariant-vs-variant components of the learned state. SplitJEPA connects representation learning with an identifiability guarantee for block structure without a decoder, and provides an operational robustness result: predictors depending only on the invariant block are locally insensitive to variant perturbations.
Method
- Learn a reconstruction-free latent representation using predictive objectives in representation space (JEPA-style) and normalization so the complete latent state is globally identifiable up to a rotation.
- Use matched invariant correspondences plus a sufficient-variation condition to force cross-block mixing terms to vanish, identifying invariant and variant subspaces without an observation decoder.
- Provide robustness via gradient isolation: downstream predictors that depend only on the invariant block have zero gradient w.r.t. variant latent factors (Theorem 3).
Limitation
The theoretical guarantees assume stationary Gaussian predictive dynamics and a full-rank variation (sufficient variation) condition (Assumption 1).
Abstract (from arXiv)
Understanding a dynamical world calls for more than a latent state that summarizes its observations: the state should also be organized into the factors that stay shared across related observations and the factors that vary between them. For example, a robot pushing a cube to a goal should take the same action when the camera shifts or the lights dim, since nothing in the scene has moved. Existing approaches to this decomposition commonly obtain it through reconstruction, so the latent variables must first explain the entire observational world before their organization can be trusted. Joint embedding predictive architectures (JEPAs) model the latent state directly and never reconstruct, yet no existing result recovers the invariant and variant parts of the state they learn. How to learn the invariant-variant structure of the latent world without paying for its reconstruction therefore remains open. To close this gap, we introduce SplitJEPA, a JEPA that jointly recovers the latent state and its invariant and variant organization directly in representation space, without any reconstruction. We prove that, under stationary Gaussian predictive dynamics and a full-rank variation condition, SplitJEPA identifies the invariant and variant subspaces up to independent block-wise isometries, without introducing an observation decoder. Since the guarantee needs no decoder, the result extends reconstruction-free latent recovery to invariant-variant block identification. Experiments on synthetic nonlinear systems and robotic manipulation tasks support the theoretical results and show their practical value for both robustness and efficiency.
Related papers
- Identifiable World Models from Pretrained Diffusion Representations
- LeHami: Symplectic Latent Dynamics for World Models
- AVL-JEPA: Preventing Causal Dynamics Information Collapse In Joint Embedding Predictive Architecture World Models
- DSReg: Provably Recovering Individual World Latents without Reconstruction
- Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
SplitJEPA: Learning Invariant and Variant Latent Worlds without Reconstruction
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.12349
Explicit source versions (1)
- v1 · 2026-10-08
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Understanding a dynamical world calls for more than a latent state that summarizes its observations: the state should also be organized into the factors that stay shared across related observations and the factors that vary between them. For example, a robot pushing a cube to a goal should take the same action when the camera shifts or the lights dim, since nothing in the scene has moved. Existing approaches to this decomposition commonly obtain it through reconstruction, so the latent variables must first explain the entire observational world before their organization can be trusted. Joint embedding predictive architectures (JEPAs) model the latent state directly and never reconstruct, yet no existing result recovers the invariant and variant parts of the state they learn. How to learn the invariant-variant structure of the latent world without paying for its reconstruction therefore remains open. To close this gap, we introduce SplitJEPA, a JEPA that jointly recovers the latent state and its invariant and variant organization directly in representation space, without any reconstruction. We prove that, under stationary Gaussian predictive dynamics and a full-rank variation condition, SplitJEPA identifies the invariant and variant subspaces up to independent block-wise isometries, without introducing an observation decoder. Since the guarantee needs no decoder, the result extends reconstruction-free latent recovery to invariant-variant block identification. Experiments on synthetic nonlinear systems and robotic manipulation tasks support the theoretical results and show their practical value for both robustness and efficiency.