Latent Dynamics · 2026-10-08
RoboJEPA: Scaling Robotic Latent World Models
Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan, Sarath Chandar, Tushar Nagarajan, Daniel Severo, Koustuv Sinha, Michal Drozdzal, Adriana Romero Soriano, Jeannette Bohg, Nicolas Ballas, Mahmoud Assran
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
RoboJEPA is a JEPA-based latent world model for robotics trained on real-robot data spanning 12 embodiments. The authors report that its imagination (latent rollout) error follows a second-order power law in training compute with verified extrapolation. They also show offline imagination error correlates with downstream robotic planning success, and that RoboJEPA can be deployed zero-shot as an agent planning toward a single goal image. They release all model checkpoints and training/robot deployment code.
Why it matters
The authors address an open scaling problem for robotic latent world models by fitting and extrapolating scaling laws for imagination error as a function of compute, enabling prediction of model quality beyond the fit scale. They connect offline latent prediction (imagination error) to downstream planning performance and use it as a proxy for real-robot evaluation. They further demonstrate zero-shot goal-image planning on real hardware and release checkpoints and code for reproducibility.
Method
- Action-conditioned latent world model: RoboJEPA deterministically predicts next latent features in one forward pass using a ViT-based predictor on top of a frozen V-JEPA 2.1 encoder, trained with an ℓ1 regression loss in the encoder’s latent space.
- Scaling-law setup: the authors fit compute-optimal frontiers and report that out-of-distribution forward-prediction (latent rollout) L1 error follows a second-order power law in training compute with verified extrapolation.
- Planning and evaluation: RoboJEPA plans by encoding a single goal image into goal features and searching over H-step action sequences using receding-horizon CEM; the authors evaluate on real-robot (DROID) and simulation (RoboCasa) and report downstream planning success.
Abstract (from arXiv)
Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.
Related papers
- The Planning Limits of Latent World Models
- One from Infinity: Actualizing Futures from Pretrained World Models into Robot Actions
- GreenWAM: A World Foundation Model for Robotics
- T2-z: Test-Time Optimization of Environment Latents for Robotic World Models
- A Review Of Robotic World Models For Dynamic Environments Based On Factor And Scene Graphs
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-07 · Indexed by Pub-AI: 2026-10-08 UTC
RoboJEPA: Scaling Robotic Latent World Models
Preprint posted: 2026-10-07 · Original record ↗
- arxiv
- 2610.10515
Explicit source versions (1)
- v1 · 2026-10-07
- Preprint available
Source date: 2026-10-07 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA's imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.