Driving · 2026-10-30
SaiVLA-JEPA: Evaluating Future Latent Prediction in JEPA-Conditioned VLA Control
Rufeng Xiao (VLA Research Team, Synthoid.ai), Ruihao Xiao (VLA Research Team, Synthoid.ai), Wei Chen (VLA Research Team, Synthoid.ai), Xinhai Sun (VLA Research Team, Synthoid.ai)
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SaiVLA-JEPA tests whether adding a JEPA-style future latent prediction objective improves closed-loop control for a vision-language-action policy that already uses frozen V-JEPA features. Two training-only residual heads predict future JEPA representations at horizons 4 and 16 and are removed at inference. On LIBERO-Plus, current JEPA conditioning yields higher success, while future supervision shows no detectable aggregate rollout gain at the evaluated training budget.
Why it matters
The authors’ results suggest that, for an already JEPA-conditioned VLA flow policy, improving latent prediction (via future representation heads) does not necessarily translate into better paired closed-loop control. This motivates evaluating latent prediction objectives through episode-paired rollouts rather than latent losses alone.
Method
- Build SaiVLA-JEPA: a VLM-conditioned DiT flow action expert receives projected features from a frozen V-JEPA encoder; inference uses the same native action sampler across variants.
- Add training-only auxiliary future prediction: two residual heads (at horizons h∈{4,16}) predict future JEPA token residuals with a tokenwise MSE auxiliary loss; heads are absent at inference.
- Evaluate causal effects using episode-paired controls (BaseFlow: no JEPA conditioning; CurrentJEPA vs FutureJEPA: same conditioning with/without future-head supervision), plus test-time adaptation variants (TCA and TPA).
Limitation
The paired CurrentJEPA–FutureJEPA comparison finds no detectable aggregate future-head gain after 25,000 training steps and is not evidence that the true effect is exactly zero; replication with independent training runs remains necessary. BaseFlow–FutureJEPA changes both JEPA conditioning and future supervision, so it does not identify the future objective’s causal contribution.
Abstract (from arXiv)
Joint-embedding predictive architectures (JEPAs) learn to predict representations rather than pixels, offering a compact way to model how visual scenes evolve. We ask whether this predictive objective improves closed-loop control when a vision-language-action (VLA) policy already uses V-JEPA features. We call our JEPA-conditioned flow policy family SaiVLA-JEPA. It combines a vision-language model with a diffusion transformer action expert and receives features from a frozen V-JEPA encoder. Two lightweight auxiliary heads predict future representations during training and are omitted at inference, leaving the action sampler unchanged. We compare policies without JEPA features, with current JEPA conditioning, and with additional future supervision on 10,030 paired LIBERO-Plus episodes. Current JEPA conditioning reaches about 82.5% success versus 71.6% without JEPA, while future supervision provides no detectable aggregate gain at the evaluated training budget. We separately evaluate SaiVLA-JEPA-TCA, which adapts latent context, and SaiVLA-JEPA-TPA, which adapts selected parameters. The context-retry gain is a system-level result under a larger attempt budget; parameter adaptation shows no consistent advantage over the frozen policy and target controls. These results motivate assessing latent-prediction objectives through paired closed-loop rollouts rather than latent loss alone.
Related papers
- Juno: Taming Predictive Latents for Vision-Language-Action Models
- V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
- Vision-Predictor-Action: Towards Efficient Embodied AI via Neuro-Symbolic JEPA Predictors
- Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State
- PLaW-VLA: Predictive Latent World Modeling for Vision-Language-Action Policies
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-10-30 · Indexed by Pub-AI: 2026-10-30 UTC
SaiVLA-JEPA: Evaluating Future Latent Prediction in JEPA-Conditioned VLA Control
First announced by arXiv: 2026-10-30 · Original record ↗
- arxiv
- 2610.38508
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-10-30 · First detected: 2026-10-30 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.38508v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Joint-embedding predictive architectures (JEPAs) learn to predict representations rather than pixels, offering a compact way to model how visual scenes evolve. We ask whether this predictive objective improves closed-loop control when a vision-language-action (VLA) policy already uses V-JEPA features. We call our JEPA-conditioned flow policy family SaiVLA-JEPA. It combines a vision-language model with a diffusion transformer action expert and receives features from a frozen V-JEPA encoder. Two lightweight auxiliary heads predict future representations during training and are omitted at inference, leaving the action sampler unchanged. We compare policies without JEPA features, with current JEPA conditioning, and with additional future supervision on 10,030 paired LIBERO-Plus episodes. Current JEPA conditioning reaches about 82.5% success versus 71.6% without JEPA, while future supervision provides no detectable aggregate gain at the evaluated training budget. We separately evaluate SaiVLA-JEPA-TCA, which adapts latent context, and SaiVLA-JEPA-TPA, which adapts selected parameters. The context-retry gain is a system-level result under a larger attempt budget; parameter adaptation shows no consistent advantage over the frozen policy and target controls. These results motivate assessing latent-prediction objectives through paired closed-loop rollouts rather than latent loss alone.