Latent Dynamics · 2026-10-20

IMPULSE: Push First, Then Predict in V-JEPA 2 Feature Space

Tingan Jin, Chung-Hsien Chou

arXiv:2610.23968PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

IMPULSE performs three calibration pushes, summarizes their outcomes with a Bayesian contact memory (8 numbers), predicts object pose via a pose pathway, and uses that pose to forecast future V-JEPA 2 feature maps. On a pre-specified held-out confirmation, the memory lowers 1-s feature-forecast NMSE by 11.6–12.8% vs the same predictor without experience (8/8 seeds). A random-Fourier pose head greatly amplifies the gain.

Why it matters

The authors test whether a few active trials improve a predictor whose targets are frozen video-encoder features, and what limits that improvement. Using an information-ladder argument, they isolate how much physics can reach the features only through motion, and they measure how pose pathway design affects the resulting feature forecast gains.

Method

  • Calibrate with three short pushes, observe noisy pose tracks, and fit post-contact normal/tangential/angular velocities plus noise estimates.
  • Infer an object-specific impulse response using a Bayesian contact memory over (log ρ, e, μ), producing 8 interpretable numbers (posterior mean response + spreads) at two lever arms.
  • Use a pose pathway to feed predicted pose into a predictor in frozen V-JEPA 2 feature space; optionally add a random-Fourier pose head to correct predicted features.

Limitation

The task is a simulated planar push with a known object shape, and the renderer shows only geometry; whether the gain carries over to real pushing videos and to 3D scenes whose hidden properties must be inferred from observed interactions remains open.

Abstract (from arXiv)

Latent world models such as V-JEPA 2 forecast in the feature space of a video encoder, but one look at an object does not reveal how hard it is to rotate, how much it bounces or how much it slides. People find out by trying: a few pokes reveal how an object responds. IMPULSE, named after the impulse response it infers, gives a predictor in frozen V-JEPA 2 feature space the same ability. A Bayesian contact memory summarizes three short calibration pushes, a predicted object pose carries it into the predictor, and a lightweight random-Fourier pose head lets the predictor use the pose. On 512 fresh objects, in a pre-specified held-out confirmation with models sealed before the data existed, the memory lowers the 1-second feature-forecast error by 11.6-12.8% relative to the same predictor without experience, for all 8 training seeds, with a 95% seed-and-object interval excluding zero. The gain also holds for 128 objects with out-of-range physics (11.7-14.0%), and the full extension beats a plain predictor by 15.5-16.6%. An information-ladder argument, exact for ideal predictors, shows that physics reaches the features only through motion, so we measure how much the pathway lets pose help. On fresh objects, the Fourier head roughly triples the gain of a perfect memory over plain pose tokens (2.8-3.2 times). Raw push measurements fed through the same pathway give a similar gain, so the Bayesian summary is not required; it condenses the trials into eight interpretable numbers. The gain is absent at contact onset and grows afterwards.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-20 UTC

preprint

IMPULSE: Push First, Then Predict in V-JEPA 2 Feature Space

Tingan Jin

Preprint posted: 2026-10-12 · Original record ↗

arxiv
2610.23968
Explicit source versions (1)
  • v1 · 2026-10-12
  1. Preprint available

    Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Latent world models such as V-JEPA 2 forecast in the feature space of a video encoder, but one look at an object does not reveal how hard it is to rotate, how much it bounces or how much it slides. People find out by trying: a few pokes reveal how an object responds. IMPULSE, named after the impulse response it infers, gives a predictor in frozen V-JEPA 2 feature space the same ability. A Bayesian contact memory summarizes three short calibration pushes, a predicted object pose carries it into the predictor, and a lightweight random-Fourier pose head lets the predictor use the pose. On 512 fresh objects, in a pre-specified held-out confirmation with models sealed before the data existed, the memory lowers the 1-second feature-forecast error by 11.6-12.8% relative to the same predictor without experience, for all 8 training seeds, with a 95% seed-and-object interval excluding zero. The gain also holds for 128 objects with out-of-range physics (11.7-14.0%), and the full extension beats a plain predictor by 15.5-16.6%. An information-ladder argument, exact for ideal predictors, shows that physics reaches the features only through motion, so we measure how much the pathway lets pose help. On fresh objects, the Fourier head roughly triples the gain of a perfect memory over plain pose tokens (2.8-3.2x). Raw push measurements fed through the same pathway give a similar gain, so the Bayesian summary is not required; it condenses the trials into eight interpretable numbers. The gain is absent at contact onset and grows afterwards.