Pub-AI: AI in Science

Latent Dynamics · 2026-09-30

What Do Latent Predictive Vehicle Representations Retain? Measuring State, Geometry, and Local Response

Enzo Nicol\'as Spotorno, Josafat Leal Filho, Ant\^onio Augusto Fr\"ohlich

arXiv:2609.32512PDF

Auto-summarized: this summary was generated by a language model from the paper’s text and has not been reviewed by an editor. Check the paper before relying on it. Benchmark numbers appear only after a human has verified them.

TL;DR

The authors present a measurement protocol for action-conditioned latent vehicle predictors with a physical readout. It separately tests retention of physical quantities, organization in latent space, one-step forecasting, and local response to small command perturbations via three matched response paths. In a case study on IPG CarMaker data, representations retain planar outputs, improved future-command inputs help 1s forecasts, but local command-response can diverge in latent space and cause regret in nearby-command ranking.

Why it matters

Latent world models are used in control/planning, but latent prediction loss alone does not guarantee recovery of physical quantities or correct sensitivity to command changes. The protocol ties each measurement lens (retention, organization, forecasting, local response) to claims about control suitability, aiming to support closed-loop evaluation of predictive latents for action-conditioned vehicle dynamics models.

Method

  • Propose a measurement protocol for any action-conditioned latent predictor with physical readouts, using an untrained-encoder reference and separate lenses for retention, organization, forecasting, and local response.
  • Retention/organization: fit frozen linear or nonlinear readouts on logged history; compare decoded outputs and neighbor organization metrics (overlap, collision, effective rank) in both output space and latent space.
  • Local response: apply matched command pulse perturbations from a common initial condition and compare plant outputs, decoded target embedding, and decoded predicted embedding across three response paths; use finite-difference directional derivatives and offline command ranking.

Limitation

The authors note that command and state vary together in the training data (closed-loop system identification limitation), so the predictor can explain futures through state and learn little about the separate effect of a command.

Abstract (from arXiv)

Models of vehicle dynamics learned from logged states and commands complement physics-based models, and latent world models, which predict in a learned representation, are used to plan and train controllers in other domains. Vehicle controllers are usually specified in physical terms: costs, limits, and references depend on position, yaw angle, speed, and yaw rate, and the optimizer compares or differentiates predicted outcomes across nearby commands. A latent model placed in such a controller must therefore let these quantities be recovered and must change its predictions with commands as the vehicle does, and prediction error on its own latent targets measures neither. We contribute a measurement protocol for action-conditioned latent predictors with a physical readout that separately tests retention, physical-neighborhood organization, forecasting, and local response to command perturbations, using an untrained-encoder reference and three matched response paths that locate errors in the representation or the predictor. In a case study of a temporal joint-embedding predictive model trained on signals logged in IPG CarMaker, the representations retain the measured planar outputs, though an untrained encoder of the same architecture retains them slightly better; future-command input improves one-second forecasts with retention nearly unchanged; and responses to small command pulses diverge from the simulator already in latent coordinates, raising regret when choosing among nearby commands in all comparisons. Updating the predictor on responses corrects them locally at a cost in forecast accuracy. Measuring retention, forecasting, and local response separately is thus what qualifies a predictive latent as a candidate model for control, and the protocol provides the basis for its closed-loop evaluation.

Related papers