Driving · 2026-10-02

Evaluating Physical Consistency and Plausibility in Generative Scenario Models for Autonomous Driving

Manasa Mariam Mammen, Zafer Kayatas, Stefan Wagner

arXiv:2610.01581PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors introduce a five-layer evaluation protocol for generative scenario models in autonomous driving, targeting both internal representation interpretability and physical executability. Layers 1–4 assess latent/feature alignment, baseline advantages, decoder feature emergence, and latent controllability via intervention. Layer 5 checks generated trajectories against vehicle dynamics constraints using a Physical Validity Rate (PVR). They demonstrate it on a guided VAE and apply it to other generative models.

Why it matters

Output-level visual realism is not sufficient for safety-critical scenario generation. The protocol aims to provide transparency on what physical quantities are encoded and where, whether latent interventions can steer outputs, and whether generated trajectories satisfy vehicle-dynamics constraints. By combining internal probing/controls with a threshold-based physical plausibility evaluation, it provides more actionable evidence for validation and diagnosis.

Method

  • Five-layer evaluation protocol for generative scenario models: Layer 1 latent-physical probing; Layer 2 baseline comparison vs PCA; Layer 3 decoder analysis via activation clustering and layer-wise probing; Layer 4 latent intervention for controllability; Layer 5 physical plausibility via vehicle dynamics constraints and PVR.
  • Layer 1 trains linear and nonlinear probes on frozen representations to predict extracted physical parameters and uses a probing gap ΔR2 = R2nl − R2lin to assess nonlinear entanglement/disentanglement.
  • Layer 5 computes accelerations and lateral jerk from generated trajectories and checks friction-circle, max lateral acceleration, and max lateral jerk thresholds; PVR is the fraction satisfying all constraints and the paper reports the relative shift Δs between generated and training distributions.

Limitation

The paper notes that the transformer and sparse/SAE baselines (and world-model-inspired GRU setup) are “simple, lightly-trained architectures intended to demonstrate the protocol’s applicability… not strong baselines,” so “the numerical values should therefore be read as illustrative rather than as benchmarks.”

Abstract (from arXiv)

Generative AI models are increasingly used for scenario generation in autonomous driving. While they can generate realistic-looking scenarios, they often provide limited transparency into learned representations and consistency with real-world vehicle dynamics. This lack of formal assurance limits their use in safety-critical validation and certification workflows. To address this aspect, we introduce a layered evaluation protocol that complements existing methods by assessing models across five layers. The first four layers inspect internal representations and network layers through kinematic alignment, statistical baseline comparison, latent controllability, and activation analysis. The fifth layer evaluates model outputs against vehicle dynamics constraints such as lateral jerk thresholds. We demonstrate the protocol on a Variational Autoencoder (VAE)-based scenario generator. Although standard output-level metrics and visualizations suggest that the generated scenarios are realistic, our protocol provides deeper insight into the extent to which the model's latent space aligns with kinematic features and whether visually plausible trajectories satisfy vehicle-dynamics constraints. We further apply the protocol to additional generative models, demonstrating its applicability beyond the VAE architecture.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC

preprint

Evaluating Physical Consistency and Plausibility in Generative Scenario Models for Autonomous Driving

Manasa Mariam Mammen, Zafer Kayatas, Stefan Wagner

Preprint posted: 2026-10-01 · Original record ↗

arxiv
2610.01581
Explicit source versions (1)
  • v1 · 2026-10-01
  1. Preprint available

    Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Generative AI models are increasingly used for scenario generation in autonomous driving. While they can generate realistic-looking scenarios, they often provide limited transparency into learned representations and consistency with real-world vehicle dynamics. This lack of formal assurance limits their use in safety-critical validation and certification workflows. To address this aspect, we introduce a layered evaluation protocol that complements existing methods by assessing models across five layers. The first four layers inspect internal representations and network layers through kinematic alignment, statistical baseline comparison, latent controllability, and activation analysis. The fifth layer evaluates model outputs against vehicle dynamics constraints such as lateral jerk thresholds. We demonstrate the protocol on a Variational Autoencoder (VAE)-based scenario generator. Although standard output-level metrics and visualizations suggest that the generated scenarios are realistic, our protocol provides deeper insight into the extent to which the model's latent space aligns with kinematic features and whether visually plausible trajectories satisfy vehicle-dynamics constraints. We further apply the protocol to additional generative models, demonstrating its applicability beyond the VAE architecture.