Latent Dynamics · 2026-10-20

See the Future, Classify the Present: Stochastic Latent World Modeling for Hyperspectral Images

Muhammad Ahmad, Gemine Vivone, Ronny H\"ansch, Ahmed Sohaib, Manuel Mazzara, Danfeng Hong

arXiv:2610.25781PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Stochastic Latent World Modeling (LWM) for hyperspectral image classification replaces static spatial-spectral embeddings with a trajectory-aware latent representation. It tokenizes spectral (per band) and spatial (per pixel) information using transformer branches, builds an environment state, then uses a stochastic latent transition model (Gaussian parameterization + recurrent refinement) to predict multi-step latent rollouts. A rollout planner adaptively aggregates the trajectory for classification.

Why it matters

The authors target limitations of static HSIC representations by introducing latent predictive structure without requiring temporal supervision. LWM learns multi-step stochastic transitions in representation space, aggregates a planned latent state for decision-making, and trains with multiple objectives including reconstruction, latent dynamics, stochastic rollout regularization, and physically motivated constraints.

Method

  • Dual tokenization: hyperspectral patches are encoded with dedicated spectral and spatial transformer branches, then fused into a unified representation with a global CLS scene embedding.
  • Latent environment state and stochastic rollouts: an environment state is formed from center, neighbor (1-ring), spectral (mean over bands), and scene features; a stochastic transition module predicts multiple future latent states via Gaussian parameterization and recurrent refinement.
  • Rollout planner for trajectory-aware classification: an adaptive rollout aggregation uses learned weights to form a planned latent state that is integrated with the base representation for final prediction.
Abstract (from arXiv)

Hyperspectral image classification (HSIC) relies on static spatial-spectral representations learned from labeled samples, limiting its ability to model structured dependencies in the latent representation space. To address this limitation, this work proposes Stochastic Latent World Modeling (LWM), a trajectory-based representation-learning framework that enriches classification through stochastic latent-state prediction. Given a hyperspectral patch, LWM tokenizes and encodes spectral and spatial information separately using dedicated transformer branches, then fuses their outputs into a unified representation. This representation is further organized into an environment state that captures center, neighborhood, spectral, and global contextual information. A stochastic latent transition module then predicts multiple future latent states through Gaussian parameterization and reparameterization, with recurrent refinement used to model dependencies across successive states. The resulting latent trajectory is adaptively aggregated by a rollout planner to construct a planned representation, which is integrated with the base representation for final classification. By extending static spatial-spectral encoding with stochastic multi-step latent-state modeling, LWM enables the classifier to exploit structured dependencies in the representation space and provides richer features for HSIC. Across five benchmark datasets, LWM achieves overall accuracies ranging from 89.21% to 99.52%, demonstrating consistent classification performance under limited-label settings.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC

preprint

See the Future, Classify the Present: Stochastic Latent World Modeling for Hyperspectral Images

Muhammad Ahmad, Gemine Vivone, Ronny Hänsch, Ahmed Sohaib, Manuel Mazzara, Danfeng Hong

Preprint posted: 2026-10-19 · Original record ↗

arxiv
2610.25781
Explicit source versions (1)
  • v1 · 2026-10-19
  1. Preprint available

    Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Hyperspectral image classification (HSIC) relies on static spatial-spectral representations learned from labeled samples, limiting its ability to model structured dependencies in the latent representation space. To address this limitation, this work proposes Stochastic Latent World Modeling (LWM), a trajectory-based representation-learning framework that enriches classification through stochastic latent-state prediction. Given a hyperspectral patch, LWM tokenizes and encodes spectral and spatial information separately using dedicated transformer branches, then fuses their outputs into a unified representation. This representation is further organized into an environment state that captures center, neighborhood, spectral, and global contextual information. A stochastic latent transition module then predicts multiple future latent states through Gaussian parameterization and reparameterization, with recurrent refinement used to model dependencies across successive states. The resulting latent trajectory is adaptively aggregated by a rollout planner to construct a planned representation, which is integrated with the base representation for final classification. By extending static spatial-spectral encoding with stochastic multi-step latent-state modeling, LWM enables the classifier to exploit structured dependencies in the representation space and provides richer features for HSIC. Across five benchmark datasets, LWM achieves overall accuracies ranging from 89.21% to 99.52%, demonstrating consistent classification performance under limited-label settings.