Latent Dynamics · 2026-10-30
Self-Supervised Latent Action-Conditioned World Modeling for Time Series
Chia-Han Kuo, Chengshuai Zhao, Huan Liu, Lo Pang-Yun Ting
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ACTS-WM is a self-supervised latent action-conditioned world model for time-series. It infers discrete latent actions from observed time-series transitions without action annotations, using an inverse/forward dynamics setup with vector-quantized actions and task embeddings. It then pretrains a JEPA-based time-series encoder and an action-conditioned predictor to model transitions in latent space, evaluated on 4 tasks across 22 datasets.
Why it matters
The authors argue that explicitly modeling factors behind transitions in time series can provide complementary supervisory signals beyond JEPA encoders that only predict latent representations without action conditioning. ACTS-WM learns latent actions directly from passive observations and uses them to condition JEPA-based world modeling for forecasting and classification across diverse datasets.
Method
- Latent action modeling from observational time-series transitions via an inverse dynamics model (IDM) and forward dynamics model (FDM), with vector quantization to obtain discrete latent actions; task embeddings distinguish forecasting horizons and classification.
- A fusion module for heterogeneous time-series: channel fusion (shared conv + multi-head self-attention + learnable-query pooling) and temporal fusion via dilated temporal convolutional networks.
- A JEPA-based encoder (adapted V-JEPA 2 to time series) plus an action-conditioned predictor, where the learned latent actions condition prediction of target time-series representations for downstream heads.
Abstract (from arXiv)
Time-series data play an important role in various tasks, such as weather forecasting, web traffic analysis, and speech recognition. Recently, pioneering studies have increasingly adopted Joint-Embedding Predictive Architecture (JEPA) based encoders to learn time-series representations through latent-space prediction. However, they often overlook the underlying factors governing transitions between time-series observations. In this work, we hypothesize that explicitly modeling the information behind changes in time series can facilitate time-series modeling. Building on this hypothesis, we propose ACTS-WM, a self-supervised latent action-conditioned world model framework for time-series analysis. First, we introduce a latent action modeling approach that infers the underlying information behind time-series transitions as latent actions, without requiring explicit action annotations. We then develop a JEPA-based encoder and an action-conditioned world model to predict target time-series representations conditioned on the learned latent actions and task information. We evaluate ACTS-WM across 4 representative tasks on 22 datasets from diverse domains, covering univariate and multivariate forecasting and classification. Comprehensive experiments demonstrate that ACTS-WM can improve downstream performance over pretrained JEPA-based encoders by learning complementary representations.
Related papers
- BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series
- Does Joint-Embedding Predictive Architecture Pretraining Help Time Series Forecasting?
- Pythia: Toward Foundation World Models for Multimodal Time Series
- ActTime: Action-Conditioned World Modeling for Time Series
- V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-10-30 · Indexed by Pub-AI: 2026-10-30 UTC
Self-Supervised Latent Action-Conditioned World Modeling for Time Series
First announced by arXiv: 2026-10-30 · Original record ↗
- arxiv
- 2610.37963
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-10-30 · First detected: 2026-10-30 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.37963v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Time-series data play an important role in various tasks, such as weather forecasting, web traffic analysis, and speech recognition. Recently, pioneering studies have increasingly adopted Joint-Embedding Predictive Architecture (JEPA) based encoders to learn time-series representations through latent-space prediction. However, they often overlook the underlying factors governing transitions between time-series observations. In this work, we hypothesize that explicitly modeling the information behind changes in time series can facilitate time-series modeling. Building on this hypothesis, we propose ACTS-WM, a self-supervised latent action-conditioned world model framework for time-series analysis. First, we introduce a latent action modeling approach that infers the underlying information behind time-series transitions as latent actions, without requiring explicit action annotations. We then develop a JEPA-based encoder and an action-conditioned world model to predict target time-series representations conditioned on the learned latent actions and task information. We evaluate ACTS-WM across 4 representative tasks on 22 datasets from diverse domains, covering univariate and multivariate forecasting and classification. Comprehensive experiments demonstrate that ACTS-WM can improve downstream performance over pretrained JEPA-based encoders by learning complementary representations.