Latent Dynamics · 2026-10-02
On the Divergence of Accuracy and Mechanism Consistency in Time Series World Models
Haochen Zhang, Jiaheng Guo, Zhen Xu, Zachary Plotkin, Nicholas Konz, Zhen Tan, Tianlong Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors formalize time series world models (TSWMs) with state, actions, and exogenous inputs, and introduce mechanism consistency: whether changing a plan moves forecasts in the declared direction of known action→state mechanisms. On eight datasets with real actions, they find accuracy and mechanism consistency diverge; directional supervision improves mechanism consistency without changing MAE.
Why it matters
World models are used to compare plans that were never executed, so their response to plan changes must be reliable. The authors’ benchmark and mechanism-consistency metric test plan reactions directly on real action time series, showing that optimizing prediction error alone can fail to ensure correct directional responses, and providing a training objective to improve it.
Method
- Formalize TSWMs with separated roles for state, controller actions, and exogenous inputs; distinguish continuous, mode, and event actions; define mechanism consistency from declared action→state direction and a quantile-shift plan perturbation.
- Benchmark systematically across prediction space (observation, frozen AE/VAE, JEPA), plan fusion (none/concat/residual/FiLM/gate/x-attn), and plan encoding over time (instant/decay/conv/attn) using seven backbones and five seeds on eight public datasets.
- Add directional supervision: a loss term penalizing wrong-signed response to shifted actions, trained without changing the network, to raise mechanism consistency while keeping MAE unchanged.
Limitation
Only five of the eight datasets are used for the mechanism-consistency metric; the two glucose-monitoring datasets have only event actions (quantile shift not defined) and one dataset’s mechanism is set aside because it acts on a timescale beyond the metric horizon.
Abstract (from arXiv)
A time series world model (TSWM) predicts a controlled system's state from its observed history and planned actions and exogenous inputs. Current approaches build forecasters with actions as covariates, trained and evaluated on prediction error under the executed plan. Yet world models compare unexecuted plans, but their responses to changed plans remain untested. We ask which design choices matter and whether accurate forecasters respond to changed plans as real systems do. We address both with a formalization and benchmark. The formalization separates state, actions and exogenous inputs, distinguishes continuous, mode and event actions, and introduces mechanism consistency, a metric built on declared action-state relations with known directions, such as a vasopressor raising blood pressure: it checks whether shifting an action moves the forecast in the declared direction. The benchmark consolidates eight public datasets with real actions from engineered infrastructure and clinical care, varying prediction space, plan fusion and plan encoding across seven backbones and five seeds. First, a frozen latent prediction space lowers MAE by 9.9% over observation space and gated output fusion lowers it by 12.7% over input concatenation on average, with both improving all eight datasets; temporal plan encoding changes average MAE by at most 2.2%. Second, prediction error and mechanism consistency diverge: the lowest-error configuration is at or below chance in consistency on four of five datasets with declared mechanisms, and no design choice avoids this. Finally, directional supervision, a loss penalizing the wrong-signed part of the response to a shifted action, significantly raises consistency on penalized mechanisms with no change in MAE. Together they give TSWMs a recipe: a frozen latent space and output-side fusion for accuracy, and a training objective for mechanism consistency.
Related papers
- ActTime: Action-Conditioned World Modeling for Time Series
- BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series
- Correct then Forecast: Observer State-Space Models for Time Series Forecasting
- When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
- One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
On the Divergence of Accuracy and Mechanism Consistency in Time Series World Models
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.01842
Explicit source versions (1)
- v1 · 2026-10-01
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
A time series world model (TSWM) predicts a controlled system's state from its observed history and planned actions and exogenous inputs. Current approaches build forecasters with actions as covariates, trained and evaluated on prediction error under the executed plan. Yet world models compare unexecuted plans, but their responses to changed plans remain untested. We ask which design choices matter and whether accurate forecasters respond to changed plans as real systems do. We address both with a formalization and benchmark. The formalization separates state, actions and exogenous inputs, distinguishes continuous, mode and event actions, and introduces mechanism consistency, a metric built on declared action-state relations with known directions, such as a vasopressor raising blood pressure: it checks whether shifting an action moves the forecast in the declared direction. The benchmark consolidates eight public datasets with real actions from engineered infrastructure and clinical care, varying prediction space, plan fusion and plan encoding across seven backbones and five seeds. First, a frozen latent prediction space lowers MAE by 9.9% over observation space and gated output fusion lowers it by 12.7% over input concatenation on average, with both improving all eight datasets; temporal plan encoding changes average MAE by at most 2.2%. Second, prediction error and mechanism consistency diverge: the lowest-error configuration is at or below chance in consistency on four of five datasets with declared mechanisms, and no design choice avoids this. Finally, directional supervision, a loss penalizing the wrong-signed part of the response to a shifted action, significantly raises consistency on penalized mechanisms with no change in MAE. Together they give TSWMs a recipe: a frozen latent space and output-side fusion for accuracy, and a training objective for mechanism consistency.