Latent Dynamics · 2026-10-14
MHWA: Multi-timescale Hierarchical World-Action Model
Pengcheng Pan, Guoqing Ma, Yuhan Zhang, Yang Chen, Yichen Liu, Ziheng Li, Shan Yu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
MHWA (Multi-timescale Hierarchical World-Action model) reduces open-loop drift in tokenized autoregressive world models by using a strided high-level context channel updated less frequently, plus context-aware routing to dynamically select a low-level Mixture-of-Experts. On a 14-game Atari subset with 10% subsampled offline data, MHWA (77M) reports IQM human normalized score (HNS) 0.884 across three seeds, outperforming JOWA-70M (0.529) and matching JOWA-150M (0.833) with fewer parameters and lower training FLOPs.
Why it matters
The authors address compounding error in autoregressive open-loop rollouts by explicitly controlling two types of interference in hierarchical world-action modeling: conceptual interference across hierarchy levels and structural interference across heterogeneous dynamics modes. The results on multi-game offline Atari show improved performance over a matched flat baseline and large training compute reduction, while maintaining comparable performance to a larger reference model.
Method
- Uses a strided High-Level Manager to infer a slow-varying context at a reduced update frequency; context is broadcast for k atomic steps, and gradients to the manager are isolated via stop-gradient.
- Implements the Low-Level Worker as an autoregressive Transformer whose feed-forward sublayers are replaced by sparse MoE layers; a gating network routes using both broadcast high-level context and the current dense token state.
- Trains with JOWA-style base world-model and offline critic objectives, plus a skip-step high-level prediction objective and an auxiliary KL load-balancing regularizer for routing.
Limitation
The authors state their experiments are scoped to multi-game offline Atari benchmark settings and do not claim indefinite stability in long-horizon generation; the long-horizon diagnostic “does not eliminate eventual manifold departure.”
Abstract (from arXiv)
Autoregressive world models have emerged as a powerful paradigm for sample efficient decision making, yet their open loop rollouts often exhibit increasing drift that leads to error accumulation even within a relatively small planning horizon. Hierarchical temporal abstraction is a natural candidate to mitigate such drift by reducing the effective prediction horizon; however, existing hierarchical designs can introduce added structural and conceptual complexity, sometimes making them difficult to control and limiting gains over flat baselines. To this end, we introduce MHWA (Multi-timescale Hierarchical World-Action model), which employs a strided high level context channel to reduce global context update frequency, effectively curbing the propagation of compounding drift. Crucially, via context aware routing, the gating network dynamically reconfigures a low level mixture of experts by using both global context and current observations, thereby enabling flexible high-low coordination, helping alleviate interference commonly observed in static hierarchies. Empirically, using RLiable's seed-by-game aggregation, MHWA (77M parameters) achieves an IQM human normalized score (HNS) of 0.884 across three training seeds on a 14-game Atari subset using 10% subsampled offline data, outperforming a matched JOWA-70M baseline (0.529) and achieving numerically comparable performance to JOWA-150M while using nearly half the parameters and 70.88% fewer training FLOPs per sequence.
Related papers
- Learning Goal-Conditioned Subgoals through Frozen Hierarchical World Models
- ActTime: Action-Conditioned World Modeling for Time Series
- AnyGoal: On World Model Generalization Beyond Expert Rollouts
- Network World Models as Environments for Algorithm Design on Complex Systems
- Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-10 · Indexed by Pub-AI: 2026-10-14 UTC
MHWA: Multi-timescale Hierarchical World-Action Model
Preprint posted: 2026-10-10 · Original record ↗
- arxiv
- 2610.14727
Explicit source versions (1)
- v1 · 2026-10-10
- Preprint available
Source date: 2026-10-10 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Autoregressive world models have emerged as a powerful paradigm for sample efficient decision making, yet their open loop rollouts often exhibit increasing drift that leads to error accumulation even within a relatively small planning horizon. Hierarchical temporal abstraction is a natural candidate to mitigate such drift by reducing the effective prediction horizon; however, existing hierarchical designs can introduce added structural and conceptual complexity, sometimes making them difficult to control and limiting gains over flat baselines. To this end, we introduce MHWA (Multi-timescale Hierarchical World-Action model), which employs a strided high level context channel to reduce global context update frequency, effectively curbing the propagation of compounding drift. Crucially, via context aware routing, the gating network dynamically reconfigures a low level mixture of experts by using both global context and current observations, thereby enabling flexible high-low coordination, helping alleviate interference commonly observed in static hierarchies. Empirically, using RLiable's seed-by-game aggregation, MHWA (77M parameters) achieves an IQM human normalized score (HNS) of 0.884 across three training seeds on a 14-game Atari subset using 10% subsampled offline data, outperforming a matched JOWA-70M baseline (0.529) and achieving numerically comparable performance to JOWA-150M while using nearly half the parameters and 70.88% fewer training FLOPs per sequence.
