Latent Dynamics · 2026-10-14

MHWA: Multi-timescale Hierarchical World-Action Model

Pengcheng Pan, Guoqing Ma, Yuhan Zhang, Yang Chen, Yichen Liu, Ziheng Li, Shan Yu

arXiv:2610.14727PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

MHWA (Multi-timescale Hierarchical World-Action model) reduces open-loop drift in tokenized autoregressive world models by using a strided high-level context channel updated less frequently, plus context-aware routing to dynamically select a low-level Mixture-of-Experts. On a 14-game Atari subset with 10% subsampled offline data, MHWA (77M) reports IQM human normalized score (HNS) 0.884 across three seeds, outperforming JOWA-70M (0.529) and matching JOWA-150M (0.833) with fewer parameters and lower training FLOPs.

Why it matters

The authors address compounding error in autoregressive open-loop rollouts by explicitly controlling two types of interference in hierarchical world-action modeling: conceptual interference across hierarchy levels and structural interference across heterogeneous dynamics modes. The results on multi-game offline Atari show improved performance over a matched flat baseline and large training compute reduction, while maintaining comparable performance to a larger reference model.

Method

  • Uses a strided High-Level Manager to infer a slow-varying context at a reduced update frequency; context is broadcast for k atomic steps, and gradients to the manager are isolated via stop-gradient.
  • Implements the Low-Level Worker as an autoregressive Transformer whose feed-forward sublayers are replaced by sparse MoE layers; a gating network routes using both broadcast high-level context and the current dense token state.
  • Trains with JOWA-style base world-model and offline critic objectives, plus a skip-step high-level prediction objective and an auxiliary KL load-balancing regularizer for routing.

Limitation

The authors state their experiments are scoped to multi-game offline Atari benchmark settings and do not claim indefinite stability in long-horizon generation; the long-horizon diagnostic “does not eliminate eventual manifold departure.”

Abstract (from arXiv)

Autoregressive world models have emerged as a powerful paradigm for sample efficient decision making, yet their open loop rollouts often exhibit increasing drift that leads to error accumulation even within a relatively small planning horizon. Hierarchical temporal abstraction is a natural candidate to mitigate such drift by reducing the effective prediction horizon; however, existing hierarchical designs can introduce added structural and conceptual complexity, sometimes making them difficult to control and limiting gains over flat baselines. To this end, we introduce MHWA (Multi-timescale Hierarchical World-Action model), which employs a strided high level context channel to reduce global context update frequency, effectively curbing the propagation of compounding drift. Crucially, via context aware routing, the gating network dynamically reconfigures a low level mixture of experts by using both global context and current observations, thereby enabling flexible high-low coordination, helping alleviate interference commonly observed in static hierarchies. Empirically, using RLiable's seed-by-game aggregation, MHWA (77M parameters) achieves an IQM human normalized score (HNS) of 0.884 across three training seeds on a 14-game Atari subset using 10% subsampled offline data, outperforming a matched JOWA-70M baseline (0.529) and achieving numerically comparable performance to JOWA-150M while using nearly half the parameters and 70.88% fewer training FLOPs per sequence.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-10 · Indexed by Pub-AI: 2026-10-14 UTC

preprint

MHWA: Multi-timescale Hierarchical World-Action Model

Pengcheng Pan, Guoqing Ma, Yuhan Zhang, Yang Chen, Yichen Liu, Ziheng Li, Shan Yu

Preprint posted: 2026-10-10 · Original record ↗

arxiv
2610.14727
Explicit source versions (1)
  • v1 · 2026-10-10
  1. Preprint available

    Source date: 2026-10-10 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Autoregressive world models have emerged as a powerful paradigm for sample efficient decision making, yet their open loop rollouts often exhibit increasing drift that leads to error accumulation even within a relatively small planning horizon. Hierarchical temporal abstraction is a natural candidate to mitigate such drift by reducing the effective prediction horizon; however, existing hierarchical designs can introduce added structural and conceptual complexity, sometimes making them difficult to control and limiting gains over flat baselines. To this end, we introduce MHWA (Multi-timescale Hierarchical World-Action model), which employs a strided high level context channel to reduce global context update frequency, effectively curbing the propagation of compounding drift. Crucially, via context aware routing, the gating network dynamically reconfigures a low level mixture of experts by using both global context and current observations, thereby enabling flexible high-low coordination, helping alleviate interference commonly observed in static hierarchies. Empirically, using RLiable's seed-by-game aggregation, MHWA (77M parameters) achieves an IQM human normalized score (HNS) of 0.884 across three training seeds on a 14-game Atari subset using 10% subsampled offline data, outperforming a matched JOWA-70M baseline (0.529) and achieving numerically comparable performance to JOWA-150M while using nearly half the parameters and 70.88% fewer training FLOPs per sequence.