Latent Dynamics · 2026-10-06

Pythia: Toward Foundation World Models for Multimodal Time Series

Xilin Dai, Hongzhou Chen, Yifan Hu, Yiding Liu, Zewei Dong, Jiang-Ming Yang

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Pythia is a context-conditioned time-series foundation world model that learns predictive latent dynamics across datasets using a joint-embedding predictive architecture. It uses a stop-gradient numerical reference to guide contextual corrections in latent space, then freezes the predictive representation and adapts a probabilistic quantile decoder on observed history and predicted world states. On MUSE, Pythia-Tiny reports 0.6879 MASE and 0.4269 WSQL, improving 6.26%/5.00% over the strongest published MUSE baseline.

Why it matters

The authors address whether reusable multimodal predictive representations can be learned before observation-space forecasting. Their two-stage approach separates predictive latent representation learning from probabilistic readout adaptation, and uses joint-embedding predictive learning plus numerical-reference guidance to incorporate entity/event/covariate context. The reported MUSE zero-shot results and controlled ablations aim to show when and why latent world states improve forecasting under heterogeneous contextual inputs.

Method

Two-stage recipe: Stage 1 learns context-conditioned predictive latent states with a joint-embedding predictive architecture, using a stop-gradient numerical reference to guide contextual correction errors (gradients update only the context pathway).
Context module refines numerical latent estimates using entity descriptions, timestamp-filtered events, and named numerical covariates; events/covariates are filtered to be no later than the prediction origin.
Stage 2 freezes the predictive system (numerical encoder/predictor/context/correction) and adapts a probabilistic quantile decoder over [observed history, special REG token, predicted future world states] using pinball loss with a parameter-anchor term.

Abstract (from arXiv)

Time-series foundation models offer a unified approach to forecasting across heterogeneous domains. Textual context and auxiliary observations provide complementary information about temporal dynamics, yet reusable multimodal predictive representations remain underexplored. We introduce Pythia, a foundation world model that learns context-conditioned latent dynamics across datasets through a joint-embedding predictive architecture. A stop-gradient numerical reference guides contextual corrections to predicted future states. A separate probabilistic decoder then adapts to the frozen predictive representation and observed history, decoupling world-model pretraining from observation-space forecasting. On MUSE, Pythia-Tiny's normalized mean absolute scaled error (MASE) and weighted sum quantile loss (WSQL) are 0.6879 and 0.4269, reducing errors by 6.26% and 5.00% relative to the strongest model evaluated in the published MUSE leaderboard. Through a series of controlled experiments, we investigate how to design a time-series world model through shared pretraining and how joint-embedding predictive learning can incorporate multimodal information. The results support separating predictive representation learning from probabilistic readout and show complementary contributions from entity descriptions, events, and covariates.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-04 · Indexed by Pub-AI: 2026-10-06 UTC

preprint