Latent Dynamics · 2026-11-10
An Evaluation of Tabular Foundation Models as Training-Free World Models
Xian Yeow Lee, Ryoichi Takase, Ahmed Farahat
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors evaluate TabFM, a pretrained tabular foundation model used training-free for action-conditioned next-state prediction via in-context regression over transition pairs. On limited transitions, they report TabFM is more accurate than LSTM/MLP trained from scratch and often beats regressors like ridge/GP, especially with long context. However, short-context rollouts degrade quickly, training on full data remains substantially better, and TabFM inference is about 3 orders of magnitude slower than an LSTM.
Why it matters
The paper tests whether a pretrained tabular “prior” can replace task-specific dynamics training for planning-like rollouts. It isolates when training-free inference helps (data-scarce settings with enough context) and when it fails (short context, multi-step error compounding). It also reports inference-time costs, relevant to whether such priors could be used inside planners or only offline.
Method
- Formulate next-state prediction as tabular in-context regression over transition pairs (x̃τ , Δxτ), querying with x̃t to predict Δx̂t and x̂t+1.
- Use TabPFN v2 off-the-shelf with default hyperparameters, issuing d sequential calls per timestep (one per state dimension); evaluate Direct-TabFM and a non-training-free Latent-TabFM.
- Compare across synthetic dynamical systems and D4RL Hopper/HalfCheetah, reporting one-step MSE and 20-step autoregressive rollout MSE over multiple context lengths, plus inference-time per one-step prediction.
Limitation
The authors exclude Latent-TabFM from the matched-budget comparison experiments because its encoder requires task-specific training, and they note that they evaluate explicit predictors only (future work includes closed-loop and pixel-based evaluation).
Abstract (from arXiv)
World models are widely considered to be a potential key component of model-based reinforcement learning and robot planning, but typically require task-specific training on substantial interaction data. We investigate whether a pretrained tabular foundation model (TabFM) can serve as a training-free dynamics model for world modeling by framing next-state prediction as in-context regression over transition pairs, and where such a model helps and where it breaks. Across synthetic dynamical systems and robot locomotion benchmarks, TabFM is more accurate than LSTM and MLP models trained from scratch on the same limited transitions in almost all settings, and among regressors fitted to the same data it is the most accurate on locomotion, while a Gaussian process is more accurate on smooth low-dimensional systems. However, models trained on the full dataset remain substantially better on most systems. When rolled out autoregressively over multiple steps, TabFM's accuracy depends strongly on how much context it is given. With a short context, its rollouts quickly degrade to the level of a trivial baseline, whereas with a long context they improve substantially and, on one locomotion task, exceed a trained LSTM. Inference is also about three orders of magnitude slower than an LSTM. Overall, in their current state, we find that training-free priors are useful as predictors when data are scarce and enough context is available, but they are not yet a substitute for trained world models in planning.
Related papers
- Causal world models under physical parameter shifts
- Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up
- The Planning Limits of Latent World Models
- Rollback: Experience Backtracking in World Models for Policy Improvement
- Beckmann World Models: Direct Terminal-Map Learning for One-Step Action-Conditioned Prediction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-09 · Indexed by Pub-AI: 2026-11-10 UTC
An Evaluation of Tabular Foundation Models as Training-Free World Models
Preprint posted: 2026-11-09 · Original record ↗
- arxiv
- 2611.11921
Explicit source versions (1)
- v1 · 2026-11-09
- Preprint available
Source date: 2026-11-09 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.11921v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World models are widely considered to be a potential key component of model-based reinforcement learning and robot planning, but typically require task-specific training on substantial interaction data. We investigate whether a pretrained tabular foundation model (TabFM) can serve as a training-free dynamics model for world modeling by framing next-state prediction as in-context regression over transition pairs, and where such a model helps and where it breaks. Across synthetic dynamical systems and robot locomotion benchmarks, TabFM is more accurate than LSTM and MLP models trained from scratch on the same limited transitions in almost all settings, and among regressors fitted to the same data it is the most accurate on locomotion, while a Gaussian process is more accurate on smooth low-dimensional systems. However, models trained on the full dataset remain substantially better on most systems. When rolled out autoregressively over multiple steps, TabFM's accuracy depends strongly on how much context it is given. With a short context, its rollouts quickly degrade to the level of a trivial baseline, whereas with a long context they improve substantially and, on one locomotion task, exceed a trained LSTM. Inference is also about three orders of magnitude slower than an LSTM. Overall, in their current state, we find that training-free priors are useful as predictors when data are scarce and enough context is available, but they are not yet a substitute for trained world models in planning.
