Driving · 2026-10-06

Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning

Andreu Matoses Gimenez, Andrei-Carlo Papuc, Chris Pek, Javier Alonso-Mora

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors build a skill-level latent world-model framework where one world-model transition corresponds to one execution of a flow-matching policy over complete skills. They compare four skill action abstractions—compressed seed, learned code, object-centric learned code, and a symbolic label—for long-horizon simulated block rearrangement (up to 14 sequential skills). The symbolic label achieves over 90% success up to six skills; without labels, the object-centric code matches it on single-skill tasks and retains half to three quarters of its success for two to five skills.

Why it matters

This paper investigates how action representations and candidate-action selection affect long-horizon planning with latent (JEPA) world models. By setting the world-model horizon to whole-skill executions (rather than control-rate steps) and isolating differences across action abstractions, it shows that symbolic precondition knowledge in the action proposal is a major contributor to planning success, while longer horizons are ultimately limited more by search than by world-model prediction quality.

Method

Train flow-matching policies on demonstrations segmented into complete skills; map planner-chosen inputs (seed/code/label) to one open-loop skill execution, treated as one world-model transition between settled scenes.
Train a JEPA-style joint-embedding predictive world model (LeWM) to predict next observation embeddings from the current embedding and the compact action (skill abstraction inputs).
Plan with receding-horizon beam/CEM search over action sequences to minimize distance between imagined state embeddings and a goal embedding; execute only the first planned skill and replan after observing the next scene.

Limitation

Beyond six skills, the authors report that the search, not the world model, limits success.

Abstract (from arXiv)

Latent world models enable robots to plan by predicting the consequences of actions. Planning long tasks with control-rate actions requires many prediction steps, which enlarges the search space and accumulates error. Skill-level actions shorten these sequences, but a symbolic skill vocabulary requires domain knowledge and labeled demonstrations. We construct skill-level actions from the inputs of a flow-matching policy trained on demonstrations segmented into complete skills. The policy maps a noise seed and an observation, optionally with a code or label, to a complete skill execution, so one execution is one world-model transition. On this mechanism we propose four action abstractions with increasing task knowledge: a compressed seed, two discrete codes learned from the demonstrations, and a symbolic label. We evaluate them with a common world-model training procedure and planning framework on simulated block rearrangement tasks that require up to 14 sequential skills. The symbolic label succeeds in over 90% of the tasks that require up to six skills and degrades beyond. Without any label, an object-centric learned code matches it on single-skill tasks and retains half to three quarters of its success on tasks of two to five skills. Ablations attribute much of the label's advantage to its planner knowing which actions are applicable, rather than to the label itself. Beyond six skills the search, not the world model, limits success. Project page: https://andreumatoses.github.io/research/flow-skill-wm

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-03 · Indexed by Pub-AI: 2026-10-06 UTC

preprint