Video World Models · 2026-10-20
EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models
X. Feng, J. Tang, Z. Lin, L. Chen, S. Shao, Q. Lu, K. Huang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors release EmbodiedWorld-200K (over 200K samples from 150 3D open-world games) for open-world embodied navigation-planning, with multi-level annotations and unified evaluation. They pretrain a VLA EWA offline on the dataset, then use a video world model as a low-cost environment proxy for interactive RL post-training with VQA-based reward scoring and DPO. They report improved performance on in-distribution and OOD evaluation splits.
Why it matters
The paper addresses an infrastructure gap for Embodied World Agents (EWAs) by providing large-scale open-world embodied-planning data, unified splits/metrics, and a reproducible two-stage training recipe (offline pretraining + world-model-based interactive RL). This directly targets open-world generalization where offline-only training is argued to be insufficient, and offers a scalable alternative to expensive real interaction.
Method
- Release EmbodiedWorld-200K: open-world embodied navigation-planning dataset with multi-level annotations (initial observation, language instruction, action sequence) and unified evaluation splits.
- Two-stage baseline: (1) offline pretraining on the dataset to build a Vision–Language–Action (VLA) EWA; (2) interactive RL post-training inside a video world model as an environment proxy.
- Construct world-model rollouts from actor-planned action sequences; derive rewards via visual question answering (VQA); refine policy with direct preference optimization (DPO) from preference pairs.
Limitation
The authors state two limitations: (1) 3D virtual games as proxies for the physical world, so factors critical for real-world deployment (e.g., fine-grained physical interaction, sensor noise, uncontrolled lighting, long-tail human–object interactions) are only partially captured; (2) limited action granularity due to a shared coarse action space (W/A/S/D movement, camera rotation, and a no-op).
Abstract (from arXiv)
Building embodied agents that can perceive, reason, and act across diverse 3D worlds is a central research direction toward artificial general intelligence. Yet most existing work remains confined to specific tasks in a single scene and transfers poorly to new environments, leaving general embodied agents for open-world settings under-explored. We refer to such agents as Embodied World Agents (EWAs), whose defining property is the ability to understand diverse visual environments and execute general actions in open-world conditions. To advance EWA research, we study open-world embodied navigation-planning and release EmbodiedWorld-200K, a large-scale dataset of over 200K samples spanning diverse scenes. Each example is equipped with multi-level annotations, ranging from low-level camera-motion trajectories to high-level semantic instructions. The dataset also provides unified evaluation splits and task-specific metrics, supporting offline pretraining and fair evaluation of EWAs. Beyond offline supervision, a defining characteristic of an EWA is its interaction with the environment; training solely on offline data is therefore insufficient, especially for generalizing to scenes not covered in the training distribution. Building dedicated real interactive environments, however, is often prohibitively expensive. To address this, we propose a novel training scheme that employs a video world model as a low-cost environment proxy and performs interactive reinforcement-learning (RL) post-training of the EWA within it. Empirically, pretraining on EmbodiedWorld-200K substantially improves the open-world embodied planning ability of the EWA, and further RL post-training inside a world model yields clear gains on out-of-distribution scenes. The dataset, baseline checkpoints, and training and evaluation code are available at https://xiaokunfeng.github.io/EmbodiedWorld-200K/.
Related papers
- Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- PrismWAM: Decoupling State Representations in World Action Models
- Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
- Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds
- Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.25959
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Building embodied agents that can perceive, reason, and act across diverse 3D worlds is a central research direction toward artificial general intelligence. Yet most existing work remains confined to specific tasks in a single scene and transfers poorly to new environments, leaving general embodied agents for open-world settings under-explored. We refer to such agents as Embodied World Agents (EWAs), whose defining property is the ability to understand diverse visual environments and execute general actions in open-world conditions. To advance EWA research, we study open-world embodied navigation-planning and release EmbodiedWorld-200K, a large-scale dataset of over 200K samples spanning diverse scenes. Each example is equipped with multi-level annotations, ranging from low-level camera-motion trajectories to high-level semantic instructions. The dataset also provides unified evaluation splits and task-specific metrics, supporting offline pretraining and fair evaluation of EWAs. Beyond offline supervision, a defining characteristic of an EWA is its interaction with the environment; training solely on offline data is therefore insufficient, especially for generalizing to scenes not covered in the training distribution. Building dedicated real interactive environments, however, is often prohibitively expensive. To address this, we propose a novel training scheme that employs a video world model as a low-cost environment proxy and performs interactive reinforcement-learning (RL) post-training of the EWA within it. Empirically, pretraining on EmbodiedWorld-200K substantially improves the open-world embodied planning ability of the EWA, and further RL post-training inside a world model yields clear gains on out-of-distribution scenes. The dataset, baseline checkpoints, and training and evaluation code are available at https://xiaokunfeng.github.io/EmbodiedWorld-200K/.