Driving · 2026-11-10
ObjectWorld: Scaling Object-Centric World Models with A Large-Scale 3D Full-Object Flow Dataset
Haonan Chen, Sicheng Isabella Lu, Haitian Li, Jiajin Wu, Hexiang Wei, Jiawei Zhang, Xin Fei, Yiwen Hou, Chenrui Tie, Chongkai Gao, Sheng Wang, Lin Shao
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ObjectWorld introduces 3D full-object flow, an object-centric motion representation with dense 3D trajectories over an object’s complete surface (including occluded regions), valid for rigid and deformable objects. The paper also presents a 1210.1-hour 3D full-object flow dataset built from real-world videos and physical simulation, plus a world model that predicts dense full-object flow from an image and task description for robot manipulation execution.
Why it matters
The authors argue that existing world-model supervision for manipulation lacks dense, unified 3D full-object motion signals needed for scalable, transferable object-centric world modeling. By defining 3D full-object flow and building a large-scale dataset with an automatic annotation pipeline and quality metrics, they provide an intermediate motion target intended to be directly actionable for downstream robot execution.
Method
- Define 3D full-object flow: dense 3D trajectories over the reconstructed complete target-object surface across time, including occluded regions, as the interface between generative prediction and robot execution.
- Build a 1210.1-hour dataset from diverse real-world videos and physical simulations; create supervision via an automatic pipeline that reconstructs object geometry and pose, then extracts full-object flow and filters annotations using spatial/temporal consistency.
- Train an object-centric world model that takes an initial RGB-D observation and a task description, reconstructs initial object geometry, and predicts dense full-object trajectories; execute them via a flow-conditioned action optimization module using a local contact patch.
Limitation
The provided text does not state any limitations of the method.
Abstract (from arXiv)
Generative world modeling from large-scale visual data offers a promising path toward scalable robot manipulation. This calls for an intermediate representation that can be supervised at scale and remains actionable for downstream control. We propose 3D full-object flow, an object-centric motion representation that assigns dense 3D trajectories to points across the complete surface of a target object, including its occluded regions. By extending visible-surface object flow to the full object geometry, this representation makes complete object motion explicit and applies uniformly to both rigid and deformable objects. To realize the potential of this representation at scale, we introduce a 1210-hour full-object flow dataset that addresses the scarcity of large-scale supervision for complete 3D object motion. The dataset integrates diverse real-world videos and physical simulations, with a scalable annotation pipeline for recovering full-object motion from real-world observations and quality metrics for systematically assessing annotation quality. We then train a world model that takes an image and a task description as input and predicts dense full-object flow over the reconstructed object geometry. Experiments show reliable annotation quality, data-scalable flow prediction, and effective downstream manipulation across novel scenes and robot embodiments. Project website with additional results and videos: https://nus-lins-lab.github.io/ObjectWorld/.
Related papers
- WorldFlow: Joint Video and 3D Motion Prediction via Large-Scale Pretraining
- DepthWorld: 3D World Model for Robot Manipulation
- UNITAS: A 3D-Native World Action Model for Embodied Manipulation
- GPWM: Modeling the World as 3D Particle Motion
- FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC
ObjectWorld: Scaling Object-Centric World Models with A Large-Scale 3D Full-Object Flow Dataset
First announced by arXiv: 2026-11-10 · Original record ↗
- arxiv
- 2611.11085
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.11085v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Generative world modeling from large-scale visual data offers a promising path toward scalable robot manipulation. This calls for an intermediate representation that can be supervised at scale and remains actionable for downstream control. We propose 3D full-object flow, an object-centric motion representation that assigns dense 3D trajectories to points across the complete surface of a target object, including its occluded regions. By extending visible-surface object flow to the full object geometry, this representation makes complete object motion explicit and applies uniformly to both rigid and deformable objects. To realize the potential of this representation at scale, we introduce a 1210-hour full-object flow dataset that addresses the scarcity of large-scale supervision for complete 3D object motion. The dataset integrates diverse real-world videos and physical simulations, with a scalable annotation pipeline for recovering full-object motion from real-world observations and quality metrics for systematically assessing annotation quality. We then train a world model that takes an image and a task description as input and predicts dense full-object flow over the reconstructed object geometry. Experiments show reliable annotation quality, data-scalable flow prediction, and effective downstream manipulation across novel scenes and robot embodiments. Project website with additional results and videos: https://nus-lins-lab.github.io/ObjectWorld/.
