Driving · 2026-10-20
ActionRoPE: action as coordinates for spatially consistent control in planar-camera game world models
Zhiyang Deng, Yuzhe Wu, Zijun Lin, Yeying Jin
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ActionRoPE represents action effects as world-coordinate rotary positional encoding in planar-camera game world models, so tokens at the same world location keep the same positional address across time. It enables spatially consistent control. The authors add CNS (Coordinate-Conditioned Noise Scheduling) to assign token-wise denoising schedules based on geometrically known vs newly revealed regions, stabilizing revisits and reducing drift.
Why it matters
The paper targets a mismatch between feature-conditioned action representations and the explicit geometry of planar-camera worlds. By binding camera displacement directly into positional coordinates, the model gains a world-coordinate system that supports precise motion commands and a known/new spatial partition for diffusion scheduling. The authors report improvements over four action-conditioning baselines and show cross-backbone and out-of-distribution transfer.
Method
- ActionRoPE: parameter-free interface that converts action-driven planar camera displacement into world-coordinate RoPE so the same world location keeps the same positional address across time.
- Known/new partition from world coordinates: define κ as whether each token’s world footprint lies in the first-frame footprint Ω0; pass κ to the network via a zero-initialized mask channel.
- CNS: Coordinate-Conditioned Noise Scheduling assigns different token-wise denoising schedules based on κ, so known regions stabilize earlier while newly revealed regions are generated with reference to them.
Limitation
ActionRoPE assumes the static world translates approximately rigidly in the image plane (true for top-down and isometric cameras that follow the player). It does not model zoom or strong parallax, and moving objects do not follow the world coordinates. Also, κ covers only the first-frame footprint, so regions that become visible later are not treated as known when the camera returns to them.
Abstract (from arXiv)
Game world models have made rapid progress toward real-time interactive generation. Yet action control is still predominantly implemented as feature conditioning, leaving the model to learn from training data how each action should move the visual world. This formulation is poorly matched to a broad class of planar-camera worlds, such as top-down and isometric games. In these worlds, player motion induces a geometrically well-defined planar camera displacement. We observe that the key to spatially consistent control in such worlds is to represent the effect of each action directly in world coordinates. To this end, we introduce ActionRoPE, a parameter-free action interface that converts action-driven camera displacement into world-coordinate rotary positions. As a result, tokens corresponding to the same world location retain the same positional address across time. These world coordinates further expose a spatial asymmetry in generation: regions covered by the initial observation are geometrically known, while newly revealed regions remain to be synthesized. We therefore propose Coordinate-Conditioned Noise Scheduling (CNS), which assigns token-wise denoising schedules according to this known-new structure. It allows observed regions to stabilize earlier, while new content is generated with reference to them. Compared with four representative action-conditioning mechanisms, our approach follows the commanded motion more faithfully and reduces the revisit drift by 66% over the strongest baseline. The same interface transfers across video backbones from 1.3B to 33B parameters and to out-of-distribution scenes. Project Page: https://teawhite.cn/ActionRoPE/
Related papers
- UniState: Unifying Memory and Intervention for Controllable World Simulation
- CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight
- DashVMC: Real-Time Discrete World Model Control in Geometry Dash
- 1Step-WAM: Your World Action Model but Faster using 1-Step Action Generation
- Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
ActionRoPE: Action as Coordinates for Spatially Consistent Control in Planar-Camera Game World Models
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.25861
Explicit source versions (1)
- v2 · 2026-10-20
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · OAI arXiv created; not header datestamp (official metadata)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Game world models have made rapid progress toward real-time interactive generation. Yet action control is still predominantly implemented as feature conditioning, leaving the model to learn from training data how each action should move the visual world. This formulation is poorly matched to a broad class of planar-camera worlds, such as top-down and isometric games. In these worlds, player motion induces a geometrically well-defined planar camera displacement. We observe that the key to spatially consistent control in such worlds is to represent the effect of each action directly in world coordinates. To this end, we introduce ActionRoPE, a parameter-free action interface that converts action-driven camera displacement into world-coordinate rotary positions. As a result, tokens corresponding to the same world location retain the same positional address across time. These world coordinates further expose a spatial asymmetry in generation: regions covered by the initial observation are geometrically known, while newly revealed regions remain to be synthesized. We therefore propose Coordinate-Conditioned Noise Scheduling (CNS), which assigns token-wise denoising schedules according to this known-new structure. It allows observed regions to stabilize earlier, while new content is generated with reference to them. Compared with four representative action-conditioning mechanisms, our approach follows the commanded motion more faithfully and reduces the revisit drift by 66% over the strongest baseline. The same interface transfers across video backbones from 1.3B to 33B parameters and to out-of-distribution scenes. Project Page: https://teawhite.cn/ActionRoPE/.