Video World Models · 2026-10-08
SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
Yunheng Liu, Ziqi Cai, Siqi Yang, Yimu Wang, Minggui Teng, Jiaming Tan, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SPW-Nav is a streaming panoramic world model for language-guided navigation. It interprets movement instructions using the previously generated panorama as camera motion, then streams continuous one-minute 2K 360° video in real time. Key ideas include spherical rotation decoupling (rotation by spherical resampling) and pose-aligned conditioning to keep translation inputs bounded, with streaming supported by multi-term memory and few-step generation.
Why it matters
The authors target the semantic gap in panoramic generation for navigation: prior panoramic generators follow predefined camera trajectories, while interactive agents often act in perspective views. SPW-Nav aims for consistent, instruction-following, streaming panoramic video that supports on-the-fly instruction switching, and evaluates it on SPW-NavBench and long-horizon closed routes.
Method
- Language-driven motion targets: a vision-language backbone localizes referenced targets in the latest generated panorama and converts each instruction into an explicit displacement or a goal target to produce a camera trajectory.
- Spherical rotation decoupling + pose-aligned conditioning: rotation is applied exactly on the sphere by resampling, leaving the video backbone to model translation; poses are expressed relative to a recent reference view to keep inputs bounded over long streams.
- Streaming with multi-term memory and few-step generation: autoregressive pyramid denoising continues from prior chunks using memory tokens (including an anchor for the first frame) and a distilled student generator.
Limitation
The multi-term memory of generated frames already keeps revisited views consistent and closes long routes with near-exact rotation. A more explicit memory (e.g., a pose-indexed 3D scene representation) could make consistency on distant revisits more reliable, although advancing memory mechanisms for video generation may also achieve this within the video itself.
Abstract (from arXiv)
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.
Related papers
- PVDepth: Panoramic Video Depth Estimation via Geometry-Aware Spatiotemporal Adaptation
- OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining
- AgiWorld: An Agentic Video World Model for Building Persistent Worlds
- Hy-Nomas: Towards Free-form Interactive World Generation
- ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-08 UTC
SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.08941
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.
