Video World Models · 2026-10-08

SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation

Yunheng Liu, Ziqi Cai, Siqi Yang, Yimu Wang, Minggui Teng, Jiaming Tan, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi

arXiv:2610.08941PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

SPW-Nav is a streaming panoramic world model for language-guided navigation. It interprets movement instructions using the previously generated panorama as camera motion, then streams continuous one-minute 2K 360° video in real time. Key ideas include spherical rotation decoupling (rotation by spherical resampling) and pose-aligned conditioning to keep translation inputs bounded, with streaming supported by multi-term memory and few-step generation.

Why it matters

The authors target the semantic gap in panoramic generation for navigation: prior panoramic generators follow predefined camera trajectories, while interactive agents often act in perspective views. SPW-Nav aims for consistent, instruction-following, streaming panoramic video that supports on-the-fly instruction switching, and evaluates it on SPW-NavBench and long-horizon closed routes.

Method

  • Language-driven motion targets: a vision-language backbone localizes referenced targets in the latest generated panorama and converts each instruction into an explicit displacement or a goal target to produce a camera trajectory.
  • Spherical rotation decoupling + pose-aligned conditioning: rotation is applied exactly on the sphere by resampling, leaving the video backbone to model translation; poses are expressed relative to a recent reference view to keep inputs bounded over long streams.
  • Streaming with multi-term memory and few-step generation: autoregressive pyramid denoising continues from prior chunks using memory tokens (including an anchor for the first frame) and a distilled student generator.

Limitation

The multi-term memory of generated frames already keeps revisited views consistent and closes long routes with near-exact rotation. A more explicit memory (e.g., a pose-indexed 3D scene representation) could make consistency on distant revisits more reliable, although advancing memory mechanisms for video generation may also achieve this within the video itself.

Abstract (from arXiv)

Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-08 UTC

preprint

SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation

Yunheng Liu, Ziqi Cai, Siqi Yang, Yimu Wang, Minggui Teng, Jiaming Tan, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi

Preprint posted: 2026-10-06 · Original record ↗

arxiv
2610.08941
Explicit source versions (1)
  • v1 · 2026-10-06
  1. Preprint available

    Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.