Driving · 2026-11-03

Xiaomi Auto World Model 2.0: Causal World Models for Real-World Driving and Embodied Intelligence

Zhenxin Zhu, Zhijian Huang, Cheng Chi, Yuru Wang, Jianhua Wu, Jiayi Guan, Yonggang Li, Mingfei Tu, JiaKui Hu, Tao Xu, Xianqi Wang, Hao Li, Yuechuan Pu, Qizeng Jia, Ruiling Yang, Hongxiao Yu, Xuhan Sheng, Jian Mao, Ruiqiang Zhang, Yu Zhu, Guang Chen, Hangjun Ye, Bing Wang

arXiv:2611.01677PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Xiaomi Auto World Model 2.0 is a streaming causal world model for Physical AI. It uses View-Temporal Attention (VTA) with multi-view rotary position encoding and a WorldRec 2.0 module that reconstructs static and dynamic 3D Gaussians in a single forward pass, enabling long-horizon view-consistent generation. It incorporates Gated Linear Attention (GLA) for efficient streaming inference and trains with a physical-interaction data pipeline covering long-tailed events.

Why it matters

The authors target two limitations of prior world models for Physical AI: (1) poor coverage of rare, safety-critical physical interactions and (2) limited support for heterogeneous observation/conditioning across variable view configurations and downstream control applications. The proposed causal, view-consistent, streaming architecture is evaluated across generation, simulation, policy learning, and WAM-style action generation.

Method

  • Architecture: View-Temporal Attention (VTA) with Multi-view Rotary Position Encoding (RoPE) for cross-view spatial dependencies and temporal dynamics; heterogeneous conditioning fused via VAE-based latent feature concatenation.
  • WorldRec 2.0: reconstructs static and dynamic 3D Gaussians from multi-view, multi-temporal observations in a single forward pass and uses renderings as explicit scene context for long-horizon generation.
  • Efficiency: Gated Linear Attention (GLA) in the temporal layers of VTA for streaming inference, reducing computation and KV-cache memory during multi-view rollouts; distillation-based post-training for stable long sequences.
Abstract (from arXiv)

We present Xiaomi Auto World Model 2.0, a unified streaming causal world model for Physical AI. It models and generates causal physical evolution in the real world, and unifies heterogeneous conditioning inputs for view-consistent generation. The architecture integrates View-Temporal Attention (VTA) to capture cross-view spatial dependencies and temporal dynamics, together with Multi-view Rotary Position Encoding (RoPE) to encode positional relationships across views and time. WorldRec 2.0 reconstructs static and dynamic 3D Gaussians from multi-view, multi-temporal observations in a single forward pass and uses their renderings as explicit scene context to preserve geometry and appearance during long-horizon generation. Conditioning inputs are fused through VAE latent concatenation in a shared representation space. We incorporate Gated Linear Attention (GLA) into the temporal layers of VTA for efficient streaming inference, reducing computation and KV-cache memory during multi-view rollouts. A physical-interaction data pipeline covers diverse interactions and long-tailed events. From Pre-train onward, these data are combined with large-scale videos and heterogeneous visual modalities to learn visual, geometric, and dynamic priors. During Mid-train, training across diverse view configurations further improves the model's ability to capture how actions and object states shape subsequent scene evolution. After distillation-based Post-train, we apply the model to closed-loop simulation, closed-loop training, and World Action Model (WAM) applications. These applications generate physically consistent future states and support world-state prediction, policy learning, and action generation for Physical AI.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC

preprint

Xiaomi Auto World Model 2.0: Causal World Models for Real-World Driving and Embodied Intelligence

Zhenxin Zhu, Zhijian Huang, Cheng Chi, Yuru Wang, Jianhua Wu, Jiayi Guan, Yonggang Li, Mingfei Tu, JiaKui Hu, Tao Xu, Xianqi Wang, Hao Li, Yuechuan Pu, Qizeng Jia, Ruiling Yang, Hongxiao Yu, Xuhan Sheng, Jian Mao, Ruiqiang Zhang, Yu Zhu, Guang Chen, Hangjun Ye, Bing Wang

First announced by arXiv: 2026-11-03 · Original record ↗

arxiv
2611.01677
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.01677v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

We present Xiaomi Auto World Model 2.0, a unified streaming causal world model for Physical AI. It models and generates causal physical evolution in the real world, and unifies heterogeneous conditioning inputs for view-consistent generation. The architecture integrates View-Temporal Attention (VTA) to capture cross-view spatial dependencies and temporal dynamics, together with Multi-view Rotary Position Encoding (RoPE) to encode positional relationships across views and time. WorldRec 2.0 reconstructs static and dynamic 3D Gaussians from multi-view, multi-temporal observations in a single forward pass and uses their renderings as explicit scene context to preserve geometry and appearance during long-horizon generation. Conditioning inputs are fused through VAE latent concatenation in a shared representation space. We incorporate Gated Linear Attention (GLA) into the temporal layers of VTA for efficient streaming inference, reducing computation and KV-cache memory during multi-view rollouts. A physical-interaction data pipeline covers diverse interactions and long-tailed events. From Pre-train onward, these data are combined with large-scale videos and heterogeneous visual modalities to learn visual, geometric, and dynamic priors. During Mid-train, training across diverse view configurations further improves the model's ability to capture how actions and object states shape subsequent scene evolution. After distillation-based Post-train, we apply the model to closed-loop simulation, closed-loop training, and World Action Model (WAM) applications. These applications generate physically consistent future states and support world-state prediction, policy learning, and action generation for Physical AI.