Latent Dynamics · 2026-11-10

WorldMind: Test-Time Scaling for Visual Spatial Reasoning through Latent World Simulation

Haoming Wang, Yongjian Fu, Ju Ren, Wei Gao

arXiv:2611.10151PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

WorldMind is a test-time scaling framework for visual spatial reasoning that performs iterative “latent world simulation.” A VLM selects spatial transformations; a world model executes them and returns aligned intermediate latent features directly to the VLM, avoiding repeated decode/re-encode through RGB. The authors train latent selection/alignment, then fine-tune with simulation-trajectory supervision and process-verifiable reinforcement learning using step-wise rewards.

Why it matters

The paper addresses deficiencies of VLMs in perspective-taking tasks requiring mental spatial transformation by using world-model latents rather than synthesized images. By exposing aligned latent features to a VLM and rewarding only valid action/latent transitions, the method aims to scale inference-time computation through active simulation and improve spatial reasoning success across multiple benchmarks.

Method

  • VLM selects camera actions from a query/observation; a pretrained world model executes them and returns latent states as features via a learnable adapter, appended to the VLM context for iterative reasoning.
  • Latent selection and alignment: identify a fixed subset of world-model latent representations across diffusion layers/timesteps, then align them to a frozen VLM with answer supervision and modality dropout.
  • Two-stage training for multi-step interaction: simulation-trajectory supervised fine-tuning on synthetic reasoning trajectories, followed by process-verifiable reinforcement learning with progress reward gated by action validity and latent/state validity.

Limitation

The authors state that the world model’s synthesis capability is bounded and may become unreliable when extrapolating far beyond the region covered by the observed views.

Abstract (from arXiv)

Vision-Language Models (VLMs) have continuous deficiency in spatial reasoning tasks that require \emph{mental spatial transformation}, such as reasoning under hypothetical changes in viewpoint or position, known as perspective-taking tasks. Existing approaches either scale linguistic reasoning or augment VLMs with static geometric representations, but neither directly enables active transformation of the represented scene. World models offer this capability, yet existing systems typically communicate through synthesized RGB images, introducing a latent-to-pixel-to-latent bottleneck. We introduce \emph{WorldMind}, a framework for test-time scaling of spatial reasoning through latent world simulation. Given a query and observation, the VLM selects useful spatial transformations, while a world model executes them and exposes aligned intermediate latent features directly to the VLM. The resulting latent evidence is incorporated into subsequent reasoning steps, enabling iterative spatial simulation without repeatedly decoding imagined states into pixels. To optimize this interaction, WorldMind further employs process-verifiable reinforcement learning that rewards reasoning progress only when the associated action and latent transition are valid. WorldMind thus scales spatial reasoning by allocating additional inference-time computation to active world simulation rather than generating more language or pixels. Across VSI-Bench, MMSI-Bench, and MindCube-Tiny, WorldMind achieves an average improvement of 16.0% over the base VLM and 8.3% over the strongest RGB-based world simulation.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

WorldMind: Test-Time Scaling for Visual Spatial Reasoning through Latent World Simulation

Haoming Wang, Yongjian Fu, Ju Ren, Wei Gao

First announced by arXiv: 2026-11-10 · Original record ↗

arxiv
2611.10151
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.10151v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Vision-Language Models (VLMs) have continuous deficiency in spatial reasoning tasks that require \emph{mental spatial transformation}, such as reasoning under hypothetical changes in viewpoint or position, known as perspective-taking tasks. Existing approaches either scale linguistic reasoning or augment VLMs with static geometric representations, but neither directly enables active transformation of the represented scene. World models offer this capability, yet existing systems typically communicate through synthesized RGB images, introducing a latent-to-pixel-to-latent bottleneck. We introduce \emph{WorldMind}, a framework for test-time scaling of spatial reasoning through latent world simulation. Given a query and observation, the VLM selects useful spatial transformations, while a world model executes them and exposes aligned intermediate latent features directly to the VLM. The resulting latent evidence is incorporated into subsequent reasoning steps, enabling iterative spatial simulation without repeatedly decoding imagined states into pixels. To optimize this interaction, WorldMind further employs process-verifiable reinforcement learning that rewards reasoning progress only when the associated action and latent transition are valid. WorldMind thus scales spatial reasoning by allocating additional inference-time computation to active world simulation rather than generating more language or pixels. Across VSI-Bench, MMSI-Bench, and MindCube-Tiny, WorldMind achieves an average improvement of 16.0% over the base VLM and 8.3% over the strongest RGB-based world simulation.