Latent Dynamics · 2026-10-14
SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning
Chenlong Wei, Hanling Wang, Mingjin Lou, Yufan Shi, Yandong Sun, Sheldon Yu, Xiaohui Zhu, Xiaobo Jin, Yiwei Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SonicWorld targets dynamic source-specific spatial audio reasoning by maintaining persistent source states from FOA recordings, then projecting source-state sequences into an Omni LLM for spatial question answering. The paper introduces Sonic-Bench (2.8K FOA clips, 28.8K QA pairs, 480 simulated indoor scenes). On Sonic-Bench, SonicWorld improves direction accuracy and reduces azimuth MAE versus Spatial-Omni; it also reports zero-shot gains on Spatial LibriSpeech.
Why it matters
The authors address linking changing acoustic observations to the correct source over time for localization, distance, and motion reasoning. By using recurrent, source-centric latent states (with learned transitions and recurrent updates) and integrating physical, learned acoustic, and semantic evidence into an Omni LLM, the approach aims to support source- and time-specific spatial question answering under overlap and motion.
Method
- Introduce Sonic-Bench for source- and time-conditioned spatial audio reasoning using FOA clips with time-aligned source trajectories and target-specific QA pairs.
- Propose SonicWorld: learn persistent source states via recurrent slot-based estimation with state transitions, trajectory supervision, and masked-state prediction; decode geometric attributes for source slots.
- Project source-state sequences into tokens for an Omni LLM; train with two-stage fine-tuning where the source-state model, native audio encoder, and original LLM weights remain frozen.
Limitation
Simulated indoor environments; stationary receiver; and offline inference due to a bidirectional temporal module that processes the complete audio clip rather than streaming perception.
Abstract (from arXiv)
Dynamic spatial audio reasoning requires a model to associate changing acoustic observations with the correct source and recover its spatial state at a queried time or interval, a challenge intensified by overlapping speech and source motion. To study this problem, we introduce Sonic-Bench, comprising 2.8K first-order Ambisonics clips and 28.8K target-specific question-answer pairs across 480 simulated indoor scenes. The benchmark covers localization, distance estimation, and motion reasoning in single- and dual-source settings. We further propose SonicWorld, which integrates physical, learned acoustic, and semantic features into persistent source states through learned transitions and recurrent updates, and projects them into an Omni LLM for spatial reasoning. Compared with Spatial-Omni on the Sonic-Bench test split, SonicWorld improves direction accuracy from 31.98% to 34.69% and reduces azimuth error from 57.22{\deg} to 48.01{\deg}. In zero-shot transfer to Spatial LibriSpeech, SonicWorld retains strong localization performance, achieving 73.98% direction accuracy and 19.29{\deg} azimuth error, compared with 27.14% and 70.74{\deg} for Spatial-Omni.
Related papers
- Audible World Models: Spatially Aware Sound Generation for 3D Worlds
- HelixWorld: A Real-time Interactive Audio-Visual World Model
- WorldSonus: Bringing Sound to Worlds
- Uruqi: Learning Spatial Cognition from Visual Experience
- Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC
SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning
Preprint posted: 2026-10-12 · Original record ↗
- arxiv
- 2610.17131
Explicit source versions (1)
- v3 · 2026-10-15
- Preprint available
Source date: 2026-10-12 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v3
Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Dynamic spatial audio reasoning requires a model to associate changing acoustic observations with the correct source and recover its spatial state at a queried time or interval, a challenge intensified by overlapping speech and source motion. To study this problem, we introduce Sonic-Bench, comprising 2.8K first-order Ambisonics clips and 28.8K target-specific question-answer pairs across 480 simulated indoor scenes. The benchmark covers localization, distance estimation, and motion reasoning in single- and dual-source settings. We further propose SonicWorld, which integrates physical, learned acoustic, and semantic features into persistent source states through learned transitions and recurrent updates, and projects them into an Omni LLM for spatial reasoning. Compared with Spatial-Omni on the Sonic-Bench test split, SonicWorld improves direction accuracy from 31.98% to 34.69% and reduces azimuth error from 57.22° to 48.01°. In zero-shot transfer to Spatial LibriSpeech, SonicWorld retains strong localization performance, achieving 73.98% direction accuracy and 19.29° azimuth error, compared with 27.14% and 70.74° for Spatial-Omni.