Driving · 2026-11-09

FATHOM: World-Model-Guided Diffusion Action-Trajectory Selection for Sparse-Marker AUV Tracking

Xiaorui Wang, Zeyu Sha, Mingyang Yang, David Chen Wang, Feitian Zhang

arXiv:2611.08898PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

FATHOM is a world-model-guided diffusion action-trajectory selection framework for sparse-marker AUV tracking under partial marker visibility. It encodes sparse LED observations with SSME, forms a predictive tracking belief with MT-RSSM using inertial data and action history, uses a diffusion action prior to generate multi-thruster candidate trajectories, predicts their tracking consequences with the learned latent dynamics, and selects the best candidate via receding-horizon objective in closed loop for real-time onboard operation without motion-capture feedback.

Why it matters

The authors target relative tracking for cooperative AUVs when active markers are deliberately sparse and can be incomplete or corrupted by occlusion, reflections, and detection noise. By combining a recurrent predictive belief (for hidden-state inference and multi-step prediction) with diffusion-based trajectory candidates (for multimodal actions) and world-model predictive selection (for task-specific choice), FATHOM aims to maintain robust tracking in real AUV experiments while running onboard in real time without motion-capture feedback.

Method

  • Structured Sparse-Marker Encoder (SSME) converts unordered, variable-cardinality LED detections into a structured representation while explicitly handling missing red-anchor cases and preserving permutation invariance among blue detections.
  • Multimodal Task-Supervised Recurrent State-Space Model (MT-RSSM) integrates sparse visual features, IMU measurements, previous actions, and temporal dynamics to form a predictive tracking belief for both inference and counterfactual prediction.
  • Diffusion Action Prior (DAP) generates multiple expert-like multi-thruster trajectories, MT-RSSM predicts future tracking consequences, and World-Model Predictive Selector (WMPS) chooses the lowest-cost candidate under a receding-horizon objective before executing only the first command.
Abstract (from arXiv)

Reliable relative tracking is fundamental to cooperative autonomous underwater vehicles (AUVs), yet remains challenging under nonlinear vehicle dynamics and degraded underwater visual conditions. Active light markers improve robustness to appearance variation, but provide deliberately sparse observations that can become incomplete or corrupted by occlusion, reflections, and detection noise during dynamic motion. To address the resulting partial-observation control problem, this paper proposes FATHOM, a world-model-guided diffusion action-trajectory selection framework for sparse-marker AUV tracking. FATHOM first integrates structured marker observations, inertial measurements, action history, and temporal dynamics into a predictive tracking belief using a multimodal task-supervised recurrent state-space model. A diffusion action prior then generates multiple expert-like multi-thruster trajectories, whose future tracking consequences are predicted through the learned latent dynamics. Rather than directly executing a generated trajectory, FATHOM selects the most favorable candidate under an explicit receding-horizon objective and continually updates the decision from onboard observations. Extensive offline analyses verify the learned predictive belief, multi-step world-model prediction, diffusion trajectory generation, and predictive selection. More importantly, comprehensive real-AUV experiments, including strong pose-based comparisons and component ablations, demonstrate robust autonomous tracking across nominal and challenging motions with naturally occurring partial marker visibility, while maintaining real-time onboard operation without motion-capture feedback.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-06 · Indexed by Pub-AI: 2026-11-09 UTC

preprint

FATHOM: World-Model-Guided Diffusion Action-Trajectory Selection for Sparse-Marker AUV Tracking

Xiaorui Wang, Zeyu Sha, Mingyang Yang, David Chen Wang, Feitian Zhang

Preprint posted: 2026-11-06 · Original record ↗

arxiv
2611.08898
Explicit source versions (1)
  • v1 · 2026-11-06
  1. Preprint available

    Source date: 2026-11-06 · First detected: 2026-11-09 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Reliable relative tracking is fundamental to cooperative autonomous underwater vehicles (AUVs), yet remains challenging under nonlinear vehicle dynamics and degraded underwater visual conditions. Active light markers improve robustness to appearance variation, but provide deliberately sparse observations that can become incomplete or corrupted by occlusion, reflections, and detection noise during dynamic motion. To address the resulting partial-observation control problem, this paper proposes FATHOM, a world-model-guided diffusion action-trajectory selection framework for sparse-marker AUV tracking. FATHOM first integrates structured marker observations, inertial measurements, action history, and temporal dynamics into a predictive tracking belief using a multimodal task-supervised recurrent state-space model. A diffusion action prior then generates multiple expert-like multi-thruster trajectories, whose future tracking consequences are predicted through the learned latent dynamics. Rather than directly executing a generated trajectory, FATHOM selects the most favorable candidate under an explicit receding-horizon objective and continually updates the decision from onboard observations. Extensive offline analyses verify the learned predictive belief, multi-step world-model prediction, diffusion trajectory generation, and predictive selection. More importantly, comprehensive real-AUV experiments, including strong pose-based comparisons and component ablations, demonstrate robust autonomous tracking across nominal and challenging motions with naturally occurring partial marker visibility, while maintaining real-time onboard operation without motion-capture feedback.