Driving · 2026-10-06

Ultrasound Operator Guidance Using World Modeling and Retrieval Based Action Planning

Noortje I. P. Schueler, Hans van Gorp, Ruud J. G. van Sloun

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose retrieval-induced latent transition models for ultrasound operator guidance. Observations are encoded with a V-JEPA 2.1 backbone into a latent space, and guidance is generated by retrieving physically recorded transitions from a reference database and performing receding-horizon planning from the live ultrasound feed alone (no pose sensing at deployment). On carotid ultrasound, they report 86% target-view reachability in retrospective closed-loop episodes vs 52% and 43% baselines.

Why it matters

The work targets operator-dependent ultrasound acquisition by turning guidance into multi-step planning in a learned latent world model. It avoids learning a parametric transition model by using nonparametric, database-retrieved transitions as the forward model, enabling receding-horizon planning without probe tracking hardware at deployment.

Method

Encode live ultrasound (four-frame clip) with a fine-tuned V-JEPA 2.1 backbone into a latent state space where anatomically related views are close; compute view probabilities for goal specification.
Build a transition database from tracked freehand acquisitions (probe poses only needed for training): each latent state stores recorded successor states and 6-DoF probe displacements.
At inference, plan with receding-horizon search by retrieving nearest latent transitions and scoring candidate trajectories for reaching the target view; issue only the first action and replan each new observation.

Limitation

Retrospective evaluation relies on recorded acquisitions, requiring each predicted probe pose to be projected onto the nearest recorded frame, introducing a coupling between the executed action and the available data.

Abstract (from arXiv)

Ultrasound is widely used, but acquisition quality is heavily dependent on the operator's knowledge and expertise. With demand for examinations outpacing the supply of trained sonographers, operator-guidance systems aim to close this gap by instructing a less trained user how to move the probe toward a target view. In this paper, we propose a retrieval-induced latent transition model for ultrasound acquisition dynamics, formulating ultrasound operator guidance as multi-step planning and retrieval in a world model. Using a V-JEPA 2.1 backbone, observations are first encoded into a latent space where anatomically related views lie close together. We then retrieve similar views from a reference database containing encoded latent states and corresponding probe positions and orientations. Rather than learning a parametric transition function, we directly use physically executed transitions from the database to establish our nonparametric, retrieval-induced transition model that supports receding-horizon planning. At deployment, guidance is generated from the live ultrasound image feed alone, without any probe tracking hardware. Applied to carotid ultrasound, the proposed planner reaches the target view in 86% of retrospective closed-loop episodes, versus 52% and 43% for representative baselines, outperforming both on every target view, including the challenging longitudinal internal and external carotid artery views. A prospective feasibility study on unseen volunteers, run in real time on a CPU using distillation, reaches 83% target-view reachability. Because planning is driven by proximity to any encodable goal latent, the same world model can navigate back to any previously acquired, patient-specific frame, supporting reproducible longitudinal imaging for e.g. perioperative or follow-up monitoring.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-05 · Indexed by Pub-AI: 2026-10-06 UTC

preprint