- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents a recipe for training an interactive ultrasound world model with a large-scale clinical dataset without pose data. Specifically, a semantic mask-conditioned US video generation model is first pretrained using the large-scale untracked US segmentation dataset. Then a pose-conditioned world model is distilled from the video generator based on synthetic pose-to-semantic mask maps simulated from segmented MR images. The main novelty is utilizing the large-scale untracked US data to train a pose-conditioned world model through self-distillation. This has practical impact, as collecting tracked US data with synchronized poses is often tedious and time-consuming.
Critical strengths
- The paper is well-organized and easy to follow, with sufficient visual demonstrations
- The proposed method is novel and practical, removing the practical burden of collecting tracked ultrasound data
- The experimental validation is solid, with sufficient quantitative and qualitative results, as well as end-to-end robotic ultrasound planning evaluation.
Critical weaknesses
- Specific training and implementation details require further clarification
- How many clinical ultrasound image frames and MRIs in total are used for training?
- How long does generating each frame take? Can the world model achieve real-time inference for robotic applications?
- What is the per-step motion scale?
- The robotic ultrasound experiment initializes the probe already close to the target (16mm). In practice, the probe is often initialized further from the target, but it is unclear whether the proposed world model still works well. The effectiveness of the planning is also constrained by the limited prediction horizon length of the world model.
Questions for authors
- See weakness 1 for questions on implementation details
- The target application of the proposed world model is visual servoing. Then what are the benefits of the proposed pipeline compared to training a world model in the mask/anatomy space? Specifically, an anatomy world model can be trained to predict the segmentation masks given the motion sequence, while initial and goal US frames can both be converted to segmentation masks with a US segmentation model. Such an anatomic world model can be trained with various segmented 3D CT/MR datasets. And the segmentation model can be trained with the clinical US dataset like CAMUS. This alternative approach might be more reasonable in practice, as the goal positions for robotic ultrasound are often specified as a 2D plane on patient anatomy instead of a US image.
- In the proposed world model, ultrasound generation is conditioned purely on the motion. But the real ultrasound images also depend on anatomical geometry. Do you observe variance in generated US images by your world model given the same motion condition?
- How diverse are the US pose trajectories in the unlabeled dataset for prior learning? Does it have to cover all possible motion trajectories that the world model needs to simulate?
We sincerely thank the reviewer for the positive assessment of our work and the constructive suggestions. We address the concerns below.
[W1, Q1, Q4] Training and Implementation Details
To address these questions, we have added detailed implementation information in the revised manuscript (Appendices E.3 and F.6), including:
[W2] Robotic Planning with Larger Initial Distances
The appropriate scanning range varies across anatomical domains. We have expanded the evaluation to 24 simulated tasks across three domains, including lower-limb tasks with an average initial distance of approximately 43 mm, substantially larger than the previously evaluated 16 mm. Closed-loop replanning enables the model to approach goals beyond a single prediction horizon. For larger-scale navigation, coarse localization using external RGB cameras could also be combined with our world model for fine-grained ultrasound guidance.
[W2] Limited Prediction Horizon
We have additionally tested 49-frame long-horizon predictions. We encourage the reviewer to watch the ▶ Long-Horizon Prediction Videos, which illustrate the model's ability to maintain coherent anatomical evolution over extended trajectories.
Future work could explore integrating 3D anatomical reconstruction with world modeling to support longer-horizon prediction.
[Q2] Why Not an Anatomy World Model?
We agree that an anatomy-only world model is a promising and potentially more lightweight solution for geometry-based planning. However, ultrasound images contain richer information than segmentation masks, including tissue appearance and acoustic characteristics. Our image-space world model aims to support broader downstream applications, such as image-quality assessment, image-based guidance, and policy learning, beyond anatomical alignment alone.
[Q3] Variability Under Identical Motion Conditions
Our world model is conditioned on both the initial ultrasound observation and the prescribed motion, rather than motion alone. Therefore, the same motion applied to different initial observations can produce different anatomical and appearance predictions, reflecting the underlying imaging context.