- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This work presented a pose-controllable ultrasound world model. The training includes 3 stages: i) pretraining with large-scale US videos without pose annotation, ii) post-training with small-scale tracked US data, and iii) self-force distillation to ensure causality. The claimed contributions include: i) enabling synthetic realistic robotic ultrasound data collection, ii) leveraging large-scale untracked US data, and iii) preserving data quality for causal autoregressive generation.
Critical strengths
- The proposed pose-controllable ultrasound world model is an interesting approach, particularly given its potential to leverage large-scale untracked ultrasound data.
- Preserving data quality during autoregressive generation is a meaningful practical contribution.
- The paper presents a broad set of experiments to validate the proposed method and its effectiveness.
Critical weaknesses
- An important line of prior work is not discussed, which may weaken the paper's claimed contributions. For example, SonoGym explores ultrasound simulation through a pipeline combining 3D CT data, pose-based 2D CT slicing, and learning-based CT-to-ultrasound generation. This approach addresses a closely related problem: generating realistic, pose-controllable ultrasound images. It may also offer advantages in computational efficiency and geometric consistency. The paper should more clearly articulate the contributions beyond this established line of work.
- The model is post-trained on a relatively small subset of EM-tracked ultrasound data. While the experiments show its ability to potentially generalize to action trajectories outside the distribution covered by this dataset, the underlying mechanism that allows this remains unclear.
- Although the learnable latent action model appears effective during pretraining, it is unclear whether this representation facilitates or complicates subsequent pose-conditioned post-training. A comparison of post-training results in Section 4.3 would better clarify the benefits of the latent actions.
Questions for authors
- How are the contributions of this work positioned against SonoGym?
- Is it possible to train robotic ultrasound policies with the proposed world model?
- Given the relatively small tracked ultrasound dataset used for post-training, how does the method generalize to action distributions and trajectories not observed during training? What mechanisms, if any, help ensure reliable predictions beyond the tracked data distribution?
- Are the learnable latent action tokens learned during pretraining beneficial for subsequent pose-conditioned post-training?
- What is the maximum sequence length the proposed world model can reliably simulate?
3 comments
Thank you again for the constructive comments. In our response above, we discussed how SonoWorld differs from SonoGym and added a long-horizon evaluation up to 18 s, among other results. We would greatly appreciate any further questions about the distinction from SonoGym or the new results and will address them promptly.
Thank you for the detailed response. Since all my major concerns have been addressed, I will raise my score.
We thank the reviewer for the constructive comments. Please find our responses below.
W1 & Q1: Contributions beyond SonoGym.
We thank the reviewer for highlighting SonoGym (Ao et al., 2025), a closely related work that we will discuss in Section 2. SonoGym provides efficient, geometry-grounded ultrasound simulation by extracting 2D slices from patient-specific 3D CT volumes and segmentation maps, followed by physics-based rendering or CT-to-ultrasound translation. However, it relies on static anatomical models without soft-tissue deformation, and its GAN-based generation does not explicitly enforce temporal consistency, as acknowledged by the authors.
SonoWorld takes a complementary, video-driven approach: (1) No patient-specific CT: SonoWorld learns from ~6.9M unlabeled clinical ultrasound frames and grounds the dynamics to 6-DoF probe actions with limited EM-tracked data, requiring no 3D anatomical model. (2) Clinical appearance and dynamics: SonoWorld learns ultrasound appearance, speckle statistics, and probe-induced image variations directly from real clinical videos rather than CT-based image formation. (3) Causal temporal modeling: SonoWorld generates each observation conditioned on the action and its generated history rather than synthesizing frames independently.
W2 & Q3: Mechanism on generalization to unseen action trajectories.
SonoWorld models action-conditioned local transitions autoregressively rather than memorizing complete trajectories. Actions are relative, egocentric 6-DoF motions in the preceding probe frame (Eqs. 3–4), and at each step the model predicts the next observation conditioned on the generated history and the action increment. Unseen trajectories are thus generated by sequentially composing local increments learned during training. Generalization may degrade when actions or anatomy fall far outside the tracked training distribution.
W3 & Q4: Benefits of latent-action pretraining.
We ran three comparisons: (i) Wan 2.1 without ultrasound pretraining, (ii) ultrasound pretraining without latent actions, and (iii) ultrasound pretraining with latent actions (ours). Causal-inference results are below (SSIM / LPIPS / PSNR).
First, latent-action pretraining consistently improves all three metrics over action-agnostic ultrasound pretraining across all five trajectory settings, confirming that it benefits the final pose-conditioned model. Second, ultrasound-domain pretraining substantially improves SSIM and LPIPS over direct Wan 2.1 initialization across all trajectories, highlighting the importance of adapting the video model to ultrasound appearance and dynamics.
Q2: Can robotic ultrasound policies be trained with SonoWorld?
Section 5.2 is a first step toward policy learning: given a current observation and a target frame, SonoWorld predicts the observation resulting from each candidate probe action and scores it by similarity to the target, correctly identifying the target-directed action in up to 74.8% of trials. Two extensions follow naturally. (1) Sampling-based planning: CEM or MPPI could optimize candidate action sequences through SonoWorld rollouts, execute the first action, and replan. (2) Model-based policy learning: a policy could interact with SonoWorld and learn from generated trajectories using task-specific rewards. We plan to validate full closed-loop policy training and sim-to-real transfer in future work.
Q5: Maximum generation sequence length.
We extended autoregressive generation to 18 s, the longest horizon our validation data support, on nine held-out EM-tracked sequences.
With our selective-eviction KV cache, quality remains stable from 3 s to 18 s (PSNR −0.17 dB, SSIM unchanged, LPIPS +0.006), whereas a local sliding window under the same memory budget degrades steadily and is worse at every horizon beyond the training length. SonoWorld thus reliably generates ultrasound video up to 18 s, which we attribute to selectively retaining attention-salient early frames.