- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposed a state-space world model for simulated ultrasound-guided robotic drilling. Specifically, a model predicts the kinematic state from the observations, and the states are propagated through a closed-form model or an MLP. The paper analyzes proposed methods for US navigation, US-guided drilling, and their sequential combination. On the methodology side, the pipeline is similar to the existing image-based MPC methods, and the main contributions are unclear. On the presentation side, the paper needs significant improvement.
Critical strengths
- The studied application is challenging but meaningful
- The paper observed a limitation of SonoGym, which is valid.
Critical weaknesses
- The overall presentation is below the ICLR conference standard
- The paper is not organized in a standard and structured way, with nested methods and results, without clear methodology and experiment sections.
- The text is generally hard to follow, with too dense, unstructured information. Some paragraphs are too long, including the abstract and several captions for several figures and tables
- The figure quality requires significant improvement: Figure 1 is unpolished with very small text; Figure 2 has inverted arrow directions
- The bolded text in the tables is misleading. In some tables, the best numbers are highlighted, and in others the proposed method is highlighted.
-
The main contribution of the proposed work is unclear. Especially, predicting state from images and then applying MPC has been investigated in many existing papers, for instance: “Vision-Based High Speed Driving with a Deep Dynamic Observer” and “Robust-Sub-Gaussian Model Predictive Control for Safe Ultrasound-Image-Guided Robotic Spinal Surgery”. Here, the second paper even focuses on the same application problem and provides more solid theoretical guarantees and analysis.
-
Applying the proposed method only to one specific simulated task limits the impact of the work
Questions for authors
- What is the main contribution of this work compared to “Robust-Sub-Gaussian Model Predictive Control for Safe Ultrasound-Image-Guided Robotic Spinal Surgery”?
- Why is the closed-form dynamic model wrong in line 205? Is it because of the tracking error from the robot, or the pose estimation uncertainty from the US images? While existing works focus on quantifying these uncertainties, the benefits of learning the pose propagation with an MLP are unclear.
- What is the conclusion of the paper? It seems the analytic propagation still achieves higher accuracy compared to MLP-based state propagation in most of the cases. Does it mean that model-based propagation is still preferred in practice?
We thank the reviewer for a careful review; we agree with much of it. Unless stated, numbers are 4 seeds × 100 environments (400 episodes) for the arm with the learned (MLP) propagator, the paper's world model. SAFE (Eq. 2) is the fraction of episodes never unsafe at any step; SAFE-state is SonoGym's own measure, the fraction of steps that are safe, as its baselines report. ins/side/rot: insertion, lateral (mm) and angle (°) errors; succ: SUCCESS.
Corrections to the submitted numbers (LB-IDT, n = 20 → 400)
Navigation: 8.3 mm / 5.2° submitted, 1.1 mm / 8.3° now. The claim "costs nothing" rested on the two equal 0.20s and does not survive: the repair recovers 90% of the teleport's SAFE (p = 0.033) and 82% of its success (p = 0.030), so a gap remains, and we withdraw the claim.
Contribution. We agree that image-to-state-then-MPC is established (Vision-Based High-Speed Driving with a Deep Dynamic Observer is an instance), and we do not claim it as our contribution. The contribution is what happens when two such controllers are composed. SonoGym places the drilling probe from ground truth, with at most 5 mm of tangential randomisation. A navigated probe hands off 7.85 mm and 32.4° of azimuth error, a channel that randomisation never covers:
Drilling tolerates only 10–15° of azimuth error (submitted §6), so larger errors lead to catastrophic failure. The navigator's readout saturates, and retraining it where it is used recovers most of the gap. On four more vertebrae, rerun from scratch, the pattern holds: the unrepaired composition is safe in at most 0.3% of episodes, and the repair recovers 88–100% of the teleport's SAFE. Full per-vertebra tables will be in the revised appendix; we are happy to share them here if helpful.
Presentation. This is fair. In the revised manuscript, we will follow standard paper-writing practice and address the concerns raised.
Q1. Robust Sub-Gaussian MPC. We will cite it. It models image-based estimation error as sub-Gaussian with bounded mean and gives closed-loop guarantees for MPC on linear dynamics; since it covers biased estimates, we do not claim our failure lies outside its noise model. The works are complementary: its guarantee is conditional on error bounds estimated in the nominal setup, where probe placement is not part of the problem. We measure what those bounds must cover once a learned navigator places the probe: a near-constant 32.4° azimuth error against a 10–15° tolerance. A bias set that wide would be very conservative, and that paper itself notes that a fully robust treatment can be infeasible. After our repair, the handoff error is 5.5°.
Q2. Why is the closed form wrong? The cause is the plant: the drill controller realises about 9% of the commanded translation per step and damps lateral and angular motion about 5× in the corridor. The learned propagator, fit to achieved poses, absorbs this. At h = 1, its depth-forecast error is 4.0 mm, nearly all of it perception; the closed form's 13.2 mm is 3.9 mm perception and 9.2 mm propagator. Because the closed form's error is a single scalar gain and the controller re-plans every step, the error rescales commands rather than misdirecting them, so the closed form controls better despite forecasting worse (41 vs 5.5 mm at h = 10): objective mismatch (submitted §7).
Q3. Is model-based propagation preferred? Where exact kinematics exist, yes; we include that arm to measure the learned propagator against it:
The difference in SAFE is not significant (p = 0.28); the difference is in accuracy. The learned propagator also transfers to new vertebrae without refitting (refitting per vertebra buys nothing consistent; ablation in the revised appendix), and every composition result holds with it.
One simulated task. Agreed. We would only add that the setting combines two SonoGym tasks, now tested in all four of its settings and on five vertebrae. We make no clinical claim.