- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents EndoWAM, a world-action model for generalizable robotic endoscopic navigation. The proposed framework leverages a pretrained video world model to extract predictive representations from a single denoising pass, which are used to condition a discrete action expert and an auxiliary future-grounding module. The grounding module reconstructs latent representations of target regions in future observations during training and is removed at inference to improve efficiency. The authors also introduce EndoMotion, a multi-procedure robotic endoscopy dataset with target annotations and viewpoint augmentation. Closed-loop experiments on physical anatomical phantoms demonstrate substantial improvements over existing imitation-learning and vision-language-action baselines, as well as alternative grounding strategies.
Critical strengths
The paper provides extensive closed-loop evaluations on a physical robotic endoscope across three anatomical phantom settings, including comparisons with established baselines, alternative grounding strategies, and ablation studies. The consistent performance improvements provide meaningful empirical support for the effectiveness of the proposed approach in robotic endoscopic navigation.
Critical weaknesses
- The action labels are generated from tracked target observations using a hand-designed centering and advancement rule. However, the paper does not evaluate this rule combined with the same fine-tuned Grounding DINO in closed-loop control. Without this direct visual-servoing baseline, it remains unclear how much the learned policy improves upon its rule-based supervision.
- The contributions of future grounding and predictive dynamics modeling are not sufficiently isolated. A matched current-frame target reconstruction baseline is missing, making it difficult to establish the benefit of future grounding specifically. The video prediction loss is also not ablated, leaving its independent contribution to the reported performance improvements unclear.
Questions for authors
- How does a direct visual-servoing controller using the same fine-tuned Grounding DINO and the action rules described in Appendix B perform in closed-loop evaluation? This comparison would help clarify the benefit of the learned policy beyond its rule-based action supervision.
- How does the proposed method perform when the Grounding DiT reconstructs current-frame target regions instead of future target regions? A controlled comparison would help establish whether the improvement comes specifically from future grounding rather than target-region reconstruction in general.
- What is the contribution of the video prediction objective to navigation performance? Could the authors provide an ablation without this objective while retaining the same backbone and grounding supervision to distinguish the benefit of predictive dynamics learning from that of pretrained visual representations?
- In Eq. 1, future visual dynamics are modeled without explicitly conditioning on robot actions, although the executed actions directly influence subsequent observations. How does this formulation capture action-dependent environmental dynamics, and what justifies the joint world-action modeling interpretation in Eq. 2?
- Future grounding reconstructs cropped target regions while excluding frames without valid target detections from its supervision. How does this objective encourage the shared representation to capture the spatial evolution of targets and support navigation under occlusion? Could the authors provide more direct evidence of these capabilities?
Thank you addressing my concerns
Thank you for the response and the additional hardware experiments, which addressed most of my concerns, I still have two points below for your further response, I will finalise my rating after your reply.