- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper tackles the well-known problems of tracking drift and memory contamination in densely sampled surgical video object segmentation. The proposed method, SurgATR, identifies unreliable propagation based on recent trajectories and, when such unreliability is detected, invokes a vision-language agent to perform spatial re-anchoring and memory rollback. Recovered hypotheses are then consolidated through reverse-trajectory validation and a global Viterbi selection. In addition, the authors introduce SurgDVOS, a new dataset comprising 558 clips from 50 procedures, with 330,246 annotated frames and 869,922 instance masks. The evaluation covers both mask-guided and referring segmentation settings on surgical as well as natural videos.
Critical strengths
The dataset could be of lasting value to the community. SurgDVOS provides dense instance trajectories that capture challenging phenomena such as occlusion, reappearance, fast motion, and distractors with similar-looking instruments, while supporting two distinct target-specification settings. Beyond serving as a conventional benchmark for ranking segmentation methods, these annotations could facilitate more systematic research on failure detection and recovery.
The joint design of spatial correction and memory recovery is well motivated and is supported by solid component-level evidence. As shown in Table 6, spatial recovery, memory recovery, and their combination achieve 63.8, 63.4, and 65.0 J&F, respectively, while cycle validation further improves performance to 65.6. These results convincingly demonstrate the complementarity of the two intervention points.
Critical weaknesses
The algorithmic advance over the closest memory-management and multi-hypothesis approaches is not established. SAM2Long [1] already maintains independent memory pathways and performs video-level selection to mitigate error accumulation. SENTRY [2] uses candidate backtracking and cycle-consistent matching to control memory contamination, while Refer-Agent [3] provides a reflection-driven agentic correction framework. Consequently, multiple hypotheses, reflection, cycle validation, and global selection do not individually establish novelty. Retrospective rollback is meaningfully different from preventive memory filtering, but direct comparisons are needed to determine whether it provides a substantial new capability. ReSurgSAM2 [4] is cited but absent from the main comparisons. [1] Ding S, Qian R, Dong X, et al. Sam2long: Enhancing sam 2 for long video segmentation with a training-free memory tree[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 13614-13624. [2] Alansari M, Michael Y, AlMarzouqi H, et al. Sentry: Sam2-enhanced neighbor-aware and temporally reasoned memory for visual tracking[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 599-619. [3] Jiang H, Liang T, Zheng W S, et al. Refer-agent: A collaborative multi-agent system with reasoning and reflection for referring video object segmentation[J]. arXiv preprint arXiv:2602.03595, 2026. [4] Liu H, Gao M, Luo X, et al. Resurgsam2: Referring segment anything in surgical video via credible long-term tracking[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2025: 435-445.
It remains unclear whether cycle validation actually tests identity consistency. The current criterion primarily evaluates reverse-trajectory reliability through area, overlap, motion, and presence cues. Consistently tracking a different but similar instrument may also produce plausible temporal evolution. Relative to SENTRY’s explicit matching against target and distractor histories, the proposed validation needs a clearer correspondence to the established target identity. Aggregate J&F improvements and removing cycle validation do not sufficiently distinguish identity correction from generic candidate-quality filtering.
Questions for authors
How does SurgATR compare against SAM2Long, SENTRY, and ReSurgSAM2 when all methods share the same backbone, initialization, and inference budget? It would also help to know which component contributes most to the reported gains—is it the recovery-candidate generation, the retrospective rollback, or the final selection step? An ablation or attribution analysis here would clarify this. With the re-anchoring prompts held fixed, how do the alternative memory strategies perform—e.g., simply retaining the memory, clearing it, selecting reliable historical frames, versus rolling back to a reliable anchor? Such a comparison would directly demonstrate whether the rollback mechanism is independently necessary. In the cycle validation, is the reverse endpoint explicitly compared against the anchor target in terms of mask or identity features? Additionally, I would suggest reporting identity switches, recovery success rate, and recovery delay separately, rather than relying solely on the aggregate J&F metric, as this would give a clearer picture of the recovery behavior. The authors should clarify the training data and action supervision used for the recovery agent, as well as whether the SurgDVOS split is performed at the patient or procedure level to avoid data leakage. It is also unclear what data the agent checkpoint evaluated on natural videos was trained on—please specify the training sources.