- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies fine-grained surgical instrument segmentation from category-name-only (“weak-text”) queries. The proposed Discover-and-Verify (DaV) framework first obtains candidate boxes from GroundingDINO. Its Confuser-aware Prompt Discriminative Grounding (CPDG) module mines local evidence, models target–confuser differences, and reranks candidates. Semantic Consistency Verification (SCV) then assesses the top candidates and rejects unsupported target hypotheses. Accepted boxes are passed to a frozen SAM3 model for mask generation and video propagation. Evaluation on EndoVis17 and EndoVis18 reports segmentation, candidate discrimination, and absent-query rejection metrics; the paper reports mcIoU of 68.34% and 82.39%, respectively.
Critical strengths
- Relevant failure modes. Category-name-only queries are practical in the surgical setting, and the confusion between visually similar instruments and the rejection of absent targets are worthwhile problems to address.
- Diagnostic evaluation. The paper reports TSA/CER and rejection accuracy/AUROC in addition to IoU/mcIoU, and separates CPDG and SCV in ablations. This provides some insight into the roles of reranking and hypothesis verification.
Critical weaknesses
-
Tables 1–2 compare only CLIPSeg (CVPR 2022), CRIS (CVPR 2022), TP-SIS (NeurIPS 2023), SurgicalSAM (AAAI 2024; the manuscript references its 2023 version), and RoSIS (2024). There is no 2025–2026 method in the main results, despite this being an ICLR 2027 submission.
-
The paper itself discusses SAMWISE (CVPR 2025) and ReferDINO (ICCV 2025) but does not compare against either; MPG-SAM 2 (ICCV 2025) is another relevant recent text-driven video segmentation method.
-
The methodological novelty appears incremental rather than a clearly differentiated new principle.
Questions for authors
-
Could the authors provide comparisons with recent text-driven video segmentation methods, such as SAMWISE, ReferDINO, and MPG-SAM 2, under the same category-name-only prompts, dataset splits, and evaluation protocol?
-
What is the core methodological novelty of the Discover-and-Verify framework? Beyond combining local evidence mining, target–confuser contrastive learning, candidate reranking, and learned rejection, do CPDG and SCV introduce a substantively new discrimination or verification mechanism? Please distinguish the methodological advances from adaptations specifically tailored to weak-text surgical instrument segmentation.
-
Could the authors provide finer-grained ablations to quantify the individual contributions of dual-path local evidence mining, confuser modeling, and latent discriminative attribute prototypes in CPDG?
-
What substantively distinguishes SCV’s “hypothesis verification” from standard supervised candidate classification or confidence-based rejection?
6 comments
Table 2: Overall comparison with methods on EndoVis17 under different prompting protocols. The highest result is bolded, while the second highest result is underlined. * denotes frame-wise evaluation under the four-fold cross-validation protocol on EndoVis17.
\hline \text{Prompt} & \text{Method} & \text{IoU} & \text{mcIoU} & \begin{array}{c} \text{Instrument Categories} \\\\ \begin{array}{rrrrrrr} \text{BF} & \text{PF} & \text{LND} & \text{VS} & \text{GR} & \text{MCS} & \text{UP} \end{array} \end{array} \\\\ \hline \text{Class prototype} & \text{SurgicalSAM} & \underline{69.94} & 67.03 & \begin{array}{rrrrrrr} 68.30 & 51.77 & 75.52 & 68.24 & \mathbf{57.63} & 86.95 & 60.80 \end{array} \\\\ \hline \begin{array}{c} \text{} \\\\ \text{} \\\\ \text{Rich Text} \\\\ \text{} \end{array} & \begin{array}{l} \text{RoSIS} \\\\ \text{TP-SIS(448)} \\\\ \text{CLIPSeg} \\\\ \text{CRIS} \end{array} & \begin{array}{r} 57.33 \\\\ \mathbf{76.45} \\\\ 65.02 \\\\ 67.83 \end{array} & \begin{array}{r} 40.65 \\\\ 54.78 \\\\ 33.42 \\\\ 38.21 \end{array} & \begin{array}{rrrrrrr} 67.34 & 39.95 & 55.85 & 45.32 & 5.10 & 26.00 & 44.99 \\\\ \underline{69.57} & \mathbf{68.91} & \mathbf{89.88} & \mathbf{82.60} & 0.00 & 72.53 & 0.00 \\\\ 51.29 & 42.27 & 49.56 & 30.12 & 9.96 & 30.69 & 20.05 \\\\ 54.87 & 50.21 & 68.33 & 50.12 & 0.00 & 43.97 & 0.00 \end{array} \\\\ \hline \begin{array}{c} \text{} \\\\ \text{} \\\\ \text{} \\\\ \text{} \\\\ \text{Weak Text} \\\\ \text{} \\\\ \text{} \\\\ \text{} \\\\ \text{} \end{array} & \begin{array}{l} \text{TP-SIS(448)} \\\\ \text{CLIPSeg} \\\\ \text{CRIS} \\\\ \text{SAMWISE} \\\\ \text{ReferDINO} \\\\ \text{MPG-SAM 2} \\\\ \text{SurgRef} \\\\ \text{\bf DaV* (frame)} \\\\ \text{\bf DaV (video)} \end{array} & \begin{array}{r} 51.97 \\\\ 42.17 \\\\ 46.23 \\\\ 45.45 \\\\ 48.33 \\\\ 50.75 \\\\ 39.73 \\\\ 67.22 \\\\ 67.91 \end{array} & \begin{array}{r} 39.28 \\\\ 29.25 \\\\ 31.41 \\\\ 32.19 \\\\ 42.01 \\\\ 42.90 \\\\ 31.20 \\\\ \underline{67.27} \\\\ \mathbf{68.34} \end{array} & \begin{array}{rrrrrrr} 58.83 & 43.26 & 70.03 & 51.63 & 0.00 & 51.23 & 0.00 \\\\ 46.49 & 33.50 & 42.61 & 28.50 & 5.83 & 29.00 & 18.83 \\\\ 51.85 & 42.30 & 69.29 & 43.71 & 0.00 & 12.70 & 0.00 \\\\ 62.33 & 15.05 & 16.69 & 39.09 & 23.81 & 58.56 & 9.80 \\\\ 52.06 & 50.74 & 68.83 & 36.41 & 35.73 & 64.25 & 36.06 \\\\ 55.11 & 48.16 & 60.80 & 30.21 & 29.74 & 46.44 & 29.84 \\\\ 35.06 & 36.73 & 41.97 & 26.60 & 28.11 & 30.04 & 19.89 \\\\ \underline{69.38} & \underline{55.09} & 67.51 & 66.62 & \underline{55.07} & \underline{94.83} & \underline{62.41} \\\\ \mathbf{70.09} & \mathbf{57.33} & \underline{69.42} & \mathbf{68.57} & 54.52 & \mathbf{94.99} & \mathbf{63.44} \end{array} \\\\ \hline \end{array}$$ ## **W2: Core Methodological Novelty** We thank you for this important question. We respectfully clarify that **our methodological contribution is an evidence-driven selective grounding principle for weak-text fine-grained video object segmentation.** In weak-text fine-grained video object segmentation, category-name-only prompts provide limited cues for distinguishing confusers (i.e., instruments that are both textually and visually similar). As shown in Fig. 1(b), GroundingDINO ranks a confuser above the true target in 51.8% of the evaluated target-confuser pairs. To address this challenge, **our method couples target-confuser-conditioned discriminative grounding with evidence-based candidate verification.** While existing studies have mitigated target ambiguity by enhancing semantic representations, cross-modal alignment, and category-level discrimination, there remains insufficient research on explicitly mining and evaluating the differential support of local evidence for targets versus confusers during candidate localization, and further verifying the validity of specific candidate hypotheses. **DaV addresses this gap through two complementary mechanisms: CPDG jointly models target-attribute support and confuser compatibility to mine and distinguish local candidate evidence, while SCV leverages this evidence to verify, re-select, or reject candidate hypotheses.** The experimental results validate the effectiveness of DaV. By jointly modeling target-attribute support and confuser compatibility, CPDG reduces the CER from 52.69%/51.80% to 33.43%/30.15% on EndoVis17/EndoVis18, indicating improved ranking of true targets over confusing candidates. By verifying multiple candidates using multi-source discriminative evidence, SCV improves the AUROC from 55.91% to 78.43% and the absent-query rejection accuracy from 80.22% to 94.13% on EndoVis17, with consistent gains on EndoVis18 (AUROC: 62.57%→80.43%; rejection accuracy: 84.09%→96.96%). **These results demonstrate the complementary contributions of CPDG and SCV: CPDG improves target-confuser discrimination, while SCV provides evidence-based candidate verification and rejection.**Q2: Core Methodological Novelty of the Discover-and-Verify Framework
We thank the reviewer for raising this important question. We clarify that the methodological contribution of DaV lies in addressing two complementary yet distinct challenges in weak-text fine-grained video object segmentation: (1) distinguishing the true target from visually similar confusers, and (2) determining whether a candidate is sufficiently supported to be accepted rather than merely being the highest-ranked proposal.
(1) Target–confuser discrimination.
Category-name-only prompts provide limited cues for distinguishing visually similar instruments. Consequently, semantic similarity alone may favor a confuser over the true target. As shown in Fig. 1(b), GroundingDINO ranks a confuser above the true target in 51.8% of the evaluated target-confuser pairs. For example, for the PF query, LND receives a higher similarity score in 33% of the evaluated cases (n=269). This exposes a limitation of target-only semantic matching: high text-region similarity does not necessarily provide sufficient evidence of target identity. Existing methods address ambiguity primarily through richer textual prompts, cross-modal alignment, or category-level representation learning, but do not explicitly couple local target-specific attribute support with compatibility against query-related confusers during candidate grounding.
To address this challenge, CPDG introduces target-confuser-conditioned discriminative grounding. Specifically, it learns latent discriminative attribute prototypes conditioned on both target and confuser representations (Eq. 10), which are optimized using visual candidate evidence through the target-confuser contrastive objective (Eq. 19). More importantly, its asymmetric local scoring (Eq. 13) jointly evaluates target-attribute support and confuser compatibility, providing discriminative evidence for candidate reranking (Eq. 14). Thus, CPDG moves beyond target-only similarity matching toward explicit local discrimination based on relative target-confuser evidence. Experimentally, CPDG reduces the Confuser Error Rate (CER) from 52.69% to 33.43% on EndoVis17 and from 51.80% to 30.15% on EndoVis18.
(2) Candidate validity verification.
Even after candidate reranking, selecting the highest-scoring proposal does not guarantee that it is a valid target. This limitation is particularly important when the queried category is absent or when the top-ranked proposal is incorrect. Existing ranking-based methods generally select the best-matched candidate, whereas category-presence-based rejection, as explored by RoSIS, primarily addresses whether a queried category is present rather than whether a specific grounding hypothesis is sufficiently supported.
To address this challenge, SCV introduces evidence-based multi-hypothesis verification. Rather than simply thresholding the original grounding confidence or accepting the highest-ranked proposal, SCV evaluates the Top-3 hypotheses retained by CPDG using fused global, local, semantic, attribute, and discriminative evidence. This separates candidate ranking from candidate validity assessment, enabling the recovery of a valid lower-ranked candidate or abstention when no hypothesis receives sufficient target-specific support. Compared with CPDG-only thresholding, SCV improves Reject Accuracy from 80.22% to 94.13% and AUROC from 55.91% to 78.43% on EndoVis17, with consistent gains on EndoVis18 (Reject Accuracy: 84.09%→96.96%; AUROC: 62.57%→80.43%).
(3) Methodological advances versus task-specific adaptations.
The methodological contributions lie in coupling target-confuser-conditioned local evidence discrimination with evidence-based multi-hypothesis verification, addressing candidate identity and candidate validity, respectively. In contrast, category-name-only prompting, text-similarity-based confuser construction, surgical instrument categories, proposal configurations, and the frozen SAM3 backend are task-specific implementations of these mechanisms. The results demonstrate that the two stages provide complementary improvements in candidate discrimination and selective rejection.
We appreciate the reviewer’s recognition of the relevance of our diagnostic evaluations and the contributions of CPDG and SCV. We have addressed the concerns and revised the manuscript accordingly. Modifications are highlighted in blue.
W1, W2, Q1: Comparisons with recent methods
We thank you for this valuable suggestion. We have substantially expanded the experimental comparisons by including recent methods: SAMWISE (CVPR 2025), ReferDINO (ICCV 2025), MPG-SAM 2 (ICCV 2025), and SurgRef (AAAI 2026). These methods represent recent advances in text-driven video segmentation. To ensure a fair and controlled comparison, we evaluate all reproduced methods under consistent dataset splits, textual inputs, and evaluation protocols.
(1) Dataset and split. We follow the dataset-specific evaluation protocols adopted by TP-SIS. For EndoVis17, we conduct four-fold cross-validation over the eight annotated training sequences, using six sequences for training and two for validation in each fold. For EndoVis18, we follow the corresponding TP-SIS split, using 11 sequences for training and the remaining four sequences for evaluation.
(2) Evaluation protocol. We evaluate predictions on the same annotated video frames using the same ground-truth masks and IoU/mcIoU implementation. We train and recompute all results on both datasets. As shown in the revised Tables 1–2, DaV achieves 67.27% and 80.46% mcIoU on EndoVis17 and EndoVis18, respectively, compared with 32.19%/38.56% for SAMWISE, 42.01%/48.24% for ReferDINO, and 42.90%/61.07% for MPG-SAM 2. This corresponds to improvements of 24.37% and 19.39% over the strongest reproduced baseline, MPG-SAM 2, on the two datasets, respectively.
These comparisons provide a direct assessment of recent RVOS methods under the weak text input. They demonstrate that fine-grained surgical instrument segmentation with weak text prompts requires more than general text-video correspondence. By explicitly discovering target-versus-confuser local evidence and verifying candidate reliability, DaV provides additional benefits in visually ambiguous surgical scenes. We have incorporated these results and the corresponding experimental details into the revised manuscript.
Table 1: Overall comparison with existing methods on EndoVis18 under different prompting protocols. The highest result is bolded, while the second-highest result is underlined.
\hline \text{Prompt} & \text{Method} & \text{IoU} & \text{mcIoU} & \begin{array}{c} \text{Instrument Categories} \\\\ \begin{array}{rrrrrrr} \text{BF} & \text{PF} & \text{LND} & \text{SI} & \text{CA} & \text{MCS} & \text{UP} \end{array} \end{array} \\\\ \hline \text{Class Prototype} & \text{SurgicalSAM} & \underline{80.33} & 58.87 & \begin{array}{rrrrrrr} \underline{83.66} & 65.63 & 58.75 & 54.48 & 39.78 & 88.56 & 21.23 \end{array} \\\\ \hline \begin{array}{c} \text{} \\\\ \text{} \\\\ \text{Rich Text} \\\\ \text{} \end{array} & \begin{array}{l} \text{RoSIS} \\\\ \text{TP-SIS(448)} \\\\ \text{CLIPSeg} \\\\ \text{CRIS} \end{array} & \begin{array}{r} 76.16 \\\\ \mathbf{81.54} \\\\ 69.86 \\\\ 72.29 \end{array} & \begin{array}{r} 44.54 \\\\ 65.48 \\\\ 39.70 \\\\ 46.04 \end{array} & \begin{array}{rrrrrrr} \mathbf{84.11} & 42.73 & 35.26 & 46.36 & 0.00 & 88.10 & 15.22 \\\\ 81.53 & 70.18 & 71.54 & \mathbf{90.58} & 21.46 & 65.57 & 57.51 \\\\ 67.25 & 39.59 & 50.72 & 47.27 & 2.92 & 79.96 & 4.22 \\\\ 73.58 & 58.20 & 47.64 & 72.14 & 4.56 & 45.99 & 20.18 \end{array} \\\\ \hline \begin{array}{c} \text{} \\\\ \text{} \\\\ \text{} \\\\ \text{} \\\\ \text{Weak Text} \\\\ \text{} \\\\ \text{} \\\\ \text{} \\\\ \text{} \end{array} & \begin{array}{l} \text{TP-SIS} \\\\ \text{CLIPSeg} \\\\ \text{CRIS} \\\\ \text{SAMWISE} \\\\ \text{ReferDINO} \\\\ \text{MPG-SAM 2} \\\\ \text{SurgRef} \\\\ \text{\bf DaV (frame)} \\\\ \text{\bf DaV (video)} \end{array} & \begin{array}{r} 67.24 \\\\ 50.40 \\\\ 64.77 \\\\ 52.57 \\\\ 55.36 \\\\ 71.45 \\\\ 40.36 \\\\ \mathbf{77.65} \\\\ \underline{77.40} \end{array} & \begin{array}{r} 44.67 \\\\ 37.00 \\\\ 49.51 \\\\ 38.56 \\\\ 48.24 \\\\ 61.07 \\\\ 32.68 \\\\ \underline{80.46} \\\\ \mathbf{82.39} \end{array} & \begin{array}{rrrrrrr} 71.39 & 42.24 & 35.73 & 50.46 & 23.01 & 73.70 & 16.18 \\\\ 66.74 & 44.80 & 36.22 & 31.25 & 2.21 & 76.56 & 1.25 \\\\ 73.20 & 41.48 & 43.73 & 35.59 & 41.13 & 81.44 & 30.00 \\\\ 67.19 & 49.26 & 47.80 & 27.73 & 12.31 & 27.64 & 37.99 \\\\ 68.00 & 41.79 & 34.64 & 43.62 & 30.24 & 86.33 & 31.09 \\\\ 86.76 & 51.52 & 33.28 & 67.86 & 56.44 & 93.74 & 37.92 \\\\ 33.14 & 22.52 & 22.63 & 36.81 & 28.74 & 61.26 & 24.12 \\\\ 81.92 & 72.28 & \underline{80.01} & 78.82 & \underline{79.50} & \underline{90.48} & \underline{80.94} \\\\ 82.17 & \mathbf{72.43} & \mathbf{83.87} & \underline{79.00} & \mathbf{79.53} & \mathbf{94.80} & \mathbf{84.90} \end{array} \\\\ \hline \end{array}$$