- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a binary image segmentation framework based on Residual-Complementary Prompting (RCP). The authors observe that, even when a general-purpose foundation model for segmentation underperforms compared to a specialized model (a "Specialist"), its predictions may still contain complementary information useful for correcting the Specialist's errors. Building on this insight, the paper shifts the objective: rather than requiring the foundation model to produce independent, accurate segmentation results, it optimizes multiple prompts to elicit segmentation hypotheses specifically designed to rectify the residual errors made by the Specialist. Overall, this work explores a valuable research direction by reframing the role of the foundation model—moving it from direct prediction toward targeted error correction for a specialized model.
Critical strengths
- The research motivation is well-grounded, introducing a meaningful perspective on the complementary collaboration between a specialist model and a foundation model. While existing adaptation and prompt-learning methods for foundation-based segmentation models typically focus on enhancing the segmentation accuracy of the foundation model itself, this work—RCP—goes a step further: it explores whether a foundation model can provide targeted assistance when a high-performing specialist model is already available.
- The method design demonstrates strong logical consistency. The three proposed modules address when to intervene, what hypotheses to generate, and how to fuse predictions, respectively; the overall framework follows a clear and coherent design logic.
Critical weaknesses
- Inconsistency in reported performance improvements.Tables 1 and 9 report a 4.4-point overall Dice gain with approximately 30% intervention, while Tables 3 and 5 show only a 5.0-point gain on intervened samples. Since non-intervened predictions remain unchanged, this would imply an overall gain of approximately 1.5 points under consistent image-wise averaging. The authors should clarify the aggregation methods and reconcile these results.
- Insufficient distinction from existing multiple-hypothesis learning methods.The set-level objective resembles Best-of-K learning, while the redundancy loss and spatial arbiter are similar to existing diversity regularization and prediction fusion methods. Stronger comparisons are needed to establish the unique contribution of specialist-relative correction.
Questions for authors
1.Could the authors clarify the relationship between the routed-subset improvements and the overall improvements reported in Tables 1, 3, 5, and 9? 2.Can the authors provide a stronger baseline that isolates the contribution of specialist-relative residual optimization? For example, a conventional best-of-K or multiple-choice learning objective could be used to train the same prompt generator without explicitly incorporating the specialist prediction into the set-level objective.
We thank the reviewer for recognizing the motivation and coherent design of RCP. Following the review, we repeated the affected evaluations, corrected the descriptions underlying the performance discrepancy, and added the requested conventional best-of-K baseline. The manuscript revisions are highlighted in blue.
Response: Parts (a) and (b) address your first and second questions, respectively.
(a) Relationship between routed-subset improvements and overall improvements: The table numbers in your question refer to the original submission. In the revised manuscript, original Tables 3, 5, and 9 correspond to Tables 4, 7, and 12, respectively; Table 1 retains its number. New Table 3 reconciles routed-subset and overall gains. Under pooled image-wise averaging, the overall gain equals the routed-subset gain weighted by the intervention fraction, because non-routed predictions remain unchanged. After repeating the affected evaluations, we identified that the original Tables 3 and 5 described a separate diagnostic subset but incorrectly labeled it as the operational routed subset underlying Tables 1 and 9. Consequently, their reported 5.0-point gain was not the routed-subset gain corresponding to the overall results. We corrected these descriptions and added the following matched decomposition in revised Table 3:
Intervention covers 239 of 798 polyp evaluation images, or 29.95%. On routed images, Dice increases from 0.760443 to 0.916000, an improvement of approximately 15.56 percentage points. Weighting this gain by the intervention fraction produces approximately 4.66 percentage points of pooled overall improvement, matching the increase from 0.855075 to 0.901664. Table 1 reports a different aggregation: mean image-wise Dice is calculated within each complete evaluation dataset, followed by an unweighted mean across datasets. For polyp segmentation, this dataset-macro score increases from 0.8818 to 0.9258, reported as an increase from 0.882 to 0.926. Original Table 9, now Table 12, uses the same dataset-macro Dice convention; its description as pooled image-wise Dice was incorrect and has been corrected. The diagnostic comparison in original Tables 3 and 5, now Tables 4 and 7, concerns the separate 200-image subset explicitly defined in Appendix A.1.4. Its improvement from 0.866 to 0.916 therefore should not be multiplied by the operational intervention rate to reconstruct the overall gain. Thus, the original discrepancy involved both an incorrect subset description and insufficiently distinguished aggregation conventions. We clarified the intervention protocol and aggregation in Sections 3.2 and 4 and the Table 1 caption, added the matched decomposition in Section 5.1 and Table 3, defined the diagnostic subset in Appendix A.1.4, and corrected the affected descriptions and captions, including Table 12.
(b) Stronger baseline isolating the contribution of specialist-relative residual optimization: We added the conventional best-of-K baseline in Section 5.2 and Table 5. Its prompt generator minimizes the segmentation loss of the best individual hypothesis, without explicitly rewarding specialist-relative residual correction. Both methods use the same specialist-guided prompt-generator architecture, K=3, frozen SAM3, training schedule, spatial-arbiter architecture, and arbiter-training procedure. They are evaluated on identical selected image IDs for each seed. The controlled difference is therefore the prompt-training objective, rather than hypothesis count or fusion architecture. The results across three random seeds are:
RCP achieves higher mean Dice on all five datasets. Table 5 additionally reports increases in residual-error coverage of 8.1–10.1 percentage points and reductions in correction redundancy of 0.15–0.18 relative to best-of-K. Final Dice is evaluated over each complete dataset, while coverage and redundancy are evaluated on routed images. These results support the contribution of specialist-relative optimization: under the same architecture and routing conditions, RCP generates hypotheses that collectively correct more specialist errors with less overlap between corrective contributions. The baseline definition, matched experimental protocol, three-seed results, and discussion have been added to Section 5.2 and Table 5.