RCP: Residual-Complementary Prompting for Specialist-Guided Binary Segmentation

Foundation segmentation models can complement task-specific specialists, but their predictions may be unreliable or redundant. We study this collaboration in binary polyp and camouflaged-object segmentation using a DINOv3-B specialist and frozen SAM3. On intervention-selected polyp cases, direct SAM3 replacement reduces Dice by 2.0 points, although a ground-truth-guided image-level choice between the specialist and direct SAM3 shows a potential 5.2-point gain. We propose Residual-Complementary Prompting (RCP), which generates SAM3 hypotheses that correct distinct specialist errors. A spatial arbiter combines the hypotheses with the specialist prediction, and a failure-aware module selects inputs for intervention. RCP improves Dice by 5.0 points over the specialist on intervention-selected cases and consistently improves performance across five polyp and three camouflaged-object benchmarks. A fixed training-calibrated threshold routes 31.2% of test inputs and captures 91.6% of specialist failures; a separate retrospective 30% budget analysis captures 92.4%. The direct-SAM3 oracle uses different candidates and does not bound RCP performance.