- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Response Preference Prompting (RePP), which reframes automatic SAM prompting from direct prompt prediction into response-based candidate selection.
At each refinement step, RePP generates several candidate prompt updates, evaluates their induced masks with a frozen SAM, and uses a learned pairwise preference model to retain the preferable response.
The paper further introduces a multi-point generator that jointly proposes several point edits to reduce the number of sequential SAM interactions.
Experiments on VOC, COCO, and NWPU show substantial gains over several adapted automatic point-prompting methods, and the preference module also improves SAC-Prompt, Matcher, and PPD when applied as a plug-in.
Critical strengths
- The central response-selection hypothesis is supported by unusually clean ablations.
The strongest evidence is Table 1: learned response preference improves AP from 23.7 with the native SAM decoder score to 59.8, approaching the 65.6 AP oracle that selects candidates using ground-truth IoU.
Table 3 further separates per-step candidate selection from final trajectory selection and shows that both contribute substantially. These results make a convincing case that evaluating realized SAM responses provides information that conventional prompt-side criteria fail to capture.
- The proposed preference mechanism appears reasonably generator-agnostic rather than being tied to the authors' candidate generator. Adding the same response preference mechanism to SAC-Prompt, Matcher, and PPD improves AP by 15.4, 5.3, and 4.0 points, respectively.
Together with the results across natural-image and remote-sensing benchmarks, this provides useful evidence that response-based determination is a reusable component rather than an isolated improvement from the multi-point generator.
Critical weaknesses
- The pairwise preference formulation does not guarantee a globally consistent ranking, although inference relies on sequential tournaments.
The antisymmetric construction in Eq. (4) guarantees that swapping a pair reverses its preference, but it does not impose transitivity across three or more responses. Consequently, the selected candidate can in principle depend on tournament ordering.
This matters (theoretically only) because candidate determination is the central component of RePP, yet the paper reports only pairwise preference accuracy and does not characterize preference cycles or ordering sensitivity.
I would view this primarily as an unresolved robustness concern rather than evidence of an actual failure.
Questions for authors
Generalization of response preference across refinement steps.
In Section 3.2 and Appendix A.1, the response preference model is trained on pairs of SAM masks generated by applying different candidate prompt updates to the same current prompt state. In other words, the model learns to compare candidate responses within a single refinement step.
However, during final selection (Section 3.4, Eq. 10), the same preference model is used to compare masks retained at different refinement steps. Unlike the training pairs, these masks are generated from different prompt states along the refinement trajectory. It is therefore unclear whether the preference model trained on within-step comparisons can reliably generalize to cross-step comparisons.
Although Table 3 shows that preference-based final selection improves segmentation performance, it does not directly establish how accurately the model ranks responses across different refinement steps.
Could the authors clarify whether the preference model maintains its ranking accuracy when comparing responses from different refinement steps? For example, reporting preference accuracy separately for within-step and cross-step response pairs, using ground-truth IoU ordering as the reference, would help assess whether the training procedure adequately supports the final selection mechanism.