- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the sensitivity of SAM-based promptable segmentation models to the spatial novelty of additional point prompts. The authors introduce the concept of Spatial-Prompt Novelty to distinguish the number of clicks from the amount of new spatial evidence they provide. The study examines this phenomenon under three settings: Zero-Novelty, where identical clicks are repeated; Controlled Low-Novelty, where distinct but spatially clustered clicks are compared with more dispersed configurations; and Naturalistic Low-Novelty, which analyzes recorded human interaction trajectories. Experiments with SAM-based models on COCO and VOC2012 reveal that repeating correct clicks can substantially degrade segmentation accuracy, with mean IoU reductions of up to 22.1 percentage points. The authors further show that closely spaced but distinct clicks can also lead to performance degradation, while greater spatial dispersion generally improves segmentation utility. An analysis of 89,924 valid human interaction events provides additional evidence that nearby click additions are associated with smaller immediate improvements. Overall, the paper highlights spatial-prompt novelty as an important factor in understanding the robustness and behavior of promptable segmentation models.
Critical strengths
-
An interesting and practically relevant perspective on promptable segmentation. Rather than proposing another segmentation architecture, this paper investigates a fundamental but relatively underexplored issue: whether additional correct clicks necessarily provide useful information and improve segmentation performance. The distinction between click count and spatial-prompt novelty offers a meaningful perspective for understanding the behavior and robustness of SAM-based segmentation models. The findings may provide useful insights for future prompt encoding, click selection, and interactive segmentation strategies.
-
Systematic experimental investigation with compelling observations. The paper investigates spatial-prompt novelty through three complementary settings, progressing from exact duplicate clicks to distinct but closely spaced clicks and finally to recorded human interactions. This experimental design helps demonstrate that the observed sensitivity is not limited to exact duplicate coordinates. The substantial performance degradation observed in some SAM-based models is particularly noteworthy. Furthermore, the analysis of a large collection of human interaction events strengthens the practical relevance of the study, while the controlled experiments provide useful evidence for understanding the effects of spatial prompt configurations.
Critical weaknesses
- Limited mechanistic understanding of the observed sensitivity. Although the paper reveals substantial performance degradation caused by redundant or closely spaced clicks, it provides limited insight into the underlying mechanisms responsible for this behavior. The analysis of prompt representations and duplicate consistency helps explain why repeated clicks can produce different outputs, but does not sufficiently explain why these changes sometimes lead to severe segmentation failures. A deeper investigation of the prompt encoder, attention mechanisms, and mask decoder would strengthen the technical contribution and provide more actionable insights for improving model robustness.
- Limited evidence of practical significance in realistic interactive segmentation. The most striking performance degradation is observed when identical positive clicks are repeated multiple times, which is an uncommon scenario in practical interactive segmentation. Although the authors extend their analysis to distinct nearby clicks and human interaction trajectories, the real-world experiments mainly establish an association between click proximity and smaller immediate improvements, rather than demonstrating comparable performance degradation. Therefore, the extent to which the severe failures observed under controlled settings translate into practical interactive segmentation remains insufficiently established.
- Limited generalizability across segmentation models and interaction settings. The main experimental findings focus on the SAM family, while different interactive segmentation architectures may encode and integrate spatial prompts in substantially different ways. The paper includes a comparison with SimpleClick, but this primarily illustrates differences in duplicate handling and interaction state. A broader evaluation of representative interactive segmentation methods, including different prompt encoding and iterative refinement strategies, would help determine whether spatial-prompt novelty is a general challenge or mainly an architectural characteristic of SAM-based models.
- Limited actionable validation of the proposed findings. While the paper identifies an interesting sensitivity phenomenon, it does not sufficiently demonstrate how the findings can be translated into improved segmentation robustness or interaction efficiency. Beyond removing exact duplicate clicks, evaluating simple novelty-aware strategies, such as distance-based prompt filtering or spatially diverse click selection, would help quantify the practical benefits of addressing this issue and strengthen the overall significance of the study.
Questions for authors
-
Mechanistic explanation of performance degradation. The experiments reveal substantial differences in sensitivity across SAM generations, particularly between SAM1 and SAM2.1/SAM3. Could the authors provide a more detailed analysis of which components of the prompt encoding and mask prediction pipeline are primarily responsible for the observed performance degradation? Such an analysis would help clarify the underlying mechanism rather than merely documenting the phenomenon.
-
Isolating the effect of spatial-prompt novelty. In the controlled low-novelty experiments, changes in click dispersion may also affect spatial coverage and the relative positions of clicks within the target object. Could the authors provide additional controlled experiments that better isolate spatial-prompt novelty from these confounding factors? This would help establish whether the observed performance differences are specifically attributable to spatial novelty.
-
Practical impact under realistic interaction. Although the analysis of human interaction trajectories shows that nearby clicks are associated with smaller immediate improvements, this does not necessarily indicate performance degradation. Could the authors further quantify how frequently low-novelty clicks actually reduce segmentation accuracy in realistic interaction sequences, and how substantial these reductions are? This would help establish the practical significance of the identified phenomenon.