Tiny Clicks Trigger Large Consequences: Exploring Sensitivity to Spatial-Prompt Novelty in SAM-Based Promptable Segmentation

Click-based point prompts provide a simple and intuitive interface for promptable segmentation, where additional correct clicks are generally expected to refine the prediction. However, the number of clicks does not necessarily reflect how much new spatial evidence they provide. We term this property Spatial-Prompt Novelty and ask a fundamental question: how sensitive are SAM-based segmentation models to different levels of spatial evidence novelty in additional clicks? We investigate this question across three progressively relaxed regimes: (1) At the Zero-Novelty limit, exact repetition adds no new spatial location yet can reduce mean IoU by up to 22.1 points. (2) Under Controlled Low-Novelty, the failure persists even when all added clicks are distinct but closely spaced, while performance consistently recovers as their spatial dispersion increases. (3) Finally, under Naturalistic Low-Novelty, 89,924 valid human interaction events show that nearby click additions occur in recorded interaction and are associated with smaller immediate gains. Further experiments reveal that seemingly minor differences in the spatial novelty of correct clicks can trigger disproportionately large changes in model behavior, exposing spatial-prompt novelty as an important but overlooked factor in SAM-based promptable segmentation.