acceptodds
Under review as a conference paper at ICLR 2027

Beyond Static Selectivity: Task-Native Access and Adaptation Stability in Promptable Segmentation Unlearning

Abstract

Current evaluation of selective concept unlearning for segmentation foundation models assesses whether a designated target is suppressed while useful non-target segmentation is preserved at a fixed post-unlearning checkpoint under a canonical inference configuration. Such static selectivity provides a meaningful qualification, but leaves open how broadly it characterizes the same target capability beyond its qualifying route and model state. We study this scope along two controlled axes: task-native access with the model fixed, and target-clean model evolution with canonical access fixed. Across a 336-run State Bank comprising SCU and six standardized forgetting/retention controls over two SAM backbones, a pre-specified criterion identifies 28 qualified model-target states. With model weights fixed, mean route-normalized residual target capability rises from 0.548 under canonical box access to 0.944 under model-ranked multimask output. In SAM2, changing only the output-resolution rule while holding the image, box prompt, and model weights fixed increases the deployed-versus-controlled IoU gap from 0.0013 in the original model to 0.203 across qualified post-unlearning states. With canonical access fixed, five target-clean adaptation updates restore 77.2% of the behavioral gap on average, while monitored utility remains within the pre-specified tolerance in 83 of 84 trajectories at step 50. In one matched fine-grained case, behavioral return from the post-unlearning state exceeds that of an ordinary-reacquisition control, while its provenance remains unresolved. These findings show that static selectivity is a condition-specific qualification rather than an access- or state-invariant characterization of post-unlearning target capability.

Primary area:
datasets and benchmarks
Keywords:
machine unlearning, promptable segmentation, segmentation foundation models, robustness evaluation, semantic segmentation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tXmGOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies whether static selectivity is sufficient to characterize concept unlearning in promptable segmentation models. The authors first qualify post-unlearning model-target states using target suppression and retained-utility criteria under a canonical access route. They then characterize the qualified states along two axes: task-native access and subsequent target-clean model evolution.

Across 336 model-target runs, 28 states qualify. With the model fixed, residual target capability increases substantially when moving from canonical box access to point and model-ranked multimask access. A fixed-input SAM2 comparison further shows that changing only the model-side output-resolution rule produces a much larger disagreement after unlearning than in the original model. With canonical access fixed, target-clean adaptation also restores a large fraction of the target capability while mostly preserving monitored utility. The paper argues that static selectivity is a condition-specific qualification rather than an access- or state-invariant characterization of unlearning.

Critical strengths
  • The qualification-first design is clear and useful. It separates the question of whether a model passes a canonical unlearning criterion from the broader question of how the qualified capability behaves under different access routes and later model updates.

  • The fixed-input SAM2 comparison is particularly convincing. The image, box prompt, and model weights are held fixed while the model-side output rule changes. The negligible gap in the original model and the much larger gap after unlearning provide evidence that unlearning can disrupt consistency across native output routes.

  • The model-evolution experiment is also informative. Target capability returns quickly under the specified target-clean adaptation protocol while retained utility is mostly preserved. The authors are appropriately cautious and do not claim that behavioral return alone proves recovery of historical target information.

Critical weaknesses
  • The main characterization cohort is narrow. Only 28 of 336 states qualify, and 27 of these states are produced by SCU. The results therefore provide strong evidence about qualified SCU states under the tested settings, but support more limited claims about selective segmentation unlearning in general. The large State Bank should not be confused with broad coverage of successful unlearning methods.

  • Some interpretations depend on comparisons that change more than one factor. The Box-versus-Point comparison changes the prompt form and the available interaction information, while the model-evolution result uses one specific five-shot, full-decoder adaptation protocol. The fixed-input SAM2 experiment addresses part of this concern, but it covers only a subset of the qualified states. In addition, target-clean support excludes direct target-positive supervision but may still contain shared visual priors or indirect information relevant to the target. Thus the experiments establish access- and state-dependence, but do not identify the mechanism or the provenance of the returning behavior.

Minor related-work comment: The positioning is somewhat incomplete. The paper discusses task-native access and downstream adaptation, but does not engage with several relevant works:

  • SHERPA: Fine-tuning Segment Anything Models with Task-relevant Guidance (ICML 2026), which studies task-relevant SAM adaptation while preserving general-purpose segmentation ability.
  • ProMISe: Promptable Medical Image Segmentation using SAM (NeurIPS Workshop 2024), which studies domain adaptation while retaining SAM’s promptable interface.
  • RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentation (NeurIPS Datasets and Benchmarks Track 2024), which shows that realistic click distributions can substantially affect interactive segmentation evaluation.

These works do not undermine the paper’s main results, but discussing them would help clarify the distinction between the present unlearning-focused characterization and prior work on SAM adaptation and access-dependent evaluation.

Questions for authors
  • For the fixed-input SAM2 comparison, could the authors clarify precisely what differs between the controlled token-0 route and the deployed route? In particular, does the comparison change only output resolution, or does it also change candidate generation or output selection?

  • How is target-clean support guaranteed to contain no information that can directly or indirectly supervise the target? For example, can target instances occur as unlabeled regions, related categories, or shared parent concepts in the support images?

  • Given that 27 of the 28 qualified states come from SCU, how should the main conclusions be scoped with respect to other unlearning methods? Would the authors agree that the strongest claim is about the instability of qualified SCU states rather than all segmentation unlearning methods?

  • The paper would benefit from clarifying how its evaluation framework differs from prior work on SAM adaptation and prompt-dependent evaluation, including SHERPA, ProMISe, and RClicks.

Overall, I find this to be an interesting and well-motivated evaluation paper. The distinction between static qualification and broader capability characterization is useful, and the fixed-input SAM2 experiment makes the central claim substantially stronger than a simple comparison of different prompts. The conclusions should remain conditional on the evaluated cohort, access routes, and adaptation protocol, but within that scope the evidence is reasonably convincing. I therefore recommend weak acceptance.

Reviewer_Zdw1OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper conducts evaluation of selective concept unlearning over SAM models. Prior works like SCU evaluate a fixed post-unlearning checkpoint under box prompting and measures the IoU of both target and non-target segmentation. The paper asks whether suppression established under the canonical access route characterizes the same post-unlearning target capability across other task-native routes. They address this question with a qualification-first study design. The authors train 336 unlearned models and examine these models along two axes: task-native access with the model fixed, and target-clean model evolution with canonical access fixed.

Critical strengths
  1. Good reporting formats. All primary results are conditioned on the model-target states selected by the static qualification protocol. Primary summaries first average across episodes within state and then aggregate across states.
  2. The authors are honest on negative results and limitations.
Critical weaknesses
  1. Bad writing style. The authors write with unclear phrases (hard to parse for people who are familiar with CV & segmentation in general). For example, "output-resolution rule", "task-native access", "target-clean", and "behavioral return" (some researchers use "relearning" and it's conceptually clear enough). At least these phrases do not appear in the SCU paper, which is the primary reference of this paper.
  2. The paper focuses on evaluation but it reports extremely incremental findings over limited methods. At the first sight, the paper includes tables with 7 methods but 27 out of 28 qualified states are from SCU. In contrast, prior works like EMMA five different concept erasure methods. The key finding in “target-clean model-evolution characterization” is closely related to Siddiqui et al. The paper should explicitly discuss the differences between its empirical findings and the ones from prior works. The research question in this paper is neither important or fundamental. It narrows down paper's contribution and potential impact a lot.
Questions for authors
  1. The paper uses a local implementation of SCU and the number differs from the SCU paper. In the original SCU paper, they report 21.5 (SAM1) and 24.6 (SAM2) Generality IoU. Is the SCU reimplementation faithful?
Reviewer_mCeWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether segmentation foundation models truly forget a target concept after unlearning, or whether the supposedly forgotten capability can still be recovered. The authors evaluate 336 experiments using SAM1 and SAM2, identifying 28 models that satisfy their unlearning criteria. The results validate that passing a standard unlearning evaluation does not mean a model has permanently forgotten a concept.

Critical strengths
  • The studied problem is interesting, as it challenges the reliability of current segmentation unlearning evaluations.

  • The experimental design is solid. The authors separately examines prompt-based recovery and recovery through model adaptation. Moreover, the experimental settings are also well controled.

  • The evaluation includes 336 candidate runs, multiple baselines, 2 SAM architectures, and sensitivity analyses.

Critical weaknesses
  • The paper is generally of limited novelty. While it introduces an interesting problem to study, it didn't introduce any proposed solution to it.

  • Whether the finetuning procedures will affect the experimental outcome is not very clear. Only a small number of support examples and a specific finetuning procedure are examined.

  • More importantly, whether recovered capabilities come from previously learned information or ordinary relearning is not well supported by the current experiments.

Questions for authors

See the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.