acceptodds
Withdrawn from ICLR 2027

SegErase: Dual-Output Concept Unlearning for Text-Prompted Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-promptable segmenters learn open-vocabulary grounding from web-scale data and can consequently be prompted to segment sensitive or proprietary concepts. Existing unlearning methods reshape a single global decision, such as a predicted label or a synthesized image. However, unlearning for segmenters is a dense prediction problem governed by two complementary outputs: pixel-wise mask logits determine where a concept is segmented, while an emission score determines whether its mask is returned. We therefore introduce \segerase, a concept-unlearning framework that jointly unlearns both outputs. Concept Attention-Guided Unlearning concentrates mask-side unlearning on pixels most associated with the target. Teacher-Relative Score Unlearning applies a bounded reduction to target scores while anchoring retained and background predictions to a frozen teacher. Extensive experiments on SAM 3 and COSINE demonstrate that \segerase attains a more favorable unlearning-retention balance than existing unlearning methods across standard concepts, unseen prompt reformulations, and multi-target settings.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine unlearning, Promptable segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1qguOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SegErase for concept unlearning in text-prompted segmentation. It combines attention-weighted suppression of target mask logits with teacher-relative reduction of emission scores, while constraining changes to retained predictions.

Critical strengths
  1. The distinction between spatial mask prediction and mask emission is well motivated. The evaluation extends beyond final emitted masks. Pre-NMS query leakage and unseen prompt reformulations provide useful evidence about residual target responses. The broader category and multi-target experiments in the appendix also strengthen the evaluation.
  2. The mean-matched weighting experiment is a useful control for CAGU. The score-objective comparison further examines the proposed rationale for limiting target-score suppression.
Critical weaknesses
  1. The practical motivation is underdeveloped. The introduction states that a deployed text interface should not return sensitive, private, or proprietary concepts, but does not establish a concrete deployment scenario or explain the additional harm caused by segmentation. It is also unclear why model-level unlearning is necessary rather than prompt or output filtering, and what protection remains if the same region can be recovered through generic or spatial prompts. A clearly defined threat model is needed to connect the proposed technical objective to its claimed practical value.
  2. The methodological advance over SCU needs sharper isolation. SCU already combines attention-level suppression, target-background supervision, and non-target preservation. SegErase differs through attention-weighted mask supervision and assignment-aware, teacher-relative score reduction, but additional controls would help establish why these particular choices are necessary. Mean-matched uniform weighting controls loss magnitude, but does not establish that the semantic information in the attention map drives the improvement.
  3. In Table 5, could the authors explicitly define AULC, Q.3, Q.5, and U.3, including their thresholds, aggregation, and normalization? Please also report score-independent residual mask overlap alongside AULC to distinguish removal of target-shaped masks from suppression of their scores.
Questions for authors
  1. In Figure 5, the identity mask includes clothing, so target-negative supervision also acts on regions that are not identity-specific. How do the authors distinguish removal of the name–identity association from suppression of the entire person mask? Please compare full-person and face-restricted unlearning masks, and evaluate preservation using both “person” and “shirt” prompts on the same images.
  2. Could the authors compare CAGU with spatially shuffled or reversed attention weights, while keeping the target-region mean weight and overall loss scale unchanged? This would help clarify whether the benefit comes from meaningful attention locations rather than nonuniform weighting alone.
  3. Could the authors evaluate SCU with its attention suppression replaced by CAGU, with TRSU added separately, and with both changes combined under the same training protocol? Comparing these variants at matched retained performance would help isolate each component’s contribution beyond SCU.
Reviewer_eRnFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SegErase, a concept-unlearning framework for text-prompted segmentation that aims to remove targeted concepts while preserving unrelated segmentation capabilities. It introduces Concept Attention-Guided Unlearning (CAGU) to suppress target-related mask predictions and Teacher-Relative Score Unlearning (TRSU) to control mask emission scores while maintaining retained predictions. Experiments on SAM 3 and COSINE demonstrate improved unlearning effectiveness and a favorable retention trade-off across object concepts, unseen prompt variations, and multi-target settings.

Critical strengths
  1. The paper identifies the importance of jointly controlling segmentation masks and emission scores for concept unlearning. The proposed CAGU and TRSU components address these complementary aspects, providing a meaningful contribution to text-prompted segmentation.
  2. Experiments on SAM 3 and COSINE demonstrate substantial improvements over existing unlearning methods. Evaluations across different concept types, unseen prompt variations, and multi-target settings further support the effectiveness and generalizability of the proposed approach.
Critical weaknesses
  1. The proposed CAGU and TRSU components primarily build upon existing attention-weighted supervision and teacher-based regularization. Their technical novelty over existing unlearning approaches, particularly SCU, could be better justified.
  2. Although SegErase achieves strong target suppression, it still exhibits residual concept leakage under unseen prompt reformulations and noticeable retention degradation on COSINE. These limitations raise concerns about the robustness and generalizability of the unlearning mechanism.
Questions for authors
  1. Clarify the fundamental differences between CAGU/TRSU and existing attention-based unlearning or teacher-regularization methods, particularly SCU, and explain which design choices contribute most to the observed improvements.
  2. Explain the residual concept leakage under unseen prompt reformulations and the larger retention degradation on COSINE. Clarify whether these limitations arise from architectural differences or the proposed unlearning objectives.
Reviewer_iGVZOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SegErase, a concept-unlearning method for text-prompted segmentation models that aims to prevent the segmentation of selected concepts while preserving performance on unrelated concepts.

Critical strengths
  • The paper identifies two outputs that matter for text-prompted segmentation: mask logits and emission scores. Jointly controlling these outputs is a reasonable approach.
  • SegErase achieves substantially lower target IoU than the tested baselines on both SAM 3 and COSINE. The component ablations also support the value of the proposed objectives.
  • The paper includes evaluations on 78 COCO categories, compositional concepts, multiple forgotten targets, and unseen prompt variants. Its pre-NMS analysis also examines target responses that may remain hidden by output filtering.
Critical weaknesses

My major concerns is the narrow application scenarios of the targeted problem. The details are in W1, W2, and Q1. Other concerns are to complete the experiments and compare with existing literature in a fair setting.

  1. Limited evidence of concept-level unlearning. The method mainly suppresses segmentation outputs. It does not show that the model loses the underlying concept knowledge. Other text prompts or visual prompts may still recover the forgotten objects.
  2. Insufficient semantic generalization. The paper tests unseen prompt reformulations but does not evaluate semantic subcategories. For example, it is unclear whether forgetting "dog" also suppresses "corgi" or "Shiba Inu." Moreover, 42.5% and 49.2% of tested prompt variants still produce successful segmentation on SAM 3.
  3. Limited benchmark coverage. The main experiments use only four COCO object categories and eight PACO concepts. Although the appendix extends evaluation to 78 COCO categories, the paper does not test on SAM 3's native SA-Co benchmark or reproduce the established SCU evaluation protocol.
  4. Incomplete forgetting–retention comparison. SegErase achieves stronger forgetting but also reduces retained performance. The paper does not compare methods at matched retention levels. Thus, it is unclear whether SegErase offers a better trade-off than existing methods.
  5. Insufficient distinction from prior work. Attention-weighted segmentation losses and teacher-guided unlearning are established techniques. The paper does not clearly distinguish its technical contributions from these methods or discuss several closely related works, including DELETE [1] and SafeDetect [2].

[1] Decoupled Distillation to Erase: A General Unlearning Method for Any Class-centric Tasks, CVPR 2025

[2] Unlearning without Forgetting: Securely Removing Targeted Concepts from Large-Scale Vision-Language Open-Vocabulary Detectors, CVPR 2026

Questions for authors
  1. Unlike image generation models, segmentation models locate objects that are already visible in the input. The paper does not clearly explain why concept unlearning is needed or what practical benefit it offers over prompt or output filtering. Why not directly reject illegal images input or filter illegal text prompts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.