acceptodds
Under review as a conference paper at ICLR 2027

MIST: Minimal Intervention Selective Token Unlearning for Co-Occurring Disease Recognition in Multi-Label Chest X-Rays

Abstract

Selective machine unlearning removes the influence of a target concept from a trained model without retraining, and is increasingly needed in medical imaging for privacy requests, annotation errors and evolving clinical standards. In multi-label chest X-ray recognition, the task is unusually hard because diseases co-occur and share visual evidence. We observe that existing methods sit at the two ends of a forgetting–retention trade-off: gradient ascent forgets the target but damages retained diseases, whereas distillation-based unlearning preserves retained diseases but barely forgets. We trace this to three causes: (i) disease evidence is entangled in shared representations, (ii) current methods act on parameters or global features and cannot localise what to forget, and (iii) nothing in their objectives protects the evidence that co-occurring diseases share. To address the above causes, we propose MIST, a novel token-level framework for Vision Transformers that is composed of three key components: (1) Disease-specific Token Selection (DTS) uses gradient-based attribution to find tokens that are informative for the target disease and not for retained diseases; (2) Selective Unlearning (SU) suppresses the target only through these tokens; and (3) Co-occurrence-Aware Retention (CAR) combines teacher–student distillation with a shared-token preservation loss on images where target and retained diseases appear together. On NIH ChestX-ray14 with patient-level five-fold cross-validation, MIST lowers Effusion AUROC from 0.870 to 0.403 while retained-disease AUROC falls by only 0.007 and co-occurrence AUROC by 0.008, which is 65% and 69% less collateral degradation than SalUn at slightly stronger forgetting, and it modifies 37% of patch tokens. Our results demonstrate that MIST is the only method ranked in the top two on all six forgetting and retention measures.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, selective forgetting, vision transformer, multi-label classification, chest X-ray, medical imaging, gradient-based attribution, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KTyxOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MIST, a patch-token-level unlearning method for Vision Transformers that selectively forgets the knowledge of a specific disease from a multi-label chest X-ray classifier. Starting from the observation that existing methods sit at the two extremes of the forgetting–retention trade-off, the method aims to limit collateral degradation by confining the intervention to a sparse part of the token space. In experiments on five diseases from NIH ChestX-ray14, the method succeeds in forgetting the target disease while limiting the accuracy drop on the diseases to be protected.

Critical strengths

The problem formulation, as a machine-learning problem, is clear. The main strength is that the paper addresses a largely unexplored setting: concept-level selective unlearning in medical imaging. In particular, the setting of forgetting only one disease in multi-label chest X-rays, where diseases co-occur, differs from existing work, and the observation that label co-occurrence places forgetting and retention in direct conflict is very reasonable. Rather than preparing a single training/test split, the evaluation is conducted by cross-validation, which makes the experiments highly reliable.

Critical weaknesses
  1. I was able to understand the technical challenge. However, I cannot understand the clinical significance of wanting to unlearn, in a pinpoint manner, a disease that co-occurs with multiple other diseases. Since the paper builds its argument on medical data, the necessity of solving this problem from a medical standpoint should be stated. Otherwise, the structure of this paper, whose introduction is specialized to the medical setting, has to be regarded as insufficient.
  2. In unlearning research, a model retrained from scratch without the forget data serves as the reference for forgetting, but this paper does not report it. Because this basic evaluation is not conducted, the effectiveness cannot be measured from multiple perspectives.
  3. Although the objective function is designed to suppress the representations associated with d_f, the evaluation is conducted on classification results, and standard baseline verifications such as relearning attacks with a small amount of data and membership inference are not performed.
  4. The parameter settings of the compared methods are fixed to a single configuration, and no comparison over a comprehensive range of parameters is conducted. This means that the discussion is very likely insufficient with respect to compared methods that may be able to adjust the trade-off.
Questions for authors
  1. Can the authors explain the problem setting from the standpoint of clinical practice? If the problem is not one whose necessity can be sufficiently appreciated, there is a possibility that the dataset used is not appropriate.
  2. Can the evaluation of forgetting be conducted not only at the classification level but also at the representation level?
  3. Regarding the evaluation metrics and evaluation methods, is it possible to conduct the evaluation under the standard settings commonly used in ordinary unlearning studies?
  4. Can experiments be conducted over a broad range of settings for the model parameters? Depending on the responses to these points, the score may go up.
Reviewer_oVEpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies selective disease unlearning in multi-label chest X-ray classification, where a target disease must be suppressed while co-occurring diseases that share visual evidence remain recognisable. The authors propose MIST, a token-level framework for Vision Transformers. It uses gradient-based attribution from a frozen teacher to select patch tokens informative for the target but not for retained diseases, suppresses the target through these tokens, and preserves retained knowledge through token-level distillation and a shared-token preservation loss on co-occurrence samples. On NIH ChestX-ray14 with patient-level five-fold cross-validation, MIST reduces Effusion AUROC from 0.870 to 0.403 while retained-disease AUROC drops by 0.007, reported as 65% less collateral degradation than SalUn at comparable forgetting. Similar trends are shown when rotating the forget target across five diseases.

Critical strengths

Meaningful problem setting with co-occurrence-aware evaluation. Selective forgetting under label co-occurrence is a realistic and underexplored challenge in multi-label medical imaging. Evaluating separately on the co-occurrence subset, rather than only on average retained performance, exposes collateral damage that standard evaluation would miss. This framing is the paper's most consequential contribution.

Simple method with clear retention gains. The token-level design is intuitive and clearly presented, and the experimental basics are sound (patient-level splits, multi-label stratification, five folds). The reduction in collateral degradation relative to SalUn (R-AUROC 0.007 vs. 0.020; C-AUROC 0.008 vs. 0.026) is clear given the reported variance, which makes the approach promising.

Critical weaknesses

The evidence does not yet support genuine forgetting and minimal intervention. The post-unlearning Effusion AUROC of 0.403 is below chance, so the ranking of positives and negatives may be inverted rather than erased; negating the logit would yield about 0.6. Because the F-AUROC metric rewards larger drops, it cannot distinguish removal from reversal, and the paper does not state which definition of forgetting it targets. Similarly, the claim that MIST modifies only 37% of patch tokens is not measured. Even when gradients pass only through selected tokens, the update is applied to encoder parameters shared by all tokens (as Eq. 2 shows), so unselected token representations may also change through self-attention. Since genuine forgetting and minimal intervention are the paper's central claims, this is the main reason for my recommendation.

The contribution of the key component and its relation to prior work are unclear. The shared-token preservation loss is what distinguishes MIST from existing distillation-based methods, but the ablation in Table 5 does not isolate it: only the full model uses λ_conf = 0.15 and 72 tokens, while all other configurations use 55 tokens. The gain could therefore come from L_Shr, the larger λ_conf, or the larger token budget. In addition, Cheng et al. (ICLR 2026, "Machine Unlearning under Retain–Forget Entanglement") identify the same limitation of evaluating retention only on average and explicitly protect retained samples correlated with the forget set. This overlaps with the paper's third identified cause (unprotected shared evidence) and contradicts the statement in Section 3.4 that no existing objective constrains the shared component. MIST's token-level mechanism differs, but its novelty needs to be established relative to this work.

Overall, the paper addresses a meaningful problem with a simple and promising approach, but I do not believe the current evidence is sufficient to support acceptance. My assessment could change depending on the responses to Q1–Q3.

Questions for authors

Q1. Verification of forgetting (additional experiment). Which definition of forgetting does MIST target: approximating a retrained model, or maximally suppressing the target? Given the below-chance AUROC, could the authors report a linear probe on the unlearned encoder to show whether target information has been removed rather than inverted?

Q2. Controlled ablation and locality (additional experiment). Could the authors report results with and without L_Shr while keeping λ_conf = 0.15 and the token budget fixed at 72? Could they also report the representation change on non-selected tokens to support the 37% claim?

Q3. Relation to Cheng et al. (clarification). How does MIST differ conceptually and empirically from methods that explicitly protect correlated retained subsets, such as Cheng et al. (ICLR 2026)?

Q4. Data and backbone details (clarification). Does the exclusion in Section 4.1 remove normal (No Finding) images? If so, how do results change when they are included? Also, Rad-DINO is based on DINOv2-base with patch size 14, which yields 256 tokens at 224 × 224. How were 196 tokens obtained?

Q5. Metrics and hyperparameters (clarification). How is C-AUROC computed on D_c, and how is H_r aggregated across retained diseases? What values were used for ρ, λ_Dist, and λ_Shr, how were they selected, and were the baselines tuned individually?

Reviewer_Wec9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MIST for selective disease unlearning in multi-label chest X-ray classification. MIST uses a frozen teacher’s gradient attributions to select tokens for target suppression, together with representation distillation and shared-token preservation to limit interference with co-occurring diseases. Experiments use patient-level five-fold cross-validation on NIH ChestX-ray14, with five diseases used as forget targets.

Critical strengths

1.The problem is relevant. Explicitly protecting co-occurring diseases addresses a difficulty in multi-label medical imaging that aggregate retention metrics can overlook. 2.The evaluation covers five patient-level folds and five forget targets. Reported mean retained AUROC drops of 0.006–0.009 suggest limited collateral degradation.

Critical weaknesses

1.AUROC degradation alone does not establish knowledge removal. Reversing prediction scores would turn an AUROC of 0.403 into 0.597; no retraining reference or representation probe is provided. 2.The local mechanism is insufficiently supported. Shared parameter updates can alter unselected tokens, while Table 5 jointly changes loss weights, token count, and shared-token preservation, obscuring individual contributions. 3.The confidence threshold does not resolve continued suppression. When �0��0�, the hinge term in Eq. (7) vanishes, but negative BCE still pushes the target probability downward.

Questions for authors

1.What retraining reference corresponds to the intended definition of successful forgetting? Can frozen-representation probes or limited relearning tests distinguish concept removal from output suppression? 2.At which layer, and through what masking operation, are forgetting gradients restricted to selected tokens? Since negative BCE remains active below the confidence threshold, how is continued oversuppression prevented? 3.With �0��0� and token count fixed, how much does shared-token preservation contribute? How does random selection perform with the same token budget? 4.Please clarify the Rad-DINO checkpoint and patch size. The official configuration uses a patch size of 14, which normally gives 256 patch tokens for a 224*224 input. How are the reported 196 tokens and 37% intervention ratio obtained?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.