acceptodds
Withdrawn from ICLR 2027

Beyond Impair–Repair: Tuning-Efficient Machine Unlearning with Anti-Overrepair

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Machine unlearning aims to remove the influence of selected training data without retraining a model from scratch while preserving performance on the remaining data. However, many approximate unlearning methods jointly optimize competing forgetting and retention objectives, introducing unlearning-specific hyperparameters that can be sensitive to the forgetting ratio, model architecture, and dataset. Repeated scenario-specific recalibration therefore adds nontrivial tuning and computational overhead, potentially reducing the practical efficiency advantage over retraining. We propose TriTune, a three-stage framework that decomposes this coupled optimization into a sequence of stage-wise, single-objective optimization problems. By avoiding directly competing objectives within each stage, TriTune simplifies hyperparameter selection and reduces sensitivity to hyperparameter choices. TriTune further introduces an adaptive anti-overrepair mechanism that prevents utility repair from restoring forgotten behavior, together with lightweight self-calibration for learning-rate initialization and stage termination. Experiments across diverse datasets, architectures, forgetting ratios, and learning tasks demonstrate effective forgetting, strong utility preservation, and reduced dependence on manual hyperparameter tuning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Tuning-Efficient Unlearning, Self-Calibration, Impair--Repair, Anti-Overrepair, Forgetting--Retention Trade-off

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_dfHgOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Not reviewed due to paper length violation

Critical strengths

None

Critical weaknesses

None

Questions for authors

None

Reviewer_2K5eOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TriTune which separates machine unlearning into three stages: forgetting, utility repair, and preventing repair from bringing back forgotten behavior. It uses held-out data as a reference and automatically adjusts learning rates and stopping points to reduce manual tuning. Experiments cover image and text classification and diffusion models, with ablations examining the role of each component.

Critical strengths
  1. Tuning costs are often overlooked when comparing unlearning with retraining. It is a good point to cut in.

  2. The ablation tests show that repair can bring back some forgotten behavior, and the proposed mechanism helps prevent this while retaining utility.

Critical weaknesses
  1. The paper exceed the page limit (9).

  2. The paper applies fixed settings across tasks, but does not report search budgets or calibration costs.

  3. Some key technical components need clearer definitions (e.g., repair-progress. gate threshold, stopping settings, and task-specific losses).

Questions for authors
  1. Could you report total tuning and calibration costs and compare methods under the same tuning budget?

  2. Could you give cleared definitions as indicated in Weakness section?

Reviewer_oqMUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces TriTune, a three-stage framework for approximate machine unlearning. Its central motivation is that existing methods often require scenario-specific hyperparameter tuning to balance forgetting and retained utility, while sequential impair–repair methods risk restoring forgotten behavior during utility recovery. TriTune addresses these challenges through three stages: forgetting alignment, utility repair, and anti-overrepair.

Critical strengths

The problem is relevant, and the separation of utility repair from forgetting preservation is intuitively appealing. The paper also reports promising results, particularly for retained accuracy and cross-scenario transfer.

Critical weaknesses

(1) This paper exceeds the page limit. According to the ICLR guidelines, “At the time of submission, the main text should be 9 pages or fewer. This limit will be strictly enforced. Papers with main text beyond the page limit will be desk-rejected.” (2) The claimed reduction in hyperparameter tuning is not sufficiently established. That is, the evaluation does not convincingly quantify the amount of tuning that TriTune actually eliminates. (3) Stage 1 aligns forget-set behavior and task risk with reference quantities derived from held-out data and the original model. The idea of replacing unbounded degradation with a reference-based target is reasonable, but the paper does not sufficiently establish that these references consistently correspond to desirable unlearning outcomes. (4) The paper explicitly states that all quantitative entries use seed 42. This is a substantial limitation because the reported improvements may depend on the selected forget set, initialization, data ordering, or optimization trajectory.

Questions for authors

Please see the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.