- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies forget–retain trade-offs in machine unlearning and proposes Tchebycheff Unlearn to construct a family of models by exploring the Pareto front. The method uses Tchebycheff scalarization with a forget-reference sweep, supported by theoretical analysis. Experiments on WMDP, TOFU, and MUSE examine empirical Pareto front coverage and the persistence of intermediate solutions compared with linear scalarization.
Critical strengths
1.The research question is meaningful and clearly motivated. Forgetting and retention can conflict, and the preferred balance varies across applications. The focus on providing a family of models with different trade-offs is well motivated.
2.The theoretical analysis is concise and supports the method’s design. The single-coordinate sweep proposition justifies varying only the forget reference and clearly distinguishes finding Pareto-optimal solutions from achieving Pareto front coverage.
3.The experiments are reasonably comprehensive and well documented. They cover multiple datasets, examine both endpoint loss pairs and training trajectories, and provide results across multiple random seeds, detailed configurations, and an explicit discussion of limitations.
Critical weaknesses
1.Behavioral forgetting and retained utility are not directly evaluated. The experiments mainly report surrogate losses, leaving it unclear whether the additional intermediate solutions correspond to better knowledge removal, reduced privacy leakage, or better preservation of model capabilities. Although the authors acknowledge this limitation, a gap remains between the application motivation and the empirical evidence.
2.The added benefit over existing methods is not sufficiently established. The core optimization procedure follows OMD–TCH, but there is no direct comparison with Tchebycheff weight sweeps at a fixed reference point. This makes it difficult to assess the independent value of the reference sweep.
3.The proposed mechanism for sustaining intermediate solutions lacks direct verification. The paper does not report trajectories of the relative weights. It therefore remains possible that the intermediate solutions persist mainly because the forget weight decreases and training shifts toward retention, rather than because of continued feedback between the two objectives.
Questions for authors
1.Could the authors report behavioral forgetting and retained utility metrics to establish whether the additional intermediate loss compromises correspond to useful behavioral trade-offs?
2.Could the authors show trajectories of the relative weights and compare against a two-stage procedure that performs forgetting followed by retention training, to clarify the mechanism behind sustaining intermediate solutions?
2 comments
The authors clarified some point yet my other concerns such as W1 remained. I thus keep my rating.
Response to Reviewer wehM
We sincerely thank the reviewer for the careful reading and constructive comments. We respond to the comments below.
W1 & Q1: Behavioral evaluation of forgetting and retained utility
We thank the reviewer for the helpful comment. Tchebycheff Unlearn guarantees better coverage of the forget–retain loss trade-off, but this does not by itself imply better behavioral trade-offs (e.g., WMDP vs. MMLU accuracy, TOFU Forget Quality vs. Model Utility), since surrogate losses and behavioral metrics need not be monotonically related. We acknowledge that this is an important open question, and we will discuss it and clarify the scope of our claims in the next version of this work.
W2: Methodological novelty
We thank the reviewer for the insightful comment. Our main contribution is to identify two limitations of linear scalarization in machine unlearning, and introduce Tchebycheff scalarization as the alternative method to address the limitations:
Tchebycheff scalarization addresses both limitations: its level sets can reach unsupported solutions, and its adaptive weights help sustain intermediate solutions. Proposition 1 varies only the forget reference and holds for any fixed retain reference, and we view Proposition 1 as a justification for a practical design choice. We will revise the introduction to make this positioning clear in the next version of this work.
W3 & Q2: Mechanism for sustaining intermediate solutions
We thank the reviewer for the helpful comment. We will report the relative-weight trajectories in the next version of this work.