- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes RADNPO, a reference-free adaptive negative preference optimization method for LLM unlearning. It contrasts forget targets with high-probability alternatives and adapts token-level forgetting strength based on target confidence and partial entropy. The method is evaluated on the TOFU and MUSE benchmarks.
Critical strengths
Valuable problem formulation. The paper observes that existing unlearning methods can reduce the probability of forget targets but do not explicitly guide how the suppressed probability mass is redistributed. This broadens the discussion from whether the target is suppressed to how the model generates responses after unlearning. The authors further suggest that excessive concentration in the post-unlearning distribution may be associated with repetitive and less diverse outputs, offering a meaningful direction for improving unlearning objectives.
Broad experimental evaluation. The authors compare multiple unlearning methods on TOFU and MUSE, assessing both forgetting effectiveness and retained utility. In addition to the main results and ablation studies, they evaluate repetition, lexical diversity, and semantic behavior, and provide supplementary analyses of hyperparameter sensitivity, random-seed variation, and performance on another model. These experiments examine the method from several perspectives and help clarify its effectiveness and limitations.
Critical weaknesses
The claim that the post-unlearning distribution may remain overly concentrated does not, by itself, establish that generation quality will decline. If probability mass is concentrated on a single reasonable and contextually appropriate alternative token, generation may remain fluent. Concentration becomes a quality problem only when it leads to repetitive, vacuous, or contextually inappropriate outputs. The authors should therefore provide empirical evidence linking excessive post-unlearning concentration to generation degradation, and show that the problem is not limited to isolated cases. Without such evidence, the paper may identify a change in distributional shape without demonstrating that it constitutes a practical limitation worth addressing.
The discussion in the third and fourth paragraphs of the Introduction is not clearly connected. The authors first argue that existing methods do not control where the probability mass suppressed from the forget target is redistributed, and then suggest that it may shift toward implausible, repetitive, or uninformative alternatives. They subsequently argue that forgetting strength does not jointly account for target confidence and distributional concentration. However, the paper does not clearly distinguish these two limitations or explain how they relate to each other. As a result, it is difficult to determine whether the central motivation is the lack of guidance for probability redistribution or the lack of adaptive forgetting strength, and how each limitation leads to observable generation problems.
The preference-optimization subsection is weakly connected to the paper’s central problem. It mainly surveys the general development of RLHF, DPO, KTO, SimPO, and ORPO, rather than focusing on the application of preference optimization to LLM unlearning and its limitations. The related-work discussion should explain more clearly how preference optimization has been used for LLM unlearning and how its limitations motivate the proposed method, rather than focusing primarily on general preference-optimization techniques.
Questions for authors
The paper argues that the post-unlearning distribution may remain overly concentrated. Does this concentration necessarily lead to degraded generation quality? Could the authors provide evidence linking distributional concentration to repetitive, uninformative, or contextually inappropriate outputs, and clarify whether this effect occurs consistently rather than only in isolated cases?
The paper appears to identify two limitations: insufficient guidance on how probability mass is redistributed after suppressing a forget target, and forgetting strength that does not adapt to the current distributional state. Which limitation is the central motivation for RADNPO, and how are the two related? In particular, what role does the adaptive forgetting strength play beyond the contrastive objective that guides probability redistribution?