- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes SeNPO, a method for knowledge unlearning in large language models that addresses implicit evocation by optimizing semantic similarity in a frozen sentence-embedding space. It uses a softplus preference loss and retain-set KL divergence to ensure forgetting while preserving model usability.
Critical strengths
- The paper introduces a novel concept of implicit evocation, where unlearning fails to remove knowledge from intermediate layers, and provides a theoretical condition for this phenomenon using the Linear Representation Hypothesis (LRH).
- SeNPO’s use of semantic-level similarity instead of token-level likelihoods is a significant innovation, reducing sensitivity to paraphrasing and adversarial suffixes compared to existing methods like NPO.
- The experiments are comprehensive, comparing SeNPO with baselines on multiple benchmarks and demonstrating strong forgetting performance while maintaining utility metrics. The results show that SeNPO outperforms other methods in retaining knowledge and fluency.
Critical weaknesses
- The theoretical analysis relies on the LRH, which assumes linear geometry in hidden states. However, the paper does not rigorously validate this assumption or address potential nonlinearities, leaving room for skepticism about the generality of the results.
- The experiments do not fully explore the impact of hyperparameters (e.g., β, γ) on performance.
- While the paper claims that SeNPO achieves "strong forgetting," it does not provide a clear definition of what constitutes "strong" forgetting or how it compares quantitatively to other methods across all metrics.
Questions for authors
- The theoretical analysis builds on the LRH with a linear hidden-state assumption, but this assumption is not rigorously checked and nonlinear effects are ignored, so the results may not generalize well.
- The experiments lack a complete investigation of how hyperparameters such as β and γ affect model performance.
- The paper claims SeNPO achieves strong forgetting, yet it fails to define this term clearly or supply full quantitative comparisons against other methods.