- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes CLEAR, a constrained multi-objective optimization algorithm for LLM unlearning. CLEAR addresses forgetting-retention trade-offs, boundary preservation, and adversarial robustness with provable convergence guarantees and experiments on benchmarks.
Critical strengths
- The paper introduces CLEAR, a novel algorithm combining Weighted-Chebyshev scalarization and multi-gradient descent to handle multi-objective LLM unlearning. This approach explicitly models forgetting, retention, boundary preservation, and robustness, addressing critical limitations of prior methods.
- The theoretical contributions include a KKT-based optimality metric and a finite-time convergence guarantee. These guarantees provide rigorous foundations for the algorithm’s performance.
- Extensive experiments on MUSE-Book and WMDP-Cyber benchmarks demonstrate CLEAR’s superiority over baselines in balancing multiple objectives. The method also enables controllable Pareto exploration via preference vectors, enhancing practical utility.
Critical weaknesses
- The theoretical analysis assumes Lipschitz continuity and linear independence constraint qualification, which may not hold for all LLM architectures. These assumptions are not rigorously validated for real-world LLMs, limiting the generalizability of the convergence guarantees.
- While the paper claims to address multi-objective trade-offs, the experiments focus primarily on quantitative metrics but lack qualitative analysis of how CLEAR handles edge cases. This weakens the demonstration of robustness in real-world scenarios.
- The paper does not discuss computational scalability or efficiency trade-offs of CLEAR compared to heuristic methods. For example, the penalty coefficients (s, r) and their tuning are not thoroughly analyzed for large-scale LLMs, which could impact practical deployment.
- The abstract and introduction briefly mention the problem of over/under-forgetting but do not explicitly clarify how CLEAR’s Pareto exploration resolves these issues. The connection between the theoretical guarantees and practical unlearning outcomes is underdeveloped.
Questions for authors
- The theoretical analysis relies on Lipschitz continuity and LICQ. These assumptions may fail on many LLM architectures and lack solid verification on real LLMs, so the convergence results cannot be widely applied.
- Although the paper aims to balance multiple objectives, its experiments mostly use numerical metrics. It does not qualitatively test CLEAR on edge cases, making its real‑world robustness less convincing.
- The work fails to compare CLEAR’s computational cost with heuristic approaches. It also lacks analysis on tuning penalty coefficients s and r for large LLMs, which hinders real deployment.