acceptodds
Withdrawn from ICLR 2027

CLEAR: Multi-Objective LLM Unlearning with Provable Performance Guarantees

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large Language Model (LLM) unlearning has emerged as a crucial paradigm for mitigating privacy and security risks by aiming to erase undesirable knowledge while preserving the LLM’s general capability. Despite recent development of unlearning algorithms, existing methods suffer from critical limitations. First, they primarily focus on the basic trade-off between forgetting and retention, neglecting other essential dimensions such as robustness. Second, they overlook the failure-sensitive nature of unlearning at the task level, generally failing to provide worst-case performance guarantees. Third, current approaches are largely heuristic, lacking rigorous theoretical foundations and finite-time convergence analysis. In this paper, we systematically address the LLM unlearning problem through the lens of constrained multi-objective optimization (MOO), which enables us to propose Chebyshev-weighted LLM unlearning algorithm (CLEAR), a new penalty-based MOO algorithm with provable performance guarantees. Specifically, we theoretically construct a rigorous Karush-Kuhn-Tucker (KKT)-based metric, based on which we establish a finite-time convergence guarantee for the proposed CLEAR algorithm, marking the first of its kind in the LLM unlearning literature. Extensive numerical experiments across multiple benchmarks, employing comprehensive evaluation metrics, demonstrate that CLEAR significantly outperforms state-of-the-art baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model, Unlearning, Multi-Objective Optimization, Adversarial Robustness, Theoretical Guarantees

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6BtVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CLEAR, a constrained multi-objective optimization algorithm for LLM unlearning. CLEAR addresses forgetting-retention trade-offs, boundary preservation, and adversarial robustness with provable convergence guarantees and experiments on benchmarks.

Critical strengths
  1. The paper introduces CLEAR, a novel algorithm combining Weighted-Chebyshev scalarization and multi-gradient descent to handle multi-objective LLM unlearning. This approach explicitly models forgetting, retention, boundary preservation, and robustness, addressing critical limitations of prior methods.
  2. The theoretical contributions include a KKT-based optimality metric and a finite-time convergence guarantee. These guarantees provide rigorous foundations for the algorithm’s performance.
  3. Extensive experiments on MUSE-Book and WMDP-Cyber benchmarks demonstrate CLEAR’s superiority over baselines in balancing multiple objectives. The method also enables controllable Pareto exploration via preference vectors, enhancing practical utility.
Critical weaknesses
  1. The theoretical analysis assumes Lipschitz continuity and linear independence constraint qualification, which may not hold for all LLM architectures. These assumptions are not rigorously validated for real-world LLMs, limiting the generalizability of the convergence guarantees.
  2. While the paper claims to address multi-objective trade-offs, the experiments focus primarily on quantitative metrics but lack qualitative analysis of how CLEAR handles edge cases. This weakens the demonstration of robustness in real-world scenarios.
  3. The paper does not discuss computational scalability or efficiency trade-offs of CLEAR compared to heuristic methods. For example, the penalty coefficients (s, r) and their tuning are not thoroughly analyzed for large-scale LLMs, which could impact practical deployment.
  4. The abstract and introduction briefly mention the problem of over/under-forgetting but do not explicitly clarify how CLEAR’s Pareto exploration resolves these issues. The connection between the theoretical guarantees and practical unlearning outcomes is underdeveloped.
Questions for authors
  1. The theoretical analysis relies on Lipschitz continuity and LICQ. These assumptions may fail on many LLM architectures and lack solid verification on real LLMs, so the convergence results cannot be widely applied.
  2. Although the paper aims to balance multiple objectives, its experiments mostly use numerical metrics. It does not qualitatively test CLEAR on edge cases, making its real‑world robustness less convincing.
  3. The work fails to compare CLEAR’s computational cost with heuristic approaches. It also lacks analysis on tuning penalty coefficients s and r for large LLMs, which hinders real deployment.
Reviewer_LQkNOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CLEAR, a constrained multi-objective optimization framework for LLM unlearning. The authors argue that practical unlearning requires jointly addressing four objectives: target forgetting, general knowledge retention, neighboring-domain preservation, and adversarial robustness. To balance these competing goals, CLEAR employs weighted Chebyshev scalarization coupled with explicit task-level constraints, optimizing the resulting formulation via a penalty-based projected gradient method.

Critical strengths
  • The paper addresses an important limitation of conventional forgetting–retention formulations by explicitly incorporating neighboring-domain preservation and adversarial robustness. These objectives capture meaningful failure modes in LLM unlearning, specifically over-forgetting and vulnerability to adversarial elicitation, and the constrained multi-objective perspective offers a principled framework for studying these competing requirements.
  • CLEAR achieves the highest aggregate scores in the reported experiments on both MUSE-Book and WMDP-Cyber. The comparison against SUM, which optimizes the same underlying objectives, is particularly informative for isolating the impact of the optimization strategy. Furthermore, additional evaluations against held-out prompt-based and relearning attacks strengthen the empirical study beyond standard forgetting–retention metrics.
Critical weaknesses
  1. The main theoretical motivation relies on weighted Chebyshev scalarization to identify and optimize the current bottleneck objective. However, the forgetting objective in Eq. (2) is defined as negative cross-entropy (which is non-positive), whereas the other three objectives are non-negative. Consequently, for any strictly positive preference vector, the maximum weighted objective can never be attained by the forgetting term. At the level of the exact scalarized objective, the Chebyshev scalarization is therefore completely independent of the forgetting preference weight, meaning that forgetting can only influence the solution indirectly through its separate feasibility constraint. This raises a fundamental question regarding the proposed preference-guided bottleneck control mechanism, specifically, its ability to explore trade-offs involving the forgetting objective.

    Moreover, Theorem 1 explicitly assumes that all objectives are strictly positive, which directly contradicts the non-positive forgetting objective used by CLEAR; thus, the theorem cannot be directly applied to the proposed unlearning formulation. In addition, there is an algebraic error in the proof of Theorem 1 (Appendix B.1): the specified preference construction does not yield the claimed equality of weighted objective values, as this step appears to require normalized reciprocal objective values rather than the weights currently written in the proof. Collectively, these issues undermine the connection between the proposed formulation, the claimed Pareto characterization, and the actual optimization algorithm.

  2. The finite-time convergence guarantee is not established by the provided proof. Although the claimed O(T−1/2)O(T^{-1/2}) convergence rate is presented as a central theoretical contribution, Appendix B.3 contains an apparent algebraic error in the final rate derivation. Specifically, Eq. line 975 bounds the average squared update direction by:

O(s+rηT)O\left(\frac{s+r}{\eta T}\right)

Under the parameterization s=r=Θ(Tξ)s=r=\Theta(T^\xi) and η=Θ(T−ζ)\eta=\Theta(T^{-\zeta}), this term scales as Tξ+ζ−1T^{\xi+\zeta-1} rather than Tξ−ζ−1T^{\xi-\zeta-1} as written in Eq. line 982. With the stated choices of ξ=1/4\xi=1/4 and ζ=1/2\zeta=1/2, the resulting bound scales as O(T−1/4)O(T^{-1/4}) instead of the claimed O(T−1/2)O(T^{-1/2}). While an alternative parameter choice might potentially recover the desired rate, the theorem is not justified by the current derivation.

The smoothness assumptions also require stronger justification. In particular, the assumed global Lipschitz continuity of the penalty-function gradient is difficult to reconcile with unbounded slack variables and nonlinear model losses; empirical smoothness estimates observed along a finite training trajectory do not establish this global property. Finally, the use of finite quadratic penalties does not by itself guarantee satisfaction of the original hard constraints. Because the theoretical result bounds an average KKT residual rather than providing a direct worst-case guarantee on task-level unlearning performance, the relationship between these two notions should be stated much more precisely.

Questions for authors
  1. How is the Chebyshev scalarization intended to control the forgetting objective given its non-positive sign?
  2. Can the authors provide a corrected proof and parameter scaling for the claimed convergence rate?
  3. Do the trained models strictly satisfy the claimed hard constraints in practice?
  4. To what extent is the empirical improvement specifically attributable to constrained Chebyshev optimization?
Reviewer_ctM5OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper highlights that moost of unlearning methods only focus on forgetting and retaining objectives independently. The paper criticizes the prior works as they separately optimize conflicting objectives and results sub-optiaml results (either over or under-unlearning). To address this, the paper reformulates machine unlearning to a constrained multi-objective optimization. The paper defines four objectives: (1) forgetting, (2) (retain data) retention, (3) preserving neighborhood knowledge and (4) robustness against adversarial probing. Each objective has a task-level tolerance constraint fif_i ≤ εi\varepsilon_i. The paper scalarize the problem with weighted-Chebyshev scalarization in epigraph form, so that a preference vector λ\lambda selects the trade-off. The optimizer targets the current worst weighted objective. The resulting problem is solved by the proposed method (CLEAR). It adds slack variables and applies a quadratic penalty to the constraints, and runs projected gradient descent jointly over the model parameters and the auxiliary variables. The experimental result shows that

Critical strengths
  1. The paper introduces constrained multi-objective optimization with strong mathematical formulations for LLM unlearning. Prior works also proposed constrained optimization, but the constraint is usually on one target loss (either forget loss or retain loss). I respect the paper's novelty and contributions.
  2. Experimental result shows that the proposed method is robust to adversaries, including prompt-base and relearning attack.
  3. Paper is well-written and its motivation, problem formulation, and objectives are clearly written. It is easy to follow even without any figures.
Critical weaknesses
  1. Contribution seems to be a bit overstated. The paper claims that it is the first work to offer non-asymptotic convergence rate. However, a prior work BLUR (Reisizadeh, Hadi, et al. 2026) proves non-asymptotic rate with similar assumptions. While the rate on this paper is better, the first seems to be an overstatement.

  2. Given a λ\lambda, CLEAR returns a stationary (KKT) point. And based on my understanding, each λi\lambda_i is a coefficient that decides relative significance of its corresponding objective. In other words, λi\lambda_i decides how small fif_i should be related to the others. Then how should a practitioner choose λ\lambda? Also what should be the magnitude of it? The paper should describe at least a heuristic to choose λ\lambda. The experiments and appendix describes λ\lambda as a searchable hyperparameter (the best λ\lambda was chosen from sweeping 100+ preference vectors and the best one was (10, 0.001, 1, 1).). If so, this significantly reduces practicality of the proposed method as these hyperparameters has to be manually searched without any principled rule.

  3. Equation 2 defines ffg=−CEf_{fg} = -CE, which has the range of (-inf, 0]. With my understanding, the constraint ffg≤εfgf_{fg} \leq \varepsilon_{fg} is then an upper bound to ffgf_{fg} and −CE-CE can grow indefinitely. So it seems that the bottleneck does not work and thus λfg\lambda_{fg} does not contribute. Also, Chevyshev assumes positive objectives and Theorem 1 also states fi>0∀i∈If_i > 0 \quad \forall i \in I. So it seems that the paper's theorem does not apply for forget loss.

  4. Theorem 1 and 2 seems to be defining λ\lambda on the simplex, but the experiments does not follow it and the ∑λ\sum\lambda varies. The paper needs to explain this.

  5. Equation 5 shows that the magnitude of λ\lambda contributes to the bottleneck rr as λ\lambda is effectively multiplied inside of the summation. Simultaneously λ\lambda's magnitude does not affect ss. What is the design principle of this? The paper needs to explain why λ\lambda contributes to rr but not ss.

  6. Fairness in comparison For each objective the paper use different datasets, meaning that CLEAR used four datasets to unlearn. However, it is not clear if baselines are also offered with same dataset. If the four datasets were only offered to the proposed method and not the baselines, then it seems to be not a fair comparison as the proposed method had this advantage.


Reisizadeh, Hadi, et al. "Blur: A bi-level optimization approach for llm unlearning." Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Questions for authors
  1. Please see the weaknesses.
  2. Why did the paper use a quadratic penalty? It is strong but cannot exactly enforces constraints at fixed s = r = 30.

I am willing to increase my score depending on the authors' response. Particularly, if my mathematical derivation is wrong, then please correct it so that I can understand and adjust the score accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.