acceptodds
Under review as a conference paper at ICLR 2027

URGE: A Unified Representation-Guided Loss Reweighting Framework for Fine-grained Machine Unlearning

Abstract

Machine unlearning is an emerging paradigm to remove specified data influences from trained models, with broad applications across image classification, diffusion models and large language models. A prominent direction is gradient-based unlearning (GU), which designs forgetting objectives and retain regularization to balance forgetting and utility. Loss reweighting further improves GU by explicitly assigning different update strengths to samples or tokens based on their forgetting needs. However, existing methods face two key limitations: (i) effectiveness: prediction-distribution-based methods rely on signals that are partially redundant with the original optimization gradients; and (ii) applicability: other methods require external auxiliary models, retain data, or task-specific designs. In response, we propose **URGE**, a **U**nified **R**epresentation-**G**uided r**E**weighting framework for GU. URGE constructs weights for each sample/token from two representation-based signals: Conditional Representation Invariance estimates its intrinsic need for forgetting, while Representation Similarity estimates its remaining need for further forgetting. By leveraging internal representations, URGE captures richer information about how samples/tokens are encoded in the model, improving unlearning effectiveness. Without requiring external models or retain data, URGE is applicable across GU methods, tasks, forget targets, and granularities. Experiments against 18 baselines across 3 tasks and 13 model-dataset settings show that URGE generally achieves a better forgetting-utility trade-off than GU and reweighting baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Loss Reweighting, Image Classification, Large Language Model, Diffusion Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ycP8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes URGE, a loss-reweighting framework for machine unlearning that uses internal model representations to determine how much each sample or token should be forgotten. It combines two signals: one measuring how stable representations are under perturbations, and another tracking how much they change during unlearning. The method can be applied to existing unlearning approaches without requiring additional models or retain data for computing the weights. Experiments on image classification, diffusion models, and LLMs show generally improved forgetting–utility trade-offs compared to existing baselines.

Critical strengths
  • URGE can be integrated into several existing unlearning methods, which is great to make it broadly applicable.

  • The proposed weighting mechanism does not require retain data or additional models, which is a practical advantage.

  • The two proposed signals are simple and easy to understand.

  • I appreciate the evaluation across three different settings: image classification, diffusion models, and LLMs.

  • The paper includes extensive ablation studies that help assess the contribution of the different components.

Critical weaknesses
  • Assumptions and applicability

The method relies on two assumptions that are not sufficiently justified: that representation invariance reflects the importance of forgetting a sample, and that changes in representations indicate successful forgetting. A model could substantially change its hidden representations while still retaining the memorized information, simply encoded differently.

Additionally, the proposed framework assumes a decomposable forgetting objective. It is unclear how URGE would apply to other gradient-based approaches, such as adversarial (i.e. AMUN), or bilevel methods. Can the authors clarify which methods URGE supports and how it could be extended to objectives that are not naturally decomposable into sample- or token-level losses? The framing of this needs to be improved in the manuscript.

  • Image classification experiments

The classification results are strong, but the MIA evaluation is insufficient. I would like to see stronger membership-inference attacks, such as LiRA or RMIA. Additionally, the average gap in the literature is usually computed also with the fourth metric of privacy, the authors should include MIA in the overall evaluation and reporting an extended average gap to retraining. Accuracy is not enough for forgetting! Additionally, it would be useful to evaluate on a more challenging dataset, such as Tiny ImageNet, since CIFAR-10 and CIFAR-100 are relatively simple, low-resolution datasets.

  • Diffusion experiments

For most baselines, forgetting accuracy is already close to 0%, making it difficult to assess whether URGE actually improves forgetting. The results mainly demonstrate better FID. I would like to see more challenging forgetting scenarios where the benefit of URGE is clearer for unlearning.

Furthermore, the diffusion experiments only evaluate class forgetting. Given the claims of general and fine-grained unlearning, I would like to see random instance-level forgetting, even with a signal forget set %. Finally, classification accuracy alone is not sufficient to establish successful forgetting. I would like to see diffusion-specific MIAs or other privacy metrics.

  • LLM experiments

The LLM results are mixed, and I find some of the claims too strong given the experimental evidence. Retain extraction is often substantially worse than retraining, and adding URGE does not always improve performance. For example, on Llama-2-7B, GA+URGE considerably worsens forgetting extraction compared to GA. Can the authors explain this behavior?

Similarly, on MUSE, URGE improves some metrics while degrading others. I would like to see stronger and more consistent privacy evaluations across the LLM benchmarks too. These mixed results should also be discussed more clearly as a limitation.

  • Validation of CRI and RS

I am not convinced that the proposed signals actually measure memorization or residual forgetting.

For CRI, high representation invariance could simply reflect robust or generalizable features rather than memorization. For example, a model might have memorized a rare fact about a fictional author, but replacing the original question with a generic prompt such as "Tell me about the author" could substantially change the representations without removing that information.

Similarly, in image classification, adding noise to model parameters might simply measure representation robustness to parameter perturbations. This is also quite different from changing the input prompt in LLMs. Why should these two approaches measure the same notion of forgetting importance?

More generally, the authors should validate CRI and RS against direct memorization and forgetting measurements and see if these actually correlate! Comparisons with simple controls, such as random weighting, token frequency, or uncertainty-based weighting, are also needed to establish whether the improvements come from the proposed signals or simply from changes in optimization.

  • Approximate unlearning and guarantees

URGE does not provide certified guarantees, indistinguishability bounds, or proof that the target information has been removed. This is understandable for an approximate-unlearning method, but I would like the authors to acknowledge this limitation more explicitly, particularly when interpreting representation changes as evidence of forgetting.

Questions for authors

Please see the section on weaknesses and tackle the points raised. In general, the idea is interesting, but I am concerened as to how much the results actually validate the assumptions/hypothesis. I am not confident that the statements on how invariance relates to forgetting are trivial and need to see more results (see above) to make sure it is correct/valid. Finally, there are some limitations in the evaluation (see above) that I would like to see tackled, and very important, the work needs robust privacy metrics (RMIA, Lira, ...) as accuracy itself is not enough to claim unlearning.

Reviewer_vpcKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an unified unlearning framework by constructing two types of scores, a static score (Conditional Representation Invariance) and a dynamic score (Representation Similarity). These scores are combined to reweight the unlearning loss of a gradient-based unlearning solution. The framework is tested without a retain dataset across many settings.

Critical strengths

The unified framework is validated on 3 tasks/models including image classification, diffusion models and LLMs across 13 model-dataset settings. The framework is simple but applicable for any gradient-based unlearning solutions.

Critical weaknesses

None

Questions for authors
  1. It seems the results are not able to support the conclusion that the approach is universally superior. For example, the results on ResNet-18,CIFAR-10 (like metric UA) and Llama2-7B (like metric ES-u) in Table 1 and 3, respectively.
  2. Even though the proposed method is gradient-based and not requiring retain data, it would be more valuable to compare with other class of unlearning methods which exhibited good performance on both of classification and generative tasks as well, like SCRUB+URGE vs Ref.1 or Ref.2.

Ref.1: DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers, NeurIPS 2025. Ref.2: Machine Unlearning via Adaptive Gradient Reweighting and Multi-stage Objective Optimization, CVPR26.

Reviewer_yojjOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The work posits that loss reweighting for unlearning is limited by effectiveness in that the existing methods may rely on partially redundant signals, or applicability wherein the methods are reliant on external models or data and proposes the method URGE which is guided by representations, eschewing the requirement for retain sets in learning the weights. Memorization is quantified using the Conditional Representation Invariance (CRI) score measured by a unit's "forming condition's" perturbation and a dynamic Representation Similarity (RS) score that allows for adaptive scaling and early stopping of the unlearning signals. The setup considers image classification, generation and text generation and is evaluated for the same.

Critical strengths
  1. The motivation is well informed. Unlearning difficulty could be quantified and then incorporated into the objective.

  2. The breadth of coverage although adapted from extant works is good. Three tasks, several baselines and settings.

  3. The runtime is reported and the method is efficient.

  4. The ablations are informative. CRI seems to be a good metric as labelled by GPT.

Critical weaknesses
  1. The method reduces to a per-unit learning rate schedule with dynamic early stopping, while the baselines are run over fixed steps (?) with no lrlr tuning raising questions regarding comparison fairness. A fair comparison could incorporate tuning and a global early stop at a matched forgetting rate (UA) or with a uniform weight as learned from URGE's mean weight and could enhance the robustness of the comparisons.

  2. The method section enumerates a procedure, not a motivation, explanation or an argument. The condition-selection principle (finest-grained condition shared by all units with the smallest sharing scope) and equation 5 are stated without any explanation as to why a representation that is stable should mean "memorized". No explanation for why and how the formulation is created and how it takes the final form and what the λ\lambda values' roles are beyond their instantiation. The noise scales aren't specified and there isn't any specification of the various hyperparameters used for URGE or the baselines.

  3. The method could delineate whether the representations component adds to the procedure or only dampens the gradient ascent procedure. The dynamic part could only be the early stop and a study of this phenomenon could help with the motivation for the work.

  4. Sample level CRI is justified only for the text-only question swap augmentation. The authors could incorporate an evaluation of the metric for the other setups as well. Additionally, the notion of stable representation under parameter noise conflates memorization with generalization. Well generalized samples could also exhibit stability and erasure may aim to preserve this generalization as quantified by certain bounds.

Questions for authors
  1. Could the authors kindly add the hyperparameter details.
  2. Could the weights be pre-set to the mean weight of the URGE method?
  3. Could the authors kindly add the baselines' parameter details.
  4. Could the authors clarify how sample-specific questions are constructed for the text-only forget data? What was the procedure and was it a LLMaaJ procedure?
  5. Could the authors report the final RS metric on the forget set for all methods? This could serve as the work's own metric as a measure of forgetting.
  6. Could the authors clarify the rationale behind the RS formulation? It looks like a monotonic transformation of the similarity score and could be much simpler.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.