acceptodds
Under review as a conference paper at ICLR 2027

CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models

Abstract

Large language model (LLM) unlearning aims to remove targeted information while preserving useful capabilities without retraining from scratch. Methods such as NPO, SimNPO, and SatImp update the full model, while FOM-UL explores selective layer updates. Suppressing target information can damage retained knowledge, and this trade-off varies across layers even under the same unlearning loss. We propose Confidence-Weighted Layerwise Unlearning (CoLU), which combines probability-weighted forgetting with retain regularization while updating a single selected transformer layer. To reduce layer-selection cost, we introduce Brief Layer Screening (BLS), which ranks short unlearning trials on a subset of the training data using a normalized forgetting-retention (NFR) score. The shortlisted candidates then undergo independent full-data unlearning, and the highest-scoring model is selected. Compared with full screening, BLS reduces total training time by 60.2% to 79.7% on TOFU while preserving much of its performance. CoLU improves both forgetting and retention over NPO, SatImp, and GA-based FOM-UL in all six TOFU model and forget-split settings, and over SimNPO in five. On MUSE Books and News, CoLU achieves the highest retained KnowMem and the PrivLeak closest to zero among the compared unlearning methods. Across the measured TOFU and MUSE settings, average GPU memory usage for single-layer unlearning is 77.4% to 79.5% lower than for full-model unlearning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large language models, machine unlearning, layer selection, parameter-efficient unlearning, knowledge retention.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tvjiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CoLU performs gradient ascent on the forget set, weighting each target token by a power of its current probability, while adding a cross-entropy regularizer on the retain set. Each run updates one complete Transformer block. BLS briefly trains every layer on a data subset, shortlists three candidates, then retrains each candidate from the original checkpoint on the full data and selects the final model by NFR. Experiments cover Phi-1.5 and Llama-2-7B on TOFU and the Books and News domains of MUSE, with results for effectiveness, screening time, and memory use.

Critical strengths
  1. The parameter constraint has practical value. Updating one layer substantially reduces peak memory. The paper also reports end-to-end screening cost rather than only the cost of a single training run.
  2. The layerwise analysis is informative. Full sweeps show that effective layers cluster near the beginning and end of the models. BLS includes at least one of the top three full-sweep layers in all six TOFU settings, making low-cost layer screening a promising direction.
Critical weaknesses
  1. Model selection and final evaluation are not independent. FS and BLS use NFR to select layers and models, while NFR is computed from the same forget and retain metrics later reported as final results. MUSE additionally selects the best checkpoint across epochs. Without independent validation and test splits, the selected best values are difficult to interpret.
  2. The method's contributions are not isolated. The token probabilities are not validated as calibrated confidence estimates, and the experiments omit WGA, uniform weighting, calibrated weighting, and controls that apply the same BLS procedure and retain regularization to baselines. The current evidence cannot determine whether the gains come from probability weighting, retain regularization, or the capacity constraint of updating one layer.
Questions for authors
  1. If layers, hyperparameters, and epochs are selected on a fully independent validation split and evaluation is performed only once on a held-out test split, do CoLU's gains over SatImp, SimNPO, and Retrain remain significant, with confidence intervals?
  2. Do the BLS screening subset, NFR evaluation set, official test set, and hyperparameter-tuning data overlap? For MUSE, which split and metric are used to select the best checkpoint?
  3. Under the same search budget and across multiple random seeds, does CWA consistently outperform WGA, uniform, rank-based, and calibrated weighting, as well as baseline objectives equipped with the same BLS procedure and retain regularization?
5 comments
AuthorsOpenReview ↗

Rebuttal to reviewer tvji

Thank you for your detailed questions about model selection and the roles of the individual components. We clarify the evaluation protocol and provide an additional validation-based layer-selection study below.

W2. Evidence for the individual components

We respectfully disagree with this characterization of the experimental evidence. Table 1 includes full-model CWA, which uses CWA and retain regularization while updating the full model. CoLU (BLS) improves both forget and retain ES over full-model CWA in five of the six TOFU settings and over full-model SatImp in all six settings. This comparison supports the benefit of combining CWA with layerwise updates. Table 3 further compares layerwise unlearning with full-model unlearning under GA, NPO, SatImp, and CWA on Phi-1.5 Forget-5%. With each layerwise result selected by NFR from a full sweep, layerwise unlearning achieves higher NFR for all four objectives. Finally, Table 11 in Appendix C shows that removing retain regularization substantially reduces retention. Together, these comparisons support the roles of each component in CoLU.

Q1(W1). Validation-based layer selection

On Phi-1.5, we partitioned the original 400-example retain evaluation set into 200 validation examples and 200 test examples. For layer selection, NFR uses retain ES from the validation subset; the test subset is excluded from this selection and used to evaluate retained performance. The forget evaluation set remains unchanged. The table reports forget ES on this set, retain ES on the 200 test examples, and NFR computed from these reported scores.

SplitMethodSelected layerForget ES ↓Retain ES (test, n = 200) ↑NFR ↑
Forget-1%GAFull model0.0990440.19327956.30
Forget-1%SatImpFull model0.0543180.57530487.96
Forget-1%SimNPOFull model0.1972030.62098479.91
Forget-1%CoLU (BLS)190.0345010.61905992.76
Forget-5%GAFull model0.1463640.27934260.07
Forget-5%SatImpFull model0.0420300.55221887.59
Forget-5%SimNPOFull model0.1644210.59427381.93
Forget-5%CoLU (BLS)20.0021480.53992689.54
Forget-10%GAFull model0.0988290.16889355.40
Forget-10%SatImpFull model0.0341470.45357880.92
Forget-10%SimNPOFull model0.1926830.56372477.76
Forget-10%CoLU (BLS)20.0024080.55838990.88

CoLU (BLS) achieves the lowest forget ES and highest NFR among the compared methods in all three splits. Its NFR exceeds SatImp by 4.80, 1.95, and 9.96 points, and SimNPO by 12.85, 7.61, and 13.12 points, respectively. On Forget-1% and Forget-10%, its retain ES is within 0.006 of SimNPO, while its forget ES is substantially lower. These results support the effectiveness of validation-based layer selection when retained performance is evaluated on separate test data.

Q2. Evaluation data and MUSE checkpoint selection

Unlearning targets information in the forget set, so we evaluate forgetting on the data designated for removal (train=test). The retain evaluation set is separate from the retain data used for optimization. In the original experiments, BLS uses retain evaluation data when scoring candidate layers. Performance-based tuning on hyperparameters also appears in previous methods such as WGA [1], which uses UWC and reports forget and retain ES across hyperparameter sweeps. Our additional study in Q1 separates the retain data used for layer selection from those used for final evaluation which further supports the effectiveness of CoLu.

For MUSE, layer and checkpoint selection use NFR from Eq. (3), computed on the final evaluation sets. NFR equally weights the mean relative reduction in forget-set VerbMem and KnowMem and normalized retain-set KnowMem. PrivLeak is reported but not used for selection. We use this NFR to rank BLS candidates and select the best checkpoint across the eight full-data unlearning epochs. We follow prior MUSE studies in selecting checkpoints based on evaluation performance. SatImp [2] selects among ten epoch checkpoints, prioritizing PrivLeak, while SimNPO [3] evaluates after each epoch and selects the best-performing model.

AuthorsOpenReview ↗

Rebuttal to reviewer tvji Cont.

Q3. Comparisons with alternative objectives and weighting schemes

Uniform weighting corresponds to GA in our formulation. Our additional three-seed results on Phi-1.5 compare CoLU with GA, NPO, SatImp, WGA, rank-based weighting, and calibrated CWA, all using BLS. CoLU achieves the highest mean NFR among the evaluated methods in all three forget splits.

MethodForget-1% NFR ↑Forget-5% NFR ↑Forget-10% NFR ↑
WGA + BLS92.6228 ± 0.484586.9112 ± 1.274889.6009 ± 0.8902
GA + BLS (uniform weighting)70.5212 ± 4.610970.5613 ± 3.201867.0881 ± 1.2646
NPO + BLS83.9244 ± 2.053882.5425 ± 0.781482.0721 ± 2.2633
SatImp + BLS89.0712 ± 3.130088.9464 ± 2.504481.6678 ± 0.3125
Rank-based + BLS84.1644 ± 5.605176.8766 ± 3.460875.6105 ± 1.7871
Calibrated CWA + BLS88.0661 ± 2.001276.2384 ± 1.854181.0271 ± 0.4637
CoLU (BLS)95.0355 ± 0.469390.0832 ± 0.918391.6610 ± 0.1553

CoLU achieves the highest mean NFR among all compared BLS-based methods across the three Phi-1.5 forget splits: 95.04, 90.08, and 91.66. These results support its improved forgetting-retention trade-off relative to the evaluated objectives and weighting schemes.

Conclusion

The component ablations support the roles of layerwise updates and retain regularization, and the additional retain validation/test split supports validation-based layer selection. The three-seed BLS comparisons show the highest mean NFR for CoLU among the evaluated methods across all three Phi-1.5 splits, supporting its combination of CWA, retain regularization, and single-layer updates with BLS for layer selection. We respectfully ask you to reconsider your assessment and raise the score in light of these clarifications and additional evidence.

Sources

[1] Wang et al. Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond. ICLR 2025. Paper.

[2] Yang et al. Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning. ICML 2025. Paper.

[3] Fan et al. Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning. NeurIPS 2025. Paper.

Dear Reviewer tvji, as the discussion period draws to a close, we would appreciate your consideration of our additional validation-based layer-selection study and three-seed BLS comparisons addressing your evaluation and component-analysis concerns. We hope this evidence supports a more favorable assessment and an increase in your score. Thank you for your time.

Reviewer_GkoqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Confidence-Weighted Layerwise Unlearning (CoLU), which combines probability-weighted gradient ascent with retain-set regularization while updating a single transformer block. To select the block efficiently, Brief Layer Screening (BLS) ranks short unlearning trials using a normalized forgetting–retention score, then fully trains three shortlisted candidates. Experiments on TOFU and MUSE report improved forgetting–retention trade-offs and substantially lower memory usage than full-model updates.

Critical strengths
  1. CoLU achieves strong retain-set extraction scores on TOFU and the highest retained KnowMem among the compared unlearning methods on both MUSE domains.
  2. The paper compares BLS with exhaustive search, gradient-based selectors, and random selection, and reports total training time rather than only the final update cost.
Critical weaknesses
  1. The framework figure looks AI-generated and should be revised to improve clarity and readability.
  2. The contribution statements would benefit from focusing on the paper’s original advances, with citations to prior work moved to the surrounding discussion.
  3. CWA is a scaled adaptation of existing probability-weighted ascent, and selective-layer updates have established precedents. The contribution is primarily their combination with trial-based screening; stronger comparisons with WGA under the same selection procedure would clarify its value.
  4. Low extraction strength is useful evidence but does not demonstrate resistance to paraphrased extraction, adversarial prompting, or relearning.
Questions for authors

see weakness

5 comments
AuthorsOpenReview ↗

Rebuttal to reviewer Gkoq

Thank you for pointing out where the presentation of our contributions and evaluation could be clearer. We clarify these points below and provide additional comparisons with WGA.

W1 and W2. Figure and contribution statement

The framework figure illustrates BLS screening, full-data unlearning of shortlisted candidates, and final model selection. We will revise the diagram with clearer arrows, and an explicit distinction between the updated layer and frozen parameters. We will also move citations to prior work from the contribution statements to the surrounding discussion so that the statements focus more clearly on our original contributions.

W3. Contribution of CWA and comparison with WGA

CWA is a scaled adaptation of WGA, with a tunable scale, A, that controls reweighing strength which give higher tuning flexibility. We compare CWA and WGA under full-model unlearning and the same BLS procedure.

full-model results on Phi-1.5 are shown below:

SplitMethodForget ES ↓Retain ES ↑NFR ↑
Forget-1%CWA0.0161570.51356786.9743
Forget-1%WGA0.0130410.47225084.1447
Forget-5%CWA0.0112310.49033485.7299
Forget-5%WGA0.0144820.44873482.3979
Forget-10%CWA0.0371660.49427384.1877
Forget-10%WGA0.0382130.48495883.4194

CWA achieves higher NFR and retain ES than WGA in all three splits, with NFR gains of 2.83, 3.33, and 0.77 points. It improves both ES metrics on Forget-5% and Forget-10%; on Forget-1%, higher retention comes with slightly higher forget ES. These results support adjusting the forgetting scale to improve the forgetting-retention trade-off.

on Phi-1.5 Forget-5%, WGA uses the same BLS screening and candidate unlearning budgets and retain regularization settings as CoLU.

MethodSelected layerForget ES ↓Retain ES ↑NFR ↑
WGA + BLS200.0350220.51294485.71
CoLU (BLS)230.0079100.54964090.39

Under this shared BLS procedure, CoLU reduces forget ES by 77.4% and increases retain ES by 0.036696 and NFR by 4.67 points relative to WGA.

Our main contributions are the CoLU framework, which combines CWA and retain regularization with single-layer updates, and BLS with superior results compare to previous methods. Table 3 supports the layerwise design, with higher NFR for layerwise GA, NPO, and SatImp than their full-model counterparts on Phi-1.5 Forget-5%. Table 4 evaluates BLS under the same CoLU objective: it achieves a 100% shortlist hit rate and 50.0% overall Recall@3 across six TOFU settings, compared with 66.7% and 27.8% for FOM-UL ratio.

W4. Evaluation beyond extraction strength

We follow the TOFU and MUSE benchmark settings used by recent state-of-the-art methods to compare forgetting and retention under common evaluation metrics. Our evaluation extends beyond extraction strength: Table 2 reports MUSE's KnowMem, which probes factual recall through questions derived from the forget corpus, providing complementary evidence relevant to paraphrased extraction beyond verbatim continuation. PrivLeak evaluates membership-inference attacks relative to a retrained reference [1]. It addresses a related concern to adversarial prompting: whether an attacker can still obtain information about the forget data after unlearning.

On MUSE Books, CoLU achieves forget-set KnowMem of 0.00 while preserving retain-set KnowMem of 61.85. On News, it reduces forget-set KnowMem from 62.93 to 12.81 while preserving retain-set KnowMem of 43.01. CoLU also achieves PrivLeak closest to zero among the compared unlearning methods in both domains. Together with the TOFU results and ablations, these findings support CoLU's effectiveness in reducing target-information extractability while preserving retained performance.

Following the benchmark settings used by prior work, our results across TOFU and MUSE demonstrate CoLU's effectiveness in reducing target-information extractability while preserving retained performance. We appreciate the suggestion to evaluate relearning; given the limited rebuttal period and its extension beyond our current evaluation scope, we leave this experiment for future investigation.

Conclusion

The additional WGA comparisons show higher NFR for CWA in the evaluated settings, while MUSE provides evidence on factual recall and membership leakage beyond extraction strength. Together with the layerwise ablations and selection results, these findings support CoLU's combination of CWA, retain regularization, and single-layer updates, and the use of BLS to select effective candidate layers with superior performance. We respectfully ask you to reconsider your assessment and raise the score based on these clarifications and additional results.

AuthorsOpenReview ↗

Rebuttal to reviewer Gkoq Cont.

Sources

[1] Shi et al. MUSE: Machine Unlearning Six-Way Evaluation for Language Models. 2024. Paper.

Dear Reviewer Gkoq, as the discussion period draws to a close, we would appreciate your consideration of our clarifications and additional WGA comparisons addressing CoLU’s contribution, alongside the evaluation beyond extraction strength. We hope these responses address your concerns and encourage you to reconsider your assessment and score. Thank you for your time.

Reviewer_Z3K1OpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This work explores targeted LLM unlearning where a single preselected layer candidate is trained with a "confidence weighted ascent" (CWA) loss. The preselection extracts K layers based on unlearning performance on a small subset of the unlearning training set, measured by a composite metric named "Normalized Forgetting–Retention (NFR)", which depends on forget and retain set performances from the target task. Each of the K layers is then independently evaluated for full unlearning on the target task using the CWA loss, leading to K unlearned models, and the best candidate is selected among them based on unlearning task performance.

Critical strengths
  1. Soundness

    • The presented approach of preselecting a layer for unlearning is promising and has been explored effectively in several prior works. The authors extend this by utilizing unlearning performance to select the layers.
    • The NFR composite metric and the CWA loss term which the authors devise are reasonable.
  2. Experiments

    • The authors present experiments with two popular benchmarks (TOFU and MUSE) and compare against a number of state of the art baselines.
    • They include several additional results of interest, comparing against other layer selection methods, full data unlearning based layer selection, along with analysis of runtime costs.
  3. Results

    • The presented results are promising and show improvement against established baselines.
Critical weaknesses
  1. Missing analysis:

    • While the authors do present fairly detailed results, they need to also include the runtime costs consumed by prior layer selection algorithms such as SLUG/SURE/FOM-UL.
    • Given that prior approaches make use of gradient importance features, they are expected to be faster since they need a single forward+backward pass.
    • Since most of them perform comparably with the approach presented here, it would be interesting to get a holistic picture on the additional cost paid for the improvement shown in this work.
  2. Statistical significance of results:

    • It is unclear if the presented gains are statistically significant since the margins are small.
    • Authors study variance from different training seeds in Appendix D, but don't compare these averages from different seeds against the baselines.
  3. Typos:

    • There are quite a few typos in the draft, listing a few below:
      • line 128: "through a brief unlearning trials"
      • line 162: "measures how each responds to"
      • Capitalize "normalized forgetting–retention" at its first mention
Questions for authors

None

2 comments
AuthorsOpenReview ↗

Rebuttal to reviewer Z3K1

Thank you for your questions about selection cost and the magnitude of the improvements. We provide measured runtimes and numerical comparisons below.

W1. Runtime cost of layer selection

On Phi-1.5 Forget-5%, the overall total runtime is 5.56 minutes for FOM-UL ratio and 5.95 minutes for both SLUG importance and SURE-inspired selection, compared with 9.89 minutes for BLS (1.66×–1.78×1.66\times\text{--}1.78\times):

SelectorOverall total runtime (min)Forget ES ↓Retain ES ↑Winner NFR ↑
BLS (ours)9.890.0079080.54964090.39
FOM-UL ratio5.560.0191820.54291889.08
SLUG importance5.950.0163940.54493889.43
SURE-inspired5.950.0163940.54493889.43

While gradient-based scores are faster to compute through a single forward–backward pass, the 1.66×–1.78×1.66\times\text{--}1.78\times runtime of BLS (still under ten minutes overall) yields much stronger unlearning. Because NFR normalizes by the original model's ES (≈0.70\approx 0.70), small NFR differences correspond to large relative reductions in residual leakage: on Phi-1.5 Forget-5%, BLS reduces Forget ES by 58.8% relative to FOM-UL ratio (0.00791 vs. 0.01918) and by 51.7% relative to SLUG and SURE (0.00791 vs. 0.01639), while also improving Retain ES.

In settings where static gradient norms miss the effective layers entirely, the performance gap is even more pronounced. Across all six TOFU settings (Table 4), BLS achieves a 100% shortlist hit rate and 50.0% Recall@3, compared with 66.7% / 27.8% for FOM-UL ratio, 33.3% / 11.1% for SLUG, and 16.7% / 5.6% for SURE. On Llama-2-7B Forget-10%, for instance, FOM-UL ratio misses the early layers and falls 4.22 NFR points behind BLS:

SelectorForget ES ↓Retain ES ↑NFR ↑
BLS (ours)0.0038790.84905998.4956
FOM-UL ratio0.0251570.79648294.2763

Here, BLS reduces residual Forget ES by 84.6% (6.5×6.5\times lower than FOM-UL ratio) while increasing Retain ES from 0.7965 to 0.8491.

Moreover, the selector comparisons above adopt the layer-selection scores from prior methods while using CoLU's single-layer CWA objective and retain regularization. Against the original FOM-UL method, reported as FU (GA) in Table 1, the performance gap is wider: on Phi-1.5 Forget-5%, FU (GA) achieves (0.249 Forget ES, 0.448 Retain ES), versus CoLU's (0.008, 0.550); on Llama-2-7B Forget-10%, it achieves (0.408, 0.585), versus CoLU's (0.004, 0.849). These results show improvements over both the adapted selection scores and the original FOM-UL method, while the runtime comparison above quantifies the additional cost relative to the adapted selectors.

W2. Magnitude of the improvements and statistical significance

W2.1. Magnitude of the improvements

The additional three-seed results on Llama-2-7B Forget-10% show substantial improvements in both forgetting and retention:

MethodForget PS ↓Retain PS ↑
NPO0.04458 ± 0.004090.52242 ± 0.06468
SimNPO0.12552 ± 0.001790.61005 ± 0.01381
SatImp (β = 0.7)0.02346 ± 0.004350.63735 ± 0.01446
CoLU (BLS)0.00540 ± 0.002000.84045 ± 0.00765

CoLU achieves the lowest mean Forget PS and highest mean Retain PS among all compared methods. Relative to SatImp, it reduces Forget PS from 0.02346 to 0.00540 (77.0%) while increasing Retain PS from 0.63735 to 0.84045 (+0.20310). Relative to SimNPO, Forget PS decreases by 95.7% and Retain PS increases by 0.23040. These three-seed results demonstrate substantial gains in both forgetting and retention over the prior state-of-the-art baselines.

The original Table 1 also shows improvements on Llama-2-7B Forget-5%: CoLU achieves Retain ES of 0.833, compared with 0.709 for SatImp and 0.687 for SimNPO, while reducing Forget ES to 0.003, compared with 0.006 and 0.133, respectively.

AuthorsOpenReview ↗

Rebuttal to reviewer Z3K1 Cont.

W2.2. Three-seed results on Phi-1.5

The table summarizes three-seed NFR results across all three Phi-1.5 forget splits from the experiments in table 1. All methods except CoLU (BLS) use full-model updates.

MethodForget-1% NFR ↑Forget-5% NFR ↑Forget-10% NFR ↑
GA59.20 ± 2.3862.59 ± 2.0255.93 ± 0.46
NPO80.74 ± 0.5881.08 ± 0.1979.06 ± 0.16
SimNPO77.42 ± 0.6280.44 ± 0.9177.62 ± 0.71
SatImp87.31 ± 1.8586.10 ± 0.3982.77 ± 1.07
WGA83.90 ± 0.7182.75 ± 0.9683.10 ± 1.31
CWA87.46 ± 0.7386.28 ± 0.2685.27 ± 0.52
CoLU (BLS)95.04 ± 0.4790.08 ± 0.9291.66 ± 0.16

These three-seed results reinforce the substantial improvements over the prior state-of-the-art baseline SatImp discussed in W2.1. On Phi-1.5, CoLU achieves the highest mean NFR in all three splits, exceeding SatImp by 7.73, 3.99, and 8.89 points, respectively. Together with the three-seed Llama-2-7B comparisons in W2.1, these results show substantial gains over SatImp across both models.

Conclusion

The runtime comparisons show that the modest 1.66×–1.78×1.66\times\text{--}1.78\times overhead of BLS reliably identifies effective layers and delivers much lower residual Forget ES and higher retention than both adapted gradient selectors and full baseline methods such as FOM-UL. In parallel, the three-seed comparisons across Phi-1.5 and Llama-2-7B confirm that CoLU's gains over prior state-of-the-art methods remain consistent across random seeds. We will also correct the noted typos.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviewer Feedback and Rebuttal Results

Dear Area Chair,

Reviewer Z3K1 highlighted our use of observed unlearning performance to select layers and found the gains over established baselines on TOFU and MUSE promising. Reviewer Gkoq noted CoLU’s strong retain-set extraction scores on TOFU and its highest retained KnowMem among the compared methods on both MUSE domains. Gkoq also highlighted the comparisons with exhaustive search, gradient-based selectors, and random selection, together with the reporting of total training time. Reviewer tvji emphasized the practical memory savings of updating a single layer and noted that BLS shortlisted at least one of the full sweep’s three best layers in all six TOFU settings.

In our rebuttal, we addressed the reviewers’ main concerns through additional experiments and clarified our evaluation protocol.

  • Z3K1. We measured total runtime and final performance on Phi-1.5 Forget-5%. In CoLU, BLS layer selection took slightly longer time for the gradient-based selectors, but reduced Forget ES by 51.7% to 58.8%, and improved Retain ES. Across three seeds, CoLU exceeded SatImp’s mean NFR by 3.99 to 8.89 points across the three Phi-1.5 splits.
  • Gkoq. We compared CWA and WGA under the same BLS procedure on Phi-1.5 Forget-5%. CoLU reduced Forget ES from 0.0350 to 0.0079 and increased Retain ES from 0.513 to 0.550 relative to WGA, improving NFR by 4.67 points. With full-model updates, CWA also achieved higher NFR than WGA on all three Phi-1.5 splits.
  • tvji. To prove generalization, we used 200 retain examples for layer selection and a separate 200 for evaluation, keeping the forget set unchanged. CoLU achieved the highest NFR among the evaluated methods on all three Phi-1.5 splits. In a separate three-seed comparison using BLS for every method, CoLU also had the highest mean NFR on each split among the objectives and weighting schemes tested.

CoLU combines confidence-weighted ascent and retain regularization to update a single selected transformer layer. CoLU improves both forgetting and retention over prior state-of-the-art baselines in most TOFU settings. On both MUSE Books and News, it achieves the highest retained KnowMem and the PrivLeak closest to zero among the compared unlearning methods. We respectfully ask you to consider our paper for acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.