- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MUSR, a router-free method for knowledge unlearning. The method add small modules per source parallel to MLP layers and modifies the MLP layers output to be a sum of the original MLP and of all introduced modules. During training a loss term pushes each module to activate only on its dedicated source and to stay zero on other sources. To unlearn a source the dedicated module’s W0 matrix is set to zero and all other modules stay intact. The method therefore requires no router.
Critical strengths
MUSR is a router-free method that wraps around the base model, so the initial model weights remain untouched for fine-tuning setups The approach is tested across different model families and training regimes, pre-training and fine-tuning, and is compared against a wide selection of baselines Almost all of the post-MUSR model’s utility is maintained after unlearning a source (Figures 3, 4) MUSR preserves model’s utility better than GRAM and APA on TOFU (Table 2)
Critical weaknesses
-
If a model doesn't produce the facts in generation, it's not clear if the knowledge is still stored in the model, but just has a smaller probability of being decoded. The experiment in Appendix, Figure 8, shows that probes on the retained modules’ outputs reach 0.911 AUC, versus 0.562 for the model trained without forget10. The relearning experiment, Figure 9, suggests the facts are not recoverable, but a white-box attacker could still determine which authors were unlearned.
-
Generalization to broader domains, for instance, unlearning the entire organic chemistry to a separate module, remains untested on large models (the paper covers a pre-training stage on a 200M model with a small K=4)
-
The storage overhead / parameter count and FLOPs scale linearly with the amount of sources. Efficiency claims (Table 16) rely on batch_size=1 tests and show that at K=200 MUSR's extra FLOPs is 12.73% vs 0.06% of GRAM
-
In the pretraining experiments, MUSR’s cross-entropy on the unlearned source after unlearning overshoots the retraining reference (3.002 vs 2.709), while GRAM (2.700) matches it more closely, which with retraining set as a standard indicates over-unlearning. The paper shows that most of the gap comes from LISP (Table 7), so the overshoot is concentrated in the source most different from most of the data. GRAM shows a similar pattern with smaller absolute changes in cross-entropy
-
Table 12 shows that on LUME with Llama-1B, MMLU drops from frozen base’s 0.455 to 0.376 with modules attached, and rises to 0.427 when the forget modules are removed. It seems that self-routing doesn’t fully suppress modules on OOD inputs, and deletion changes the model’s behavior on non-source prompts
Because of weaknesses 4 and 5 I am leaning to rejection.
Update: the authors have provided experiments resolving weaknesses 4 and 5
Questions for authors
-
How do you check retention of facts that are partially overlapping with unlearned knowledge? If a fact is duplicated for two different authors, would unlearning one of the authors do anything with the other?
-
Could you provide more detailed analysis on whether modules fire on non-source queries, for instance MMLU questions?
-
Could you report the per-source retraining cross-entropy for Table 1, to verify that the overshoot relative to retraining comes mainly from LISP?
-
Extension to safety: RMU is used as a baseline, but it would be curious to know how MUSR compares against RMU on WMDP benchmark
4 comments
Q1: Unlearning facts shared across sources
We evaluated both naturally overlapping information and explicitly shared facts on TOFU. In the original data, we identified retained questions whose answers share an entity with forget10, such as a country mentioned for both a retained and an unlearned author. After unlearning, their recall remains at , decreasing by only .
We also constructed cases where retained and unlearned authors share the same attribute value. For example, if authors A and B both speak Spanish, deleting A should preserve “B speaks Spanish” while unlearning “A speaks Spanish”. After unlearning, recall on these facts is for retained authors and for unlearned authors, compared with and for retraining. These results support MUSR’s ability to unlearn author-specific associations while preserving shared information about retained authors. We have added the results in Appendix E.9, Table 24.
Q4: Comparison with RMU on WMDP
To address the reviewer’s request for a comparison with RMU on hazardous knowledge, we ran additional experiments in our realistic pretraining setting. WMDP tests knowledge already present in pretrained models, whereas our fine-tuning setup removes information learned through added modules around a frozen backbone. Our pretraining setting instead introduces the domain data during modular training. We use it as a controlled proxy for WMDP because it includes cybersecurity, biology, and nuclear-science sources relevant to hazardous knowledge, while providing a retrained reference for each removed source.
We therefore compare MUSR with RMU, GradDiff, NPO, and SimNPO. RMU preserves retained performance but raises cross-entropy on the unlearned sources to , compared with for retraining. NPO reaches after optimization steps per deletion request. MUSR reaches through module deletion alone, with an average cross-entropy change of only on retained data. We have added these comparisons in Appendix C.3, Table 9.
We hope that the revised pretraining results and the MMLU analysis address the two concerns that led to the rating, and we would appreciate it if the reviewer would consider raising the score in that case. Please let us know if further clarification would be helpful.
References: [1] Stanford Alpaca: An Instruction-following LLaMA model, Taori et al., 2023
I thank the authors for additional experiements. I believe my main concerns have been addressed (MUSR no longer overshoots retraining by a large margin and it's MMLU performance is closer to frozen). I will raise my score
We thank the reviewer for the constructive feedback and for raising the score. Your comments helped us improve the pretraining removal results and the preservation of MMLU performance. We are glad these changes address your main concerns and are happy to answer any further questions.
We thank the reviewer for the detailed feedback. We appreciate the recognition of MUSR's retained utility and the evaluation across models and training settings.
The white-box membership probe detects a residual trace after deletion, but requires access to internal module activations and labeled examples for training the probe. Our black-box membership inference evaluations consider a common deployment setting with output-only access and show no significant evidence of leakage after unlearning (Figure 3(d) and the per-model results). This difference in results arises because the white-box probe examines the effect of negative samples on the retained modules. During training, these modules learn to remain quiet on forget-source examples, and that pattern persists after the corresponding source modules are removed. To test whether this trace helps recover the deleted answers, we additionally ran two relearning attacks that fine-tune the model on forget data after unlearning. Neither attack recovers the answers faster than the corresponding control trained without forget10.
We agree that domain breadth and model scale are evaluated separately in our experiments. The 200M-parameter pretraining experiment assigns LISP, biology, nuclear science, and cybersecurity to separate removable modules, with an average cross-entropy change of only 0.002on retained data after deletion. Our larger-model experiments evaluate source-level removal on Llama-3.1-8B and Qwen2.5-7B with 200 modules on TOFU. These experiments together demonstrate domain-level removal during pretraining and source-level removal at larger model sizes.
To address the reviewer’s comment that our efficiency measurements used only batch size 1, we ran additional experiments with batch sizes up to 256 on Llama-1B and Llama-8B. At K=200, MUSR has lower generation latency than GRAM at every tested batch size from 1 to 256 on both Llama backbones. At batch size 256, MUSR takes 6.425 seconds versus 6.677 for GRAM on Llama-1B, and 23.647 versus 24.616 on Llama-8B. Dense evaluation avoids the module-selection operations of routed evaluation. We have added the measurements in Appendix F, Table 26.
We addressed the pretraining overshoot by training the backbone alone on a fraction of the shared core batches. Previously, the backbone was always trained with the source modules active. We now switch off all modules on 80% of shared core batches, reducing the backbone's dependence on their outputs. Source batches retain the same module-specific training rule.
With this change, MUSR reaches an average unlearned-source cross-entropy of 2.764±0.010, compared with 2.711±0.002 for retraining (Table 1). The per-source comparison is reported in Table 8 in Appendix C.2 and summarized below.
The gap to retraining is now 0.079 for LISP. Source learning remains strong, with pre-unlearning cross-entropy of 2.468, compared with 2.519 for GRAM. The revised training therefore brings unlearning closer to retraining while preserving source learning. We have updated Table 1 and the per-source heatmap in Table 7 with these results. The training change and its ablation are described in Appendix C.2, Table 8 of the revised manuscript.
The MMLU decrease also happens under standard adaptation to LUME. On Llama-3.2-1B-Instruct, joint fine-tuning reaches 0.348±0.010, retraining reaches 0.330±0.008, and GRAM reaches 0.395±0.012, compared with 0.455 for the frozen base. To address this for MUSR, we extended the output penalty with general-text negatives from Alpaca [1], reducing module output norms on MMLU from 18.8% to 5.3% of their own-source norms.
This brings MMLU close to the frozen base while preserving Retain and Forget performance, without additional inference computation. We have added the analysis and results in Appendix D.3, Table 18.