- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Carrier-mediated Unlearning Behaviour Transfer (CUT) and "Forget without Seeing", bringing concepts from subliminal learning to the practice of machine unlearning and performing unlearning on personalised language models from an unlearned teacher. The CUT method uses the difference between a model and its unlearned variant to learn a steering control vector and to generate a synthetic training dataset for downstream student models which induces the desired unlearning while maintaining any personalisation adjustments it has undergone. CUT is evaluated against 3 research questions, which discuss how well unlearning is transferred, how much personalisation is retained, and compares CUT against a strong TA baseline. Furthermore, the method has been analysed for data leakage.
Overall this paper makes a novel contribution by bringing subliminal learning to machine unlearning, which is an interesting and promising direction for future research. However, I have substantial questions/clarifications before I can strongly recommend acceptance - and given sufficient discussion in a response from the authors, I would be very open to increasing my score.
Critical strengths
- The problem is well motivated, and well presented with good structuring. Forget without Seeing and CUT are well-positioned to be useful in practical settings.
- The CUT procedure is intuitively introduced and well defined, with a clear structure and rationale as to why it works. Using a directional prior before optimising the vector is a smart and interesting way to bias the learning.
- Experiments and ablations are thorough and cover many bases and are well presented, clearly describing and outlining what each experiment contributes to the justification of CUT.
- The combination of subliminal learning and steering vector learning applied to machine unlearning is an interesting direction to study, with large potential for both fields.
- Performing experiments where the unlearned teacher was produced through a variety of methods was very appreciated, and strengthened the results greatly. Using a retrained model as the "unlearned" teacher would be a good additional experiment but is not necessary.
Critical weaknesses
- The results section is lacking the
forget qualitymetric of TOFU, which is one of the main metrics used to analyse how well a model has forgotten something. Additionally, experiments are reported over 3 seeds, but no statistical significance is mentioned or described. Finally, the raw-p metric is not defined in the main text making it difficult to interpret the results. - The comparison to Task Arithmetic is interesting because TA is a much simpler method, which also doesn't require the forget set. The paper would benefit from a deeper dig into this - how well does TA retain personalisation relative to CUT?
- Discussion of unlearning transfer and personalization preservation deserve separate analysis and discussion as this was difficult to untangle. The evaluation tasks (TOFU, and LaMP-3) would also benefit from brief discussion and why they were chosen.
- In section 5.3.2 it is not clear how much information about the forget set is contained with the steering vector and if the steering vector can be recovered from the constructed pairs
- The provided code does not contain trained model checkpoints, dataset split, the optimisation schedule for the steering vector, or complete training recipes nor metric code. This means I cannot run experiments to independently verify your results. Any attempt by me to reproduce your results would not be possible here - could the authors please provide this?
- It is not immediately clear in the main text how the prompts are synthetically constructed, and the appendix discussion does not clarify why they are constructed like this. The main text would benefit from some discussion on this.
- A concern that comes to mind with such a technique for unlearning is it's reversibility - if someone took all the PTCs and flipped
chosenandrejected, then trained on this, would this recover the unlearned behaviour? - A very mild point is that this work is limited to language model evaluations, and this limitation is not discussed by the authors and the title mis-advertises the work as general unlearning.
Questions for authors
- In this study, the student models are of the same architecture as the teacher model - does CUT generalise to distillation between non-uniform architecture student-teacher setups?
- How does the setting of "Forget without Seeing" differ from others which study unlearning without access to the forget set such as PBU?
- It is not clear how large the generated dataset corpus must be in order to use CUT, how many paired-text carriers must be produced?
17 comments
Author response to Reviewer aYdG (2/6): W3–W4
W3: Separating unlearning transfer from personalization preservation and explaining the benchmarks
We apologize that the original manuscript briefly introduces the benchmarks and their evaluation roles in Section 5.1 (page 6) but does not explain them sufficiently clearly. We provide a more detailed description of the benchmarks and the rationale for their selection below. TOFU is a synthetic QA benchmark for LLM unlearning, containing 200 fictional authors with 20 QA pairs per author, for a total of 4,000 pairs. The questions cover biographical knowledge such as birth background, family relationships, writing genres, works, and awards. The forget set contains QA about designated authors, while the retain set contains QA about the remaining authors. TOFU also provides Real Authors and World Facts to evaluate retention of real-author knowledge and general factual knowledge. Fictional entities make the source of target knowledge and the scope of forgetting explicit and controllable, making TOFU suitable for evaluating entity-specific unlearning behaviour. TOFU
LaMP-3 is the personalized product-rating prediction task in the LaMP benchmark, built from Amazon product reviews. Each example contains a review whose rating must be predicted, its 1–5-star rating, and a user profile consisting of historical reviews and ratings. The task requires the model to use this history to capture the user’s rating habits and preferences and predict the rating of the current review. The benchmark provides both user-based and time-based splits to evaluate personalization for new users and prediction of future interactions for existing users, respectively. MAE and RMSE measure rating prediction error. LaMP
TOFU and LaMP are established, widely used benchmarks for LLM unlearning and personalization, respectively, and have been adopted in subsequent work such as OpenUnlearning and OPPU. Their clear task definitions and established metrics allow us to evaluate the two objectives separately: TOFU tests whether upstream unlearning behaviour transfers, while LaMP-3 tests whether existing personalization is preserved.
W4: Information encoded in the steering vector and its recoverability
We apologize that this point was not sufficiently highlighted in the original manuscript. We would like to clarify that steering-vector recovery and the corresponding privacy attacks were already evaluated in Sections 5.3.2 and 5.3.3 (page 8), Appendix E.2.1 (page 27), and Appendix E.4.1 (page 31). We summarize the methods and results below to make both recoverability and forget-set information exposure explicit.
The steering vector can be approximately recovered from the Paired-Text Carrier. We freeze the model parameters and use only the carrier’s prompt/chosen text to learn an intervention direction, amplitude, and layer window by minimizing cross-entropy on the chosen completions. The recovered direction has a cosine similarity of 0.84785 with the true direction; see Appendix E.2.1.
We use this recovered vector in the privacy attacks reported in the main text. Content inversion in Section 5.3.2 recovers no complete target name, predefined factual keyword, or complete answer, with a maximum answer ROUGE-L F1 of 0.231. Update-role inference in Section 5.3.3 achieves an AUROC of 0.563. These results show that the recovered vector contains some forget-set information, but the tested attacks do not reconstruct original forget examples and only weakly distinguish forget from retain roles.
We further evaluate a generative attack with stronger access by providing the true vector, which remains upstream and is not delivered downstream under the normal protocol. Unlike the main-text inversion attack, which searches for content through iterative token replacement, this attack guides generation using token-probability differences between the pre- and post-update models and those induced by vector interventions. Each condition generates 160 outputs. Maximum answer ROUGE-L F1 is 0.25926 with the recovered vector and 0.38889 with the true vector. Neither condition recovers a complete target name or complete answer; see Appendix E.4.1 and Table A13.
Access to the true vector increases textual overlap but still does not recover complete target content. These direct attacks show that the steering vector retains unlearning-related behavioural signals and is recoverable, but those signals do not translate into disclosure of complete target identities or original answers under the tested attacks. The corresponding revised locations are Appendices F.2.1 and F.4.1, including Table A22.
Author response to Reviewer aYdG (3/6): W5–W6
W5: Code and reproducibility materials
We have supplemented the anonymous code repository with model checkpoints, fixed data splits, the steering-vector optimization schedule, training configurations, and metric computation code.
W6: Synthetic prompt construction and its rationale
We apologize that this point was not sufficiently highlighted in the original manuscript. We would like to clarify that Section 4.2 (page 5) already describes the carrier-generation pipeline, while Appendix B.2 and Table A1 (pages 15–16) provide the complete prompt templates, sampling rules, and examples. We explain the design rationale and construction process more explicitly below.
Design basis. Our prompt construction draws on the template-based, randomized construction of synthetic data in Subliminal Learning and Subliminal Steering. The former uses random numerical prefixes and code templates to construct generation tasks. The latter programmatically combines context, generation instructions, quantity requirements, and output formats from fixed template pools. We adapt this approach to fictional-author QA unrelated to the forget targets. Subliminal Learning, Subliminal Steering, Appendix A.2.
How prompts are constructed. We programmatically combine randomly generated fictional-author names with predefined question templates. First and last names are sampled separately from fixed name lists. Questions are drawn from ten categories of biographical QA templates covering writing genres, parents’ occupations, works, awards, birthplaces, writing styles, and related topics. Each prompt requests an answer of approximately 55–85 words that naturally incorporates two randomly sampled neutral words. It explicitly requires the experiences, book titles, places, and awards to be fictional. For example:
Elian Iver is an entirely fictional author. In approximately 72 words, answer: “What language does he usually write in, and why?” Invent the relevant experiences and naturally include pressure and durability. Do not imitate real people or existing works.
Why we use this construction. New names and fictional settings keep the prompts separate from the original forget entities and QA. Retaining a biographical QA format similar to TOFU lets us test a focused question: can non-target text alone transfer unlearning behaviour downstream without providing target examples? The ten question categories cover different QA content, while the requested word count controls the expected answer length. Neutral words introduce content variation within each template. Beyond changing author names, we require answers to incorporate randomly sampled words naturally, encouraging different narrative material and phrasing within the same question type. This is intended to reduce dependence on fixed response patterns. The chosen and rejected completions in each pair use identical lexical constraints, so these constraints introduce no additional difference between the two generation conditions.
How prompts produce a training signal. For each prompt, the same Original Model generates chosen and rejected completions with and without steering, respectively, using matched randomness to reduce sampling noise. These labels indicate generation conditions, not the factual correctness of the fictional answers. The downstream model learns the steering-induced behavioural change from these pairs without receiving the target authors’ original QA. Validity filtering and signal selection that preserves category coverage produce a Paired-Text Carrier containing 3,072 pairs. Only the three text fields—prompt, chosen, and rejected—are delivered. We have expanded Section 4.2 to explain the prompt form and design rationale; the detailed construction is in Appendix C.2 and Table A2 of the revised manuscript.
Author response to Reviewer aYdG (1/6): W1–W2
We thank the reviewer for the constructive feedback. We address the concerns and questions point by point below. References to our manuscript use the original submission’s numbering unless the revised manuscript is explicitly specified.
W1: Forget Quality, statistical significance, and the definition of raw-P
We address Forget Quality (FQ), statistical significance, and the definition of raw-P separately below.
1. Additional FQ results and analysis. We added FQ evaluations on Qwen2.5-7B with TOFU forget01 using the same eight Personalized Endpoints as the main experiments. Recompiled Student results follow the main-experiment endpoint aggregation and are averaged across three training seeds; see Appendix I.1, Table A25 of the revised manuscript:
We also observed and analysed an interesting pattern: some Recompiled Students achieve higher FQ than their corresponding Unlearned Teachers. FQ evaluates whether the distribution of relative probabilities assigned to fact-preserving paraphrases of correct answers versus perturbed incorrect answers matches that of the retain-only reference. It does not simply measure whether the probability of the original answer decreases. Even after substantially reducing that probability, a model can still favour paraphrased correct answers over incorrect candidates more strongly than the retain-only reference, resulting in low FQ. We observe this pattern in some Unlearned Teachers, while their corresponding Recompiled Students achieve higher FQ.
Drawing on subliminal learning, we offer a possible explanation. CUT repeatedly transmits the same implicit direction through diverse fictional QA, providing a consistent training signal across examples. This may extend the update from local suppression of specific answers to changes in features shared across different formulations. Such changes could reduce the excessive preference for paraphrased correct answers over incorrect candidates, bringing the distribution closer to the retain-only reference.
With both SimNPO and NPO, the Recompiled Student achieves higher FQ than the Unlearned Teacher. In these two settings, CUT therefore not only transfers target-answer suppression but also improves unlearning quality as measured by TOFU. We will further investigate the mechanism behind this observation and the conditions under which it holds.
2. Statistical significance. Standard deviations have been added to Tables A24 and A6 in the revised manuscript’s appendix.
3. Definition of raw-P. We apologize that this point was not sufficiently highlighted in the original manuscript. We would like to clarify that Section 5.1 (page 6) already describes raw-P as reference-answer probability, and Appendix C.2.1 (page 21, Equation A20) provides the complete definition. Specifically, we compute the geometric mean of the conditional token probabilities for each reference answer, then take the arithmetic mean across QA examples. This measures the model’s propensity to predict the reference answers while reducing the influence of answer length on comparisons. In the revised manuscript, the complete definition is in Appendix D.2.1, Equation A20.
W2: Personalization preservation compared with Task Arithmetic
We apologize that this point was not sufficiently highlighted in the original manuscript. We would like to clarify that the comparison of CUT and TA in terms of personalization preservation was already reported in Section 5.2.2 (page 7), Figure 3, and Appendix Table A4. We restate the LaMP-3 results below to make this comparison explicit. Both methods use the same upstream SimNPO Unlearned Teacher and the same eight Personalized Endpoints, at the original deployment settings:
CUT reduces MAE and RMSE by 21.05% and 11.17%, respectively, relative to TA, and improves accuracy by 5.95 percentage points. CUT’s MAE is also close to the pre-update value of 0.26190, demonstrating better preservation of existing personalization while transferring unlearning behaviour. The corresponding results are in Table A6 of the revised manuscript.