- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies probabilistic relearning attacks on unlearned LLMs. It introduces GRPO with a binary fact-disclosure reward to increase relearning information's exposure and evaluates attack results on questions across Harry Potter, TOFU, and WMDP. It also provides finite-sample leakage bounds and an idealized analysis of amplification and comparisons with single-response SFT.
Critical strengths
- Relevant problem setup: The paper highlights that single-response evaluation and sampling without subsequent attacks may not be sufficient for attacks. Introducing GRPO as a multi-turn attack is a very interesting and intuitive idea.
- Clear objective: The design of the reward provides an intuitive connection between reinforcement learning and target-fact exposure probability. Also, this paper provides proof for corresponding ideas and theoretical grounds.
- Useful empirical evidence: Several evaluation and cross-attribute probes examine recovery beyond training answers, while retained-utility and attack performance also help emphasize the balance of the proposed methods.
Critical weaknesses
- Several notations and proof problems: In Line 162, the denominator is $\bar{\sigma}$, when all rewards in the group are the same, it could become 0 for a denominator. In Line 272, the meaning of $\eta$ is not defined. The Clopper–Pearson interval in lines 1113–1114 seems to correspond to Lemma 1/Theorem 1 instead of proposition 1. In line 887, the proposition 3 didnt show up earlier.
- Novelty issues: This paper is built on the relearning attack that is proposed by [1] and introduce GRPO with grouped sampling and a binary fact-disclosure reward. The shift from single response supervision to GRPO's group response is interesting, but it remains unclear that this forms a methodological advance beyond a straightforward application of an existing RL algorithm. For example, could the two-stage approach, SFT warm up + GRPO RL, achieve stronger attack performance?
- Experiments: Hardware requirements and training timeare not reported for SFT and GRPO. Ablations on GRPO group size could be helpful. Adding experiments on MUSE mught help evaluate generalization to corpus-level forgetting and verbatim memorization beyond current QA-focused evaluation. Also there are several works specifically focusing on robustness to relearning attacks, following up on SAM+NPO. Adding them could be helpful on evaluating further performance.
[1] Hu, Shengyuan, et al. "Unlearning or obfuscating? jogging the memory of unlearned llms via benign relearning." International Conference on Learning Representations. Vol. 2025. 2025.
Questions for authors
See weaknesses. Specifically,
- Could SFT+GRPO improve overall performance than merely using GRPO or SFT?
- What's the computational overhead compared to SFT?
- What's the ablation results on using different group size in GRPO?
- What's the result on MUSE dataset and more recent works that focus on robustness to relearning attacks?
3 comments
W3/Q2. Computational overhead and hardware requirements
All TOFU experiments (Llama-3.2-1B-Instruct, LoRA rank 8) were run on a single NVIDIA RTX A6000 (48 GB) GPU. The wall-clock training time and peak memory usage are:
| Attack | GPU-minutes | Peak memory (GB) |
|---|---|---|
| SFT | 0.4 | 8.1 |
| GRPO () | 6.2 | 14.0 |
| SFT + GRPO | 7.2 | 16.0 |
Although GRPO incurs higher relative cost than SFT due to generating on-policy rollouts per prompt, its absolute cost remains very low (6.2 minutes and 14 GB VRAM on a single GPU), making it practical for standard unlearning robustness audits. We have added these hardware and runtime details to Appendix D.4.
W3/Q3. GRPO group-size ablation
As suggested, we evaluated GRPO across group sizes on TOFU SimNPO checkpoint A.
| Group size | Leak@128 /20 | Keyword matches /2,560 |
|---|---|---|
| 4 | 16 | 1,635 |
| 8 | 16 | 1,424 |
| 16 | 17 | 1,723 |
| 32 | 18 | 1,791 |
Increasing the group size monotonically improves Leak@128 coverage from 16/20 (at ) to 17/20 (at ) and 18/20 (at ), with also achieving the highest total keyword-match count (1,791/2,560). Larger groups reduce variance in the group-normalized advantage estimate and increase the likelihood of sampling at least one positive-reward disclosure on harder prompts, whereas offers a strong balance between recovery and runtime.
W3/Q4. Results on MUSE and recent robustness work to relearning attack.
To evaluate generalization to corpus-level forgetting beyond QA benchmarks, we tested SFT and GRPO on MUSE using both standard (SimNPO) and sharpness-aware relearning-resilient (NPO-SAM, Fan et al., 2025a) checkpoints, measuring held-out ROUGE-L recovery:
| Unlearning Method | Unlearned (No attack) | SFT | GRPO |
|---|---|---|---|
| NPO-SAM | 0.441 | 0.488 | 0.524 |
| SimNPO | 0.486 | 0.502 | 0.549 |
GRPO achieves substantially higher held-out ROUGE-L recovery than SFT on both SimNPO (0.549 vs. 0.502) and the relearning-resilient NPO-SAM checkpoint (0.524 vs. 0.488).
Combined with our evaluations on NPO-SAM and TAR-Bio-v2 on WMDP (Table 4), RULE-NPO on TOFU (Table 2), and the newly added WGA, SatImp, and PDU results (shown in Response to Q1 above), these findings demonstrate that probabilistic policy optimization consistently recovers more residual information than single-reference SFT even against recent defenses designed to resist relearning and probabilistic decoding.
We also identified two additional recent works studying robustness to relearning attacks following up on SAM+NPO:
- Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization (Wu et al., 2025)
- Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter (Xiao et al., 2026)
Unfortunately, (1) does not provide public code or checkpoints, and (2) does not release unlearned model checkpoints or provide code for the TOFU or MUSE datasets. Because training unlearning defenses from scratch requires substantial hyperparameter optimization that is difficult to validate fairly during the rebuttal period, we focus our empirical evaluation on publicly released unlearned checkpoints and have added both works to Section 6 (Related Work) and Section 5 (Discussion) as important directions for future evaluation.
Response by Authors
Dear Reviewer TJoK,
As the interactive discussion period is drawing to a close, we would like to kindly ask whether our response has adequately addressed your concerns. Please let us know if any further clarification or additional information would be helpful.
Best regards,
Authors
Response to Reviewer TJoK
We thank the reviewer for recognizing the novelty of the problem we study, the clarity of our objectives, and the experimental evidence supporting our claims.
W1. Notations and proof problems
Thank you for identifying these notation and cross-reference issues. We address each point explicitly:
a) Zero reward variance (line 162)
The displayed advantage normalization omitted the numerical safeguard used in implementation. We now write the stabilized expression explicitly, where εnum is a small positive constant. Ai=σˉ+εnumri−rˉ,εnum>0. When all rewards in a group are identical (ri=rˉ for all i), every numerator is zero, so Ai=0 and the group contributes no reward-based policy-gradient signal (only the KL regularization term remains active).
b) Notation near Equation (11) (line 272)
We now explicitly define η>0 at Equation (11) as the step-size parameter of the idealized KL-proximal update (distinct from the neural optimizer learning rate).
c) Clopper–Pearson reference in Table 5 caption (lines 1113–1114)
We have corrected the caption of Table 5 to reference Lemma 1 (which defines the Clopper–Pearson interval endpoints) and Theorem 1 (which states its finite-sample coverage guarantee), rather than Proposition 1.
d) Amplification reference in Appendix A.5 (line 887)
We have corrected "Proposition 3" on line 887 to "Theorem 3", as Equation (23) follows directly from the convergence bound in Equation (13) using the matched coefficient βΔ.
We have also audited all remaining theorem and equation cross-references across the paper and appendix.
W2/Q1. Methodological contribution and sequential SFT + GRPO results
Methodological and theoretical advance: Beyond applying GRPO, our work makes two core contributions:
Sequential SFT + GRPO evaluation: Following your suggestion, we evaluated a two-stage SFT warm-up + GRPO pipeline across four unlearning methods (
WGA,SatImp,PDU, andSimNPO):SFT + GRPOdoes not consistently outperform standaloneGRPO: while warm-up improves recovery onSatImp(19/20vs.18/20) and greedy recovery onPDU(18/20vs.16/20), standaloneGRPOachieves equalLeak@128coverage and slightly higher mean leakage onWGA,PDU, andSimNPO. This aligns with Theorem 4: because SFT warm-up initially concentrates policy mass toward a single reference string, it can reduce sampling diversity prior to RL exploration while adding extra compute overhead.