acceptodds
Withdrawn from ICLR 2027

A Conditional Joint Matching of Response Profiles for LLM Unlearning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Honouring a deletion request means making a language model behave like one retrained without the deleted facts, but retraining per request is too costly. Reference-based methods instead move each forgotten fact's likelihood to the level of never-seen facts, reducing the fact to one number. Yet memorisation also decouples a fact's probes, the phrasings that test it: on the TOFU benchmark, the correlation between direct and paraphrased likelihoods falls from 0.91 on never-seen facts to 0.15 on memorised ones, a drop that no order-preserving per-probe target can undo. Our intuition is that a forgotten fact should resemble comparable never-seen facts on all its probes at once. We propose conditional joint matching: each fact becomes a response profile of log-likelihoods under six probes, and entropic optimal transport couples each forgotten profile with never-seen profiles of similar relation, length and difficulty to build that profile's target. Unlike per-probe targets, these whole-profile targets carry the never-seen correlation, and training stops by a rule that needs no retrained model. On TOFU at 1B parameters, the method has the lowest mean per-fact distance to the retrained model among ten target assignments and four published objectives, 0.785 nats against 0.906 for unconditional per-probe matching, but only that comparison survives correction for multiple testing. Shuffling reference probes across facts within relation and difficulty strata gives a statistically indistinguishable 0.797, so the gain cannot come from how probes pair beyond those strata. Closer per-fact agreement, however, brings more leakage under sampling; the method also fails TOFU's Forget Quality test and trails a single-number target on a second model family. These results establish that forgetting must restore how probes co-vary across facts, not just a likelihood level, and that never-seen facts supply this structure but not the retrained model's level, so a retrain-free estimate of that level is the missing piece.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, LLM unlearning, retrain reference, response profiles, conditional optimal transport, Sinkhorn coupling, energy distance, membership inference, relearning attacks, TOFU, MUSE, evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JnqnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper describes how a language model responds to a fact using six probes. It then matches the facts to be unlearned onto never-seen facts through conditional optimal transport. Training and model selection need no access to the retrained model. The method improves some per-fact likelihood metrics. However, sampling leakage becomes worse, and the independent benefit of joint matching and its stability across models remain insufficiently supported.

Critical strengths
  1. The multi-probe view has value. It shows that a single answer probability cannot fully measure unlearning.
  2. The internal ablations are fairly thorough, and the paper reports metric conflicts and failed results honestly.
  3. Training and model selection do not depend on the retrained model, and the procedure is clear.
Critical weaknesses
  1. The metric gains do not turn into better behavioral unlearning. JOINT reaches a 73% sampling leakage rate. That is 40.6 percentage points above the retrained model, and it also fails the Forget Quality test.
  2. The core mechanism lacks direct support. Against conditional per-coordinate matching and against pairing shuffled within condition, JOINT shows no significant advantage on either. The necessity of joint coupling is therefore hard to confirm.
  3. The reference target carries an unresolved bias. Never-seen facts and the retrained model sit at different likelihood levels. A more accurate match to that reference can therefore retain more of the information marked for deletion.
  4. The cross-model advantage is not stable. A simple scalar objective does better on Qwen, and full joint matching has not been verified on larger models.
  5. The comparison with the closest recent method is not direct enough. REGUN ports only the reference target. That cannot replace a full comparison against the original ReGUn method.
Questions for authors
  1. At the same leakage rate, does JOINT still reach a lower per-fact error?
  2. Which result directly supports joint coupling over conditional information alone?
  3. Why does the simple scalar objective do better on Qwen?
  4. Can the likelihood bias of the reference target be corrected without the retrained model?
  5. For the roughly 25% of forget facts that never enter training, is leakage more severe?
  6. How does a change in the stopping threshold affect leakage and retention?
  7. Can the comparison against the original ReGUn use the same budget and the same stopping rule?
  8. When a real deletion request has no never-seen facts from the same distribution, how does the method build its reference set?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Jnqn (1/2): weaknesses

Thank you for the detailed and direct review. It is the review that changed the paper most: your points on the reference bias (W3, Q4) and on ReGUn (W5, Q7) led to new experiments on three further seeds with their own retrains (in blue: Section 3.5 (p. 6), Section 5.6 (p. 9), Appendix E.7 (p. 24) with Table 16 (p. 25)). We agree with most of your points and say so plainly below. This comment answers the weaknesses; (2/2) answers the questions.

W1. The metric gains do not turn into better behavioural unlearning; JOINT leaks most and fails Forget Quality. We agree with the observation: on the reported score JOINT leaks most among the arms of Table 2 (p. 7), and it fails Forget Quality. The five-seed results are unchanged. Three things are new.

  • A correction of our own text. Section 5.1 described leak@k as a sample entailing the true answer, but the reported score counts a fact when the true answer entails a sample, which also counts vague and truncated samples. The sentence is corrected (p. 8) and both directions are now reported on the further seeds. When a sample must entail the true answer, JOINT exceeds the retrain by 0.08 and JOINT-ANCHORED stays within 0.05 of it; NPO and SimNPO stay at or below the retrain.
  • The cause. Token by token, JOINT is no closer to the retrain than the model before unlearning (4.55 against 4.41 nats): a profile coordinate is a mean over the answer's tokens, so matching it leaves open where in the answer the unlikelihood sits.
  • A first remedy. Targets set per token (TOKEN-ANCHORED) lower the per-fact distance to 0.570 nats against JOINT's 0.799, significant after Holm correction. Its samples are truncated (a median of 3 words against the retrain's 26), so we do not claim that it reproduces the retrain's behaviour.

No arm of ours passes Forget Quality; Section 5.6 (p. 9) and the Limitations (Section 6 (p. 9)) say so.

W2. The core mechanism lacks direct support. We agree, and the new arms support your reading rather than ours. A regression on relation, length and difficulty with no transport (REGRESS) is not separated from JOINT (0.831 against 0.799 nats, closer on 1 of 3 seeds), and with anchoring the regression matches the coupled arm (0.687 against 0.675). We changed the paper accordingly: Section 5.6 (p. 9) states that the gain comes from what the targets condition on, not from the transport, the Limitations open with this point, and the abstract's conclusion now says "indicate" where it said "establish".

W3. The reference target carries an unresolved bias. We agree that it was unresolved; the submitted abstract called it "the missing piece". This was the most productive point of the review for us, because Section 3.5 (p. 6) now estimates the level without a retrain. Part of the bias is composition: under the base model the forget facts are 0.32 nats harder than the reference facts. JOINT-ANCHORED therefore starts each target from the fact's own profile under the base model and takes from never-seen facts only the shift that fine-tuning induces (Eq. (3) (p. 6)). Before training, these targets sit 0.06 nats from the retrain on DIRECT against 0.26 for JOINT's. After training, JOINT-ANCHORED is closer to the retrain than JOINT on 3 of 3 seeds (0.675 against 0.799 nats, not significant at three seeds). The remaining DIRECT bias of every arm is in the "bias" column of Table 16 (p. 25).

W4. The advantage is not stable across models; larger models are unverified. We agree, and the new runs do not remove this limit; we prefer to state it plainly. On two further Qwen2.5-1.5B seeds, SCALAR, JOINT, JOINT-ANCHORED and TOKEN-ANCHORED reach 1.117, 1.127, 1.141 and 1.039 nats, with no arm ahead on both seeds, so the gain from anchoring at 1B does not transfer within the same six epochs (Section 5.6 (p. 9)). Models above 1.5B remain untested (Section 6 (p. 9)).

W5. REGUN ports only the reference target, not the original ReGUn objective. We agree, and we ran it. ReGUn is defined for classifiers, so we transcribed its loss to next-token prediction: every answer position of a forget question is distilled onto the frozen model's next-token distribution, averaged over held-out answers of the same relation class, with our budget and stopping rule (Appendix E.7 (p. 25)). It reaches 3.678 nats per fact against JOINT's 0.799, because it overshoots the retrain's level. We note that this is our transcription, not the authors' code.

Deleted text. The abstract's and the conclusion's "a retrain-free estimate of that level is the missing piece" is replaced by the anchoring result.

AuthorsOpenReview ↗

Response to Reviewer Jnqn (2/2): questions

This comment answers your eight questions, each with data from the three further seeds. Locations refer to the revised PDF; every arm is in Table 16 (p. 25).

Q1. At the same leakage rate, does JOINT still reach a lower per-fact error? We scored every epoch of JOINT and of the published objectives (Appendix E.7 (p. 26), "Equal leakage"). Against SimNPO no comparison exists: along its epochs JOINT never comes down to SimNPO's leakage. Against UNDIAL, where the leakage ranges overlap, JOINT is closer per fact by 0.20 nats.

Q2. Which result directly supports joint coupling over conditional information alone? No result does on per-fact error, and the paper now says so. The within-strata shuffle is not separated from JOINT, REGRESS (no transport) is not separated from JOINT, and REGRESS-ANCHORED matches JOINT-ANCHORED. What the evidence supports is the conditioning, which now includes the fact's own base profile (Section 5.6 (p. 9)).

Q3. Why does the simple scalar objective do better on Qwen? We have one measured difference, not a full explanation. On Qwen2.5-1.5B JOINT brings the forget facts up to the reference level only at epoch 5 or 6 of its six epochs, against epoch 2 at 1B, so its budget ends as the level is reached and little is left for matching individual facts (Section 5.6 (p. 9)). On the two further seeds the scalar's lead is also not reproduced on both seeds (W4). We ran the arms with twice the epochs as well; the ordering stayed inconsistent across the two seeds, so we report no claim from it.

Q4. Can the likelihood bias of the reference target be corrected without the retrained model? Largely, yes, by anchoring each target on the fact's own profile under the model before fine-tuning (Section 3.5 (p. 6)). Before training, the DIRECT target error falls from 0.26 to 0.06 nats, and the per-fact target error from 0.613 to 0.520 (six seeds). After training, the per-fact distance to the retrain falls from 0.799 to 0.675 nats, and to 0.570 with per-token targets, which is significant after Holm correction.

Q5. For the roughly 25% of forget facts that never enter training, is leakage more severe? Yes. On the 101 forget facts that lack a core probe and enter no arm, JOINT's excess leak is +0.49 against +0.35 on the others, higher on each of six seeds (Appendix E.7 (p. 26), "Facts without a profile").

Q6. How does the stopping threshold affect leakage and retention? It does not, within a wide range. For every threshold from 0.02 nats to no limit, the rule selects the same epoch in each JOINT run, so leakage and retention do not move (Appendix E.7 (p. 26), "Stopping threshold").

Q7. Can the comparison against the original ReGUn use the same budget and stopping rule? Yes; see W5 in (1/2). The row "ReGUn's objective" of Table 16 (p. 25) uses our budget and stopping rule.

Q8. When a deletion request has no never-seen facts from the same distribution, how does the method build its reference? We tested a generated reference. Another model (Qwen2.5-7B-Instruct) invents 36 fictitious people who are not authors and writes question-answer pairs about them; after the probe checks, 123 facts build targets and 55 feed the stopping rule. This reference costs JOINT 0.266 nats, but JOINT-ANCHORED only 0.044, because anchoring asks the reference only for the shift that fine-tuning induces, not for the level (Appendix E.7 (p. 25), "Generated reference").

Presentation (rated 2). We agree. A readability pass rewrote sixty-two passages of the main text (counted by a script, in blue): Section 1 (p. 1) states the paper in one sentence, one example fact runs through Sections 1, 3.1 and 5.1, captions open with their takeaway, and Algorithm 1 (p. 5) has inputs, an output and a comment on every step. One term that your review uses changed: "coupling" now always means the transport plan, and what you call joint coupling is "sharing one coupling across coordinates" (Section 5.4 (p. 8), Section 6 (p. 9)). An example from Section 5.3 (p. 8). Before: "COORDINATE's monotone maps keep the forget set's pairing between phrasings in its targets". After: "COORDINATE's increasing maps keep the forget set's correlation in its targets".

In summary. We accept the review's central criticisms, and the paper is now explicit about each. In response, the revision adds a retrain-free estimate of the level that lowers the per-fact distance significantly, a measured cause of the behavioural gap, a direct comparison with ReGUn's objective, and an answer with data to each of your eight questions. We hope that these additions address a good part of your concerns about soundness, and we would be grateful if you would take them into account in your final assessment. Thank you again; the review changed the paper substantially.

Dear Authors,

Thank you for the detailed rebuttal and the additional experiments, including three new seeds with corresponding retrains. I read the full rebuttal and revised PDF. The new results answer most of my questions and leave the main concerns below unresolved.

The experiments do not establish a benefit from joint transport. The title, abstract, and Section 3 present entropic OT as the main contribution. The controls give similar results.

  • REGRESS scores 0.831 nats versus 0.799 for JOINT (p = 0.72, Table 16).
  • REGRESS-ANCHORED scores 0.687 versus 0.675 for JOINT-ANCHORED (Table 16).
  • The within-strata shuffle and COORDINATE-COND show no significant difference from JOINT (p = 0.75 and 0.54, Table 6).

COORDINATE is the only control that differs significantly from JOINT, and it removes both coupling and conditioning. As the authors acknowledge in Q2, the experiments leave the contribution of transport unresolved. DIRECT scores for memorised facts are near the floor (mean 0.07 nats, Table 7), which could explain much of the drop in its correlation with QPARA from 0.91 to 0.15. SCALAR recovers most of the correlation with one supervised coordinate (0.740 versus 0.092 for the untouched model and 0.873 for the retrain, Table 5). Proposition 1(b) applies only to the targets (Section 5.3).

Therefore, I will keep my rating.

AuthorsOpenReview ↗

Response to Reviewer Jnqn (follow-up)

Thank you for reading the rebuttal and the revised PDF in full, and for this precise follow-up. We agree with your reading of every number you cite; they are the paper's own controls. We differ only on what the paper rests on, so we state that as exactly as we can.

1. Transport is not shown to be necessary, and the paper does not rest on it. We agree that no control separates JOINT from REGRESS, from the within-strata shuffle or from COORDINATE-COND. The revised paper says so in Section 5.4 (p. 8), in Section 5.6 (p. 9), where the gain is "not from the transport", and at the head of the Limitations (Section 6 (p. 9)). We also accept that Section 3 gives the coupling most of its space.

What the method proposes is the target. Each forgotten fact gets a target for its whole profile, conditioned on what is known about the fact apart from its memorisation and estimated from never-seen facts; a two-sided loss approaches that target, and training stops by a rule that needs no retrain. Transport is one way to compute the target and ridge regression is another. That the two are not separated suggests that the target, not the solver, carries the result. Two results about the target are significant after Holm correction:

  • Conditioned whole-profile targets against unconditional per-probe targets: 0.785 against 0.906 nats on five seeds, closer on each (p = 0.01, Table 2 (p. 7)).
  • Targets conditioned further, on each token's likelihood under the base model: 0.570 against 0.799 nats on three further seeds, closer on each (p < 0.01, Table 16 (p. 25)), at the cost of truncated samples.

The loss must also be two-sided: a variant that only raises NLL ends at 2.749 nats (two seeds, Table 4 (p. 18)).

2. The floor. We agree that this is the likely mechanism, and the paper claims no other: memorisation drives DIRECT to the floor (mean 0.07 nats, Table 7 (p. 20)), and its correlation with QPARA falls from 0.91 to 0.15. The consequence is what the method acts on. At the floor, a fact's own value no longer says where that fact would sit had it not been trained on, so that place has to come from information taken before memorisation. The difficulty covariate is exactly that, DIRECT under the base model, and it is the one penalty whose removal hurts (0.866 against 0.797 nats at seeds 3 and 4, Table 4 (p. 18)). Anchoring on the fact's base profile (Section 3.5 (p. 6)) takes the same step further. In our reading, your observation is therefore the reason why conditioning matters.

3. SCALAR and Proposition 1(b). We agree on both. Lifting DIRECT off the floor restores much of the correlation by itself, and Section 5.3 (p. 8) states that the proposition constrains targets, not trained models. This is why the paper's criterion is the per-fact distance, with correlation as the diagnostic that motivates it: SCALAR regains a correlation of 0.740 and still ends 1.031 nats from the retrain per fact, against 0.811 and 0.785 for JOINT (five-seed means, Table 5 (p. 19); this pair was not tested).

We understand your decision and do not ask you to credit the transport. We hope that the narrower claim, a retrain-free per-fact target whose gain comes from what it conditions on, can be weighed on its own evidence in the final discussion. Thank you again; your review made the paper more accurate.

Reviewer_iRFLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a machine unlearning method called "Conditional Joint Matching," which aims to ensure that, after removing a specific subset of data, a model’s response behavior reverts to a state as if it had never been trained on that data. The authors point out that most existing methods focus solely on matching individual likelihood values, failing to account for the interdependencies that arise when a single fact is probed multiple times. To overcome this limitation, the proposed method represents each fact as a multi-dimensional response profile and employs "covariate-conditioned optimal transport" to align the facts targeted for unlearning with data the model has never encountered. Experimental results on the TOFU benchmark demonstrate that, compared to various baselines, this method achieves likelihood distance metrics—pertaining to individual facts that are closer to those of a model retrained from scratch without the forget set .

Critical strengths

Traditional methods focus primarily on the likelihood of individual training answers; however, this paper points out that there exists an underlying statistical structure among different ways of asking about the same fact. It is an area worthy of further investigation.

The paper does not simply equate the reduction of negative log-likelihood (NLL) across the board with the objective of "forgetting." Its "two-sided loss" effectively addresses the issue where different probes may require opposing update directions, making it a more targeted approach than simply suppressing the probability of an answer.

Critical weaknesses

1.This method employs six probes to characterize each fact; most of these probes primarily capture different linguistic realizations of the same underlying fact. Although the inclusion of inverse and perturbation probes introduces additional variation, it remains unclear whether these six dimensions are sufficient to fully delineate the joint structure of factual memory. As I understand it, these appear to be merely six distinct ways of expressing the same piece of knowledge—and, clearly, there are far more than six possible ways to express any given piece of knowledge. I would therefore appreciate it if the authors could clarify the motivation behind this design.

2.JOINT shows a clear improvement over COORDINATE; however, this performance gap narrows significantly once conditioning is introduced. In particular, JOINT-SHUFFLED-COND performs almost identically to JOINT, which undermines the argument that "preserving the original joint structure across probes is the key factor driving performance gains." I would appreciate it if the authors could provide an explanation for this experimental result.

Questions for authors

See the "weakness"

3 comments
AuthorsOpenReview ↗

Response to Reviewer iRFL

Thank you for the thoughtful review, and for recognising the two-sided loss and the structure among different ways of asking about one fact. Your two questions were precise, and each led to new measurements on three further seeds with their own retrains (in blue: Section 3.1 (p. 3), Section 3.5 (p. 6), Section 5.6 (p. 9), Appendix E.7 (p. 24) with Table 16 (p. 25)).

W1. Why six probes, and are they sufficient to describe how a fact is memorised? Motivation. We labelled this poorly: the probes are not six rewordings of one question. Each tests a distinct property of how a model holds a fact, following the probe taxonomy of earlier evaluation work (Cohen et al., 2024; Wang et al., 2025): recall of the trained answer (DIRECT), robustness to the wording of the question (QPARA) and of the answer (APARA, ALT), reversal, with the answer as prompt and the question as target (INVERSE), and preference for the true answer over false ones (PERT). Before any unlearning they already separate memorised from never-seen facts: the correlation between DIRECT and QPARA is 0.91 on never-seen facts and 0.15 on memorised ones (panel (a) of Figure 1 (p. 2)).

Sufficiency. We agree that six probes cannot exhaust the ways of expressing a fact, and the paper does not claim so. We have now measured how far they go. The probe generator wrote twenty candidate rephrasings per forget question and answer, and the paper's checks keep 10.6 and 15.1 per fact. Under the retrain, a single question rephrasing has an intraclass correlation of 0.93 across facts and an answer paraphrase 0.79, so one probe per kind is a reliable coordinate, though not a noise-free one. The many rephrasings also show what six probes miss: for 50% of facts JOINT leaves some question rephrasing more than 0.5 nats likelier than under the retrain. Targets set per token (Section 3.5 (p. 6)) bring that share to 23%.

Change. Section 3.1 (p. 3) states the design; the rephrasing protocol and numbers are in Appendix E.7 (p. 25) and the "rephr." column of Table 16 (p. 25).

W2. JOINT-SHUFFLED-COND performs almost as well as JOINT; please explain this result. You are right, and thank you for pressing on it. To keep the record accurate, the paper does not argue that preserving the joint structure across probes drives the gain: Section 5.4 (p. 8) states in bold "The mechanism control fails", and the abstract says that the gain "cannot come from how probes pair within those strata". What was missing was the explanation, which is now measured (Appendix E.7 (p. 26), "What the shuffle keeps"). The strata that the shuffle preserves, relation class by difficulty tercile, carry 30 to 60% of each core coordinate's reference variance. The mean correlation between core coordinates is 0.39 overall and 0.19 within strata. As a result the targets of JOINT and of JOINT-SHUFFLED-COND correlate at 0.97: most of what links the probes is already carried by the covariates, so shuffling within strata changes the targets little.

Two new arms confirm this from the other side. A regression of the reference profile on the covariates, with no transport (REGRESS), is not separated from JOINT (0.831 against 0.799 nats, closer on 1 of 3 seeds), and with anchoring the regression matches the coupled arm (0.687 against 0.675). Section 5.6 (p. 9) therefore states that the gain comes from what the targets condition on, and the abstract now says "indicate" where it said "establish".

What does improve agreement is conditioning on more. Anchoring each target on the fact's own profile under the model before fine-tuning lowers the per-fact distance to the retrain from 0.799 to 0.675 nats on 3 of 3 seeds, and to 0.570 with per-token targets, which is closer on every seed and significant after Holm correction.

Change. Section 3.5 (p. 6), Section 5.6 (p. 9), Appendix E.7 (p. 24). The five-seed results are unchanged.

A correction of our own text. Section 5.1 described the leakage score with the entailment direction reversed. The sentence is corrected (p. 8) and both directions are reported in Table 16 (p. 25).

Presentation (rated 2). We agree. A readability pass rewrote sixty-two passages of the main text (counted by a script, in blue): Section 1 (p. 1) states the paper in one sentence, one example fact runs through Sections 1, 3.1 and 5.1, captions open with their takeaway, and Algorithm 1 (p. 5) has inputs, an output and a comment on every step. The passage you asked about is one of them. Before: "Neither half of the gap over COORDINATE holds on every seed [...] both change sign". After: "Neither part of the gap over COORDINATE is positive on every seed [...] each is negative on one seed".

We hope that this clarifies the design and explains the result you asked about, and we would be grateful if you would consider the new evidence in your final assessment. Thank you again.

I sincerely thank the authors for their response, which has, to some extent, addressed my concerns. However, the motivation behind the "six probes" design remains unclear. Furthermore, the authors have made extensive revisions to the original manuscript (highlighted in blue). In my view, the initial submission does not yet meet the threshold for acceptance; therefore, I have decided to maintain my original score.

AuthorsOpenReview ↗

Response to Reviewer iRFL (follow-up)

1. The motivation for the six probes

You read the probes as six realisations of one fact, and by construction that is what they are. The design rests on one observation: a model treats those realisations differently when it has memorised the fact than when it has never seen it, and that difference is what unlearning has to remove.

What a probe is for. The method must say where each forgotten fact should end up, and it reads that from never-seen facts. A forgotten fact and a never-seen fact have different answers, so their tokens cannot be compared. What can be compared is how likely the model finds a standard test of each fact. Each probe is one such test, so six probes turn every fact into six comparable numbers.

Why one test is not enough. If the six were interchangeable, memorisation would move them alike. It does not. Before unlearning, the gap between never-seen and memorised facts is +2.54 nats on DIRECT, +2.30 on QPARA, +1.06 on APARA, +1.31 on ALT, +0.59 on PERT and -0.45 on INVERSE (Table 7 (p. 20)): the gaps differ by more than 5× in magnitude and one has the opposite sign. The tests also stop moving together: DIRECT and QPARA correlate at 0.91 across never-seen facts and at 0.15 across memorised ones (panel (a) of Figure 1 (p. 2)). A target for the trained wording alone therefore leaves the other tests wrong. With the same transport, matching DIRECT alone ends 1.115 nats from the retrain per fact and matching all six 0.906 (five-seed means, Table 2 (p. 7)).

Why these six. Each changes one part of the question-answer pair and keeps the rest: nothing (DIRECT), the wording of the question (QPARA), the wording of the answer (APARA, ALT), the direction, with the answer as prompt (INVERSE), and the truth of the answer (PERT). These kinds of test follow the probe taxonomy of earlier evaluation work (Cohen et al., 2024; Wang et al., 2025), and they are the kinds on which the paper is evaluated: TOFU's own paraphrases and perturbations and a reworded inverse template (Section 5.1 (p. 7)). The method is thus trained on the kinds of behaviour it is evaluated on, but on separately written instances: training and evaluation share only DIRECT and one QPARA probe.

Why six and not more. Six is not a claim of completeness, and Section 3.1 (p. 4) now says so. It is one test per kind, with two for the wording of the answer, because every required probe costs facts: a fact missing one leaves the objective, and the five required probes keep 75% of forget and 70% of reference facts. The revision measures what this choice gives and what it misses (Appendix E.7 (p. 25), Table 16 (p. 25)). Under the retrain, a single question rephrasing has an intraclass correlation of 0.93 across facts and an answer paraphrase 0.79, so one probe per kind is a reliable measurement, though not a noise-free one. The limit is that for 50% of facts JOINT leaves some question rephrasing more than 0.5 nats likelier than under the retrain. The number six is also not built into the method: its cost and loss run over whichever coordinates a fact has, and ALT is absent for a quarter of facts (Appendix C.1 (p. 14)).

2. The extent of the revision

We understand the concern and respect your wish to judge the paper as submitted. The title, the procedure of Algorithm 1 and the five-seed results are unchanged. The blue text is of three kinds:

  • Rewording for clarity, since all three reviews rated presentation at 2: sixty-two passages of the main text, among them a more cautious conclusion in the abstract.
  • One addition, anchored targets on three further seeds (Section 3.5 (p. 6), Section 5.6 (p. 9), Appendix E.7 (p. 24)). The submitted abstract ended by calling a retrain-free estimate of the retrain's level "the missing piece"; the addition supplies and tests that estimate, and reports where it falls short.
  • One corrected sentence, on the direction of the leakage score (p. 8).

ICLR's discussion phase provides one additional page for such changes, and our general comment lists each with its location.

We hope this makes the design clear, and we would be grateful if it could inform your final assessment. If any step remains unclear, we will gladly address that point directly. Thank you again for your time.

Reviewer_2tnVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper targets knowledge unlearning problem in Large Language Models. It argues that as a fact is memorized by the model it breaks the correlation between different phrasings. It then proposes to make forgotten facts look like never-seen facts across many different phrasings (and not just push down the likelihood). The authors represent each fact by 6 different NLL-based probes and use optimal transport to match it against a set of never-seen facts, which then become regression targets for forgotten fact NLL. On TOFU, the method achieves the best per-fact RMSE (but is significantly better only than per-coordinate matching), but a small difference with the shuffled-conditioned version may suggest that improvement may primarily be attributed to conditioning and not necessarily joint matching. The method also loses to other published baselines on FQ and excess leak.

Critical strengths

The main strengths of the paper are:

  • The overall direction and observations made by the authors are solid. The authors correctly identify some of the drawbacks of other well-known techniques and evaluations (for example, that population-based metrics like FQ are blind to per-fact errors). They creatively combine difficulty and length conditioning, building fact profiles based on multiple fact rephrasing, rely on matching to never-seen facts, use non-monotonic loss and an oracle-free stopping criterion.
  • The empirical results are catiously promising with respect to the author's own metrics, but some of the observations suggest that it could be conditioning and not profile matching that is responsible for the observed gains.
  • The empirical validation is sufficiently deep (authors measure distance to the retrain in several forms, look at behavioral forgetting, robustness, etc.) and honest. The statistical analysis is careful (using multiple seeds, paired t-tests with Holm correction, etc). The authors also use FQ and excess leak on top of their own proposed metrics (and it is these established metrics where their results are currently less convincing).
  • Theoretical analysis in Section 4 appears valid and gives a good conceptual view of what different methods that match never-seen facts can and cannot establish.
  • The authors provided code and claimed that it can be used to reproduce the main results of the paper (not verified).
Critical weaknesses

The main weaknesses of the paper, in my opinion are:

  • While the underlying premise that a single phrasing of a fact is insufficient is correct, the authors use fact profiles of just six probes, several of which correspond to a single generated rephrasing. Each component is therefore likely to be high-variance, and moreover compares different strings across different facts. As a result, using such profiles as queries or targets for optimal transport may be quite noisy. In my opinion, when targeting NLL, a fact profile should represent a distribution of likelihoods over a sufficiently rich variety of rephrasings. Matching entire distributions rather than 6-dimensional vectors would reduce this noise, and the training objective could then minimize a distance between distributions rather than regress the NLLs of arbitrarily chosen phrasings. In addition, a heavy tail of high-likelihood rephrasings would immediately signal that the fact is preserved in at least some phrasings. While this could be expensive in practice, posing the problem this way would be much more defensible.
  • This approach shares a common problem with some other techniques -- it relies on using a retain set to retain other knowledge and avoid affecting facts other than forgotten ones. This not only requires keeping and training on potentially very large amounts of data, but may also not be able to protect against a more "localized" effect of forgetting. Intuitively, if the forgotten facts lie on a localized, lower-dimensional manifold in data space, the concentrated suppression of these facts may also affect their neighborhood, while the retain loss, spread more uniformly over the whole corpus, constrains this neighborhood only weakly (the paper shows some evidence of this with NLL on never-seen facts from the same domain that drifts during unlearning).
  • Assigning each forgotten fact an exact NLL target by relying on optimal transport is conceptually questionable. It is not clear how the fact's retrain NLL can be inferred (as a specific target) from other facts. It feels like a distributional or interval constraint would be more appropriate than a point target.
  • The empirical results presented in the paper do not show a convincing improvement. Only the gap to coordinate-wise matching appears to be statistically significant (on RMSE only, though JOINT is consistently closer on correlation structure). One control experiment shuffling reference profiles within relation and difficulty strata performs almost equally well. At the same time, on a second model family a single-number target outperforms the proposed method. It is also a little concerning that the method performs worse on some behavioral metrics (like sampling leakage, Forget Quality). This suggests that closeness to the retrain on per-fact NLL may not be a good proxy for forgetting.
Questions for authors
  • Since JOINT and JOINT-SHUFFLE-COND are reasonably close, how does the proposed method compare with targets regressed directly from relation, length and base-model difficulty, but no optimal transport?
  • Is it possible to evaluate on many rephrasings per probe type and report the per-fact tail: the most likely rephrasing, compared with the retrain? Even at evaluation time only, this could potentially reveal whether some phrasings survive unlearning.
  • The proposed method uses never-seen reference facts from the same distribution as the forget set. Beyond subsampling the same pool, have the authors tested a reference from an entirely different source?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 2tnV (1/2): weaknesses

Thank you for the careful and constructive review, and for your kind words on the direction, the statistical care and the analysis of Section 4. Each of your four weaknesses and three questions led to an experiment or a change, on three further seeds with their own retrains (in blue: Section 3.5 (p. 6), Section 5.6 (p. 9), Appendix E.7 (p. 24) with Table 16 (p. 25)). Your third weakness led to the main addition of the revision. This comment answers the weaknesses; (2/2) answers the questions.

W1. Six probes are few and noisy; a profile should be a distribution over rephrasings, whose tail would reveal a surviving fact. We agree that six probes are a small sample of a fact's phrasings, so we measured how much this matters. The probe generator wrote twenty candidate rephrasings per forget question and answer; the paper's checks keep 10.6 and 15.1 per fact, none a training probe. Two results follow. First, the noise of a single probe is moderate: under the retrain, one question rephrasing has an intraclass correlation of 0.93 across facts and one answer paraphrase 0.79. Second, the tail you describe is real for our method: for 50% of facts JOINT leaves a question rephrasing more than 0.5 nats likelier than under the retrain. The new per-token targets bring that share to 23%. We did not train on a distribution over rephrasings; we agree that it is the more defensible formulation and regard it as the natural next step.

Change. Section 3.1 (p. 3) now says what the six probes test and that they do not exhaust a fact's phrasings; the protocol and numbers are in Appendix E.7 (p. 25) and in the "rephr." column of Table 16 (p. 25).

W2. Reliance on a retain set; a localised effect around the forget set. We agree on both counts. The method trains on retain data, as do the objectives we compare with, and the Limitations (Section 6 (p. 9)) now say so. On the localised effect, the paper's evidence is the drift on never-seen facts of the same domain: in Table 9 (p. 21) ("model shift on never-seen") it lies between -0.056 and +0.016 nats per coordinate. We agree that this is a measurement on one benchmark and not a guarantee, and we ran no new experiment on this point.

W3. A point target inferred from other facts is questionable; an interval would fit better. We agree, and this point shaped the revision most. First, the target is no longer inferred from other facts alone: JOINT-ANCHORED starts from the fact's own profile under the base model and takes from never-seen facts only the shift that fine-tuning on the rest of the domain induces (Eq. (3) (p. 6)). Before training these targets sit 0.06 nats from the retrain on DIRECT, against 0.26 for JOINT's. Second, we built the interval you suggest, a split-conformal interval around the anchored target. It covers the retrain's value in 41, 72 and 86% of cells at nominal 50, 80 and 90%, so it is somewhat too narrow, and the paper says so. An arm trained to the 50% interval (INTERVAL-ANCHORED) reaches 0.709 nats, between JOINT (0.799) and JOINT-ANCHORED (0.675).

Change. Section 3.5 (p. 6); "Controls" in Appendix E.7 (p. 25).

W4. The gains are not convincing; the method is worse on sampling leakage and Forget Quality. We agree with this reading of the five-seed results, which are unchanged, and with your conclusion that per-fact closeness alone is not a sufficient proxy for forgetting; Section 5.5 (p. 9) says so. Three things are new.

  • A significant gain. On the further seeds JOINT-ANCHORED is closer to the retrain than JOINT on 3 of 3 seeds (0.675 against 0.799 nats, not significant at three seeds), and TOKEN-ANCHORED reaches 0.570, closer on every seed and significant after Holm correction.
  • A cause. Token by token, JOINT is no closer to the retrain than the model before unlearning (4.55 against 4.41 nats): a profile coordinate is a mean over the answer's tokens, so matching it leaves open where in the answer the unlikelihood sits.
  • A correction of our own text. The reported leak@k counts a fact when the true answer entails a sample, which also counts vague and truncated samples; Section 5.1 described the converse. Both directions are now reported. When a sample must entail the true answer, JOINT exceeds the retrain by 0.08 and JOINT-ANCHORED stays within 0.05 of it.

What remains, and is stated in the paper: no arm of ours passes Forget Quality, NPO and SimNPO still leak least, and on Qwen2.5-1.5B the gain from anchoring does not transfer within the same six epochs.

Change. Section 5.6 (p. 9); the metric sentence in Section 5.1 (p. 8) and the caption of Table 10 (p. 22). Deleted: the abstract's and the conclusion's "a retrain-free estimate of that level is the missing piece", replaced by the anchoring result.

AuthorsOpenReview ↗

Response to Reviewer 2tnV (2/2): questions and presentation

Q1. Targets regressed on relation, length and base-model difficulty, without optimal transport. We ran this as the arm REGRESS and report the answer, although it does not favour the transport: 0.831 nats against JOINT's 0.799, closer on 1 of 3 seeds, so the two are not separated. With anchoring the answer is the same: REGRESS-ANCHORED (0.687) matches JOINT-ANCHORED (0.675). Section 5.6 (p. 9) therefore now states that the gain comes from what the targets condition on, which now includes the fact's own base profile, not from the transport. We think your question sharpened the paper. In the same spirit, the abstract's conclusion is now an indication and no longer an established necessity: "These results indicate that forgetting should restore how probes co-vary across facts, at least across strata".

Q2. Many rephrasings and the per-fact tail. Done at evaluation time, as you suggest; see W1 in (1/2). The "rephr." column of Table 16 (p. 25) gives, for every arm and every published objective, the share of facts with a question rephrasing more than 0.5 nats likelier than under the retrain.

Q3. A reference from an entirely different source. We tested a generated reference. Another model (Qwen2.5-7B-Instruct) invents 36 fictitious people who are not authors and writes question-answer pairs about them; after the probe checks, 123 facts build targets and 55 feed the stopping rule. This reference costs JOINT 0.266 nats, but JOINT-ANCHORED only 0.044, because anchoring asks the reference only for the shift that fine-tuning induces and not for the level (Appendix E.7 (p. 25), "Generated reference"). This also bears on your third weakness: the anchored targets depend far less on what the other facts are.

Presentation (rated 2). We agree that the paper asked too much of the reader. A readability pass rewrote sixty-two passages of the main text (counted by a script, in blue). Section 1 (p. 1) states the paper in one sentence and introduces one example fact that Sections 3.1 and 5.1 reuse, every main-text caption opens with its takeaway, and Algorithm 1 (p. 5) now has inputs, an output and a comment on every step. One example, from Section 3.3 (p. 4). Before: "A criterion that sees only the population of profiles cannot tell which fact holds which profile (the fact assignment)". After: "no criterion on the set of profiles, energy distance and Sinkhorn divergence included, can tell which fact holds which profile, so evaluation is per fact".

In summary. Your review asked whether the point target, the six probes and the transport are sound. The revision answers each with a measurement. The target is now anchored on the fact itself and comes with an interval whose coverage is reported. The probes' reliability and their tail are measured. The transport is shown not to be the source of the gain, and the paper says so. We hope that these additions address your concerns about soundness, and we would be grateful if you would take them into account in your final assessment. Thank you again for a review that improved the paper.

I would like to thank the authors for such a detailed response and additional experiments. New experiments help to shed some additional light on the proposed technique and its promise. Unfortunately, I think this additional evidence does not make the original proposed approach based on optimal transport more convincing (it even seems to confirm that the transport is not what drives the gain) and the removal of the facts is also still not demonstrated (the new analysis of rephrasing shows that for a large fraction of facts some rephrasing of the question still makes the answer noticeably likelier than under the retrained model). JOINT-ANCHORED is an interesting extension, but its effectiveness is not yet established (it is not significantly better than JOINT, the only significant variant produces truncated answers, and the gain does not transfer to the second model family). Since this approach has just been introduced during the discussion phase, I think it would benefit from deeper analysis. I am leaning to keeping my score unchanged.

AuthorsOpenReview ↗

Response to Reviewer 2tnV (follow-up)

Thank you for the follow-up, and for seeing promise in the new experiments. You write that you are leaning, so we take your four reasons in turn. On each we agree with the facts, which are the paper's own, and we respectfully differ on what follows from them.

1. "The transport is not what drives the gain." Agreed. The arm that your first question asked for established it, and Section 5.6 (p. 9) states it. We do not think that it makes the approach less convincing, for three reasons.

  • The regression you proposed keeps every ingredient that your review listed as a strength (conditioning on difficulty and length, profiles over several phrasings, never-seen facts as the reference, a non-monotonic loss, an oracle-free stopping rule). It replaces only the solver that combines them, and its distance is not separated from JOINT's (0.831 against 0.799 nats). The result therefore does not hinge on one solver.
  • No control that replaces or perturbs the coupling has a lower mean than JOINT: 0.833 (COORDINATE-COND) and 0.797 (within-strata shuffle) against 0.785 on five seeds (Table 2 (p. 7)), and 0.831 (REGRESS) against 0.799 on the three further seeds. The data show that the transport is not necessary; they do not show that it is worse.
  • The main result replicated. On three new seeds with their own retrains JOINT scores 0.799 nats against 0.785 before (Appendix E.7 (p. 25)), and the published objectives are again farther (1.015 to 3.032 nats, Table 16 (p. 25)).

2. "The removal of the facts is still not demonstrated." We agree, and the paper does not claim it: Section 2 (p. 3) says "we claim only restored likelihoods". We differ on what the rephrasing test shows. It is the test that your second question asked for, a worst case over the 10.6 rephrasings kept per fact, and no objective we compared has a small share while staying near the retrain. UNDIAL leaves a share of 0.74. NPO and SimNPO reach 0.12 and 0.18 only by overshooting: their DIRECT NLL ends 2.20 and 0.91 nats above the retrain's, and they sit 3.032 and 1.753 nats from the retrain per fact. Suppression beyond the retrain is not the retrain's behaviour either, and it is detectable (Section 1 (p. 1)). The anchored arms lower both quantities at once: the share falls from 0.50 for JOINT to 0.31 for JOINT-ANCHORED and 0.23 for TOKEN-ANCHORED, and the per-fact distance falls with it (Table 16 (p. 25)).

3. "The effectiveness of JOINT-ANCHORED is not yet established." Your three facts are in the paper, and we agree that the evidence is early. Each deserves one qualification.

  • Significance. Each of the four anchored arms is closer than JOINT on all three seeds (Table 16 (p. 25)). A Holm-corrected paired test on three seeds has little power, and one arm passes it nonetheless. The level estimate is also measured before any training, on six seeds: the anchored targets sit 0.06 nats from the retrain on DIRECT against 0.26, and 0.520 against 0.613 per fact (Section 5.6 (p. 9)).
  • Truncation concerns TOKEN-ANCHORED only. JOINT-ANCHORED's samples have a median of 40 words (JOINT 45, retrain 26). It is closer to the retrain than JOINT on the DIRECT bias (-0.23 against -0.59), on the rephrasing share and on stating leak (-0.04 against +0.08), at the cost of more entailed leak (+0.50 against +0.37).
  • Second family. On two seeds no arm is ahead on both, the scalar baseline included (1.117, 1.127, 1.141 and 1.039 nats), so this test ranks no arm. It limits the scope of the claim, which the paper states; it is not evidence against anchoring.

4. "It was introduced during the discussion phase." We agree that it would benefit from deeper analysis, and the paper presents it as a first step with its limits. It is there because the reviews asked for it: your third weakness asked how a fact's retrain level can be inferred from other facts and suggested an interval, and anchoring and the interval arm are our answer. The submitted method and its five-seed results are unchanged and do not depend on it. We would be grateful if an extension made in reply to the reviews were weighed as such, and not against the submitted method.

Your review called the validation deep and honest. We tried to keep that standard: every new result that does not favour the method is in the paper. We hope these points can still be weighed in your final assessment, and we thank you again.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision and of what the reviews changed

We sincerely thank the three reviewers and the area chair. The reviews were careful and fair, and their main criticisms were accurate: the joint coupling was not shown to matter beyond the conditioning, the reference carried a level bias, and per-fact closeness to the retrain did not become behavioural forgetting. Each point led to an experiment or a change. The revised PDF marks every change in blue; the five-seed results are unchanged.

The paper in one sentence. A forgotten fact should resemble comparable never-seen facts on several probes at once; we match six-probe response profiles to never-seen profiles by a covariate-conditioned coupling and reach the lowest mean per-fact distance to the retrain among the objectives compared (0.785 nats), with no retrained model in training or stopping.

What the reviews added (three further seeds, each with its own retrain).

  1. A retrain-free estimate of the level, in answer to the level bias (Section 3.5 (p. 6), Section 5.6 (p. 9)). Each target is anchored on the fact's own profile under the model before fine-tuning, and never-seen facts supply only the shift that fine-tuning induces. Before training, the DIRECT target error falls from 0.26 to 0.06 nats. After training, the per-fact distance falls from 0.799 to 0.675 nats (closer on 3 of 3 seeds), and to 0.570 with per-token targets, which is closer on every seed and significant after Holm correction.
  2. A measured cause for the gap between profile agreement and behaviour. Token by token, JOINT is no closer to the retrain than the model before unlearning (4.55 against 4.41 nats); the per-token targets address exactly this.
  3. Every control the reviewers asked for, reported whichever way it came out (Appendix E.7 (p. 24), Table 16 (p. 25)): regression targets without transport and an interval target with its coverage (Reviewer 2tnV), many rephrasings per fact (2tnV, iRFL), a reference generated from another source (2tnV, Jnqn), ReGUn's own objective under our budget and stopping rule, leakage on forget facts outside training, the stopping threshold and equal-leakage comparisons (Jnqn).

A correction of our own text. Section 5.1 described leak@k as a sample entailing the true answer. The reported score counts a fact when the true answer entails a sample, which also counts vague and truncated samples. The sentence and the caption of Table 10 (p. 22) are corrected, and both directions are now reported. When a sample must entail the true answer, JOINT exceeds the retrain by 0.08 and the anchored arm stays within 0.05 of it.

What remains open, and is stated in the paper. No arm of ours passes Forget Quality. The coupling is not separated from the conditioning: a regression without transport does as well, so the paper now attributes the gain to what the targets condition on, and the abstract says "indicate" where it said "establish". The per-token arm produces truncated samples (a median of 3 words against the retrain's 26). On Qwen2.5-1.5B the gain from anchoring does not transfer within the same six epochs.

Presentation. All three reviews rate the presentation 2, and we agree that the paper asked too much of the reader. A readability pass rewrote sixty-two passages of the main text (counted by a script from the sources before and after the pass). Section 1 (p. 1) gives the paper in one sentence with its number, one example fact runs through Sections 1, 3.1 and 5.1, every main-text caption opens with its takeaway, and Algorithm 1 (p. 5) lists its inputs and output and comments every step. Each term has one meaning: "coupling" is only the transport plan, and the two leakage scores are named entailed and stating leak. Each reply quotes one passage before and after.

Other changes. Section 3.1 (p. 3) states what the six probes test. Section 3.3 (p. 4) gives the rank correlations that Proposition 1 concerns (0.12 on memorised facts, 0.88 on never-seen facts). The Limitations (Section 6 (p. 9)) add the retain-data requirement, and the ethics statement the compute of the new runs.

Deleted text. The abstract and the conclusion no longer say that "a retrain-free estimate of that level is the missing piece". To keep the page count, the main text drops sentences that the appendix or a float already gives: the probe-check rules (Appendix C.3 (p. 15)), the list of what the coupling must handle (Appendix C.1 (p. 14)), the MUSE-News observation behind the retain term (Appendix E.6 (p. 24)), the reason the live reference fails (Appendix E.5 (p. 24)), the method's requirements (the input line of Algorithm 1 (p. 5)) and the per-arm seed counts (Table 2 (p. 7)).

Code and records of the new analyses will be released with the paper. We are grateful for the time given to this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.