- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes CURE (Counterfactual Unlearning via Ratio Estimation). The key idea is to identify the ratio/weight of the forget set's contribution in the desired retain-only model. To do so, the paper uses an auxiliary model that amplifies the forget influence. CURE estimates the forget-source weight by tracking how much more likely each prefix is under the auxiliary model than under the factual model and updates it autoagressively. CURE then use the estimated weight to recover the retain-only next-token distribution. CURE is empirically evaluated on TOFU, MUSE, and RealToxicityPrompts.
Critical strengths
- The paper raises an important perspective. To recover the model or distribution after unlearning, we may first need to identify the weight/ratio the forget set play in generating the output sequences.
Critical weaknesses
-
The paper is not clearly presented and it is difficult to follow. This makes it hard to accurately assess the contributions and soundness of the paper.
- Although Secs 2-5 of the paper describe what is done, they consistently do not describe why it is done and do not provide intuition, a top down overview, or an explanation of each theorem's significance for the readers. In my opinion, significant rewriting may be needed.
- How is different from ? Is the goal of unlearning to recover ?
- Sec 2.1 suggests that the weight must be known to uniquely recover .
- Sec 2.2 suggests a way to set but does not justify why. How is chosen or known? (Eq 16 is still unclear) The subsection then abruptly introduces an auxiliary model that amplifies the forget distribution.
- In Sec 3, the concept "decoder state" is suddenly introduced. Why is the recovery operator and introduced and as defined in equation 8? seems to be designed to recover based on eq (6) and (7). is not even used in the main paper.
- It is hard to understand and comment on the remaining theoretical results due to the lack of clarity. The theoretical results seem to bound the error in estimating and
- A better explained algorithm pseudocode in the main paper would be useful
-
The experiment results can be more complete. See questions below.
Questions for authors
- Address the questions and concerns in weakness 1. Additionally, explain the intuition/significance of each theorem and design choice.
- Discuss the computation cost/complexity of CURE as compared to other unlearning methods.
- Beyond the usual unlearning metrics in Table 1 and 2, is it possible to report how CURE’s next-token predictions match those of the retrain model or report the approximation error? This would better relate the experiment results to the theory section.
- Are the experiments per section of the table repeated for different seeds? No uncertainty estimate is reported.
- if the claim is a better forget-utility tradeoff, it would be better to plot forget against retain ROUGE in a graph to compare methods across iterations as well.
- On what dataset or forget-retain split is CURE expected to perform best or worst? When is CURE preferred over other methods?
Other suggestions
- Bold and underline the best and second best result in each column of the experiment results
4 comments
Response to Reviewer C4MQ (2/4)
Q1(d). Why introduce controlled amplification
Controlled amplification provides a second observation of the same source components with a different forget contribution. For , combining that observation with the factual mixture eliminates the unknown forget conditional and gives
This equation explains the design: reveals the direction introduced by amplification, while the contextual coefficient determines how far to reverse that movement. Calibrated prefix density ratios supply the coefficient. The auxiliary and ratio estimator therefore provide complementary information for identifying the retain prediction.
Theorem 5.1 connects this construction to auxiliary training: it characterizes the data conditional under which the reverse-KL objective realizes the amplified observation. The result gives a precise training condition for producing the second observation used by the recovery rule.
Q1(e). Recovery and state-update operators
Each generation step has two tasks. The recovery operator first uses the current source weight to correct the next-token prediction. After a token is selected, the prefix changes, so the next correction must incorporate the new source evidence. The update operator performs this second task:
\qquad \ell_t=\frac{p_A(y_t\mid h_t)}{p_D(y_t\mid h_t)}.$$ A token more likely under the amplified observation increases the unprojected state; a token less likely under it decreases the state. The recursion consequently changes the correction strength along the observed prefix. The reference decoder evaluates these operations with neural likelihood ratios, and Theorem 3.1 quantifies how state-estimation error affects the recovered prediction. ### Q1(f). The theorem-to-algorithm connection The results form the following progression: | Question | Role of the result | |---|---| | What information is missing? | Theorem 2.1 relates the unidentified contextual weight to ambiguity in the retain prediction. | | What additional observations suffice? | Theorem 4.1 gives recovery from controlled amplification, calibrated BOS information and prefix ratios. | | What must auxiliary training realize? | Theorem 5.1 characterizes the required auxiliary-data conditional. | | How does approximation affect the practical target? | Theorem 3.1 separates the errors contributing to disagreement with retraining. | The strict reference decoder follows this sequence: initialize at BOS; update through prompt tokens; evaluate both next-token distributions; apply recovery and probability repair; select a token; and update that beam's state. The strict upper margin keeps the inverse finite, while repair handles negative coordinates caused by approximation. On the same 20 QA examples per split, the historical decoder and complete strict reference yield: | Decoder | Forget ROUGE | Retain ROUGE | |---|---:|---:| | Historical CURE | 32.133 | 99.000 | | Complete strict reference | 83.989 | 77.950 | This shared-subset comparison directly tests how decoder choice affects the forgetting–retention balance. The historical decoder achieves lower forget ROUGE and higher retain ROUGE, linking the high-retention result to a concrete decoding configuration.Response to Reviewer C4MQ (3/4)
Q2. Preparation, inference and implementation cost
CURE keeps the factual model fixed, trains an auxiliary, and performs additional model evaluation during decoding. Table 8 reports the following TOFU-10% system measurements:
| Method | Preparation, min | Inference, min | Peak inference memory, GB |
|---|---|---|---|
| DPO | 88 | 35 | 15 |
| NPO | 54 | 35 | 15 |
| GA | 62 | 35 | 15 |
| ULD | 36 | 72 | 21 |
| CURE | 7 | 50 | 54 |
The measurements locate the preparation–inference trade-off: CURE takes 7 minutes of preparation and 50 minutes of inference, compared with 54 and 35 minutes for NPO. With beam width , response length and vocabulary size , probability correction adds arithmetic. The archived implementation also performs full-vocabulary sorting, adding , and recomputes complete prefixes without incremental KV reuse. Model-forward cost additionally depends on prefix length and model size.
Reusing an identical auxiliary forward result reduces two auxiliary calls per native step to one. In the reuse experiment, generation time falls from 4.2899 to 3.5731 seconds, a 16.7% reduction, with all 66 checked token sequences unchanged. The sequence comparison ties the speed gain to preserved outputs, showing that this optimization removes redundant computation.
Q3. Direct next-token comparison with retraining
We compare factual and CURE next-token distributions with the retain-only retrained predictor at common prefixes. The evaluation uses 100 QA examples per split and measures TV after probability repair; lower values indicate closer agreement:
| Split | n QA | TV factual | TV CURE |
|---|---|---|---|
| Forget | 100 | 0.394915 | 0.841886 |
| Retain | 100 | 0.010667 | 0.006092 |
On Retain, CURE reduces TV by 0.004575, approximately 42.9% relative to the factual model. This directly answers whether the retained predictions become closer to the retrained predictor: the correction improves next-token agreement on Retain. On Forget, TV increases by 0.446971, placing the corrected prediction farther from that reference. The comparison ties empirical evaluation to the practical recovery target defined in the theory.
Q4. Repeated training and uncertainty
To test whether the retention result persists across auxiliary-training seeds, the added study repeats complete auxiliary training with seeds 0, 101 and 202, using 20 epochs and 8,000 updates per run. Each trained auxiliary is evaluated on the same 25 QA examples per split:
| Training seed | Forget ROUGE | Retain ROUGE |
|---|---|---|
| 0 | 38.537 | 100.000 |
| 101 | 32.894 | 100.000 |
| 202 | 33.542 | 98.857 |
| Mean ± sample SD | 34.99 ± 3.09 | 99.62 ± 0.66 |
Across the three trained auxiliaries, retained ROUGE averages 99.62 with a sample standard deviation of 0.66, while forget ROUGE averages 34.99 with a sample standard deviation of 3.09. Keeping the evaluation questions fixed makes this a direct check of sensitivity to auxiliary training. High retained ROUGE recurs in all three runs, addressing the concern that retention depends on one favorable auxiliary-training seed. Appendix D separately reports 100 controlled synthetic trials per sample size.
Response to Reviewer C4MQ (4/4)
Q5. Checkpoint trajectories
The checkpoint comparison follows CURE auxiliary epochs 2, 5, 10 and 20 and full-model NPO steps 1, 4, 16 and 32 on the same ten forget and ten retain QA examples:
| Method / checkpoint | QA per split | Forget ROUGE | Retain ROUGE |
|---|---|---|---|
| CURE consistency .5 / epoch2 | 10 | 38.3019 | 89.1935 |
| CURE consistency .5 / epoch5 | 10 | 29.4113 | 92.5329 |
| CURE consistency .5 / epoch10 | 10 | 33.471 | 99.2593 |
| CURE consistency .5 / epoch20 | 10 | 41.4255 | 100 |
| NPO full / step1 | 10 | 99.3849 | 100 |
| NPO full / step4 | 10 | 42.998 | 61.861 |
| NPO full / step16 | 10 | 4.4463 | 4.4302 |
| NPO full / step32 | 10 | 0 | 0 |
Forget and retain ROUGE at saved checkpoints, using the same 10 QA examples per split. Labels indicate auxiliary-training epochs for CURE and optimizer steps for NPO.
CURE's retained ROUGE increases from 89.19 at epoch 2 to 100 at epoch 20, while forget ROUGE varies non-monotonically. NPO moves toward stronger suppression together with reduced retained ROUGE. Among these saved checkpoints, CURE at epoch 20 and NPO at step 4 have comparable Forget ROUGE (41.43 versus 43.00), while Retain ROUGE is 100.00 versus 61.86. This comparison demonstrates CURE's retention advantage at this tested forgetting level. Following both scores across saved checkpoints also makes the effect of operating-point selection explicit.
Q6. Dataset differences and deployment choices
The analysis predicts favorable recovery conditions when the auxiliary is well aligned with the controlled observation, initialization is calibrated, and relevant prefixes have adequate coverage and retain-source support. The inverse becomes more sensitive as the source posterior approaches one or the amplification parameter approaches zero. This explains why forget-dominated prefixes are difficult: the inverse amplifies model and state-estimation errors as the retain-source contribution shrinks.
The TOFU results show how behavior varies with the deletion split:
| Forget split | CURE forget / retain ROUGE-L | Retain-only forget / retain ROUGE-L |
|---|---|---|
| 1% | 46.7 / 99.5 | 42.7 / 98.9 |
| 5% | 38.1 / 93.4 | 39.2 / 98.9 |
| 10% | 38.3 / 98.4 | 39.6 / 99.2 |
Among these runs, the 5% split has the largest retained-score deficit, 5.5 percentage points; the 10% run is close to the reference on both generation scores. On MUSE, retained utility is 69.53 versus 69.05 for retraining on Books, and 52.1 versus 55.7 on News. These comparisons answer the suitability question in terms of retained behavior: the current configuration achieves near-reference utility on Books, with larger retention gaps on News and TOFU-5%.
CURE is useful when an application needs to preserve the factual checkpoint and can accommodate an auxiliary model and a custom decoder. Its intervention operates during decoding, with the factual weights unchanged. Inference-memory and latency requirements can be assessed using the resource measurements in our response to Question 2.
Other suggestion. Table markings
For MUSE privacy deviation, values closer to zero rank higher; the other columns follow the metric directions stated in their captions. The retain-only row provides the target behavior against which the unlearning methods are compared. These criteria give a consistent basis for marking the best and second-best entries across columns.
Response to Reviewer C4MQ (1/4)
Q1(a). The practical target and the ideal distribution
The practical unlearning target is qR=A(Dr), the predictor trained on retained data under a specified training procedure, including its initialization and randomness. The distribution pR is the ideal retain-source component in the mixture model. Introducing pR makes it possible to study which predictive observations identify the retained contribution and to derive a corresponding recovery rule.
Theorem 3.1 connects this identification result to the practical target. Equation (11) bounds the discrepancy between the repaired prediction and qR using model-realization, auxiliary-alignment and state-estimation errors, repair distortion, and ϵR=∥qR−pR∥1. This gives a quantitative connection between the identifiable retain-source component and the practical retraining target. The response to Question 3 evaluates next-token agreement with qR directly.
Q1(b). Why the contextual weight is needed
The contextual weight resolves an ambiguity in the observed predictions. Consider a two-token illustration with pD=(0.6,0.4) and pF=(0.8,0.2). Under pD=(1−r)pR+rpF, the choice r=1/4 yields pR=(8/15,7/15), whereas r=1/2 yields pR=(0.4,0.6). Both explanations produce the same observed pD and pF.
Theorem 2.1 identifies the missing information: specifying a feasible weight determines which retain prediction is intended. This motivates the next step of the method—obtaining additional information through a controlled auxiliary observation and calibrated prefix ratios. The required weight concerns the current prefix, so it generally differs from the fraction of training examples designated for deletion.
Q1(c). Prompt posteriors and initialization
In the calibrated construction, α(x) is derived by processing the prompt from a known BOS source prior α0. With known amplification parameter κ and the assumed joint observation laws,
1+κα(x)=(1+κα0)pD(x)pA(x).
Prompt prefill determines the source weight for the first response token by incorporating the prompt's source evidence. The same likelihood-ratio update then adapts the correction strength as the response develops. For ideal response likelihood ratios and the unprojected state, Equation (16) quantifies how the initial error is multiplied by the response-prefix density ratio.
The reference initialization a0=β/(κλ)=0.5 is specified by the training weights; under BOS calibration it represents α0. The archived decoder uses a differently parameterized state initialized at zero and omits prompt prefill.