acceptodds
Under review as a conference paper at ICLR 2027

When Can First-Order Models of Fine-Tuning Bound Forgetting?

Abstract

Fine-tuning a language model on new data can make it forget facts that it should keep. We ask whether measurements taken at the start of a fine-tuning run can bound, for each protected fact, the probability that the run makes the model forget it. In LoRA fine-tuning with stochastic gradient descent on models from 0.6B to 14B parameters, a first-order response model estimated by finite-difference probes predicts changes of per-fact margins with correlation 0.974–0.998. Predictions of forgetting built on this model nevertheless failed; in Qwen3-0.6B, forgetting requires parameter changes far outside the region in which the model was validated. The probes can, however, bound the probability that a margin first falls below a boundary near zero: we derive Freedman and Azuma first-passage bounds for a linear surrogate of the margin and test on new runs whether they hold for the model. The bounds contain a term R that measures how much the response coefficients change during the run. The simplified Freedman bound, which sets R = 0, certified most facts but was violated in 14 of 112 conditions, and in these conditions every fact on which it was violated had R ≥ a, where a is the distance of the fact's margin to the boundary. The complete Freedman bound certifies only facts with R < a, and it held in every condition. On the violated facts, the spread of the margin across test runs was a median of 14.6 times the prediction of the response model, so the failures are breakdowns of the model, and in our data they occurred only where R ≥ a. We found this pattern post hoc and tested it in two preregistered confirmatory studies with 43 new conditions: the complete bound held in all of them, and the simplified bound failed there on only 3 facts, each with R ≥ a. A screen that decided for a whole condition whether to use the simplified bound failed on new starting states. First-order models of fine-tuning can thus bound forgetting on the facts whose response coefficients change by less than their distance to the boundary.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
catastrophic forgetting, fine-tuning, LoRA, concentration inequalities, first-passage bounds, knowledge retention

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_VMf4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether first-order models of fine-tuning can yield probabilistic guarantees on individual fact forgetting. The authors build a first-order response model via finite-difference probes along a reference fine-tuning trajectory, deriving first-passage probability bounds from Freedman's and Azuma's inequalities. A key finding is that highly accurate local linear approximations do not necessarily predict forgetting over an entire fine-tuning trajectory. The paper introduces a coefficient-change term (R) and identifies the condition (R < a), where (a) represents the distance to the forgetting boundary, as critical for obtaining nontrivial bounds. Experiments across several language models and fine-tuning configurations support this observation.

Critical strengths
  • The paper moves beyond ranking facts by forgetting risk and asks whether individual forgetting probabilities can be bounded before repeated fine-tuning runs—an interesting and practically relevant direction.

  • The coupling and non-identifiability results clarify why accurate local first-order approximations may fail to characterize long-horizon training behavior. The coefficient-change term RR offers a useful perspective on the reliability of local response models.

  • The authors evaluate multiple model families and scales, report unsuccessful hypotheses, and distinguish post-hoc findings from prospective confirmatory experiments, strengthening the credibility of the empirical analysis.

Critical weaknesses
  1. Theoretical guarantees apply to a linear surrogate, not actual model trajectories. The proposed Freedman and Azuma bounds are formally derived for a linear surrogate of the margin dynamics. Extending them to actual model trajectories requires a uniform approximation-error bound, which remains unverified (Section 4.3). Consequently, the reported validity on real models is empirical rather than theoretically guaranteed, limiting the strength of the central certification claim.

  2. Empirical reliability may stem from fact exclusion rather than the probabilistic bound. The complete Freedman bound certifies only 23.8% of eligible facts, whereas the simple condition R<aR < a covers approximately 31% of facts, none of which were forgotten in the reported test runs. This suggests that empirical reliability may primarily arise from excluding unstable facts rather than from the variance-dependent probability bound itself. A clearer analysis of the additional information provided by probabilistic certification would strengthen the contribution.

  3. Limited statistical power undermines the failure-localization evidence. The evaluation typically uses 32 test runs per condition. As acknowledged in Appendix I, under Bonferroni correction, a bound of 0.05 is rejected only after at least eight forgetting events, making smaller violations difficult to detect. Moreover, the confirmatory studies contain only three significantly violated facts for the simplified bound, so the evidence supporting the failure-localization criterion remains limited.

  4. Probe cost may outweigh practical benefits. The finite-difference probes require repeated trajectory replay, with computational cost growing approximately quadratically with the training horizon. The reported probe cost is substantial relative to directly running multiple fine-tuning trials. It remains unclear when the proposed certification provides a meaningful practical advantage over empirical risk estimation or simpler screening methods.

  5. Limited experimental scope restricts generality. The experiments focus on LoRA fine-tuning with SGD, short training horizons, and factual recall tasks with four candidate answers. The authors also observe that the linear response approximation breaks down under Adam. It is therefore unclear whether the framework generalizes to longer-horizon fine-tuning, commonly used adaptive optimizers, or more realistic knowledge-retention settings.

Questions for authors
  1. Can the authors provide empirical estimates or useful upper bounds for the approximation remainder between the actual margin trajectory and the linear surrogate? This would clarify when the theoretical guarantees can transfer to real models.

  2. What practical benefit does the complete Freedman bound provide beyond screening facts using (R < a)? Can the authors compare their calibration and risk discrimination under matched coverage?

  3. How sensitive are the conclusions to the number of test runs, particularly for detecting moderate violations that the current statistical tests may miss?

  4. Could the authors clarify the practical settings in which the cost of finite-difference probing is preferable to direct Monte Carlo estimation, especially when the same starting state is not reused extensively?

1 comment

We thank the reviewer for the thorough review and the detailed questions.

Q1: the remainder. Measured in the new studies (general response, point 2, table). For the 162 recorded facts that the complete bound certified, it stayed below a−Ra-R in 1,365 of 1,368 fact-runs.

Q2: calibration and discrimination at matched coverage.

  • Matched coverage: general response point 1, first table.
  • AUROC for forgetting in at least one run: R/aR/a 0.945, simplified value 0.929, margin 0.909.
  • Calibration with 128 to 256 runs per condition:
FactsThird studyFourth study
R<aR<a, any value: forgetting rate per run00
R≥aR \ge a, simplified value below 0.001: rate / mean value7.7×10−37.7\times10^{-3} / 1.4×10−41.4\times10^{-4}8.1×10−48.1\times10^{-4} / 1.3×10−41.3\times10^{-4}

Q3: number of test runs. The violations found grow with the number of runs, all on R≥aR \ge a (general response point 3, table). With 128 and 256 runs, the new studies found 28 violated facts, all with R≥aR \ge a, and no violation of the complete bound or the two-stage rule.

Q4: when probing beats simulation. See general response point 5, table.

  • When a starting state is used once, a single run followed by an evaluation is cheapest.
  • The probes pay off for small probabilities, for bounds on sets of facts, and when forgetting must be bounded before any run.

W5: scope.

  • Adam is not covered (Appendix B).
  • At longer horizons, the bounds held but certify few facts (Appendix K).
  • LoRA on bf16 weights and 128 facts per condition behaved like the main setting.
  • We tested only four candidate answers, although the margin applies to any finite set.

We hope these results address the reviewer's concerns, and we are happy to answer further questions.

Reviewer_JRc9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when local first-order models can bound forgetting during LoRA fine-tuning. It derives martingale bounds for a linear surrogate and evaluates their transfer to actual models. Empirically, all detected failures of the simplified bound occur where coefficient change (R) exceeds the margin distance (a).

Critical strengths
  1. The distinction between accurate local responses and reliable trajectory predictions is valuable, supported by experiments across multiple model sizes and families.

  2. The research question is interesting.

Critical weaknesses
  1. The guarantees concern a linear surrogate. Transfer to actual models requires an unquantified uniform remainder, while (V=2K) lacks a validated coverage guarantee. Appendix G also reports underestimated observed variability. Consequently, reliable per-fact probability certification remains unestablished, despite the encouraging empirical results.

  2. Independent confirmation is limited: the confirmatory studies contain only three violated facts, and the per-fact tests have low power near the nominal risk level. Moreover, Table 2 shows zero observed forgetting for the simpler (R<a) filter, which covers more facts than the complete bound. The added practical value of the numerical probability bounds therefore remains unclear.

Questions for authors
  1. How are coefficients between probe times handled when justifying the pathwise inequality (|D_t|\le R) in Appendix A.3?

  2. Can independent evaluation establish the numerical bounds’ value beyond the (R<a) filter, with uncertainty estimates accounting for dependence among facts sharing test runs?

4 comments

We thank the reviewer for the precise technical questions.

Q1: coefficients between probe times in ∣Dt∣≤R|D_t| \le R (Appendix A.3).

  • The surrogate holds the coefficients constant on each grid interval: every step in [s,s+ws)[s, s+w_s) uses qt,sq_{t,s}.
  • With this definition the bound is exact: each step contributes at most 2ηG0∥qt,s−qT,s∥2\eta G_0 \|q_{t,s}-q_{T,s}\|, and the wsw_s equal terms of an interval give the factor wsw_s.
  • The variation of the actual coefficients within an interval belongs to the remainder in Assumption 1, which we now measured.
  • Section 4 and Appendix A.3 now state this.

Q2: value beyond R<aR<a, with uncertainty that accounts for dependence. See general response points 1 and 3. All our uncertainty statements allow for dependence:

  • run-level counts (0 of 8,048 runs forgot a fact of the union-bound sets);
  • the bootstrap over whole starting states;
  • the set criteria of the new studies, which test whether a run forgets any fact of the set. They held in every condition.

W2: three confirmatory violated facts; low power. The confirmatory evidence now rests on 31 violated facts, all with R≥aR \ge a (general response part 1, second table). The base-rate test gives p=2.0×10−5p=2.0\times10^{-5}, and the within-condition test p=0.02p=0.02. With 256 runs, the third study detects a forgetting rate of 0.12 at power 0.8.

W1: V=2KV=2K.

  • κ=2\kappa=2 was fixed before the first study and never tuned (Section 4).
  • The realized quadratic variation of the surrogate was at most VV in at least 88.2% of fact-runs in every condition.
  • The noise outside the probed directions appears in the remainder, which we report separately (general response, point 2).

We hope these answers address the reviewer's concerns, and we are glad to discuss further.

As the rebuttal phase is approaching its end, we would kindly like to follow up. We hope our response is helpful.

May we kindly ask whether the response resolves the concerns? If it does, we would be grateful if the reviewer would consider reflecting this in the assessment. If any concern remains, we would very much welcome further discussion: we are happy to clarify any point or provide additional detail in the remaining time. Thank you for your time and effort in reviewing our work.

Thank you for the clarifications. These address several of my original concerns. However, my overall assessment remains limited by the modest methodological novelty and contribution. Therefore, I will maintain my original rating.

Thank you for the follow-up, and we are glad the clarifications addressed several of your concerns. Briefly, on novelty: forgetting predictors rank examples, and mitigation methods change the update. Our method instead gives each fact an upper bound on its forgetting probability, computed before training and without changing the update. The main finding also reaches beyond forgetting. Influence functions and linearizations are usually validated over short horizons; we show that an accurate first-order model fails over a run, and that a condition computed before training (R < a) identifies exactly the facts it still describes reliably. This held in every experiment, including the new ones. We appreciate your time and the thoughtful review.

Reviewer_gi9BOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether measurements taken at the start of a LoRA/SGD fine-tuning run can upper-bound, for each protected fact, the probability that the run drives the fact’s margin below zero. A first-order response model, estimated by finite-difference probes along 8 gradient-derived directions, predicts margin changes with correlation 0.974–0.998 on models from 0.6B to 14B parameters.

Critical strengths
  1. The methodology is unusually rigorous and transparent. It includes preregistered prospective studies on new starting states and frozen analysis scripts. The paper also candidly reports failed predictions (Table 4), a failed screening rule, protocol deviations, and corrected bugs.

  2. The paper shows that an accurate local model can still fail to predict run-level forgetting, which is a useful observation about the limits of local models of training.

Critical weaknesses

The bounds add little beyond the R < a rule. The complete bound certifies only facts with R < a. Table 2 shows that simply certifying every fact with R < a covers more facts (31.0% vs. 23.8%) with the same zero forgetting events, and the margin-threshold baseline also never failed. The formal guarantee covers only the surrogate, Assumption 1 is unverified, and the variance estimate is too low for most facts (Appendix G). The complete bound therefore holds because its certified facts lie far from the boundary (a − R exceeds 3 SD for 99% of them), not because its derivation applies. It certifies none of the facts that are actually at risk.

Questions for authors
  1. What does the Freedman probability add over the rule “certify iff R < a”? Is there any setting where the complete bound certifies facts with non-negligible empirical forgetting rates, i.e., where the bound is close to tight?

  2. Can the probe cost be reduced (e.g., fewer perturbation times or directions) while preserving validity? How does it scale to horizons of thousands of steps and to Adam?

2 comments

We thank the reviewer for the positive assessment of the methodology and the clear questions.

"The bounds add little beyond the R<aR<a rule"; the margin baseline also never failed. The selection R<aR<a is the central finding of the paper. The submission states in Section 7.4 that it, more than the variance estimate, makes the complete bound valid. R<aR<a is exactly the condition under which the complete bound is nontrivial, and the revision presents it as the answer to the title question.

The margin baseline is a useful comparison. It passed every test, but its facts were forgotten where the complete bound's were not:

Margin threshold: fact-runs, eventsComplete bound: fact-runs, events
112 earlier conditions18,232, 534,256, 0
Third study7.9% certified, 2 facts forgotten1.9% certified, 0
Fourth study9.8% certified, 1 fact forgotten23.8% certified, 0

At matched coverage, the probes (RR or a/Va/\sqrt{V}) separate facts that the raw margin does not (general response, point 1). The reviewer's reading, that the complete bound holds because its facts lie far from the boundary, is now measured directly: the remainder stayed below a−Ra-R in 99.8% of recorded fact-runs.

Q1: what the Freedman value adds; is the bound ever close to tight?

  • It adds bounds on sets of facts and small probabilities (general response, points 1 and 5).
  • On the facts it certifies, the values are conservative, as the revision reports:
    • Facts with R<aR<a were forgotten 0 times, against 22.5 events allowed by the union bounds.
    • Forgetting that is not negligible happens on facts with R≥aR \ge a, where the model breaks down. There, the spread across runs is 14.6 times the prediction, and forgetting is 54 times the simplified value in the third study.
  • The facts at risk are exactly those on which the first-order model breaks down. Tight values there would need a model that stays accurate along the run, and the paper reports this as a measured boundary of first-order models.

Q2: fewer probes; long horizons; Adam.

  • Probes at s=0s=0 alone (16 replays at any horizon) held prospectively.
  • All eight directions are needed: with k=4k=4, forgetting events fell on certified facts (Table 17).
  • Long horizons (general response, point 4): the cost stays flat, but the share of facts with R<aR<a falls from 46.9% at T=40T=40 to 12.5% at T=160T=160, even with full probes.
  • Under Adam, the probe responses stop scaling linearly after about ten steps (Appendix B), so the method covers SGD.

We hope this clarifies the contribution, and we would welcome further questions.

Thanks, I will keep my positive score.

Reviewer_rxPzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article investigates whether finite difference probes can be used to provide an upper bound on the probability that training will cause the model to forget each protected fact before LoRA+SGD fine-tuning begins. The author first discovered that the first-order response model was very accurate in predicting small disturbances (correlation coefficient 0.974-0.998), but based on this, the prediction of forgetting throughout the entire training process failed; So we turned to the linear proxy derivation of the margin, and Azuma and Freedman crossed the boundary for the first time, which contained a term R that measured the drift of the response coefficient during training. Among the 112 conditions of the 0.6B to 14B models, the simplified Freedman boundary ignoring R was violated in 14 conditions, and all violated facts satisfy R ≥ a (a is the distance from the reference margin to the boundary); Retaining the complete boundary of R only verifies the fact that R<a, which holds true under all conditions, and remains true in two pre registration confirmation studies (43 new conditions). Deciding whether to use the simplified boundary filtering scheme based on the entire condition will fail at the new starting state.

Critical strengths
  • The report on pre registration, post analysis, and failure outcomes (including screening rules for failures and early predictions) is very candid, and readers can clearly distinguish which conclusions have been prospectively tested and which are only discovered after the fact.
  • The results grouped by R/a showed a clear monotonic trend: none of the 1436 facts with R<a were forgotten, and 63.1% were forgotten when R/a ≥ 16, which has practical reference value as an empirical observation.
Critical weaknesses
  • The paper clearly distinguishes between the formal guarantee of linear agents and empirical testing on real models, but the abstract and conclusion still tend to summarize the results as being able to constrain the forgetting probability of real models. Due to the unquantifiable residual terms and lack of validation of the variance upper bound hypothesis, the current evidence is insufficient to support factual probability authentication on real models.
  • The experimental setup is relatively narrow (40 step SGD, 64 facts, 4-bit LoRA), and only about 20% of the facts are verified in the complete boundary. At T=80, the probe cost (equivalent to 68 test runs) has exceeded the 59 required for direct simulation, and the demonstration of practical value is not sufficient.
  • Only using R<a can achieve higher coverage and zero forgetting. Therefore, the current experiment directly supports a safe subset screening rule, rather than verifying the effectiveness of the Freedman numerical probability bound itself; The incremental value of the latter relative to simple screening is still unclear.
Questions for authors
  • How much worse is the ability of "R<a" to distinguish forgetting compared to filtering directly based on the reference margin a, under the condition of layering or matching authentication ratios based on a?
  • If the Freedman formula is not used and only "R<a" is used as the certification standard, what is the difference between the conclusion and now?
  • The conditions of different learning rates under the same model, data and seed may share the initial status. Can you report violations and certification rates in terms of the initial status?
2 comments

We thank the reviewer for the careful summary and the precise questions.

W1: "the abstract and conclusion still tend to summarize the results as being able to constrain the forgetting probability of real models." We have made the scope explicit in both places. The abstract now ends "The guarantee is formal for the surrogate and empirical for the model", and the conclusion says the same. We also measured the remainder that connects the two (general response, point 2).

W2: narrow setup, about 20% coverage, probe cost.

  • Coverage: the complete bound excludes facts with R≥aR \ge a, where the first-order model breaks down. The two-stage rule certifies about 31% of facts, both over the earlier conditions and in the fourth study.
  • Setup: the fourth study adds LoRA on bf16 weights and 128 protected facts, and every rule held. Appendix K adds horizons of 160 and 320 steps.
  • Cost: with probes at s=0s=0, T=80T=80 costs 16 replays instead of 68 run-equivalents, and this variant held prospectively.

Q1: R<aR<a against the margin aa, stratified or at matched coverage. Matched coverage and the within-condition comparison are in general response point 1. Stratified by quintile of aa (Table 15, post hoc):

Margin aaR<aR<a: factsR<aR<a: forgottenR≥aR \ge a: factsR≥aR \ge a: forgotten
(0, 0.18]230890369
(0.18, 0.38]189072435
(0.38, 0.52]348056421
(0.52, 0.62]40205114
(0.62, 0.79]474043910

The prospective within-condition comparison was undecided, because too few facts were forgotten at the lower learning rates.

Q2: conclusions with "R<aR<a" alone.

  • The validity results are the same.
  • R<aR<a gives a label, while the Freedman values give probabilities that add up to bounds on sets of facts. These held in every new condition.
  • R<aR<a is the condition a−R>0a-R>0 under which the complete bound is nontrivial, so the rule comes from the bound.

Q3: results by starting state.

  • Correct: conditions that differ only in learning rate share a starting state, and all CounterFact conditions of one model share the pretrained model. Section 3 now says so.
  • Over the 46 starting states, the complete and Azuma bounds held in all, and the simplified bound failed in 8 (Appendix L).

We hope the revision addresses the reviewer's concerns, and we are glad to answer further questions.

As the rebuttal phase nears its end, we would like to follow up. We hope our response has addressed your concerns. If so, we would appreciate your consideration of an updated score. If any concerns remain, we would be happy to clarify them. Thank you for your time and thoughtful review.

Comments on the paper

4 comments

General response (1/3): new experiments

We thank all four reviewers for their careful and constructive reviews. Three questions recur:

  • what the Freedman value adds over the rule "certify iff R<aR<a";
  • how the guarantee for the linear surrogate transfers to the model;
  • how strong the evidence on new runs is.

To answer them, we ran three new experiments with new random seeds, plus one extra run at a longer horizon. For each, the settings, the statistical tests and the analysis code were fixed before the first run. The revised PDF reports them in Section 7.4 and Appendices J and K, and shows all text added or changed since the submission in blue.

Results of the new experiments. The bounds held in every new condition.

Experiment 1Experiment 2Longer horizons
Conditions12114 (+1 with full probes)
Learning ratesevent-rich, where the simplified bound had failed0.008 to 0.0120.008, 0.012
Horizon TT; test runs per condition40; 25640; 128160 and 320; 64
Eligible facts with R<aR<a4.4%31.0%0% to 6.2%
Complete bound: certified, forgotten1.9% (9 facts), 023.8% (153), 01 fact, 0
Two-stage rule: certified, forgotten4.4% (21), 031.0% (199), 02 facts, 0
Conditions where the bound was exceeded: complete / two-stage / union bounds / s=0s=0 probes0 / 0 / 0 / 00 / 0 / 0 / 00 / 0 / - / 0
Simplified bound: violated facts (with R≥aR \ge a)23 (23)5 (5)3 (3)
  • The two-stage rule takes the simplified Freedman value for facts with R<aR<a.
  • Experiment 2 also covers LoRA on bf16 weights and 128 protected facts per condition; every rule held there.
  • One planned comparison was inconclusive: within conditions, R<aR<a against the facts with the largest margins. Too few facts were forgotten at these learning rates (1 among the largest margins).

Where the simplified bound fails (Table 12). We test the location against the base rate, the share of certified facts that have R≥aR \ge a. That share is high within a single experiment, so the test gains its power from pooling:

Set of conditionsViolated factsof which R≥aR \ge aR≥aR \ge a among certified factsFisher ppWithin-condition pp
Experiment 1232392.7%0.160.22
Experiment 25562.2%0.090.17
All four sets of new runs (two in the submission, two new)313170.8%2.0×10−52.0\times10^{-5}0.02
All 135 conditions626263.7%5.8×10−135.8\times10^{-13}3.4×10−63.4\times10^{-6}

Over all 135 conditions, facts with R<aR<a were forgotten 0 times in 75,824 (fact, test run) pairs.

General response (2/3): what RR and the values add; the remainder

1. What RR and the Freedman values add (Section 7.6; post hoc on the 112 earlier conditions). That the selection R<aR<a carries the validity of the complete bound is a finding of the paper. The submission states it in Section 7.4 (7.5 in the revision): "the exclusion of R≥aR \ge a, more than the variance estimate, therefore makes the complete bound valid".

  • RR comes out of the derivation.
  • R<aR<a is exactly the condition a−R>0a-R>0 under which the complete bound is nontrivial.

This is the paper's answer to the title question. Two further analyses show what the probes and the Freedman values contribute beyond the reference margin.

The probes add information beyond the reference margin. We compared selections of the same number of facts:

Selection of 1,436 factsFacts later forgottenEvents
R<aR<a00
Largest margin aa (one threshold for all conditions)1354
Largest aa within each condition33
Largest a/Va/\sqrt{V} within each condition00
  • A bootstrap over the 46 starting states puts the first difference at 12 facts (95% interval 2 to 28).
  • Much of that difference lies between conditions, since RR grows with the learning rate.
  • Within conditions, both RR and VV, which come from the probes, separate facts that the raw margin does not.

The values add up over facts.

  • A label of 0.05 on every fact with R<aR<a sums to a median of 0.68 per condition.
  • With the two-stage values, sets containing 30.6% of eligible facts carry a union bound of 0.05 on forgetting any of their facts.
  • Over 135 conditions, 0 of 8,048 test runs forgot a fact of these sets, against 22.5 allowed. The bounds are conservative, and this count holds for any dependence between facts.

Role of the variance term. In our data, facts with R<aR<a were forgotten 0 times whatever their Freedman value. So the variance term sets the size of the values, and with it the union bounds over sets, rather than which facts survive. The per-fact rule, with a threshold fitted on earlier data, certifies more facts than the complete bound, and the revision reports both.

2. The guarantee is formal for the surrogate; we measured the remainder (Section 7.4). The abstract, discussion and conclusion now say that the guarantee is formal for the surrogate and empirical for the model. From the projected clipped gradients recorded at every step, and from 16 further minibatches at the same parameters, we rebuilt each run's surrogate path:

Experiment 1 (event-rich)Experiment 2 (lower rates)
Realized quadratic variation ≤V\le V (lowest share over conditions)96.9%88.2%
Remainder / (a−R)(a-R) for facts with R<aR<a (median, range over conditions)0.09 to 1.030.01 to 0.34
Facts certified by the complete bound: remainder <a−R< a-R144 of 144 (fact, run) pairs (9 facts)1,221 of 1,224 (153 facts)

So the remainder of Assumption 1 grows with the learning rate, and the facts that the bound certifies lie beyond it. This is why the bound holds on the model.

General response (3/3): power, cost, horizons, presentation

3. Power and dependence. Our dependence-aware statements are:

  • the run-level counts above;
  • the bootstrap over whole starting states;
  • results by starting state: over 46 states for the 112 earlier conditions, the complete bound held in all, and the simplified bound failed in 8.

The detected violations grow with the number of test runs, and all of them stay on R≥aR \ge a (Table 16, post hoc):

Test runs used8162432
Simplified bound: violated facts (all with R≥aR \ge a)18283134
Complete bound: violating conditions0000

With 256 runs, a fact with value 0.05 is flagged at power 0.8 once its forgetting rate reaches 0.12 (0.305 with 32 runs).

4. Probe cost and horizons (Appendix K, Table 17). Probing at s=0s=0 alone costs 2k=162k=16 replays at any horizon. Post hoc on the 112 earlier conditions, it held everywhere and certified 20.8% of facts, against 23.8% with full probes, and it held on new runs in every new experiment. Coverage shrinks with the horizon. Same starting state (Qwen3-0.6B, learning rate 0.008):

HorizonProbesProbe cost (replays)Eligible facts with R<aR<a
T=40T=40every grid time4046.9%
T=160T=160every grid time7212.5%
T=160T=160s=0s=0 only16 (measured: 17 test runs)6.2%
T=320T=320s=0s=0 only16 (measured: 16 test runs)0%

The bounds held at long horizons but certify few facts, a limit of first-order models that Section 8 states. All eight directions are needed: with k=4k=4, forgetting events fell on certified facts.

5. Simulation (Appendix L, Table 13). Simulation used the first 59 runs to certify (value 0.0495), and every rule was scored on the remaining runs:

RuleExp. 1: certifiedExp. 1: forgotten laterExp. 2: certifiedExp. 2: forgotten later
Simulation, 59 runs75.1%3888.3%10
Per-fact rule13.0%027.7%0
Two-stage rule4.4%031.0%0
Complete bound1.9%023.8%0

Simulation suits a 0.05 statement when about 60 runs per starting state are affordable. The probes give values mostly below 0.001 (about 3,000 runs by simulation), bounds on sets of facts, and decisions before any run.

6. Presentation.

  • The abstract and introduction state the answer (R<aR<a).
  • Section 4 defines the surrogate between probe times.
  • Section 5 adds the two-stage rule.
  • Section 7.6 gathers what RR and the values add.
  • The screened bound is described more briefly.
  • In the revised PDF, text added or changed since the submission is shown in blue.
AuthorsOpenReview ↗

Summary of the discussion and revisions

We thank the reviewers for a constructive discussion. Below we summarize the paper, the concerns raised, and how the revision addresses them; changes are marked in blue in the revised PDF.

The paper in one paragraph. A first-order model of fine-tuning can be very accurate locally (correlation 0.974 to 0.998) and still fail to predict forgetting over a run. We show that it can nevertheless bound the probability of forgetting for each fact before training, and we identify when: when the drift R of the model's response coefficients is smaller than the fact's distance a to the decision boundary. Across all experiments, every failure of the simplified bound occurred on a fact with R ≥ a, and facts with R < a were never forgotten.

Strengths noted by the reviewers

  • Rigorous and transparent methodology, with failed hypotheses reported (rxPz, gi9B, VMf4).
  • The gap between local accuracy and run-level prediction is a useful observation, supported across model sizes (gi9B, JRc9, VMf4).
  • The coefficient-change term R is a useful perspective, and R/a shows a clear monotonic relation to forgetting (rxPz, VMf4).

Where the discussion stands. gi9B kept a positive score. JRc9 noted that our clarifications addressed several original concerns, and VMf4 acknowledged the value of the new experiments. The remaining points are novelty (JRc9) and the scope of the guarantee (VMf4), which we address below.

Contribution and novelty

  • A new kind of output. Forgetting predictors rank examples by risk (Jin & Ren, 2024; 2025), and mitigation methods change the update (EWC, replay, null-space projection). We give each fact an upper bound on its probability of being forgotten, computed before training and without changing the update.
  • Where first-order models of training can be trusted. Influence functions and linearizations are usually validated over short horizons. We show that an accurate first-order model fails to predict a run, and we identify a condition, R < a, computed before training, that separates the facts it describes reliably from those it does not. The condition held in every experiment.
  • Method. The bound is a first-passage (maximal martingale) bound with an explicit coefficient-drift term R. A measurement of the linearization error then turns the guarantee for the surrogate into a bound for the real model, with a failure rate that holds on held-out runs.

Main concerns and how we addressed them

Concern (reviewers)What we didEvidence (revised PDF)
What the Freedman bound adds beyond filtering by R < a (all)R < a is the bound's own condition; we compared it with margin-based screening, forgetting predictors and Monte CarloAt equal coverage, a margin threshold includes 13 later-forgotten facts, R < a none. The values combine over facts: union bounds cover 28.5% of facts, against 1.8% for a filter. AUROC 0.941 for R/a, against 0.880 for margin drop and 0.575 for a gradient-interference forecast (Sec. 7.6, Tables 13, 14, 19)
Guarantee covers the linear surrogate only (all)Measured the linearization error on recorded runs and computed the real-model bound U(a−R−γ) + δWith half the margin reserved for the error, the real-model bound stays ≤ 0.05 for 76.5% of certified facts; δ estimated on half of the runs holds on the held-out half (2.0% vs. upper bound 3.7%) (App. L, Table 18)
Few violations on new runs, low power (JRc9, VMf4)Two new experiments with new seeds and 128 to 256 runs per condition31 of 31 violations across the four sets of held-out runs have R ≥ a (p = 2.0×10⁻⁵); facts with R < a: 0 forgetting events in 82,352 (fact, run) pairs (Sec. 7.4, Table 12)
Probe cost, long horizons (rxPz, gi9B, VMf4)Probes at s = 0 only; runs at 160 and 320 steps; re-probing every 40 stepsProbes at s = 0 cost 16 replays at any horizon and stay valid. Coverage shrinks with the horizon (46.9% to 12.5% of facts with R < a), and re-probing keeps the bound valid in every window (App. K, Tables 11, 20)
Setup and scope (rxPz, VMf4)Added LoRA on bf16 weights and 128 facts per conditionAll rules held; Adam stated as out of scope with the measured reason (App. B, J; Sec. 8)
Clarity (all)Restructured; clarified technical detailsSec. 3 to 5, App. A.3, L

Limitations we state in the paper

  • The guarantee is formal for the linear surrogate. For the real model it rests on measured quantities, because no current analysis bounds the linearization error of LLM fine-tuning in advance.
  • Coverage shrinks with the training horizon.

The paper tells practitioners which facts a first-order model can protect before training, and the new experiments show that this holds on new runs and seeds, across model families and at longer horizons. We thank the reviewers again for their comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.