- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines whether first-order models of fine-tuning can yield probabilistic guarantees on individual fact forgetting. The authors build a first-order response model via finite-difference probes along a reference fine-tuning trajectory, deriving first-passage probability bounds from Freedman's and Azuma's inequalities. A key finding is that highly accurate local linear approximations do not necessarily predict forgetting over an entire fine-tuning trajectory. The paper introduces a coefficient-change term (R) and identifies the condition (R < a), where (a) represents the distance to the forgetting boundary, as critical for obtaining nontrivial bounds. Experiments across several language models and fine-tuning configurations support this observation.
Critical strengths
-
The paper moves beyond ranking facts by forgetting risk and asks whether individual forgetting probabilities can be bounded before repeated fine-tuning runs—an interesting and practically relevant direction.
-
The coupling and non-identifiability results clarify why accurate local first-order approximations may fail to characterize long-horizon training behavior. The coefficient-change term offers a useful perspective on the reliability of local response models.
-
The authors evaluate multiple model families and scales, report unsuccessful hypotheses, and distinguish post-hoc findings from prospective confirmatory experiments, strengthening the credibility of the empirical analysis.
Critical weaknesses
-
Theoretical guarantees apply to a linear surrogate, not actual model trajectories. The proposed Freedman and Azuma bounds are formally derived for a linear surrogate of the margin dynamics. Extending them to actual model trajectories requires a uniform approximation-error bound, which remains unverified (Section 4.3). Consequently, the reported validity on real models is empirical rather than theoretically guaranteed, limiting the strength of the central certification claim.
-
Empirical reliability may stem from fact exclusion rather than the probabilistic bound. The complete Freedman bound certifies only 23.8% of eligible facts, whereas the simple condition covers approximately 31% of facts, none of which were forgotten in the reported test runs. This suggests that empirical reliability may primarily arise from excluding unstable facts rather than from the variance-dependent probability bound itself. A clearer analysis of the additional information provided by probabilistic certification would strengthen the contribution.
-
Limited statistical power undermines the failure-localization evidence. The evaluation typically uses 32 test runs per condition. As acknowledged in Appendix I, under Bonferroni correction, a bound of 0.05 is rejected only after at least eight forgetting events, making smaller violations difficult to detect. Moreover, the confirmatory studies contain only three significantly violated facts for the simplified bound, so the evidence supporting the failure-localization criterion remains limited.
-
Probe cost may outweigh practical benefits. The finite-difference probes require repeated trajectory replay, with computational cost growing approximately quadratically with the training horizon. The reported probe cost is substantial relative to directly running multiple fine-tuning trials. It remains unclear when the proposed certification provides a meaningful practical advantage over empirical risk estimation or simpler screening methods.
-
Limited experimental scope restricts generality. The experiments focus on LoRA fine-tuning with SGD, short training horizons, and factual recall tasks with four candidate answers. The authors also observe that the linear response approximation breaks down under Adam. It is therefore unclear whether the framework generalizes to longer-horizon fine-tuning, commonly used adaptive optimizers, or more realistic knowledge-retention settings.
Questions for authors
-
Can the authors provide empirical estimates or useful upper bounds for the approximation remainder between the actual margin trajectory and the linear surrogate? This would clarify when the theoretical guarantees can transfer to real models.
-
What practical benefit does the complete Freedman bound provide beyond screening facts using (R < a)? Can the authors compare their calibration and risk discrimination under matched coverage?
-
How sensitive are the conclusions to the number of test runs, particularly for detecting moderate violations that the current statistical tests may miss?
-
Could the authors clarify the practical settings in which the cost of finite-difference probing is preferable to direct Monte Carlo estimation, especially when the same starting state is not reused extensively?
We thank the reviewer for the thorough review and the detailed questions.
Q1: the remainder. Measured in the new studies (general response, point 2, table). For the 162 recorded facts that the complete bound certified, it stayed below a−R in 1,365 of 1,368 fact-runs.
Q2: calibration and discrimination at matched coverage.
Q3: number of test runs. The violations found grow with the number of runs, all on R≥a (general response point 3, table). With 128 and 256 runs, the new studies found 28 violated facts, all with R≥a, and no violation of the complete bound or the two-stage rule.
Q4: when probing beats simulation. See general response point 5, table.
W5: scope.
We hope these results address the reviewer's concerns, and we are happy to answer further questions.