acceptodds
Under review as a conference paper at ICLR 2027

A Training-Dynamics View of Catastrophic Overfitting: Understanding and Prevention

Abstract

We study the underlying mechanism of catastrophic overfitting, a phenomenon in which models overfit to weak adversarial examples and lose true robustness, from the perspective of training dynamics. While prior work explains *when* catastrophic overfitting occurs through input-space quantities, the training-dynamics mechanism by which parameter updates produce the sudden collapse has received comparatively little attention. Catastrophic overfitting is a failure of *training*, and what corrupts training is the parameter update; we therefore analyze the *mixed Hessian*, the operator that carries an input-space perturbation error into the parameter-update error. We provide analytical and interventional evidence that rapid amplification of the mixed Hessian drives the parameter-update misalignment underlying catastrophic overfitting, and show that directly suppressing its spectral norm prevents the collapse. Based on this insight, we propose a novel KL divergence-based regularizer that stabilizes training dynamics and effectively prevents catastrophic overfitting. Our method achieves the strongest robustness among single-step methods and, when combined with multi-step adversarial training, further improves robustness over multi-step training alone, indicating that mixed-curvature stabilization is a general principle rather than a single-step remedy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Attack, AI Safety, Trustworthy AI, Catastrophic Overfitting, Training Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HFGeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies catastrophic overfitting (CO) in single-step adversarial training from a training-dynamics view. The authors write the gap between the FGSM and PGD parameter updates as a first-order term, connecting to a mixed input–parameter Hessian. They split this term into three factors: spectral norm of the mixed input–parameter Hessian, the mismatch between the PGD perturbation and the FGSM perturbation, and an alignment factor. Tracking these factors during training, they find that the spectral norm of the Hessian grows by orders of magnitude at CO onset. Intervention experiments suggest a positive feedback loop between spectral norm of the Hessian and the parameter-gradient norm, and penalizing the Hessian alone prevents CO. Based on this, they propose a KL consistency regularizer between an input-perturbed and a weight-perturbed prediction. The proposed method achieves improvement on the single-step robustness on CIFAR-10/100 and small gains when added to PGD-10 and TRADES.

Critical strengths

The paper asks a clear and relevant question: how an input-space attack error turns into a harmful parameter update. The mixed Hessian is a natural lens for this, and the analysis goes beyond correlation by using targeted interventions with a matched control. The proposed KL regularizer is simple to implement. It achieves the improved robustness among the single-step baselines and also improves multi-step methods such as PGD-10 and TRADES.

Critical weaknesses

The key concern of the paper is that the curvature/Hessian view of CO is not new, and the paper's analysis adds little insight beyond switching to another Hessian block. CO has been explained through curvature for years, for example: Kim et al. (AAAI 2021) linked CO to decision boundary distortion and a highly curved loss surface. Ortiz-Jimenez et al. (TMLR) tracked the top input-Hessian eigenvalue and showed that curvature explosion drives CO. GradAlign and ELLE both regularize local linearity, and ELLE explicitly ties its regularizer to loss curvature.

Moreover, the paper does not explain the key open questions about CO: The most puzzling features of CO are its suddenness and why it affects single-step training specifically. The paper does not address either:

  • The positive feedback argument (eq. 9–11) is only shown to be "consistent with" amplification. It does not explain when or why amplification starts, or why multi-step training avoids it. The paper itself only hypothesizes this last point (Sec. 5.2).
  • The interventions (Table 2) push σ1(H_xθ) upward directly and observe that the misalignment rises. Since the misalignment scales with σ1 by eq. (8), this is expected and does not show that naturally occurring CO is caused by σ1 growth rather than accompanied by it. For example, both could be driven by growing logit or weight scale.

References:

Questions for authors

See Weaknesses

2 comments

Thank you. We clarify the distinct role of the mixed Hessian and the purpose of the intervention, and add evidence that targeted amplification accelerates CO. We also connect measured KL-induced mixed-Hessian suppression to the corrected theorem.


What the mixed Hessian adds to input-curvature accounts.

Both quantities are second derivatives, but they are different operators. HxxH_{xx} maps an input perturbation to a change in the input gradient, entirely within input space. The mixed block HxθH_{x\theta} maps that perturbation to a change in the parameter gradient, transferring input-space attack error into the parameter-update space. Our analysis makes this connection explicit: for d=δF−δPd=\delta_F-\delta_P, gF−gP≈Hxθdg_F-g_P\approx H_{x\theta}d. This explains how an inaccurate inner attack changes the training update.


UM amplification precedes CO.

For each training update, we measure UM on the batch used for that update, apply the update, and measure UM again on the same batch. Test FGSM/PGD accuracy is also measured before and after the update. The raw pre-update measurements are:

Update (pre)UMPGD-10 (%)
16600.75939.61
16630.99339.49
16671.30939.08
16701.63935.18

Between updates 1660 and 1667, UM amplifies by 72.5%, while PGD-10 accuracy decreases by 0.53 pp (1.34% relative to its initial 39.61%). The subsequent FGSM–PGD gap reaches 13.17 pp at update 1718 and 41.57 pp at 1793. UM amplification therefore precedes the collapse of robustness, providing temporal evidence for UM's role in the causal sequence examined by the interventions below (See Revised Fig. 2(c)).

This progression also clarifies the sudden appearance of CO. With batch size 128, one epoch comprises 352 updates; 1660→1793 spans only 133 updates (0.38 epoch). Our iteration-level measurements resolve UM amplification accumulating before the large FGSM–PGD separation, a progression that appears sudden at epoch resolution. This short timescale is consistent with the within-epoch CO transitions reported by [Andriushchenko and Flammarion (2020, Sec. 4)].

Iteration-level accumulation and epoch-level abruptness. The paired measurements in revised Fig. 2(c) resolve UM amplification over successive updates before the robustness gap becomes large. Epoch-level plots compress these intervening changes into a sharp transition. The feedback and acceleration interventions below connect this temporal progression to mixed-Hessian amplification. The importance of finer temporal resolution is also supported by [Li et al. (2020, Sec. 3.3)].


Sec. 4.3: from positive feedback and UM amplification to accelerated CO.

The intervention tests why mixed curvature grows, not merely the mixed-Hessian-to-UM relation represented by the corresponding arrow in Eq. (5). Sec. 4.3 and original Table 2 established bidirectional amplification between the loss-gradient norm and σ1(Hxθ)\sigma_1(H_{x\theta}), together with the resulting UM amplification. We now extend this causal test to CO itself: targeted mixed-Hessian amplification accelerates the collapse.

Starting from the same checkpoint, we maximize the mixed singular value with an adversarial CE anchor, then resume FGSM training. Controls use no intervention, CE descent alone, or a random displacement matched to the targeted intervention's parameter-displacement norm. Ratios below are post/pre intervention.

Checkpoint interventionMixed ratioGrad. ratioUM ratioCO
None1.001.0001.00360
CE descent only1.730.9821.38390
Sigma ascent + CE anchor1.991.0201.61310
Matched random displacement1.010.9880.96460

The targeted intervention amplifies the mixed singular value to 1.99× and UM to 1.61×, and advances CO from update 360 to 310. This extends the original feedback result to a robustness collapse: mixed-Hessian amplification contributes causally to earlier CO (See Revision Sec. 4.3/Table 3)


Sec. 4.4: suppressing the mixed singular value prevents CO.

Sec. 4.4 provides additional causal evidence through direct suppression. Penalizing σ1(Hxθ)\sigma_1(H_{x\theta}) prevents CO and yields PGD-10 accuracy 47.44±1.21%. This intervention targets the mixed singular value directly and is independent of the KL analysis. Together with the acceleration experiment, it establishes complementary intervention outcomes: amplifying mixed curvature accelerates CO, while suppressing it prevents CO.


Conclusion on the mechanism.

UM amplification precedes CO; the decomposition in Fig. 3 identifies spectral amplification as a major factor in UM amplification; the original interventions establish positive feedback with the loss gradient; the added intervention accelerates CO; and direct suppression in Sec. 4.4 prevents it. Together, these temporal and interventional results support mixed-Hessian amplification as a causal driver of CO.

Theorem 1: corrected expansion and connection to mixed-Hessian suppression.

We acknowledge the mixed-term sign error and apologize for the incorrect statement. We correct Theorem 1 and its proof. At the common prediction pp, let Jx,JθJ_x,J_\theta be probability Jacobians, W=diag⁡(1/p)W=\operatorname{diag}(1/p), A=Jx⊤WJxA=J_x^\top WJ_x, B=Jθ⊤WJxB=J_\theta^\top WJ_x and C=Jθ⊤WJθC=J_\theta^\top WJ_\theta. For u=ηxu=\eta_x and b=ηθb=\eta_\theta, the corrected quadratic is

=\tfrac12u^\top Au-b^\top Bu+\tfrac12b^\top Cb, \qquad \mathrm{KL}=R_2+o(\|(u,b)\|^2).$$ The predictive Fisher $F=[A,B^\top;B,C]\succeq0$ has the standard mixed blocks; the negative cross term comes from the difference between the two perturbed predictions. The KL objective used in training is unchanged. **The KL-regularized method already suppresses mixed-Hessian growth in submitted Fig. 4(a); Fig. 4(b) relates the final mixed singular value to PGD robustness.** The scale ablation below further measures this suppression across input and parameter perturbations. These *empirical findings motivate the corrected connection between KL and mixed curvature*: Fix the model, batch and $u$, and draw $b\sim\mathcal N(0,s^2I)$. The signed cross term has zero expectation because $\mathbb E_b b=0$, while $\mathbb E_b[bb^\top]=s^2I$ turns the parameter quadratic into $s^2\operatorname{tr}C/2$. Therefore $$Q:=\mathbb E_b R_2(u,b) =\tfrac12u^\top Au+\tfrac{s^2}{2}\operatorname{tr}C.$$ The PSD Fisher structure gives $\|Bu\|^2\le(u^\top Au)\operatorname{tr}C$. Multiplying by $s^2$, taking square roots and applying AM–GM yields $$s\|Bu\|\le\sqrt{(u^\top Au)s^2\operatorname{tr}C}\le Q.$$ Thus, for fixed $s>0$, **$Q/s$ is an upper bound on directional mixed-Fisher sensitivity**. Expected local KL therefore regularizes a bound on mixed coupling through its input and parameter sensitivity terms. The corrected theorem supplies this local bound for $B$; **Fig. 4 and the scale ablation directly measure suppression of the actual loss mixed Hessian $H_{x\theta}$**. The corrected statement and proof appear in Theorem 1 and Appendix A. --- ### Scale ablation: joint KL suppresses mixed-Hessian growth. To empirically show how joint KL regularization suppresses the actual loss mixed Hessian, we vary $\eta_x=a\delta_F$ and per-coordinate Gaussian parameter std $s$ across the grid below, using the same starting checkpoint and 1600 FGSM updates with KL weight 50. The results are: |Input factor $a$|Parameter std $s$|Final $\sigma_1(H_{x\theta})$|CO| |---|---:|---:|:---:| |0|0|90.203|Yes| |0|1.00e-06|289.497|Yes| |0|7.55e-05|56.239|Yes| |0|3.02e-04|0.820|No| |0|1.21e-03|0.867|Yes| |0|4.83e-03|1.047|Yes| |0|6.04e-03|0.846|No| |0|7.25e-03|0.711|No| |1/8|0|0.829|No| |1/8|3.78e-05|0.681|No| |1/8|1.51e-04|0.826|No| |1/8|6.04e-04|0.750|No| |1/4|0|0.531|No| |1/4|1.00e-06|0.472|No| |1/4|7.55e-05|0.427|No| |1/4|3.02e-04|0.421|No| |1/4|1.21e-03|0.470|No| |1/4|4.83e-03|0.407|No| |1/4|6.04e-03|0.402|No| |1/4|7.25e-03|0.416|No| |1/2|0|0.282|No| |1/2|1.51e-04|0.240|No| |1/2|6.04e-04|0.224|No| |1/2|2.42e-03|0.239|No| **All 13 joint input–parameter configurations suppress mixed-Hessian growth:** final $\sigma_1(H_{x\theta})$ is **0.224–0.826**, compared with **90.203 without KL**. At fixed $a=1/4$, adding parameter noise $s=3.02\times10^{-4}$ reduces final mixed sigma from **0.531 to 0.421 (20.6% using unrounded values)**, demonstrating a contribution from parameter perturbation beyond input perturbation alone. **All 13 joint configurations also avoid CO.** The parameter-only rows quantify the effect of parameter scale when input perturbation is zero. These measurements connect the **corrected KL analysis** to **actual mixed-Hessian suppression.** Revision Sec. 5.3/Table 7 provides matched input–parameter comparisons.
Reviewer_DZExOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper evaluate catastrophic overfitting in adversarial training from the training dynamics perspective. It analysis how the mixed Hessian affects parameter updates and how its rapid increase can lead to a loss of adversarial robustness. Also, a KL divergence regulariser is proposed to stabilise training and prevent overfitting.

Critical strengths

There are several experiments including comparisons with adversarial training methods and different perturbation settings. The proposed regulariser shows improvements in robustness on the CIFAR datasets.

Critical weaknesses

I have the following concerns about the proposed model:

  1. My main issue is the mathematical formulation in Theorem 1. In Eq. 13, the KL is computed between predictions obtained by perturbing the input and model parameters separately. But, as I understand, the second-order expansion should contain negative mixed terms, while Eq. 15 presents them as positive. I am not sure about the correctness of the derivation, because this theorem is the main justification for the regulariser that proposed.

  2. In Eq. 17, the KL regulariser is introduced to eliminate the mixed Hessian. However, Theorem 1 only describes a local approximation based on the Fisher information matrix. Indeed, I am not convinced how minimising this KL guarantees a reduction in the largest singular value of the mixed Hessian, which also claimed to be the cause of overfitting mostly. In addition, the interventions in Table 2 modify the mixed Hessian norm or gradient norm, which also affect other properties of the network. Therefore, I do not think these experiments are sufficient to establish that the mixed Hessian is the main reason of catastrophic overfitting

  3. My another concern with Eq. 17 is that the input and parameter perturbations have different dimensions and scales, but there is no mechanism to balance their contributions. As I observed, probably this can cause the KL to mostly control one space instead of the mixed curvature as claimed. I think this is a limitation of the formulation that needs strong justification.

Questions for authors

Please refer to the weaknesses.

2 comments

Thank you for the careful assessment of the KL expansion, perturbation scales and causal interpretation. We strengthen the causal evidence, correct Theorem 1, and report the input–parameter scale ablation of actual mixed-Hessian suppression.


UM amplification precedes CO.

For each training update, we measure UM on the batch used for that update, apply the update, and measure UM again on the same batch. Test FGSM/PGD accuracy is also measured before and after the update. The raw pre-update measurements are:

Update (pre)Actual-batch UMTest PGD-10 (%)FGSM–PGD gap (pp)
16600.75939.614.04
16630.99339.494.04
16671.30939.084.69
16701.63935.185.86

Between updates 1660 and 1667, UM amplifies by 72.5%, while PGD-10 accuracy decreases by 0.53 pp (1.34% relative to its initial 39.61%). The subsequent FGSM–PGD gap reaches 13.17 pp at update 1718 and 41.57 pp at 1793. UM amplification therefore precedes the collapse of robustness, providing temporal evidence for UM's role in the causal sequence examined by the interventions below. Revised Fig. 2(c) shows the paired pre/post measurements and a past-only EMA.

This progression also clarifies the sudden appearance of CO. With batch size 128, one epoch comprises 352 updates; 1660→1793 spans only 133 updates (0.38 epoch). Our iteration-level measurements resolve UM amplification accumulating before the large FGSM–PGD separation, a progression that appears sudden at epoch resolution. This short timescale is consistent with the within-epoch CO transitions reported by [Andriushchenko and Flammarion (2020, Sec. 4)].


Sec. 4.3: from positive feedback and UM amplification to accelerated CO.

The intervention tests why mixed curvature grows. Sec. 4.3 and original Table 2 established bidirectional amplification between the loss-gradient norm and σ1(Hxθ)\sigma_1(H_{x\theta}), together with the resulting UM amplification. We now extend this causal test to CO itself: targeted mixed-Hessian amplification accelerates the collapse.

Starting from the same saved checkpoint, we maximize the mixed singular value with an adversarial CE anchor, then resume FGSM training. Controls use no intervention, CE descent alone, or a random displacement matched to the targeted intervention's parameter-displacement norm. Ratios below are post/pre intervention.

Checkpoint interventionMixed ratioGrad. ratioUM ratioCO
None1.001.0001.00360
CE descent only1.730.9821.38390
Sigma ascent + CE anchor1.991.0201.61310
Matched random displacement1.010.9880.96460

The targeted intervention amplifies the mixed singular value to 1.99× and UM to 1.61× their pre-intervention values, and advances CO from update 360 to 310. The CE-only and matched-random controls reach CO at 390 and 460. This extends the original feedback result to a robustness collapse: mixed-Hessian amplification contributes causally to earlier CO. Revision Sec. 4.3/Table 3 reports this added experiment.


Sec. 4.4: suppressing the mixed singular value prevents CO.

Sec. 4.4 provides additional causal evidence through direct suppression. Penalizing σ1(Hxθ)\sigma_1(H_{x\theta}) prevents CO and yields PGD-10 accuracy 47.44±1.21%, versus 0.19±0.10% for FGSM (original Table 3; revision Table 4). This intervention targets the mixed singular value directly and is independent of the KL analysis. Together with the acceleration experiment, it establishes complementary intervention outcomes: amplifying mixed curvature accelerates CO, while suppressing it prevents CO.


Conclusion on the mechanism.

UM amplification precedes CO; the decomposition in Fig. 3 identifies spectral amplification as a major factor in UM amplification; the original interventions establish positive feedback with the loss gradient; the added intervention accelerates CO; and direct suppression in Sec. 4.4 prevents it. Together, these temporal and interventional results support mixed-Hessian amplification as a causal driver of CO.

Theorem 1: corrected expansion and connection to mixed-Hessian suppression.

We acknowledge the mixed-term sign error and apologize for the incorrect statement. We correct Theorem 1 and its proof. At the common prediction pp, let Jx,JθJ_x,J_\theta be probability Jacobians, W=diag⁡(1/p)W=\operatorname{diag}(1/p), A=Jx⊤WJxA=J_x^\top WJ_x, B=Jθ⊤WJxB=J_\theta^\top WJ_x and C=Jθ⊤WJθC=J_\theta^\top WJ_\theta. For u=ηxu=\eta_x and b=ηθb=\eta_\theta, the corrected quadratic is

=\tfrac12u^\top Au-b^\top Bu+\tfrac12b^\top Cb, \qquad \mathrm{KL}=R_2+o(\|(u,b)\|^2).$$ The predictive Fisher $F=[A,B^\top;B,C]\succeq0$ has the standard mixed blocks; the negative cross term comes from the difference between the two perturbed predictions. The KL objective used in training is unchanged. **The KL-regularized method already suppresses mixed-Hessian growth in submitted Fig. 4(a); Fig. 4(b) relates the final mixed singular value to PGD robustness.** The scale ablation below further measures this suppression across input and parameter perturbations. These *empirical findings motivate the corrected connection between KL and mixed curvature*: Fix the model, batch and $u$, and draw $b\sim\mathcal N(0,s^2I)$. The signed cross term has zero expectation because $\mathbb E_b b=0$, while $\mathbb E_b[bb^\top]=s^2I$ turns the parameter quadratic into $s^2\operatorname{tr}C/2$. Therefore $$Q:=\mathbb E_b R_2(u,b) =\tfrac12u^\top Au+\tfrac{s^2}{2}\operatorname{tr}C.$$ The PSD Fisher structure gives $\|Bu\|^2\le(u^\top Au)\operatorname{tr}C$. Multiplying by $s^2$, taking square roots and applying AM–GM yields $$s\|Bu\|\le\sqrt{(u^\top Au)s^2\operatorname{tr}C}\le Q.$$ Thus, for fixed $s>0$, **$Q/s$ is an upper bound on directional mixed-Fisher sensitivity**. Expected local KL therefore regularizes a bound on mixed coupling through its input and parameter sensitivity terms. The corrected theorem supplies this local bound for $B$; **Fig. 4 and the scale ablation directly measure suppression of the actual loss mixed Hessian $H_{x\theta}$**. The corrected statement and proof appear in Theorem 1 and Appendix A. --- ### Scale ablation: joint KL suppresses mixed-Hessian growth. To empirically show how joint KL regularization suppresses the actual loss mixed Hessian, we vary $\eta_x=a\delta_F$ and per-coordinate Gaussian parameter std $s$ across the grid below, using the same starting checkpoint and 1600 FGSM updates with KL weight 50. The results are: |Input factor $a$|Parameter std $s$|Final $\sigma_1(H_{x\theta})$|CO| |---|---:|---:|:---:| |0|0|90.203|Yes| |0|1.00e-06|289.497|Yes| |0|7.55e-05|56.239|Yes| |0|3.02e-04|0.820|No| |0|1.21e-03|0.867|Yes| |0|4.83e-03|1.047|Yes| |0|6.04e-03|0.846|No| |0|7.25e-03|0.711|No| |1/8|0|0.829|No| |1/8|3.78e-05|0.681|No| |1/8|1.51e-04|0.826|No| |1/8|6.04e-04|0.750|No| |1/4|0|0.531|No| |1/4|1.00e-06|0.472|No| |1/4|7.55e-05|0.427|No| |1/4|3.02e-04|0.421|No| |1/4|1.21e-03|0.470|No| |1/4|4.83e-03|0.407|No| |1/4|6.04e-03|0.402|No| |1/4|7.25e-03|0.416|No| |1/2|0|0.282|No| |1/2|1.51e-04|0.240|No| |1/2|6.04e-04|0.224|No| |1/2|2.42e-03|0.239|No| **All 13 joint input–parameter configurations suppress mixed-Hessian growth:** final $\sigma_1(H_{x\theta})$ is **0.224–0.826**, compared with **90.203 without KL**. At fixed $a=1/4$, adding parameter noise $s=3.02\times10^{-4}$ reduces final mixed sigma from **0.531 to 0.421 (20.6% using unrounded values)**, demonstrating a contribution from parameter perturbation beyond input perturbation alone. **All 13 joint configurations also avoid CO.** The parameter-only rows quantify the effect of parameter scale when input perturbation is zero. These measurements connect the **corrected KL analysis** to **actual mixed-Hessian suppression.** Revision Sec. 5.3/Table 7 provides matched input–parameter comparisons. For $P$ parameters, the Gaussian vector norm is $s\sqrt P$. At $a=1/4$, the initial input-relative perturbation norm is 3.69% in normalized coordinates; matching the parameter-relative norm gives $s=3.02\times10^{-4}$, versus 0.0122% parameter-relative norm for $s=10^{-6}$. The grid tests the parameter contribution over both the original and larger relative scales, and the mixed-Hessian measurements quantify the resulting suppression.
Reviewer_zPiqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies catastrophic overfitting in single-step adversarial training, in which neural networks trained with FGSM attacks can overfit to training-time FGSM perturbations but fail to generalize to stronger PGD perturbations. The paper argues that the mixed Hessian HxθH_{x \theta} is the driver, supported by both theoretical analysis and controlled empirical experiments. The authors further introduce a KL-divergence consistency regularizer between predictions for a perturbed input and predictions under perturbed model parameters, which effectively mitigates catastrophic overfitting.

Critical strengths
  • The paper provides a new perspective to study catastrophic overfitting. Understanding how input-space attack errors propagate into parameter updates is a promising research direction.

  • The paper has a clear structure for readers to follow, starting with a controlled study on mixed Hessian, theoretical analyses, and progressing to the development of the KL-regularizer.

  • The performance of the proposed method (FGSM adversarial training + KL regularizer) is compelling, outperforming prior baselines by a noticeable margin.

Critical weaknesses
  • I do not see how the equality in Equation 8 is established. The SVD of the mixed Hessian should depend on projections onto all right singular vectors, weighted by their singular values, not just the leading one. Since this is a key theoretical result that supports the paper's central claim that the mixed Hessian drives catastrophic overfitting, the authors should correct the decomposition and clarify its implications.

  • In Equation 15, the off-diagonal terms should have a negative sign. If we conduct a second-order Taylor expansion on DKL(pθ(y∣x+ηx)∣∣pθ+ηθ(y∣x))D_{KL} (p_\theta (y | x + \eta_x) || p_{\theta+\eta_{\theta}}(y|x)) at (ηx,ηθ)=(0,0)(\eta_x, \eta_{\theta}) = (0,0), the mixed derivative terms should be −12ηxFxθηθ−12ηθFθxηx-\frac{1}{2} \eta_x F_{x \theta} \eta_\theta - \frac{1}{2} \eta_\theta F_{\theta x} \eta_x. The missing negative sign raises concerns about the validity of Theorem 1. Theoretically, it is not clear whether minimizing the proposed KL regularization term really suppresses the spectral norm of the mixed Hessian.

  • The paper presents evidence aligned with a relationship between mixed-Hessian behavior and catastrophic overfitting, but the analysis does not rigorously establish that the mixed Hessian is the central causal driver.

Questions for authors
  1. Please clarify the above concerns regarding the theoretical claims.

  2. The proposed KL-regularized FGSM adversarial training is about twice as slow as the standard FGSM baseline. Can the authors further improve the computational efficiency of their method while maintaining robust accuracy?

2 comments

Thank you for identifying the manuscript error in Eq. (8) and the sign error in Theorem 1. We correct both and strengthen the temporal and interventional evidence connecting mixed-Hessian amplification to CO.


Eq. (8): correcting our manuscript's description of α\alpha.

In the paper, we incorrectly described α\alpha as alignment with the top right singular vector. We sincerely apologize for this error. The quantity used in our measurements is the coefficient ratio below, which our evaluation implementation computes. With d=δF−δPd=\delta_F-\delta_P and d^=d/∥d∥\hat d=d/\|d\|, the corrected definition is

=\sqrt{\sum_i\left(\frac{\sigma_i}{\sigma_1}\right)^2(v_i^\top\hat d)^2}, \qquad \|H_{x\theta}d\|=\sigma_1(H_{x\theta})\|d\|\alpha.$$ Here $v_i$ are the input-space right singular vectors and $\alpha\in[0,1]$. The coefficient incorporates **all singular modes**. We correct the manuscript's definition and Eq. (8) accordingly. The decomposition separates spectral amplification, attack-error magnitude and the all-mode coefficient; its connection to UM uses the linearization $\|g_F-g_P\|\approx\|H_{x\theta}d\|$. --- ### UM amplification precedes CO. For each training update, we measure UM on the batch used for that update, apply the update, and measure UM again on the same batch. Test FGSM/PGD accuracy is also measured before and after the update. The measurements are: |Update (pre)|Actual-batch UM|Test PGD-10 (%)|FGSM–PGD gap (pp)| |---|---:|---:|---:| |1660|0.759|39.61|4.04| |1663|0.993|39.49|4.04| |1667|1.309|39.08|4.69| |1670|1.639|35.18|5.86| Between updates 1660 and 1667, **UM amplifies by 72.5%**, while PGD-10 accuracy decreases by **0.53 pp (1.34% relative to its initial 39.61%)**. The subsequent FGSM–PGD gap reaches 13.17 pp at update 1718 and 41.57 pp at 1793. **UM amplification therefore precedes the collapse of robustness**, providing *temporal evidence* for UM's role in the **causal sequence** examined by the interventions below. Revised Fig. 2(c) shows the paired pre/post measurements and a past-only EMA. **This progression also clarifies the sudden appearance of CO.** With batch size 128, one epoch comprises 352 updates; 1660→1793 spans only **133 updates (0.38 epoch)**. Our iteration-level measurements resolve UM amplification accumulating before the large FGSM–PGD separation, a progression that appears sudden at epoch resolution. This short timescale is consistent with the within-epoch CO transitions reported by [Andriushchenko and Flammarion (2020, Sec. 4)]. --- ### Sec. 4.3: from positive feedback and UM amplification to accelerated CO. The intervention tests *why mixed curvature grows*. Sec. 4.3 established bidirectional amplification between the loss-gradient norm and $\sigma_1(H_{x\theta})$, together with the resulting UM amplification. **We now extend this causal test to CO itself: targeted mixed-Hessian amplification accelerates the collapse.** Starting from the same checkpoint, we maximize the mixed singular value with an adversarial CE anchor, then resume FGSM training. Controls use no intervention, CE descent alone, or a random displacement matched to the targeted intervention's parameter-displacement norm. Ratios below are post/pre intervention. |Checkpoint intervention|Mixed ratio|Grad. ratio|UM ratio|CO| |---|---:|---:|---:|---:| |None|1.00|1.000|1.00|360| |CE descent only|1.73|0.982|1.38|390| |Sigma ascent + CE anchor|1.99|1.020|1.61|310| |Matched random displacement|1.01|0.988|0.96|460| **The targeted intervention amplifies the mixed singular value to 1.99× and UM to 1.61× their pre-intervention values, and advances CO from update 360 to 310.** The CE-only and matched-random controls reach CO at 390 and 460. This extends the **original feedback result** to a **robustness collapse**: mixed-Hessian amplification **contributes causally** to earlier CO. Revision Sec. 4.3/Table 3 reports this added experiment. --- ### Sec. 4.4: suppressing the mixed singular value prevents CO. Sec. 4.4 provides **additional causal evidence through direct suppression.** Penalizing $\sigma_1(H_{x\theta})$ prevents CO and yields PGD-10 accuracy **47.44±1.21%**. This intervention targets the **mixed singular value directly** and is *independent of the KL analysis.* Together with the acceleration experiment, it establishes complementary intervention outcomes: **amplifying mixed curvature accelerates CO, while suppressing it prevents CO.** --- # Conclusion on the mechanism. UM amplification *precedes* CO; the decomposition in Fig. 3 identifies spectral amplification as a *major factor* in UM amplification; the original interventions *establish positive feedback* with the loss gradient; the added intervention *accelerates* CO; and direct *suppression* in Sec. 4.4 *prevents* it. **Together, these temporal and interventional results support mixed-Hessian amplification as a causal driver of CO.**

Theorem 1: corrected expansion and connection to mixed-Hessian suppression.

We acknowledge the mixed-term sign error and apologize for the incorrect statement. We correct Theorem 1 and its proof. At the common prediction pp, let Jx,JθJ_x,J_\theta be probability Jacobians, W=diag⁡(1/p)W=\operatorname{diag}(1/p), A=Jx⊤WJxA=J_x^\top WJ_x, B=Jθ⊤WJxB=J_\theta^\top WJ_x and C=Jθ⊤WJθC=J_\theta^\top WJ_\theta. For u=ηxu=\eta_x and b=ηθb=\eta_\theta, the corrected quadratic is

=\tfrac12u^\top Au-b^\top Bu+\tfrac12b^\top Cb, \qquad \mathrm{KL}=R_2+o(\|(u,b)\|^2).$$ The predictive Fisher $F=[A,B^\top;B,C]\succeq0$ has the standard mixed blocks; the negative cross term comes from the difference between the two perturbed predictions. The KL objective used in training is unchanged. **The KL-regularized method already suppresses mixed-Hessian growth in submitted Fig. 4(a); Fig. 4(b) relates the final mixed singular value to PGD robustness.** The scale ablation below further measures this suppression across input and parameter perturbations. These *empirical findings motivate the corrected connection between KL and mixed curvature*: Fix the model, batch and $u$, and draw $b\sim\mathcal N(0,s^2I)$. The signed cross term has zero expectation because $\mathbb E_b b=0$, while $\mathbb E_b[bb^\top]=s^2I$ turns the parameter quadratic into $s^2\operatorname{tr}C/2$. Therefore $$Q:=\mathbb E_b R_2(u,b) =\tfrac12u^\top Au+\tfrac{s^2}{2}\operatorname{tr}C.$$ The PSD Fisher structure gives $\|Bu\|^2\le(u^\top Au)\operatorname{tr}C$. Multiplying by $s^2$, taking square roots and applying AM–GM yields $$s\|Bu\|\le\sqrt{(u^\top Au)s^2\operatorname{tr}C}\le Q.$$ Thus, for fixed $s>0$, **$Q/s$ is an upper bound on directional mixed-Fisher sensitivity**. Expected local KL therefore regularizes a bound on mixed coupling through its input and parameter sensitivity terms. The corrected theorem supplies this local bound for $B$; **Fig. 4 and the scale ablation directly measure suppression of the actual loss mixed Hessian $H_{x\theta}$**. The corrected statement and proof appear in Theorem 1 and Appendix A. --- ### Scale ablation: joint KL suppresses mixed-Hessian growth. To empirically show how joint KL regularization suppresses the actual loss mixed Hessian, we vary $\eta_x=a\delta_F$ and per-coordinate Gaussian parameter std $s$ across the grid below, using the same starting checkpoint and 1600 FGSM updates with KL weight 50. The results are: |Input factor $a$|Parameter std $s$|Final $\sigma_1(H_{x\theta})$|CO| |---|---:|---:|:---:| |0|0|90.203|Yes| |0|1.00e-06|289.497|Yes| |0|7.55e-05|56.239|Yes| |0|3.02e-04|0.820|No| |0|1.21e-03|0.867|Yes| |0|4.83e-03|1.047|Yes| |0|6.04e-03|0.846|No| |0|7.25e-03|0.711|No| |1/8|0|0.829|No| |1/8|3.78e-05|0.681|No| |1/8|1.51e-04|0.826|No| |1/8|6.04e-04|0.750|No| |1/4|0|0.531|No| |1/4|1.00e-06|0.472|No| |1/4|7.55e-05|0.427|No| |1/4|3.02e-04|0.421|No| |1/4|1.21e-03|0.470|No| |1/4|4.83e-03|0.407|No| |1/4|6.04e-03|0.402|No| |1/4|7.25e-03|0.416|No| |1/2|0|0.282|No| |1/2|1.51e-04|0.240|No| |1/2|6.04e-04|0.224|No| |1/2|2.42e-03|0.239|No| **All 13 joint input–parameter configurations suppress mixed-Hessian growth:** final $\sigma_1(H_{x\theta})$ is **0.224–0.826**, compared with **90.203 without KL**. At fixed $a=1/4$, adding parameter noise $s=3.02\times10^{-4}$ reduces final mixed sigma from **0.531 to 0.421 (20.6% using unrounded values)**, demonstrating a contribution from parameter perturbation beyond input perturbation alone. **All 13 joint configurations also avoid CO.** The parameter-only rows quantify the effect of parameter scale when input perturbation is zero. These measurements connect the **corrected KL analysis** to **actual mixed-Hessian suppression.** Revision Sec. 5.3/Table 7 provides matched input–parameter comparisons. **Computational cost.** Our runtime is approximately twice that of unregularized FGSM (2.11 vs. 1.03 h), which undergoes CO. **For assessing the efficiency of CO prevention, the relevant comparison is with methods that prevent CO:** ours takes **2.11 h**, versus **3.46 h for GradAlign** and **2.63 h for ELLE** (original Table 4; revision Table 5). Our method therefore prevents CO while requiring less training time than these CO-prevention baselines.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Mixed-Hessian Amplification: Causal Evidence and CO Prevention

Our contributions are a training-dynamics explanation of CO and a joint KL regularizer motivated by this mechanism. The mixed Hessian explains how attack errors propagate into parameter updates:

Loss-gradient amplification
             ⇅  Positive feedback [3]
Mixed-Hessian spectral amplification
             ↓  Major factor in UM [2]
UM amplification
             ↓  Temporal precedence [1]
Catastrophic overfitting

Together, temporal precedence and interventions that amplify or suppress mixed curvature support mixed-Hessian amplification as a causal driver of CO.

  • [1] Temporal evidence — revised Fig. 2(c)
    UM amplification → subsequent robustness collapse. Iteration-level progression → sudden appearance at epoch resolution.

  • [2] Decomposition — Eq. (8); Fig. 3
    Mixed-Hessian spectral amplification → a major factor in UM amplification. The corrected gain coefficient incorporates all singular modes.

  • [3] Feedback and acceleration — Sec. 4.3; revised Tables 2–3
    Original interventions: mixed singular value ⇄ loss-gradient norm, with UM amplified downstream.
    Added controlled intervention: targeted mixed-Hessian amplification → earlier CO.

  • [4] Direct prevention — Sec. 4.4
    Direct mixed-spectral penalty → CO prevention, providing additional causal evidence independently of the KL regularizer.

  • [5] From mechanism to regularization

    • Theory: corrected expected local KL quadratic → an upper bound on directional mixed-Fisher sensitivity (Theorem 1; Appendix A).
    • Evidence: training measurements and scale ablations → actual mixed-Hessian suppression, with parameter perturbations contributing beyond input perturbations (Fig. 4; revised Table 7).
    • Benefit: improved single-step robustness and further gains in multi-step training (revised Tables 5–6).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.