acceptodds
Under review as a conference paper at ICLR 2027

Damped Filtering for Representation Conditioned Adversarial Robustness

Abstract

Adversarial training typically applies a single ambient geometry to all local input directions, while manifold-aware defenses often require an unavailable hard tangent–normal decomposition. We propose Representation-Conditioned Complementary Damped-Filter Regularization (R-CDFR), which uses the singular spectrum of the classifier's representation Jacobian to allocate regularization according to local representation gain. Complementary damped filters define a high-gain pass branch and a low-gain residual branch. Although the exact Jacobian-null component of the input loss gradient vanishes, finite damping yields a nontrivial row-space residual that emphasizes low-gain directions. R-CDFR combines spectrum-wide inverse-metric slope control, concentrated residual-gradient suppression, and finite-step predictive consistency along pass-filtered perturbations. All operators are computed matrix-free using Jacobian–vector and vector–Jacobian products. We derive a local loss-growth bound for the induced perturbation family and relate the three regularizers to complementary local and finite-step quantities. Extensive experiments show consistent robustness gains across datasets, training recipes, convolutional and transformer backbones, and analytic tasks with known directional geometry.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
adversarial robustness, adversarial training, representation-conditioned regularization, anisotropic robustness, damped filtering, Jacobian spectrum

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_QapUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Representation-Conditioned Complementary Damped-Filter Regularization (R-CDFR), which uses complementary damped filters over the representation-Jacobian gain spectrum to allocate adversarial regularization across local directions, yielding consistent robustness gains without a hard directional partition. Extensive experiments show consistent robustness gains across datasets, training recipes, convolutional and transformer backbones, and analytic tasks with known directional geometry.

Critical strengths
  1. The paper consider a interesting research problem.
  2. The experimental study is broad, which is conducted on multiple scenarios.
  3. The paper provides both theoretical and empirical analysis to support the proposed method.
  4. The paper provides the ablation study to show the effectiveness of the proposed method.
Critical weaknesses
  1. Baseline comparisons are insufficient and lack competitive recent methods. The paper does not provide sufficiently comprehensive comparisons with recent competitive adversarial defense methods. The two most recent baselines included in the experiments are from 2024 and 2022, respectively, while stronger methods available on RobustBench are not adequately considered.
  2. The proposed method remains less competitive than the current state of the art. Although R-CDFR achieves the highest robust accuracy among the selected baselines, its performance still falls behind current state-of-the-art methods reported on RobustBench.
  3. The manuscript contains several formatting and presentation issues. Seemingly transparent text appears in multiple places throughout the paper. Additionally, the term "R-LipACR" appears in Figure 9 but is never introduced or explained in the main text.
  4. The font size in Figure 3 is too small, making the figure difficult to read.
Questions for authors

See weakness

2 comments

Thank you for recognizing our research problem, broad empirical study, and theoretical and ablation analyses. In our revision, we promoted a 200-epoch comparison with competitive baselines (including DHAT, ICCV 2025 [1]) into Sec. 6.6 (Table 4), standardized all labels and embedded PDF text, and expanded Figure 3 into a wide two-panel layout for readability.

Regarding the review summary, our paper proposes R-CDFR (Representation-Conditioned Complementary Damped-Filter Regularization), allocating regularization continuously over the representation Jacobian spectrum; ATFeaturePure and the vector field in the summary belong to a different submission. Below we address all four concerns for R-CDFR.

W1. Comparisons with recent competitive baselines

To establish performance alongside stronger recent recipes, Sec. 6.6 (Table 4) and App. F.1 (Table 13) evaluate controlled 200-epoch cosine runs across tuned TRADES, TRADES-AWP, TRADES-AWP + EMA/CutMix, and DHAT (ICCV 2025) [1]. All runs use WRN-34-10, no extra/synthetic data, validation selection on a held-out split, and raw-pixel AutoAttack at ℓ∞,ϵ=8/255\ell_\infty, \epsilon=8/255 (App. B.4, Table 7). Within this protocol, DHAT retains its 4/2554/255 feature-inverse generation and frozen natural ResNet-18 Grad-CAM mask, and we explicitly distinguish this matched adaptation from the original DHAT setup.

Table R10. Competitive 200-epoch WRN-34-10 recipes and paired R-CDFR additions (Sec. 6.6, Table 4; mean ± SD %).

Method (200-ep cosine)RunsC10 CleanC10 AAC100 CleanC100 AA
Tuned TRADES584.36±0.2784.36\pm0.2755.63±0.4155.63\pm0.4167.21±0.3867.21\pm0.3834.71±0.5334.71\pm0.53
+ R-CDFR584.12±0.3084.12\pm0.3057.96±0.4857.96\pm0.4867.05±0.4167.05\pm0.4137.45±0.6137.45\pm0.61
TRADES-AWP583.61±0.3183.61\pm0.3156.91±0.3856.91\pm0.3867.84±0.3667.84\pm0.3635.96±0.5235.96\pm0.52
+ R-CDFR583.47±0.2983.47\pm0.2958.48±0.4658.48\pm0.4667.71±0.3967.71\pm0.3937.83±0.5837.83\pm0.58
TRADES-AWP + EMA/CutMix382.67±0.2882.67\pm0.2858.02±0.3458.02\pm0.3468.11±0.3368.11\pm0.3337.18±0.4637.18\pm0.46
+ R-CDFR382.79±0.2582.79\pm0.2559.31±0.3759.31\pm0.3768.20±0.3168.20\pm0.3138.54±0.5138.54\pm0.51
DHAT [1], adapted585.42±0.2985.42\pm0.2959.08±0.4259.08\pm0.4269.08±0.4169.08\pm0.4138.77±0.5738.77\pm0.57
+ R-CDFR585.17±0.3385.17\pm0.3359.57±0.3759.57\pm0.3768.86±0.4668.86\pm0.4639.38±0.5439.38\pm0.54

Adding R-CDFR improves mean AA across all four recipes: +2.33/+2.74+2.33/+2.74 pp on Tuned TRADES, +1.57/+1.87+1.57/+1.87 pp on TRADES-AWP, and +1.29/+1.36+1.29/+1.36 pp on EMA/CutMix (winning every paired seed), plus +0.49/+0.61+0.49/+0.61 pp on DHAT (reaching 59.5759.57% / 39.3839.38% AA, with Clean shifts of −0.25/−0.22-0.25/-0.22 pp). Table 4 and App. Table 13 report 95% paired Student-tt intervals, which are positive on C100 DHAT ([0.12,1.10][0.12, 1.10]) and span zero on C10 DHAT ([−0.07,1.05][-0.07, 1.05]).

[1] K. Zhang, J. Weng, S. Li, and Z. Luo. Towards Adversarial Robustness via Debiased High-Confidence Logit Alignment. ICCV, 2025, pp. 2783–2792.

W2. Contextualizing performance vs. RobustBench state of the art

Fixed-backbone results without extra data must be distinguished from top RobustBench entries using millions of synthetic diffusion images, larger backbones (WRN-70-16), and longer budgets. Our revision separates system operating points (57.4357.43% / 36.9836.98% AA for 50-ep WRN-34-10 in Table 1; 59.5759.57% / 39.3839.38% AA for 200-ep no-extra-data DHAT + R-CDFR in Sec. 6.6, Table 4) from the modular effect of Jacobian-spectrum filtering, grounded in Sec. 5 via our spectral-scale ambient bound (Cor. 5.2) and smoothness-free finite-step margin bound (Prop. 5.3). At 1.19×1.19\times TRADES epoch time vs. 1.45×1.45\times for TART (App. Table 8), R-CDFR is an efficient plug-in regularizer alongside clean-accuracy and compute trade-offs.

W3. Formatting, embedded text, and consistent naming

We removed the legacy R-LipACR title from App. Figure 9, standardized R-CDFR across all panels, legends, captions, and text, and audited the PDF via rendering and text extraction so no stray or low-opacity labels remain.

W4. Font size and readability of Figure 3

We replaced the narrow inline wrapfigure (0.30\linewidth) with a wide two-panel Figure 3 in Sec. 6.4, enlarging all axis titles, ticks, and legends. Panel (a) shows the CDF of 1D decision-changing distances along normalized residual-filtered gradients, and Panel (b) pairs it with pass-filtered KL consistency across step sizes ϵP\epsilon_P, distinguishing this directional probe from unrestricted adversarial distances.

Thank you for the rebuttal. Some of my concerns have been addressed, and I will raise my score.

Reviewer_3mpcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes R-CDFR, a representation-conditioned regularization framework for adversarial robustness. The method uses the representation Jacobian to distinguish directions with different levels of representation change and allocates regularization through complementary damped filters. It combines two gradient-based penalties with predictive consistency regularization. The paper provides a local theoretical analysis and evaluates the method across multiple datasets, architectures, and training settings.

Critical strengths

An interesting approach to directional robustness. The proposed framework uses continuous representation gain to allocate regularization, avoiding a hard division of input directions into separate subspaces. The formulation offers a useful perspective on how representation-dependent sensitivity can inform adversarial training.

Broad experimental evaluation. The paper includes standard adversarial evaluations, component ablations, alternative regularization controls, and experiments across convolutional and transformer architectures. Improvements also persist when the method is added to a stronger training recipe, providing encouraging evidence of practical value.

Critical weaknesses

The interpretation of the proposed regularization remains insufficiently established. The regularizers depend on the learned representation, whose scale and geometry can change during training. Consequently, reductions in the regularization terms may partly reflect changes in the representation rather than improvements in adversarial robustness. The external-representation controls are helpful, but further clarification is needed to establish that the proposed mechanism operates as intended during joint training.

The empirical evidence requires clearer reconciliation and contextualization. Some reported sensitivity and residual-energy measurements appear inconsistent with their stated relationship, making the mechanism analysis difficult to interpret confidently. In addition, the improvement over the main baseline is substantially larger than the gain under the stronger training recipe. Although the latter result remains positive, the paper should more clearly establish the magnitude and reliability of the benefit under competitive, well-tuned training conditions.

Update after rebuttal: The corrections and additional experiments substantially address the concerns above. I have raised my rating to weak accept.

Questions for authors
  1. How does the method control changes in representation scale during training, and what evidence distinguishes improved robustness from changes in the representation used to define the regularizers?
  2. Can you clarify the relationship between the reported residual sensitivity and residual-energy measurements, including any differences in normalization or aggregation?
  3. What explains the difference in improvement between the main comparison and the stronger training recipe? How consistently does the method improve robustness when the underlying baseline is well tuned?
5 comments

Thank you for your review. Our revision fixes DRD_R, adds scale/trajectory controls, and expands 200-ep recipes. All section, table, and figure numbers cited below refer to the revised manuscript PDF.

W2/Q2. Residual sensitivity and energy

For r=Rλgr=R_\lambda g, SR=∥r∥2S_R=\Vert{}r\Vert{}_2 and DR=∥r∥22=SR2D_R=\Vert{}r\Vert{}_2^2=S_R^2 satisfy DˉR=SˉR2+Varemp[SR]≥SˉR2\bar{D}_R=\bar{S}_R^2+\mathrm{Var}_{\mathrm{emp}}[S_R]\ge\bar{S}_R^2 within each run (Sec. 6.4; App. B.7, Eq. (23)). An export bug filled DRD_R with ρR=E[∥Rλg∥22/(∥g∥22+η)]\rho_R=\mathbb{E}[\Vert{}R_\lambda g\Vert{}_2^2/(\Vert{}g\Vert{}_2^2+\eta)] instead of Eq. (9)'s unnormalized ∥Rλg∥22\Vert{}R_\lambda g\Vert{}_2^2. Recomputing unnormalized DRD_R directly on test checkpoints updates Table 3 & App. Tables 11–12 (AA% / SRS_R / DR±SDD_R\pm\text{SD}; 5 runs C10/C100, 3 TinyIN & controls):

VariantC10C100TinyIN
Full R-CDFR57.43 / 1.08 / 1.47±0.1236.98 / 1.71 / 3.64±0.2730.82 / 2.21 / 6.10±0.43
w/o Lspec\mathcal{L}_{\mathrm{spec}}52.14 / 1.22 / 1.96±0.1732.41 / 1.92 / 4.65±0.3426.73 / 2.48 / 7.76±0.58
w/o Lres\mathcal{L}_{\mathrm{res}}49.38 / 1.58 / 3.62±0.2929.26 / 2.42 / 8.31±0.5723.81 / 3.14 / 13.92±0.91
w/o Lcons\mathcal{L}_{\mathrm{cons}}53.47 / 1.13 / 1.60±0.1433.58 / 1.79 / 4.02±0.3127.64 / 2.34 / 6.96±0.52
Ambient GP54.91 / 1.30 / 2.32±0.1934.28 / 2.01 / 5.21±0.39—
Hard thr.55.86 / 1.21 / 1.85±0.1635.06 / 1.95 / 4.79±0.36—
Top-kk56.34 / 1.16 / 1.71±0.1535.61 / 1.86 / 4.36±0.33—

All runs obey DˉR≥SˉR2\bar{D}_R\ge\bar{S}_R^2; removing Lres\mathcal{L}_{\mathrm{res}} spikes SR,DRS_R,D_R and drops AA most.

W1/Q1. Scale vs. robustness gains

Rescaling h↦ch,cϕ(z)↦cϕ(z/c)h\mapsto ch, c_\phi(z)\mapsto c_\phi(z/c) alters fixed-λ\lambda terms while preserving fθf_\theta and pixel AA. Sec. 6.4 & Apps. I, L decouple scale:

  1. Fixed/external hh (Tables 20–21). Freezing 200-ep TRADES hψh_\psi in 50-ep head training keeps +1.94/+3.29+1.94/+3.29 pp AA over fixed-hh TART (56.58% / 35.76%). Under frozen clean-ERM WRN-34-10, SRrefS_R^{\mathrm{ref}} drops 1.57→1.291.57\to 1.29 (C10) and 2.43→1.952.43\to 1.95 (C100).
  2. Normalized Gram (App. I.2, Eq. (34), Table 23). With s2=tr(J⊤J)/ms^2=\mathrm{tr}(J^\top J)/m (16 Rademacher probes in autograd), Gˉ=J⊤J/s2,Aˉ=Gˉ+λˉId\bar{G}=J^\top J/s^2, \bar{A}=\bar{G}+\bar{\lambda}I_d are invariant to J↦cJJ\mapsto cJ, getting 57.18±0.62% / 36.61±0.74% C10/C100 AA (80.03% / 67.12% Clean; Table 7) vs. 57.43% / 36.98% for R-CDFR.
  3. Shared-basis energy (App. I.3, Table 24). In frozen ERM singular bins on C10 (5 runs, TART=1), R-CDFR low-/high-gain energy is 0.67±0.06 / 0.85±0.05 (33% vs. 15% cut), vs. 0.79±0.05 / 0.81±0.04 for Ambient GP and 0.86±0.06 / 0.89±0.05 w/o Lres\mathcal{L}_{\mathrm{res}}.
  4. Theory & 40-d study (Sec. 5; App. L, Tables 27–30). Cor. 5.2 (Eq. (15)) includes M=∥J∥22+λM=\sqrt{\Vert{}J\Vert{}_2^2+\lambda} in C=MSP+SRC=MS_P+S_R (J↦cJJ\mapsto cJ raises MM); Prop. 5.3 (Eq. (16)) bounds pass error by min⁡{1,2κP/μ2}\min\lbrace 1,2\kappa_P/\mu^2\rbrace via Pinsker. In our 5-seed, 40-d ReLU study (Eq. (36), Table 30), Ambient GP over-cuts high-gain energy (0.388; low 1.130; 70.31% acc) and w/o Lres\mathcal{L}_{\mathrm{res}} cuts low-gain by 7.6% (0.924; high 0.654; 70.62%), whereas R-CDFR and Norm-Gram cut low-gain by 33.9% (0.661) and 38.4% (0.616), keep high-gain (0.667 / 0.625), lower E[C]\mathbb{E}[C] (3.46→3.03 / 2.53), and get 70.78% acc (Base AT: 71.25%).

W2/Q3. Gains on competitive 200-ep recipes

In Table 1 (50-ep step decay), standalone TRADES stalls at 31.25% C10 AA from early inner-KL curvature; any R-CDFR subset lifts AA to 49.38%–53.47% (Table 3) and R-CDFR to 57.43% (+2.05/+3.78 pp vs. TART). In 200-ep cosine runs, Tuned TRADES converges (55.63% / 34.71%). Sec. 6.6 (Table 4) & Table 13 give paired 200-ep WRN-34-10 runs (ℓ∞,8/255\ell_\infty, 8/255):

200-ep Recipe (Runs)C10 AAC10 Δ\Delta [95% CI]C100 AAC100 Δ\Delta [95% CI]
Tuned TRADES (5)55.63→57.96+2.33 [1.88,2.78]34.71→37.45+2.74 [2.21,3.27]
TRADES-AWP (5)56.91→58.48+1.57 [1.17,1.97]35.96→37.83+1.87 [1.36,2.38]
+ EMA/CutMix (3)58.02→59.31+1.29 [0.70,1.88]37.18→38.54+1.36 [0.60,2.12]
DHAT [1], adapted (5)59.08→59.57+0.49 [-0.07,1.05]38.77→39.38+0.61 [0.12,1.10]

R-CDFR wins all paired runs (5/5, 3/3) on TRADES recipes (EMA/CutMix Δ\Delta: +1.02, 1.38, 1.47 C10; +1.15, 1.22, 1.71 C100), tapering on DHAT (4/5 C10, CI spans 0; 5/5 C100) from regularization overlap at 1.19×1.19\times TRADES time (Table 8).

[1] K. Zhang, J. Weng, S. Li, and Z. Luo. Towards Adversarial Robustness via Debiased High-Confidence Logit Alignment. ICCV, 2025, pp. 2783–2792.

Thank you for the corrections and additional experiments. These substantially address my concerns, and I will raise my score.

Thank you very much for taking the time to review our response and revised manuscript, and for your reassessment. Your constructive feedback has greatly strengthened the rigor and clarity of our work.

Reviewer_z1CUOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes R-CDFR to address the limited ability of conventional adversarial training to distinguish among input directions. It uses complementary damped filters to assign weights according to how strongly each direction affects the model’s internal representation, and adds three terms to a base adversarial-training objective: an inverse-metric spectral-slope regularizer, a damped residual-gradient penalty, and a finite-step pass-filtered consistency term. On CIFAR-10, R-CDFR achieves 61.63% accuracy under PGD-20 and 57.43% under AutoAttack; on CIFAR-100, the corresponding figures are 39.15% and 36.98%. All four are the highest among the methods compared in the main paper. The authors also measure pass- and residual-filtered loss sensitivities (SP and SR) and predictive KL divergence under a small pass-filtered perturbation (GP). R-CDFR has the lowest values for all three diagnostics in the comparison. Component ablations further show that removing any one of the three terms reduces accuracy under attack.

Critical strengths

(1)Under matched experimental settings, R-CDFR achieves the highest robust accuracy among the compared methods across multiple datasets and attacks. Its advantage therefore does not depend on a single dataset or attack. The paper provides extensive robustness evaluations, with strong results in the main comparisons. (2)The method is clearly motivated. The complementary damped filters assign weights continuously according to representation gain, without requiring a hard split between directions. The three regularizers target loss slope, low-gain residual gradients, and predictive stability under finite perturbations, respectively. The component ablations show that each term contributes to the reported robustness gains.

Critical weaknesses
  1. Inconsistent reporting across robustness evaluations. Tables 1 and 14 report different CIFAR-10 clean accuracies for PGD-AT (62.86% versus 71.92%) and AWP (66.99% versus 73.55%). Changing the evaluation norm alone should not change clean accuracy for the same checkpoint. If different checkpoints or training protocols were used, these differences need to be specified; otherwise, the reported entries require correction. This ambiguity weakens confidence in the cross-norm comparison.
  2. Insufficient specification of the representation defining the filters. The method depends directly on the representation Jacobian, but the implementation details do not clearly identify the WRN-34-10 layer used or specify whether pooling or feature normalization is applied before computing the Jacobian. These choices determine the spectrum and the resulting filter weights. Although fixed-reference controls and additional architectures are evaluated, sensitivity to representation-layer choice within the main backbone remains unclear, limiting reproducibility and practical guidance.
  3. Dependence on representation scale is insufficiently characterized. The filter weights depend on the relative scale of squared Jacobian singular values and the damping parameter. However, Table 4 uses the same damping value across datasets without reporting their relative scales. Rescaling the representation while compensating in the classifier head can preserve predictions but alter the filters and regularization penalties. The CIFAR-10 damping sweep provides useful sensitivity evidence, but does not resolve this parameterization dependence or establish comparable spectral allocation across datasets.
  4. Training and baseline configurations are not sufficiently detailed for reproduction. Appendix B specifies the training duration and R-CDFR-specific hyperparameters, but does not provide a complete configuration for the main matched experiments, including the exact optimizer settings, learning-rate schedule, batch size, training-attack parameters, and baseline-specific hyperparameter selection. Sharing an optimizer family and training budget does not by itself establish that each baseline is adequately configured. The stronger-recipe experiment is a useful additional control, but does not remove the need to document the main comparison precisely.
  5. The theoretical analysis provides a limited explanation of the reported finite-radius robustness gains. Proposition 5.1 bounds local loss growth over a representation-filtered perturbation family with the filters fixed at the reference input. It does not directly establish a bound for the standard finite-radius adversarial threat models used in the main evaluation. Moreover, the gradient-Lipschitz constant and remainder term are not quantified experimentally, and the finite-step consistency term is complementary rather than explicitly incorporated into the bound. The analysis therefore motivates the regularizers, but leaves the connection between optimizing the full objective and improving standard adversarial robustness largely empirical.
Questions for authors

(1)In Tables 1 and 14, the Clean accuracies for PGD-AT on CIFAR-10 are 62.86% and 71.92%, respectively; for AWP, they are 66.99% and 73.55%. Changing the attack norm alone should not affect Clean accuracy. For each of these methods, were the same trained models evaluated in both tables? If so, please check the reported values; if not, please clarify the differences in training or model selection. (2)Could you specify which layer of WRN-34-10 is used as the internal representation, and whether pooling or normalization is applied before computing its Jacobian? How sensitive are the results to this choice? (3)Table 4 uses λ = 10-3 for all three datasets, while Figure 6 studies its sensitivity on CIFAR-10. Could you report the typical scale of σi2 relative to λ on each dataset and explain how the representation scale is controlled? Rescaling the internal representation while compensating in the classifier head could leave predictions unchanged but alter the filter weights.

3 comments

Thank you for your constructive review; we address each concern below. All section, table, and figure numbers cited below refer to the revised manuscript PDF.

W1/Q1. Cross-norm clean accuracy

Tables 1, 17, and 18 test the same 5 models/method; fixing a legacy script mapping for PGD-AT/AWP Clean in Tables 1 and 18 (attack scores unchanged) unifies Clean (%):

MethodCleanAA (ℓ∞\ell_\infty)AA (ℓ2\ell_2)PR-20
PGD-AT71.92±0.7871.92\pm0.7847.34±0.8347.34\pm0.8362.86±0.8362.86\pm0.8345.12±0.7945.12\pm0.79
AWP73.55±0.7573.55\pm0.7555.31±0.8255.31\pm0.8266.99±0.8266.99\pm0.8251.74±0.7351.74\pm0.73
R-CDFR79.97±0.3679.97\pm0.3657.43±0.7157.43\pm0.7172.94±0.7172.94\pm0.7155.84±0.6955.84\pm0.69

W2/Q2. Representation layer and BN

WRN-34-10 uses hψ=GAP(ReLU(BN(z3)))∈R640h_\psi=\mathrm{GAP}(\mathrm{ReLU}(\mathrm{BN}(z_3)))\in\mathbb{R}^{640}, fθ=Whψ+bf_\theta=Wh_\psi+b (Sec. 4.2; App. B.3, Eq. (22)), no ℓ2\ell_2 norm, raw [0,1][0,1] derivatives, and frozen running BN stats in regularizers for per-input independence (keeping BN/model gradients). At λ=10−3\lambda=10^{-3} (Table 6, joint layer/scale check), C10/C100 AA is 55.96±0.6955.96\pm0.69/35.11±0.8735.11\pm0.87 pre-BN/ReLU (40,96040{,}960-d), 56.38±0.6556.38\pm0.65/35.82±0.8135.82\pm0.81 pre-GAP (40,96040{,}960-d), and 57.43±0.7157.43\pm0.71/36.98±0.7836.98\pm0.78 post-GAP (640640-d).

W3/Q3. Representation scale and damping

For h(c)=c h,cϕ(c)(z)=cϕ(z/c)h^{(c)}=c\thinspace h, c_\phi^{(c)}(z)=c_\phi(z/c) (c>0c>0), J↦cJJ\mapsto cJ gives ∑iai2/(c2σi2+λ)\sum_i a_i^2/(c^2\sigma_i^2+\lambda) and ∑iλ2ai2/(c2σi2+λ)2\sum_i \lambda^2 a_i^2/(c^2\sigma_i^2+\lambda)^2 (App. I.2, Eqs. (32)–(33)). On 128 val inputs at λ=10−3\lambda=10^{-3}, σi2/λ\sigma_i^2/\lambda quantiles (P10/Med/P90) and pass fraction deffd_{\mathrm{eff}} straddle 11 (Table 22): C10 (0.080/1.832/42.070.080/1.832/42.07; 0.5780.578), C100 (0.062/1.336/29.830.062/1.336/29.83; 0.5380.538), TinyIN (0.037/0.916/22.960.037/0.916/22.96; 0.4880.488). With s2=tr(J⊤J)/ms^2=\mathrm{tr}(J^\top J)/m (16 Rademacher probes), Gˉ=J⊤J/s2,Aˉ=Gˉ+λˉId\bar{G}=J^\top J/s^2, \bar{A}=\bar{G}+\bar{\lambda}I_d (Eq. (34), Table 23) is invariant to J↦cJJ\mapsto cJ, getting 57.18±0.6257.18\pm0.62%/36.61±0.7436.61\pm0.74% C10/C100 AA (vs. 57.4357.43%/36.9836.98%) plus +1.94/+3.29+1.94/+3.29 pp fixed-hh gains over TART (Table 20); anisotropic changes stay unnormalized.

W4. Training protocols and baseline tuning

Sec. 6.1 and App. B.4 (Table 7) give all specs: WRN-34-10, SGD (0.90.9 mom., 5×10−45\times10^{-4} wd, batch 128128, LR 0.10.1; 5050-ep 0.1×0.1\times drops at 7575%/9090% vs. 200200-ep cosine), crop/flip, 1010-step ℓ∞\ell_\infty (8/2558/255, step 2/2552/255), raw-pixel PGD-20/AA at ep 50/200. On a 45k/5k split (seed 2027, isolated RNGs), val PGD-40-CE (5 starts, seeds 0–2) picks βTRADES=6∈{3,6,9}\beta_{\mathrm{TRADES}}=6\in\lbrace 3,6,9\rbrace, γAWP=0.005∈{0.001,0.005,0.01}\gamma_{\mathrm{AWP}}=0.005\in\lbrace 0.001,0.005,0.01\rbrace (ep 11), and scale {0.5,1,2}\lbrace 0.5,1,2\rbrace (1×1\times TRADES; 0.5×0.5\times AWP, EMA/CutMix, DHAT: λ1=λ2=1\lambda_1=\lambda_2=1, inv. 4/2554/255, RN18 mask ω=0.5\omega=0.5). In 200200-ep paired runs (Sec. 6.6, Table 4; Table 13), R-CDFR lifts Tuned TRADES AA (55.63±0.41→55.63\pm0.41\to 57.96±0.4857.96\pm0.48%, +2.33+2.33 pp C10; 34.71±0.53→34.71\pm0.53\to 37.45±0.6137.45\pm0.61%, +2.74+2.74 pp C100), AWP (+1.57/+1.87+1.57/+1.87), EMA/CutMix (+1.29/+1.36+1.29/+1.36), and DHAT (+0.49/+0.61+0.49/+0.61).

W5. Ambient and finite-step bounds

Ambient bound (Sec. 5, Cor. 5.2; App. A.1). For M=∥J∥22+λM=\sqrt{\Vert{}J\Vert{}_2^2+\lambda} and HH-Lipschitz ∇ℓ\nabla\ell on B2(x,ϵ)B_2(x,\epsilon), ∥g∥2≤MSP+SR\Vert{}g\Vert{}_2\le MS_P+S_R gives Eqs. (15), (19) (ϵ=d ϵ∞\epsilon=\sqrt{d}\thinspace\epsilon_\infty for ℓ∞\ell_\infty; J↦cJJ\mapsto cJ raises MM in C=MSP+SRC=MS_P+S_R): sup⁡∥δ∥2≤ϵℓ(x+δ,y)≤ℓ(x,y)+ϵ(MSP+SR)+H2ϵ2,Rϵ≤R0+ϵMˉLspec+ϵLres+Hˉ2ϵ2.\sup_{\Vert{}\delta\Vert{}_2\le\epsilon}\ell(x+\delta,y)\le\ell(x,y)+\epsilon(MS_P+S_R)+\tfrac{H}{2}\epsilon^2,\quad \mathcal{R}_\epsilon\le\mathcal{R}_0+\epsilon\bar{M}\mathcal{L}_{\mathrm{spec}}+\epsilon\sqrt{\mathcal{L}_{\mathrm{res}}}+\tfrac{\bar{H}}{2}\epsilon^2. Finite-step bound (Sec. 5, Prop. 5.3; App. A.2). With margin μ=py(x)−max⁡j≠ypj(x)>0\mu=p_y(x)-\max_{j\ne y}p_j(x)>0 and κP(x)=Eu[KL(pθ(x)∥pθ(x+δP))]\kappa_P(x)=\mathbb{E}_u[\mathrm{KL}(p_\theta(x)\Vert{}p_\theta(x+\delta_P))], Pinsker's inequality gives smoothness-free Eq. (16) (Fisher form in Eq. (21)): Pr⁡u[y^(x+δP)≠y]≤min⁡{1,2κP(x)/μ2},Pr⁡D,u[y^(x+δP)≠y]≤Pr⁡D[μ≤μ0]+2Lcons/μ02.\Pr_u[\hat{y}(x+\delta_P)\ne y]\le\min\lbrace 1,2\kappa_P(x)/\mu^2\rbrace,\quad \Pr_{\mathcal{D},u}[\hat{y}(x+\delta_P)\ne y]\le\Pr_{\mathcal{D}}[\mu\le\mu_0]+2\mathcal{L}_{\mathrm{cons}}/\mu_0^2. Diagnostics (App. L, Tables 27–30). In our 5-seed, 40-d ReLU study (r=ℓ(x+δ,y)−ℓ(x,y)−g⊤δr=\ell(x+\delta,y)-\ell(x,y)-g^\top\delta, H^=2[r]+/∥δ∥22\hat{H}=2[r]_+/\Vert{}\delta\Vert{}_2^2), R-CDFR remainder shares at 1, 2, 4, 8/255 are 2.16%, 4.25%, 8.28%, 15.68%. At 8/255 (98% crossings), R-CDFR cuts g⊤δg^\top\delta (0.162→0.1490.162\to 0.149), [r]+[r]_+ (0.030→0.0240.030\to 0.024), Q0.9(H^)Q_{0.9}(\hat{H}) (2.22→1.682.22\to 1.68), and CC (3.46→3.033.46\to 3.03) vs. Base AT. At ϵP=0.25\epsilon_P=0.25 (Table 29), pass error is 3.98±0.433.98\pm0.43% vs. bound 18.10±1.5018.10\pm1.50% (0 violations), and Lres\mathcal{L}_{\mathrm{res}} cuts shared-basis Elow/Elow0E_{\mathrm{low}}/E_{\mathrm{low}}^0 (0.924→0.6610.924\to 0.661; 0.616 norm.) while keeping EhighE_{\mathrm{high}} (0.667 vs. 0.654).

Reviewer_z1CUOpenReview ↗

Assessment after the author response

Thank you for the detailed response. It provides useful clarifications and additional evidence that substantially alleviate several of my concerns. My assessment is as follows.

  1. Cross-norm reporting. The explanation of a legacy script mapping error directly addresses the discrepancy in clean accuracy. I appreciate the clarification that the same trained models were evaluated and that the attack accuracies are unchanged. Please ensure that the corrected clean means and standard deviations are consistent across all affected tables.
  2. Representation and implementation.The specification of the 640-dimensional post-GAP representation, the absence of feature normalization, differentiation with respect to raw inputs, and the treatment of BN running statistics addresses the main implementation ambiguities I raised. The reported comparisons across three representation locations also provide useful initial senitivity evidence. I therefore substantially reduce the weight of this concern.
  3. Representation scale. The spectrum statistics and the scale-normalized Gram control are meaningful additions. The reported robustness gains under the normalized control suggest that the benefit is not confined to the original representation scale. However, invariance of the normalized filtering operators should be distinguished from invariance of the complete regularized objective, particularly its inverse-metric term. Dependence on more general anisotropic reparameterizations remains a limitation.
  4. Training and baseline tuning.The response supplies considerably more detail about optimization, training attacks, validation, and baseline hyperparameter selection. The reported improvements over tuned baselines strengthen the empirical case.
  5. Theoretical scope.The proposed ambient and finite-step bounds, together with the remainder diagnostics, appear relevant to my original concern. The low-dimensional diagnostics are useful supporting evidence, but do not by themselves establish that the bounds are informative for the image models at the evaluated attack radii. I therefore retain a narrower concern about the quantitative connection between the theoretical bounds and the reported standard adversarial robustness. Overall, the response improves my assessment of the implementation clarity and empirical support. I remain positive about the complementary filtering approach and its reported robustness gains. I maintain my rating of 3 (Weak acceptance), while retaining reservations about reporting consistency and the quantitative scope of the theoretical explanation.

Thank you very much for your thorough reassessment, constructive feedback, and continued support of our work. We confirm that the corrected clean means and standard deviations are fully synchronized across all affected cross-norm tables (Tables 1, 17, and 18) in the revised manuscript. We also appreciate your insightful remarks distinguishing operator-level normalization from full objective invariance under general reparameterizations, as well as clarifying the quantitative scope of the theoretical bounds on high-dimensional image models, both of which we will highlight explicitly in the final revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.