acceptodds
Under review as a conference paper at ICLR 2027

Learning Dynamics Reveal Adversarial Distance and Membership Risk

Abstract

Deep neural networks memorize individual training samples to different degrees, with consequences for both adversarial robustness and privacy. Yet sample-level adversarial distance and membership vulnerability are expensive to measure at scale because they typically require iterative attacks, auxiliary models, or repeated retraining. We show that both quantities are encoded in the learning trajectory itself. We study Cumulative Sample Loss (CSL) and Cumulative Sample Gradient (CSG), which summarize how a sample's loss and input gradient evolve during training and can be collected with negligible overhead. Theoretically, we first establish a direct relation between CSL and CSG. We then derive two-sided bounds on adversarial distance from sample loss and input-gradient statistics and extend these results to trajectory-level quantities. Under stability, generalization, and model-bias assumptions, we yield a formal connection between memorization, adversarial vulnerability, and privacy leakage. These results motivate efficient sample-level vulnerability estimators from training dynamics. Our adversarial-distance proxy improves mislabeled-sample detection and achieves near-perfect duplicate detection on CIFAR-10 and CIFAR-100. For privacy auditing, we introduce CSL-Diff, which identifies samples vulnerable to LiRA, AttackR, and RMIA without auxiliary models at deployment and consistently outperforms prior artifact-based predictors. Together, our results turn learning dynamics into a reusable signal for auditing robustness, privacy, and dataset quality at the cost of standard training.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Robustness, Adversarial Distance, Learning Dynamics, Cumulative Sample Loss, Cumulative Sample Gradients, Membership Inference Attacks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gKiROpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper links signals obtained from the learning dynamics to privacy (specifically membership inference) and robustness concerns. As these signals also relate to memorizations, the paper shows how memorizations, privacy and robustness are all related to each other. The paper provides experiments to support its theory.

Critical strengths
  1. The paper finds a new, cheaper method to evaluate how much a sample is vulnerable to privacy and robustness attacks. The method relies on signals obtained from the learning dynamic of the learning algorithm, so it is cheap to obtain them during training.

  2. As the same signals are used to both privacy and robustness, the paper links the two topics using those signals and the notation of memorization.

  3. The paper provides various experiments to support its theory and showing how to use the signals to define a metric that measures how much a sample is vulnerable to privacy attacks.

  4. I really like that every theorem has an interpretation.

Critical weaknesses
  1. Regarding background and assumptions - they seem really strong. Are there any models that satisfy the assumptions needed for the theorems? especially the L-bounded loss (equation 5).

  2. line 234 - δmin\delta_{min} and δmax\delta_{max} are unclear to me. In the end, only the last checkpoint of the model is published, so why would one care about all the δ\delta during the training phase? It seems to me that only δT\delta_T is relevant.

  3. equation 13, why this is the interesting direction of the inequalities? It seems to me that it's more interesting to prove an upper bound to δmax\delta_{max} and a lower bound to δmin\delta_{min}.

  4. I don't agree with Theorem's 2 interpretation. δmin\delta_{min} is the weakest point during training phase, but the deployed model will have δT≥δmin\delta_T \geq \delta_{min}, so the upper bound may not apply to δT\delta_T. Similary, the lower bound may not apply to δT\delta_T since δT≤δmax\delta_T \leq \delta_{max}.

  5. Is there a way for the attacker to estimate CSL and CSG? if not, the bounds are not exposed to the attacker and he has no additional information to exploit.

Overall, I liked the paper and I will gladly raise my score based on the rebuttal.

Questions for authors
  1. Equation 7 - there is no ziz_i in the equation, but rather xix_i. I guess it's a typo.

  2. Section 5 - is the perturbation δ\delta defined per sample, or is it global? If it's related to a specific sample ziz_i, consider adding a symbol so it would be more understandable, for example δzi\delta_{z_i}.

  3. Section 5 - I don't fully understand the terms "necessary threshold" and "sufficient threshold". Consider adding a short explanation to the terms. I see that they are defined in the appendix, so maybe consider adding a reference.

I want to stress that rejecting my suggestions here will not affect my final score, they are just friendly suggestions that you may or may not agree with.

4 comments

We thank the reviewer for their time and efforts in reviewing our work. We answer the questions below.

1. The background and assumptions seem really strong; are there any models that satisfy the assumptions needed for the theorems, especially the L-bounded loss (equation 5)?

A: Theorems 1 and 2 do not require LL-bounded loss, stability, or generalization. They require only smoothness and a non-negative loss l≥0l\ge0, which unbounded Cross-Entropy satisfies. Corollary 1(i) additionally requires error stability β\beta, through the memorization bound of Ravikumar et al. (2025), but not LL-bounded loss. LL-bounded loss is required only in Theorem 3 and Corollary 1(ii), to normalize the loss following Theorem 2 of Yeom et al. (2018).

Networks with smooth activations (e.g. softplus, GELU, SiLU) combined with softmax Cross-Entropy satisfy the smoothness assumptions for finite weights. Cross-Entropy is bounded whenever predicted probabilities stay bounded away from zero, which holds for finite weights on the bounded input domain [0,1]d[0,1]^d. In practice, networks trained with weight decay plateau at a finite maximum loss. Nearly all the assumptions, including the LL-bounded loss, were adapted from the work on CSL, CSG, privacy and memorization by Ravikumar et al. (2024; 2025; 2026), where they are justified in more detail. ReLU networks do not satisfy global smoothness, which is why our experiments use the theory to derive empirical proxies.

2. In line 234, δmin⁡\delta_{\min} and δmax⁡\delta_{\max} are unclear because only the last checkpoint of the model is published, so why would one care about all the δ\delta during the training phase when only δT\delta_T is relevant?

A: We agree that δT\delta_T is what matters for the deployed model, and our single-checkpoint bounds (Eq. 12 at t=Tmax⁡t=T_{\max}) address it directly. These are also the only bounds an attacker can use, since they only have access to the final checkpoint.

The trajectory bounds are meant for the defender. The model developer has the full training trajectory, and hence CSL and CSG, at no extra cost. A small upper bound on δmin⁡\delta_{\min} flags a sample that came close to the decision boundary during training, typically a hard or memorized sample, which the developer can audit and mitigate before release. The lower bound on δmax⁡\delta_{\max} certifies the best margin the sample reached. Moreover, when the released checkpoint is chosen by early stopping, δmin⁡\delta_{\min} and δmax⁡\delta_{\max} bracket the distance at whichever checkpoint is selected. We will clarify this in the revision.

3. For equation 13, why is this the interesting direction of the inequalities when it seems more interesting to prove an upper bound to δmax⁡\delta_{\max} and a lower bound to δmin⁡\delta_{\min}?

A: We agree that the reviewer's directions would be more informative, since they would bound every δt\delta_t, including δTmax⁡\delta_{T_{\max}}. The current direction is forced by the fact that CSL and CSG are averages over the trajectory. To average the per-iteration upper bound, δt\delta_t must be replaced by a quantity independent of tt, and only δt≥δmin⁡\delta_t\ge\delta_{\min} keeps the inequality valid. An upper bound on δmax⁡\delta_{\max} would need control of every individual checkpoint, which an average cannot provide: one checkpoint with a vanishing gradient can make δt\delta_t arbitrarily large while barely changing CSG. The same argument limits the lower bound to δmax⁡\delta_{\max}. In addition, nearly every sample is misclassified at initialization, so δmin⁡≈0\delta_{\min}\approx0 and a lower bound on it would be vacuous over the full trajectory.

The current bounds are still meaningful for the defender, as described above. We will add this as a remark and as a limitation of our work.

4. I don't agree with Theorem 2's interpretation because while δmin⁡\delta_{\min} is the weakest point during training, the deployed model will have δT≥δmin⁡\delta_T\ge\delta_{\min} so the upper bound may not apply to δT\delta_T, and similarly, the lower bound may not apply to δT\delta_T since δT≤δmax⁡\delta_T\le\delta_{\max}.

A: The reviewer is correct that the trajectory-based bounds for δmin⁡\delta_{\min} and δmax⁡\delta_{\max} do not apply to δT\delta_T. The bounds for δT\delta_T are our Single-Checkpoint bounds in Theorem 2 (Eq. 12), which serve both the defender and the attacker. Since the attacker only has access to the final checkpoint, these are the only bounds they can compute: the lower bound provides safety guarantees for the defender, and the upper bound provides attack-budget guarantees for the attacker. The trajectory-based bounds can only be computed by the defender. We will update the interpretation of Theorem 2 to clarify this.

5. Is there a way for the attacker to estimate CSL and CSG? If not, the bounds are not exposed to the attacker and he has no additional information to exploit.

A: No. An external attacker cannot compute CSL or CSG, since both require the training trajectory. With white-box access to the final checkpoint, the attacker can only compute the loss and input gradient that enter the single-checkpoint bounds. This works in the defender's favour: our framework is designed as a zero-cost auditing tool for the model developer, not as an attack tool. The developer obtains CSL and CSG for free during training, can identify records vulnerable to external C&W or shadow-model MIA attacks, and can mitigate them before publishing the final checkpoint. Note that the attacker's inability to compute the bounds does not remove the vulnerability itself, since attacks such as C&W and LiRA exploit it directly, but with a significantly higher computation budget.

6. In Equation 7, there is no ziz_i in the equation, but rather xix_i, which appears to be a typo.

A: Thank you for catching this. Since z⃗i=(x⃗i,yi)\vec z_i=(\vec x_i,y_i) denotes the input-label tuple, we will explicitly write z⃗i=(x⃗i,yi)\vec z_i=(\vec x_i,y_i) in Equation 7 to avoid confusion.

7. In Section 5, is the perturbation δ\delta defined per sample, or is it global? If it's related to a specific sample ziz_i, consider adding a symbol so it would be more understandable, for example δzi\delta_{z_i}.

A: Yes, δt\delta_t, δmin⁡\delta_{\min}, and δmax⁡\delta_{\max} in Section 5 are defined per sample z⃗i\vec z_i. We will update the notation to δt(z⃗i)\delta_t(\vec z_i), δmin⁡(z⃗i)\delta_{\min}(\vec z_i), and δmax⁡(z⃗i)\delta_{\max}(\vec z_i).

8. In Section 5, I don't fully understand the terms "necessary threshold" and "sufficient threshold", so consider adding a short explanation or a reference to the appendix.

A: These are derived in Lemma 1 (App. B.2.1), and we will add the following short explanation. In KK-class classification with Cross-Entropy loss, loss values fall into three regimes. Below the necessary threshold (l<Lcl<L_c), the sample is always correctly classified because the true class holds over half the probability, so crossing LcL_c is necessary for misclassification. In between (Lc≤l<LuL_c\le l<L_u), misclassification is not guaranteed and depends on how the remaining probability is split among the other K−1K-1 classes. Above the sufficient threshold (l≥Lul\ge L_u), the true-class probability drops below 1/K1/K, so by the pigeonhole principle another class wins and crossing LuL_u is sufficient for misclassification.

I thank the authors for their answer. All my concerns are resolved. I have raised my score to 4

We sincerely thank the reviewer for their time and efforts to read and respond to our Rebuttal.

Reviewer_aqpUOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether sample-level learning dynamics can estimate untargeted adversarial robustness and membership inference risk. Building on CSL and CSG, the authors derive an inequality connecting the two under smoothness assumptions. They then construct upper and lower bounds on untargeted L_2 adversarial distance using checkpoint-level loss and input gradients, extending these estimates across the training run. Under stability, generalization, and model-bias assumptions, the paper attempts to bound optimal membership inference attack advantage using CSL.

Empirical evaluations in the paper mainly focus on ResNet and WideResNet architectures on CIFAR-10 and CIFAR-100. The authors report that an adversarial-distance proxy effectively detects mislabeled inputs and identifies duplicates with conflicting labels, and they also propose CSL-Diff, a metric tracking loss changes between early and late training phases to rank records vulnerable to LiRA, AttackR, and RMIA. The result reports high detection AUROC and stronger Precision@k than several loss-trajectory baselines.

It's worth noticing that CSL and CSG themselves seem to come from prior work.

Critical strengths

The paper does have some strengths:

  1. The workflow repurposes loss and gradient histories already generated during standard training. If substantiated, using these signals to audit datasets and models without running iterative attacks or training shadow ensembles would save substantial computing cost.

  2. The empirical detection and ranking experiments show competitive results across multiple noise rates, CIFAR benchmarks, and membership attack setups. The ability of CSL-Diff to surface high-risk records remains an interesting empirical result, even if its theoretical justification is flawed.

Critical weaknesses

1. Theorem 3 fails for fixed target samples under the stated generalization assumptions.

The generalization condition in Eq. 2 guarantees low expected error across random draws of the full dataset. In Appendix B.3, around Eqs. 65–67, this condition is applied to training sets conditioned on containing one specific record ziz_i, without proving that the guarantee holds uniformly or conditionally.

This gap exposes the theorem to an immediate counterexample. Just consider a binary classification task without informative features where labels follow Y∼Bernoulli(p)Y \sim \text{Bernoulli}(p) with p=10−6p = 10^{-6}, evaluated under bounded squared error. Let the learning algorithm always predict the constant Bayes-optimal rate pp, ignoring the training set entirely. Here, the stability parameter is β=0\beta = 0, model bias is Δ=0\Delta = 0, and optimal MIA advantage is strictly 00.

With m=100m = 100, γ=10−6\gamma = 10^{-6}, and ξ=0.01\xi = 0.01, a random training set contains zero positive instances with probability (1−p)100≈0.9999(1 - p)^{100} \approx 0.9999. On that dominant subset, the gap between empirical and population risk falls well within γ\gamma, satisfying Eq. 2. However, if the target record ziz_i is fixed to the rare positive class, its loss is (1−p)2≈1(1 - p)^2 \approx 1. Evaluating Eq. 95 from the appendix produces κ≈5.96×10−6\kappa \approx 5.96 \times 10^{-6}, while the left-hand side of Eq. 14 yields roughly 9.8×10−39.8 \times 10^{-3}. Because the true MIA advantage is 0, the inequality fails.

As written, the theorem cannot guarantee privacy bounds for an arbitrary, fixed record. If the authors intended an expectation over ziz_i, or if the claim requires uniform convergence over all conditional subsets, the paper needs a revised formal statement and an explanation of whether any sample-specific privacy guarantees actually survive. Furthermore, the proof assumes bounded loss, whereas the experiments rely on unbounded cross-entropy. Sadly, the paper also provides no theoretical derivation for the CSL-Diff heuristic actually used in the benchmarks.

2. Theorem 2 ignores edge cases and unverified conditions.

In Eq. 12, Δt(u)=Lu−ℓt\Delta_t^{(u)} = L_u - \ell_t. Whenever ℓt>Lu\ell_t > L_u, the difference Δt(u)\Delta_t^{(u)} turns negative. Under those values, the discriminant condition Ct2≥2ρxΔt(u)C_t^2 \geq 2\rho_x\Delta_t^{(u)} is satisfied trivially, but the resulting upper-bound formula can return a negative value. Distance metrics cannot be negative. If a sample already satisfies ℓt>Lu\ell_t > L_u, the authors' own definitions indicate it is already misclassified, requiring separate handling rather than direct substitution into a positive-root formula. The text also does not handle cases where Δt(c)≤0\Delta_t^{(c)} \leq 0 or Ct=0C_t = 0.

For a non-negative function with ρx\rho_x-Lipschitz gradients, basic optimization theory dictates that Ct2≤2ρxℓtC_t^2 \leq 2\rho_x\ell_t. Pairing this property with the paper's upper-bound condition forces ℓt≥Lu/2\ell_t \geq L_u / 2. Consequently, the bound breaks down for common, low-loss training samples learned with high confidence. On top of that, standard networks with ReLU activations violate the global gradient and Hessian smoothness properties assumed throughout Section 3. I think the paper should explicitly separate formal bounds from heuristic estimators and document the actual percentage of training points that meet these criteria. High rank correlation with an empirical attack does not make an inequality valid.

3. Corollary 1 contains flawed derivations and misinterprets its own bounds.

In Corollary 1(i), Eq. (104) in the appendix invokes Theorem 5.5 from Ravikumar et al. (from ICML 2025). That result bounds memorization averaged over the data distribution. The authors convert it into a pointwise statement for a single record ziz_i over optimization noise alone, without offering a proof for the stronger claim or addressing the missing normalization factor LL.

In Corollary 1(ii), near Eq. 116 in Appendix B.4.3, the argument moves from:

Δmia≤Cmiaδsup+Bδsup2+Aδsup3\Delta_{\text{mia}} \leq C_{\text{mia}}\delta_{\text{sup}} + B\delta_{\text{sup}}^2 + A\delta_{\text{sup}}^3

directly to:

δsup≤ΔmiaCmia\delta_{\text{sup}} \leq \frac{\Delta_{\text{mia}}}{C_{\text{mia}}}

This algebraic step is invalid because the higher-order terms Bδsup2B\delta_{\text{sup}}^2 and Aδsup3A\delta_{\text{sup}}^3 are non-negative. Dropping them reverses the inequality direction. The degree-reduction argument as written is mathematically broken.

Sadly, the paper's interpretation of this result seems to be also incorrect. The authors claim that as MIA advantage rises, the lower bound on adversarial distance shrinks, thereby forcing the sample's actual adversarial distance to drop. A decreasing lower bound relaxes a constraint, and does not force the underlying parameter downward.

4. Discrepancies between theory and code reproducibility.

I happened to discover that, in validation_experiments/train_eval_full.py, the training dataset is formed once using [(transform_train(img), tgt) for img, tgt in raw_train]. As a result, random crops and flips are applied a single time during initialization rather than refreshing every epoch. CSL and CSG are tracked against these statically transformed images, while Carlini-Wagner (C&W) distances are measured against separate, unaugmented inputs. I also didn't see the channel normalization steps described in Appendix C.1.

During backpropagation, the implementation scales batch gradients via B * x.grad to estimate individual input gradients. In architectures using batch normalization, operations across the batch are coupled. Because:

B∇xi(1B∑jℓj)=∑j∇xiℓj≠∇xiℓiB\nabla_{x_i}\left(\frac{1}{B}\sum_j \ell_j\right) = \sum_j \nabla_{x_i}\ell_j \neq \nabla_{x_i}\ell_i

the calculated CSG values need not equal the independent per-sample gradient norms specified in Theorem 1.

Furthermore, while Eq.10 defines trajectory statistics using a non-uniform stopping distribution TT, the code simply sums raw losses and gradients across sequential epochs without sampling weights. The C&W attack script uses torch.atanh(2 * x_clamped - 1 + 1e-6), which moves outside the valid real domain of inverse hyperbolic tangent whenever an input pixel equals 1. If an attack fails to find a successful adversarial example, the returned distance may retain its initialization value (max_eps=6.0). Because the implementation does not return a separate attack-success flag, failed or censored searches can be mixed with valid distance estimates, potentially biasing the reported adversarial-distance statistics.

Questions for authors
  1. Theorem 3 and conditional assumptions: How do you justify applying a high-probability generalization bound over random training sets to a dataset conditioned on a fixed sample ziz_i? How does the current theoretical setup resolve the constant-predictor counterexample detailed above? If the theorem requires taking an expectation over ziz_i or assuming a uniform or conditional generalization guarantee, please provide the corrected statement and clarify what individual-sample privacy guarantees remain.

  2. Theorem 2 applicability: How does the formulation handle boundary conditions where Δt(u)≤0\Delta_t^{(u)} \leq 0, Δt(c)≤0\Delta_t^{(c)} \leq 0, or Ct=0C_t = 0? Can you report the percentage of CIFAR training samples that satisfy the prerequisite Ct2≥2ρxΔt(u)C_t^2 \geq 2\rho_x\Delta_t^{(u)} under clearly specified estimates or bounds for the smoothness constant ρx\rho_x? Please also clarify whether any of these statements hold as certified bounds for the ReLU networks used in the experiments under the stated smoothness assumptions, or if they should be framed strictly as empirical proxies.

  3. Corollary 1 proofs: What allows the expectation-over-samples memorization bound from ICML 2025 to apply to an individual sample in Corollary 1(i), and how are the assumptions and loss normalization of the cited result addressed? How do you resolve the invalid inequality step in Appendix B.4.3 for Corollary 1(ii)? Lastly, why should a shrinking theoretical lower bound imply a drop in the actual adversarial distance?

  4. Code and experimental execution: Were the results in Section 6.1 generated using the submitted training script, including static one-time data augmentation and missing channel normalization? How do you account for gradient coupling across BatchNorm layers when approximating individual sample gradients with B * x.grad? How does the epoch-wise accumulation of CSL/CSG correspond to the non-uniform stopping distribution TT in Eq.10? Also take care of the atanh domain error in the C&W implementation and the treatment of failed attacks retaining the default max_eps = 6.0, including the attack success rates.

7 comments

We thank the reviewer for their time and efforts in reviewing our work. We answer the questions below.

Part 1: Theorem 3 and conditional assumptions

1. How do you justify applying a high-probability generalization bound over random training sets to a dataset conditioned on a fixed sample ziz_i? How does the current theoretical setup resolve the constant-predictor counterexample? If the theorem requires an expectation over ziz_i or a uniform or conditional generalization guarantee, please provide the corrected statement and clarify what individual-sample privacy guarantees remain.

A: We sincerely thank the reviewer for identifying this measure-theoretic gap regarding Eq. 2. The reviewer is correct: applying an unconditional high-probability generalization bound to a dataset SiS_i conditioned on a fixed, worst-case outlier ziz_i breaks the inequality, as the constant-predictor counterexample demonstrates.

Corrected Statement & Resolution: We take the expectation over the target sample zi∼Dz_i\sim\mathcal{D}, so that it is now an unconditioned draw and Eq. 2 applies. The revised Theorem 3 bounds the expected privacy risk:

m−2m2Ezi∼D,S,ϕ[CSL(z⃗i)Tmax⁡]≤LmEzi∼D,t∼T[AdvMIA∗(w⃗t,z⃗i)]+κ.\frac{m-2}{m^2}\mathbb{E}_{z_i\sim\mathcal{D},S,\phi}\left[\frac{CSL(\vec z_i)}{T_{\max}}\right]\le\frac{L}{m}\mathbb{E}_{z_i\sim\mathcal{D},t\sim\mathbf{T}}\big[Adv^{\ast}_{MIA}(\vec w_t,\vec z_i)\big]+\kappa .

This absorbs the counterexample: the large loss of the 10−610^{-6} outlier is suppressed by its tiny probability weight. The left-hand side becomes ≈9.8×10−9\approx9.8\times10^{-9}, below κ≈5.96×10−6\kappa\approx5.96\times10^{-6}, so the inequality holds even with zero advantage.

Remaining Privacy Guarantees: Deterministic individual-sample guarantees do not survive this correction, and the bound cannot be restricted to a sub-population of D\mathcal{D} without reintroducing the same conditioning issue. The observation that high-CSL records are individually the most vulnerable is supported empirically (Fig. 1, Table 1).

2. The proof assumes bounded loss, whereas the experiments rely on unbounded cross-entropy.

A: We acknowledge the gap between the LL-bounded assumption and unbounded Cross-Entropy. It affects only Theorem 3 and Corollary 1(ii), where it needs the LL-bounded assumption. Theorems 1, 2 and Corollary 1(i) need only l≥0l\ge0, which is satisfied by CE loss. We adopt the standard justification from the literature we build on (Ravikumar et al., 2024, 2025, 2026): CE is bounded whenever predicted probabilities stay bounded away from zero, which holds for finite weights on the bounded input domain [0,1]d[0,1]^d. In practice, networks trained with weight decay plateau at a finite maximum loss. We will explicitly separate our formal bounds from the applied CE proxies in the revised text.

3. The paper provides no theoretical derivation for the CSL-Diff heuristic actually used in the benchmarks.

A: We clarify that CSL-Diff is not directly derived from Theorem 3. It is an empirical heuristic inspired by our trajectory framework, designed to subtract early-stage generalized fitting and isolate late-stage memorization. We will explicitly state its status as a heuristic extension in the revised text.

Part 2: Theorem 2 applicability

4. How does the formulation handle boundary conditions where Δt(u)≤0\Delta^{(u)}_t\le0, Δt(c)≤0\Delta^{(c)}_t\le0, or Ct=0C_t=0?

A: We thank the reviewer for identifying these edge cases. Our formulation implicitly assumed Δt(c)\Delta^{(c)}_t, Δt(u)\Delta^{(u)}_t and CtC_t to be strictly positive, and we apologize for not stating this. The boundary cases resolve as follows. If lt≥Lul_t\ge L_u (Δt(u)≤0\Delta^{(u)}_t\le0), the sufficient threshold is crossed, the sample is already misclassified, and δt=0\delta_t=0 by definition (Appendix Eq. 17). If lt≥Lcl_t\ge L_c (Δt(c)≤0\Delta^{(c)}_t\le0) and the sample is still correctly classified, the lower bound trivially becomes 0. If Ct=0C_t=0, the term Δt(c)/Ct\Delta^{(c)}_t/C_t in B~t\tilde B_t is undefined. Assumption 6 alone then gives δt≥2Δt(c)/ρx\delta_t\ge\sqrt{2\Delta^{(c)}_t/\rho_x} for lt<Lcl_t<L_c, and since the upper-bound prerequisite fails, no upper bound is claimed.

Corrected Statement of Theorem 2 (Single-Checkpoint Bounds):

Assume l≥0l\ge0 and smoothness assumptions 4 and 6. For a sample z⃗i\vec z_i at iteration tt: if lt≥Lul_t\ge L_u, then δt=0\delta_t=0. Otherwise, let Ct=∥∇x⃗il(w⃗S,tϕ,z⃗i)∥2C_t=\|\nabla_{\vec x_i}l(\vec w^{\phi}_{S,t},\vec z_i)\|_2, Δt(c)=Lc−lt\Delta^{(c)}_t=L_c-l_t and Δt(u)=Lu−lt\Delta^{(u)}_t=L_u-l_t. If lt≥Lcl_t\ge L_c, then δt≥0\delta_t\ge0. If lt<Lcl_t<L_c and Ct>0C_t>0, let B~t=ρx2+ρ6Δt(c)Ct\tilde B_t=\frac{\rho_x}{2}+\frac{\rho}{6}\frac{\Delta^{(c)}_t}{C_t}; then

δt≥2Δt(c)Ct2+4B~tΔt(c)+Ct.\delta_t\ge\frac{2\Delta^{(c)}_t}{\sqrt{C_t^2+4\tilde B_t\Delta^{(c)}_t}+C_t}.

If lt<Lcl_t<L_c and Ct=0C_t=0, then δt≥2Δt(c)/ρx\delta_t\ge\sqrt{2\Delta^{(c)}_t/\rho_x}. If Ct2≥2ρxΔt(u)C_t^2\ge2\rho_x\Delta^{(u)}_t (which requires Ct>0C_t>0 and lt≥Lu/2l_t\ge L_u/2), then

δt≤2Δt(u)Ct+Ct2−2ρxΔt(u).\delta_t\le\frac{2\Delta^{(u)}_t}{C_t+\sqrt{C_t^2-2\rho_x\Delta^{(u)}_t}}.

For the trajectory bound (Eq. 13), the linear inequality is applied only at iterations where this prerequisite holds; if any checkpoint has lt≥Lul_t\ge L_u, then δmin⁡=0\delta_{\min}=0 trivially.

5. Can you report the percentage of CIFAR training samples that satisfy the prerequisite Ct2≥2ρxΔt(u)C_t^2\ge2\rho_x\Delta^{(u)}_t under clearly specified estimates or bounds for ρx\rho_x? Please also clarify whether any of these statements hold as certified bounds for the ReLU networks used in the experiments, or if they should be framed strictly as empirical proxies.

A: As noted at the beginning of Section 6, the proxies dˉadv,T\bar d_{adv,T} and dˉadv,1\bar d_{adv,1} are framed as empirical proxies, not certified bounds for the deployed networks. We agree this distinction needs more prominence and will expand on it in Section 6. As the reviewer points out, ReLU networks violate global smoothness because of their piecewise linear nature; the bounds are certified only for smooth activations such as softplus or GELU. Assuming smoothness to derive functional relationships is a standard theoretical relaxation (e.g. Ravikumar et al., 2024; 2025; 2026). Following this methodology, our theory provides the algebraic form of the proxies, which we validate empirically. Because computing ρx\rho_x for deep networks is intractable at scale, we omit it to preserve the zero-overhead nature of the proxies, since we do not claim them as certified bounds.

The reviewer's derivation that the prerequisite forces lt≥Lu/2l_t\ge L_u/2 is consistent with our experimental design. This condition fails for well-learned samples, which is why the single-checkpoint proxy (T=1T=1) is used only for the experiments on mislabeled and duplicate samples (Secs. 6.2, 6.3), which resist learning and retain comparatively high loss. For normal samples, we use the trajectory-level proxy instead.

Part 3: Corollary 1 proofs

6. What allows the expectation-over-samples memorization bound from ICML 2025 to apply to an individual sample in Corollary 1(i), and how are the assumptions and loss normalization of the cited result addressed?

A: We sincerely thank the reviewer for catching this. The expectation-based memorization bound cannot be applied deterministically to a single, fixed sample ziz_i. To resolve this, we apply Ezi∼D\mathbb{E}_{z_i\sim\mathcal{D}} to both sides of the inequality before isolation (Eq. 109, Appendix B.4.2). This brings Ezi,ϕ[CSL(z⃗i)]\mathbb{E}_{z_i,\phi}[CSL(\vec z_i)] into the right-hand side, matching the form of the ICML 2025 bound and allowing a valid substitution.

Regarding loss normalization, the general Theorem 5.5 in Ravikumar et al. carries a 1/L1/L factor. Because our framework uses Cross-Entropy, we rely on their CE-specific bound (Eq. 14 of Ravikumar et al. 2025), which eliminates the 1/L1/L factor: Ezi,ϕ[mem(z⃗i)]≤Ezi,ϕ[CSL(z⃗i)]+β−ξ\mathbb{E}_{z_i,\phi}[mem(\vec z_i)]\le\mathbb{E}_{z_i,\phi}[CSL(\vec z_i)]+\beta-\xi. Its error-stability assumption β\beta is added to the corollary.

Corrected Statement of Corollary 1(i):

For non-negative cross-entropy losses under the memorization framework of Ravikumar et al. (2025), provided Ct2≥2ρxΔt(u)C_t^2\ge2\rho_x\Delta^{(u)}_t for all tt and ϕ\phi, the expected product of the absolute minimum adversarial distance δinf⁡(z⃗i)=inf⁡ϕmin⁡tδt\delta_{\inf}(\vec z_i)=\inf_\phi\min_t\delta_t and the CSG is upper-bounded by expected memorization:

Ezi∼D[δinf⁡(z⃗i)Eϕ[CSG(z⃗i)]]≤2(LuTmax⁡−Ezi,ϕ[mem(z⃗i)]+β−ξ)\mathbb{E}_{z_i\sim\mathcal{D}}\Big[\delta_{\inf}(\vec z_i)\mathbb{E}_\phi[CSG(\vec z_i)]\Big]\le2\Big(L_uT_{\max}-\mathbb{E}_{z_i,\phi}[mem(\vec z_i)]+\beta-\xi\Big)

7. How do you resolve the invalid inequality step in Appendix B.4.3 for Corollary 1(ii)?

A: We thank the reviewer for their careful reading. The reviewer is correct that dropping non-negative higher-order terms from Δ≤Cδsup⁡+Bδsup⁡2+Aδsup⁡3\Delta\le C\delta_{\sup}+B\delta_{\sup}^2+A\delta_{\sup}^3 would reverse the inequality. The confusion stems from a notational shortcut in Appendix B.4.3(after Eq 116): the intermediate steps were written with δsup⁡\delta_{\sup} instead of the exact root δ∗\delta^{\ast}. The step uses the same root-bounding technique as Theorem 2 (Appendix B.2).

Since A,B≥0A,B\ge0 and C>0C>0, the cubic is strictly increasing on [0,∞)[0,\infty). If δ∗\delta^{\ast} is the exact root of Cδ+Bδ2+Aδ3=ΔC\delta+B\delta^2+A\delta^3=\Delta, monotonicity guarantees δsup⁡≥δ∗\delta_{\sup}\ge\delta^{\ast}. For the exact root, all terms are non-negative, so Cδ∗≤ΔC\delta^{\ast}\le\Delta, giving δ∗≤Δ/C\delta^{\ast}\le\Delta/C. Substituting this into one factor of the cubic term, exactly as in Theorem 2 (Eqs. 44--45), gives

Δ≤Cδ∗+(B+AΔC)(δ∗)2=Cδ∗+B~mia(δ∗)2.\Delta\le C\delta^{\ast}+\Big(B+A\frac{\Delta}{C}\Big)(\delta^{\ast})^2=C\delta^{\ast}+\tilde B_{mia}(\delta^{\ast})^2 .

Solving for the positive root yields

δsup⁡≥δ∗≥2ΔmiaCmia2+4B~miaΔmia+Cmia,\delta_{\sup}\ge\delta^{\ast}\ge\frac{2\Delta_{mia}}{\sqrt{C_{mia}^2+4\tilde B_{mia}\Delta_{mia}}+C_{mia}},

which is exactly Eq. 16, with the inequality direction preserved. We will correct the notation in Appendix B.4.3 to use δ∗\delta^{\ast} and cross-reference Appendix B.2.

8. Why should a shrinking theoretical lower bound imply a drop in the actual adversarial distance?

A: The reviewer is correct: a shrinking lower bound relaxes a constraint rather than forcing the actual distance downward. Our original phrasing ("fundamentally limits the possible adversarial distance") was imprecise. We will revise the Corollary 1 takeaways to state that a shrinking lower bound reduces the guarantees of the defender, as originally stated in the takeaways.

Reviewer_TQavOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies cumulative sample loss (CSL) and cumulative sample gradient (CSG), and links them to a sample's adversarial distance and to its membership-inference risk. The CSG is bounded by CSL using Lipschitz gradient and Hessian. The adversarial distance is also bounded from the cross-entropy thresholds. The experiments using ResNet and CIFAR also validate the claims.

Critical strengths

The experiments/evaluations in the paper is broad. It checks CSL-Diff against multiple methods, and covers ranges of FPR and shadow budgets. It's a meaningful empirical contribution.

Critical weaknesses
  • It seems there are some divergence between the theorems and the paper's main claim. The theorem 1 is a direct adaptation from Nesterov & Polyak 2006, but its replation to the experiment is not shown. With ρx\rho_x-smooth non negative loss, ∥∇l∥2≤2ρxl\Vert\nabla l\Vert^2\leq 2\rho_x l, what does ρ\rho do here? Besides, this bound is an upper-bound, how does it explain the near-linear relation?
Questions for authors
  • How does the bound on the minimum or maximum of δt\delta_t over the training trajectory imply anything about the distance used in Table 1?
  • How was the nn for CSL-diff chosen? Was it fixed?
2 comments

We thank the reviewer for their time and efforts in reviewing our work. We answer the questions below.

1. There is some divergence between the theorems and the paper's main claim.

A: We agree that the submission did not clearly separate the formal statements from the empirical proxies built on them. The theorems give conditional relations under the stated assumptions, while dˉadv,T\bar d_{adv,T}, dˉadv,1\bar d_{adv,1} and CSL-Diff are empirical estimators whose algebraic form is inspired from these relations. In the revision, each theorem will state exactly what it bounds, the proxies will be labelled as estimators at the start of Section 6 along with the existing discussion, "validating" will become "consistent with", and the Table 1 header "(Theoretical)" will become "(Proxy)".

2. Theorem 1 is a direct adaptation from Nesterov & Polyak 2006, but its relation to the experiment is not shown.

A: Theorem 1 predicts that a sample can have a large CSG only if it also has a large CSL: for any fixed ϵ\epsilon, every sample lies below a line in the CSL--CSG plane. This is the structure seen in the CSL vs. CSG panel of Figure 1, where low-CSL samples also have low CSG and the high-CSG samples are the high-CSL ones, and it is what the CSL--CSG correlation in Table 1 quantifies. We will state this link explicitly in Section 6.1.

3. With ρx\rho_x-smooth non-negative loss, ∣∣∇l∣∣2≤2ρxl||\nabla l||^2 \le 2\rho_x l, what does ρ\rho do here?

A: We thank the reviewer. Theorem 1 expresses a linear relation between CSG and CSL, which the reviewer's inequality does not directly provide. Since we define CSG as the expected gradient norm E[∣∣∇l∣∣]E[||\nabla l||] and not the expected squared norm, taking the expectation of ∣∣∇l∣∣2≤2ρxl||\nabla l||^2\le2\rho_x l bounds E[∣∣∇l∣∣2]E[||\nabla l||^2], which is not CSG. A CSG--CSL relation can be obtained from it only after taking square roots and applying Jensen's inequality, which gives a square-root form:

CSG(z⃗i)Tmax⁡≤2ρxCSL(z⃗i)Tmax⁡.\frac{CSG(\vec z_i)}{T_{\max}}\le\sqrt{2\rho_x\frac{CSL(\vec z_i)}{T_{\max}}}.

Our linear form comes from the Nesterov--Polyak expansion evaluated at a fixed perturbation scale ϵ\epsilon. This makes the per-iteration bound linear in ll, so it passes directly through the expectation over the trajectory. The term ρ6ϵ2\frac{\rho}{6}\epsilon^2 in Eq. 11 is a result of this expansion; it is non-negative and does not affect the validity of the bound. The two relations are consistent: for each ϵ\epsilon, Eq. 11 is a tangent line to the square-root curve, and minimizing over ϵ\epsilon recovers the reviewer's bound. Since the square-root version is also a valid relation, we will add it as a remark after Theorem 1.

4. The bound is an upper-bound, so how does it explain the near-linear relation?

A: We agree with the reviewer, an upper bound does not mathematically predict a linear relationship. Theorem 1 establishes an upper bound on CSG in terms of CSL, and the near-linear correlation we observe is an empirical phenomenon occurring within this feasible space. What the bound does guarantee is one direction: a large CSG is only possible with a large CSL. To correct our overstatement, the revision will replace "validating" with "consistent with" (Sec. 6.1, Takeaways) and clarify that the linear CSL--CSG relationship is an empirical observation constrained by our bound.

5. How does the bound on the minimum or maximum of δt\delta_t over the training trajectory imply anything about the distance used in Table 1?

A: Formally, it does not. The trajectory bound (Eq. 13) bounds δmin⁡\delta_{\min}, while Table 1 uses the final-checkpoint C&W distance dadv≥δmin⁡d_{adv}\ge\delta_{\min}. As noted at the start of Section 6, Table 1 evaluates dˉadv,T\bar d_{adv,T} as an empirical proxy that mirrors the algebraic form of the bound. Because we do not compute ρx\rho_x to verify the prerequisite Ct2≥2ρxΔt(u)C_t^2\ge2\rho_x\Delta^{(u)}_t, it serves purely as an applied estimator. For this reason we evaluate rank agreement (Spearman) and cosine similarity rather than treating it as a calibrated distance. The bound that applies to the deployed model is the single-checkpoint bound (Eq. 12) at t=Tmax⁡t=T_{\max}. We will rename the column to "(Proxy)" and state this distinction explicitly.

6. How was the n for CSL-diff chosen, and was it fixed?

A: Yes, it was fixed. The previous state-of-the-art, LT-IQR (Pollock et al., 2025), computes the interquartile range (Q0.75−Q0.25Q_{0.75}-Q_{0.25}), capturing volatility over the middle 50% of the loss trajectory. We fixed n=40n=40 and n=50n=50 a priori to cover similarly substantial portions of training, but shifted the focus to contrasting the extremes of optimization: early-stage underfitting versus late-stage memorization. With n≤50n\le50 the two windows never overlap. The same values were used for every dataset, attack, shadow budget and FPR without tuning, and both outperform LT-IQR in every setting.

We thank the reviewer again for the constructive review. We wanted to briefly summarize the changes made in response to your comments and share the updated materials.

Theory vs. proxies. The revision states exactly what each theorem bounds, labels dˉadv,T\bar d_{adv,T}, dˉadv,1\bar d_{adv,1} and CSL-Diff as empirical estimators, replaces "validating" with "consistent with", and renames the Table 1 header to "(Proxy)".

Role of ρ\rho and Theorem 1. We add the square-root form of the CSG–CSL relation obtained from your inequality as a remark after Theorem 1, and clarify that the linear form (Eq. 11) is a family of tangent lines to it.

Table 1 and CSL-Diff. We clarify that the trajectory bound does not formally bound the final-checkpoint C&W distance, and that n=40n=40 and n=50n=50 were fixed a priori and used unchanged across all settings.

Code and experiments. All implementation issues raised during the discussion are fixed, and the affected experiments have been re-run.

The corrected code, the new results, and a standalone document with the revised theorems and complete proofs (revised_theory.pdf) are available at: "https://anonymous.4open.science/r/Rebuttal_for_ICLR27-DC85/" We would be happy to clarify anything further. If our responses have addressed your concerns, we would appreciate it if you could take them into account in your final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.