acceptodds
Under review as a conference paper at ICLR 2027

More Noise, Less Damage: Enhanced Noise Injection Against Gradient-Inversion in Federated Learning

Abstract

Gradient inversion attacks expose a key vulnerability of federated learning: even when raw data remain on local devices, shared gradients can still leak sensitive information about training samples. Existing defenses typically mitigate this risk by perturbing gradients before transmission, but these perturbations often interfere with optimization and reduce model performance. In this paper, we show that stronger noise injection need not lead to a proportional loss in utility when the noise is introduced in a structured way. Building on this insight, we propose TIDE, a structured noise injection framework designed to improve the privacy–utility trade-off. TIDE injects larger noise at each training step while exploiting a telescoping effect that causes perturbations to partially cancel over the course of training. As a result, it provides stronger protection for shared gradients with smaller perturbations to the optimization dynamics. We theoretically show that, on smooth nonconvex objectives at fixed learning rate (), TIDE's residual stationarity floor scales as whereas standard DPSGD incurs an floor; equivalently, with optimally tuned step sizes TIDE attains a strictly faster convergence rate compared to DPSGD's . This separation enables TIDE to maintain better optimization stability despite using larger noise at each step. Empirically, TIDE remains effective with noise levels up to larger than those used in standard approaches, while consistently achieving better model performance. Extensive experiments across multiple datasets and against a range of inversion attacks confirm that TIDE significantly enhances gradient privacy while preserving competitive test accuracy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
gradient inversion defense, DPSGD, noise injection

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_Y6oyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TIDE. It adds fresh Gaussian noise to clipped per-example gradients and maintains a client-level exponential moving average correction. The aggregated perturbations telescope into boundary terms. Under smoothness, exact gradients, inactive clipping and a fixed learning rate, the paper derives an O(η²) stationarity upper bound and a tuned O(T^(-2/3)) bound, compared with O(η) and O(T^(-1/2)) bounds for independent noise. It also gives an exact-gradient-preimage distance bound and a conditional matrix-Gaussian sensitivity analysis. Experiments cover CIFAR-10/100, Tiny-ImageNet, ViT fine-tuning, partial participation and several inversion attacks, including a TIDE-aware multi-round objective. The main tables suggest useful accuracy retention at larger noise levels. However, they do not establish a general privacy advantage at a matched end-to-end privacy budget. I recommend weak rejection: the practical idea is promising, but the security interpretation, novelty positioning and inconsistent experimental reporting require substantial revision.

Critical strengths
  1. Simple mechanism with a useful restricted optimization analysis. The identity sum_t(ξ̄_t + e_(t-1)) = (e_(-1) - e_(T-1))/(1-β) is transparent, and the auxiliary-iterate argument in Appendix A.2 explains why temporally correlated perturbations can reduce optimization damage. The stationary initialization and explicit dependence on β make the construction understandable and implementable. This is a credible basis for further study, although it does not itself prove privacy.
  2. Broad empirical evidence of utility retention. At σ = 0.003, Table 1 reports CIFAR-10 accuracy of 91.16 ± 0.17% for TIDE versus 87.69 ± 0.39% for DP-SGD; CIFAR-100 gives 68.57 ± 0.50% versus 62.86 ± 0.41%. Appendix I extends utility evaluation to 100 non-IID clients, and Appendix F explicitly considers temporal correlation through a covariance-aware attack. Including these settings, clipping ablations and perceptual metrics is valuable. The gains over the stronger DP-Step-Decay baseline are smaller, and the privacy results remain specific to the evaluated attacks.
Critical weaknesses
  1. The claimed security and theoretical scope exceed what is established. Theorem 3.1 bounds the distance to an exact solution of G_w(x′) = g′. If the noisy gradient lies outside the gradient map's image, the defined distance is infinite; approximate inversion may still recover x accurately. For an attack with residual ρ, the same argument gives only max(0, (||g′ - G_w(x)|| - ρ)/l_g), so a perturbation-norm bound alone is insufficient. Appendix A.6 usefully distinguishes conditional sensitivity energy from subsampled RDP, but no full-transcript (ε, δ) comparison is reported. Table 2 even gives a 34.1-fold conditional divergence ratio at T = 200, full participation and 10-fold TIDE noise; its q < 1 rows are not privacy guarantees. Larger marginal noise therefore does not establish stronger overall privacy. Appendix F tests only T = 10, lacks a GLS positive control and omits crucial tuning details. Its raw-gradient forward model also needs reconciliation with active clipping. Separately, Appendix A.2 analyzes exact gradients without active clipping, momentum or learning-rate decay, whereas Appendix B uses all these practical features. Mini-batch sampling noise does not telescope with the injected noise. The abstract should describe bounds for this restricted model rather than a demonstrated general convergence-rate separation for practical DP-SGD.
  2. Novelty and empirical reliability are insufficiently resolved. At β = 0, TIDE's first-difference noise matches the Anti-PGD construction up to scaling/initialization; Orvieto et al. (ICML 2022) and Koloskova et al. (NeurIPS 2023) are directly relevant but absent from the references. The latter already studies correlated-noise optimization, including nonconvex objectives. Appendix A.3 mentions DP-FTRL/matrix mechanisms, yet there is no substantive comparison establishing the incremental contribution. Reporting conflicts further weaken the evidence: Section 5.4 gives ViT accuracy 90.52% versus 83.17%, but Table 8 gives 89.27% versus 86.65% at σ = 0.03. The 27.45% DP result is at σ = 0.1 in Table 8, not 0.3, and Table 9 gives TIDE SSIM 0.0470, not 0.034. All five SSIM means and standard deviations for TIDE at σ = 0.01 are identical across Tables 1a and 1b. These require source-log verification, not an assumption of misconduct. Algorithm 1 line 4 also uses σ where Equation (1) uses σ_p, changing variance by a factor of B. Without resolving the released object, noise normalization and experiment provenance, the comparisons cannot be reliably reproduced.
Questions for authors

Clarifications and evidence that could change the recommendation:

  1. What does the server actually receive: all per-example gradients, a batch mean, or local model updates? Please reconcile Algorithm 1 line 4 with Equations (1)-(2), and identify whether every table uses σ or σ_p, the clipping threshold, and the training/attack batch sizes. How does Appendix A.6 model the shared batch state for that exact transcript?
  2. Can the safety argument bound approximate reconstruction error under the stated attacker, and can privacy/utility be compared at a common full-training (ε, δ), adjacency and sampling protocol? Otherwise, please narrow the privacy claims explicitly to the tested attacks.
  3. Please provide corrected ViT results and source logs for the repeated CIFAR SSIM rows. Were DP-SGD and DP-Step-Decay attack metrics intentionally reused, and at which checkpoint/noise level were they measured?
  4. What are the GLS attack's λ, γ, optimizer, restarts, sample count and clipping-aware forward model? Does it succeed on an undefended or weak-noise positive control? A longer observation horizon would materially test the claimed temporal robustness.
  5. What is new relative to Anti-PGD and correlated-noise/DP-FTRL analyses? Please separate the EMA/defense contribution from prior noise-cancellation ideas and explain which convergence claims survive mini-batch noise and active clipping. Please also report main-benchmark seed/sample counts, error-bar definitions, training lengths, aggregation details and checkpoint selection.
4 comments
AuthorsOpenReview ↗

Response to Reviewer Y6oy

We thank the reviewer for the detailed and technically insightful feedback. We have refined the theoretical statements, expanded the experimental comparisons and attack evaluations, and verified and corrected the reported numerical results. We address the theoretical, experimental, and reproducibility questions in detail below.

W1 & Q2. Approximate inversion theorem and privacy analysis

We refined the gradient-matching reconstruction objective to allow an explicit matching residual. For a clean gradient map GwG_w, defended observation g′g', and tolerance rr, define

dw(r)(x;g′)=inf⁡x′: ∥Gw(x′)−g′∥≤r∥x′−x∥.d_w^{(r)}(x;g')=\inf_{x':\,\|G_w(x')-g'\|\le r}\|x'-x\|.

Under lgl_g-Lipschitz continuity, the geometric argument gives

dw(r)(x;g′)≥[∥g′−Gw(x)∥−r]+lg.d_w^{(r)}(x;g')\ge \frac{[\|g'-G_w(x)\|-r]_+}{l_g}.

When g′=Gw(x)+ξg'=G_w(x)+\xi, the numerator becomes [∥ξ∥−r]+[\|\xi\|-r]_+. The Gaussian reconstruction-distance estimate is stated with the same residual tolerance. The revised formulation explicitly relates perturbation magnitude, achievable gradient-matching accuracy, and reconstruction displacement.

Appendix A.6 analyzes the conditional sensitivity energy of a sequence of released batch-averaged gradients, given a target sample's specified training-step participation pattern. It includes full- and partial-participation examples and accounts for the correlations induced by the client-local EMA correction. The reported conditional divergence energies characterize these participation patterns; the empirical privacy assessment reports reconstruction performance for the specified inversion attacks. A full-training privacy budget also depends on the adjacency relation, sampling protocol, and composition of the observed transcript. The new tree DP-FTRL training comparison uses the stated nominal noise scales and is interpreted as an optimization-utility comparison.

W1 & Q4. Covariance-aware multi-round reconstruction and attack protocol

We expanded multi-round reconstruction evaluation with covariance-aware GLS applied to T=10T=10 and T=20T=20 consecutive gradient releases from the same fixed batch of four images. The attack uses λ=1\lambda=1, Adam with learning rate 0.060.06, and five attack evaluations per condition. SSIM is reported as mean ±\pm standard deviation across those evaluations:

DefenseGLS T=10T=10GLS T=20T=20
None0.6329 ±\pm 0.34610.6358 ±\pm 0.3476
DP-SGD (σ=0.003\sigma=0.003)0.1763 ±\pm 0.08630.4628 ±\pm 0.2065
TIDE (σ=0.03\sigma=0.03)0.0140 ±\pm 0.00440.0355 ±\pm 0.0132

The undefended controls yield SSIM above 0.630.63 at both horizons, establishing an effective reconstruction reference. Longer observation horizons increase reconstruction SSIM for both defended configurations. TIDE maintains low SSIM at the evaluated noise scale. We report TGIAs-RO and GLS in separate Appendix tables and describe the consecutive same-batch observation protocol and the attack's stated optimization settings. The experimental comparisons use the distinct reported noise scales for DP-SGD and TIDE.

W2 & Q3. Numerical corrections and experimental provenance

We verified the result tables and corrected the affected numerical descriptions. For ViT-B/16 on CIFAR-100 at σ=0.03\sigma=0.03, the accuracies are 89.27% for TIDE and 86.65% for DP-SGD, and TIDE's CI-Net SSIM is 0.0470. The DP-SGD accuracy of 27.45% corresponds to σ=0.1\sigma=0.1.

The duplicated TIDE σ=0.01\sigma=0.01 CIFAR reconstruction values came from inserting the CIFAR-100 measurements into the CIFAR-10 row. The corrected CIFAR-10 means for GI, IG, iDLG, CI-Net, and DGGI are 0.0267, 0.0180, 0.0186, 0.0281, and 0.0235, respectively; the CIFAR-100 measurements are retained.

The inversion attacks are evaluated at model initialization. At that point, DP-SGD and DP-Step-Decay use the same gradient perturbation mechanism and noise scale, so their reported attack metrics use the same initialization-stage attack measurements. Model-training accuracy is evaluated separately. The manuscript clarifies the shared measurements, the batch-level σ\sigma convention, and the corresponding experimental provenance.

AuthorsOpenReview ↗

Response to Reviewer Y6oy

W2 & Q5. Contribution, convergence analysis, and experimental evidence

Prior work and contribution. We expanded Related Work to include Anti-PGD (Orvieto et al., 2022), Koloskova et al. (2023), and DP-FTRL/matrix-factorization methods. Anti-PGD's first-difference noise corresponds to TIDE's β=0\beta=0 case up to scaling and initialization. Koloskova et al. provide a general linearly correlated-noise framework recovering the same asymptotic order in the corresponding fixed-step setting. TIDE develops an EMA-parameterized, client-local realization for gradient-inversion defense and studies reconstruction resistance and training utility. The reported multi-round ablation identifies β=0\beta=0 as the weakest reconstruction-defense setting among the evaluated values.

Direct correlated-noise baseline. We added a binary-tree DP-FTRL comparator using the same CIFAR training protocol and nominal noise-scale parameter. Three-seed best-validation accuracy (%, mean ±\pm sample SD) is:

Datasetσ\sigmaDP-SGDTree DP-FTRLTIDE
CIFAR-100.00387.69 ±\pm 0.3991.37 ±\pm 0.2091.16 ±\pm 0.17
CIFAR-1000.00362.86 ±\pm 0.4168.11 ±\pm 0.3468.57 ±\pm 0.50
CIFAR-100.03—87.60 ±\pm 0.2090.60 ±\pm 0.45
CIFAR-1000.03—60.12 ±\pm 0.0767.46 ±\pm 0.30

At σ=0.003\sigma=0.003, TIDE and tree DP-FTRL achieve comparable utility; at σ=0.03\sigma=0.03, TIDE has 3.00/7.34 percentage-point higher validation accuracy on CIFAR-10/CIFAR-100, respectively.

Convergence assumptions and stochastic gradients. The theorem uses fixed learning rate and inactive clipping to isolate the impact of injected temporally correlated noise. For conditionally unbiased mini-batch gradients g^t=∇L(wt)+ζt\widehat g_t=\nabla L(w_t)+\zeta_t, satisfying E[ζt∣Ft]=0\mathbb E[\zeta_t\mid\mathcal F_t]=0 and E[∥ζt∥2∣Ft]≤V\mathbb E[\|\zeta_t\|^2\mid\mathcal F_t]\le V, the Appendix extends the stationarity bound with additional 2lηV2l\eta V and lηVl\eta V terms for TIDE and DP-SGD, respectively, under the proof's constants. The corresponding injected-noise terms scale as O(η2)O(\eta^2) for TIDE and O(η)O(\eta) for DP-SGD in the simplified setting. The principal experiments additionally evaluate active clipping and scheduled learning rates.

Theory-aligned training controls. We added no-clipping/fixed-LR 0.30.3 results (200 epochs, three clients, three seeds), reporting best-validation accuracy (%, mean ±\pm sample SD):

Datasetσ\sigmaDP-SGDTIDE
CIFAR-100.00384.25 ±\pm 0.4888.24 ±\pm 0.57
CIFAR-1000.00358.29 ±\pm 0.7166.09 ±\pm 0.23
CIFAR-100.0331.66 ±\pm 0.6687.16 ±\pm 0.54
CIFAR-1000.034.66 ±\pm 0.3860.67 ±\pm 0.37

These fixed-step comparisons complement the theorem's convergence-speed bound with same-epoch-budget training outcomes. The CIFAR training setup uses batch size 128, 200 epochs, three clients, and three training seeds; the reported training accuracy is best validation accuracy. Single-round reconstruction uses ten target batches of size four (40 images) at a shared initialized sigmoid-activated ResNet-18. GLS SSIM error bars are calculated across five attack evaluations. The manuscript and supplementary materials contain the relevant evaluation protocol and implementation.

Q1. Released gradient, noise normalization, and sensitivity protocol

Clients communicate one defended batch-averaged gradient per training step and retain their EMA correction states internally. Algorithm 1 now distinguishes the client-side computation of per-example gradients, used for clipping and batch aggregation, from the final communicated gradient:

e_t=\beta e_{t-1}-(1-\beta)\bar\xi_t.$$ The experimental $\sigma$ specifies the **batch-level Gaussian innovation scale**; the equivalent independent per-example construction uses $\sigma_p=\sqrt B\,\sigma$. The CIFAR training evaluation uses **batch size 128**, while inversion is conducted **at initialization** using a shared sigmoid-activated ResNet-18, **batch size four**, and **ten target batches (40 images)**. This attack batch size allows informative reconstruction at a manageable joint-inversion difficulty. The training protocol specifies the clipping threshold and scheduled-learning-rate configuration, while the fixed-LR, inactive-clipping experiment separately tests the assumptions used in the convergence proof. Appendix A.6 measures the **conditional sensitivity energy of the time-ordered sequence of released batch means** for a sample participating at designated training steps. The calculation includes the covariance generated by the client-local EMA correction and examines full and partial participation scenarios. This identifies the release transcript and participation patterns underlying the conditional divergence calculation.

Thank you for the detailed response and additional experiments. Some of my concerns have been addressed.

However, I believe two major concerns remain insufficiently resolved. First, the paper still lacks a privacy analysis covering the full training process and a privacy–utility comparison under the same end-to-end (ϵ,δ)(\epsilon,\delta) privacy budget. The existing conditional sensitivity analysis and empirical attack results are insufficient to fully support the paper's broader privacy claims. Second, although the additional convergence analysis and experiments are valuable, the claimed convergence-rate advantage has not been sufficiently established under practical training conditions involving mini-batch sampling noise, gradient clipping, and learning-rate schedules. Furthermore, while the authors have further clarified the differences from existing correlated-noise methods, the incremental theoretical contribution remains limited.

I appreciate the authors' efforts to improve the paper. Nevertheless, these remaining concerns continue to affect my overall assessment. Therefore, I will maintain my original score.

Thank you for your follow-up and for acknowledging our additional analyses and experiments. We would like to further clarify the following three points.

1. End-to-end privacy accounting. While tree-based DP-FTRL constrains cumulative noise variance to O(log⁡T)O(\log T), TIDE can retain O(T)O(T) growth with a substantially reduced coefficient and bounded relative sensitivity energy by generalizing its error-correction update to et=βet−1−γξte_t=\beta e_{t-1}-\gamma\xi_t, with β+γ<1\beta+\gamma<1. Within this generalized framework, DP-SGD and the original TIDE correspond to γ=0\gamma=0 and β+γ=1\beta+\gamma=1, respectively.

For β+γ<1\beta+\gamma<1, the inverse noise-generation matrix exhibits exponentially decaying entries, and the conditional sensitivity energy ratio in Appendix A.6 satisfies

(1-q)\left(1+\frac{\gamma^2}{1-(\beta+\gamma)^2}\right) +q\left(\frac{1-\beta}{1-\beta-\gamma}\right)^2.$$ This upper bound is independent of the training horizon. For example, with $\beta=0.5$, reducing $\gamma$ from $0.5$ to $0.45$ reduces the 34.1-fold conditional divergence ratio highlighted by the reviewer to 0.862 under the same setting ($T=200$, full participation, and 10-fold noise scale), while still eliminating 99% of the asymptotic cumulative noise variance relative to independent noise of the same scale. This provides a principled extension of TIDE toward balancing gradient reconstruction resistance, optimization utility, and end-to-end differential privacy. We further empirically validate this generalized formulation on CIFAR-10 with $\beta=0.5$ and $\gamma=0.45$: | Method | $\sigma=0.003$ | $\sigma=0.03$ | | --- | --- | --- | | DP-SGD | $87.69\pm0.39$ | - | | Original TIDE | $91.16\pm0.17$ | $90.60\pm0.45$ | | Generalized TIDE | $91.17\pm0.05$ | $90.56\pm0.32$ | Generalized TIDE achieves nearly identical test accuracy to the original TIDE while substantially reducing its conditional sensitivity energy. These theoretical and empirical results demonstrate the flexibility of TIDE in balancing noise cancellation and transcript sensitivity, providing a principled direction toward end-to-end privacy–utility optimization. **2. Convergence under practical training conditions.** Thank you for bringing up this remaining concern. We have provided two additional theoretical analyses in Proposition A.5 (*Convergence with unbiased mini-batch gradients*) and Proposition A.6 (*Synchronous fixed-weight client aggregation*). The former explicitly incorporates mini-batch gradient variance into the convergence bounds of both TIDE and DP-SGD, while preserving TIDE's control over accumulated injected noise. The latter establishes that the noise cancellation property extends to synchronous multi-client training. Our main experiments incorporate active gradient clipping and scheduled learning rates, and the additional experiments with inactive clipping and fixed learning rates further validate the theoretical predictions under the corresponding assumptions. Together, these results establish the optimization benefit of temporal noise cancellation in the analyzed settings and demonstrate its empirical effectiveness under practical training configurations. **3. Theoretical contribution.** We thank the reviewer for highlighting the connections to Orvieto et al. (ICML 2022) and Koloskova et al. (NeurIPS 2023). We would like to emphasize three distinctions that clarify the contributions of TIDE. First, TIDE generalizes the Anti-PGD mechanism of Orvieto et al. through an EMA-based error-correction framework. Anti-PGD corresponds to the special case of $\beta=0$, while TIDE allows $\beta\in[0,1)$, providing an additional degree of freedom for controlling temporal noise correlations. Second, while the general convergence framework of Koloskova et al. can encompass TIDE's noise structure, our theoretical analysis addresses a different objective: gradient inversion defense. Specifically, Theorems 3.1 and 3.2 jointly characterize the instantaneous perturbation governing gradient reconstruction and the accumulated perturbation governing optimization. This connects reconstruction resistance with convergence behavior within a unified, explicitly parameterized mechanism. Third, our empirical findings demonstrate that $\beta=0$ does not necessarily provide the best reconstruction resistance (Table 9). This observation is particularly important because it shows that the special case considered in prior studies is not always the most favorable choice in practice. We hope these clarifications further highlight the motivation and contributions of our work, and we appreciate your constructive feedback throughout the discussion.
Reviewer_wteuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses defenses against gradient inversion attacks in federated learning. Existing differential-privacy-style defenses add independent noise at every step, and this noise accumulates over the course of training. The authors propose TIDE, a noise injection mechanism that applies a large perturbation at each step while making consecutive noise terms partially cancel over the trajectory, so that the cumulative distortion stays bounded.

Critical strengths
  • The perspective is interesting. Instead of asking how much noise to add at each step, the paper asks how noise accumulates along the trajectory.

  • The paper provides theoretical analysis.

Critical weaknesses
  • The correlated noise update is not new. Related-noise constructions already exist in the DP-FTRL and matrix-factorization DP literature. The paper only briefly positions itself against this line of work and does not include a direct empirical comparison with those methods.

  • It remains unclear how effective the proposed method is against adaptive attacks.

  • The theory and the experiments are disconnected. The theoretical guarantees are derived under assumptions (clipping inactive, fixed step size) that do not match the experimental setup (clipping enabled, scheduled learning rates).

  • There is no comparison of convergence speed between methods.

  • The code is not open-sourced.

Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer wteu

We thank the reviewer for recognizing the motivation and theoretical contributions of our work. In response to the suggestions, we have strengthened the comparison with existing correlated-noise methods, extended the convergence analysis, and added experiments under theory-aligned training settings and adaptive attacks. We provide detailed responses below.

W1. Related correlated-noise constructions and DP-FTRL baseline

We expanded Related Work to discuss DP-FTRL/matrix-factorization approaches, Anti-PGD (Orvieto et al., 2022), and Koloskova et al. (2023). The first-difference perturbation used in Anti-PGD appears as the β=0\beta=0 case of TIDE up to scaling and initialization. Koloskova et al. provide a correlated-noise convergence framework that recovers the same order for the corresponding simplified optimization setting. TIDE develops this noise-correction family into a client-local, EMA-parameterized gradient-inversion defense and studies reconstruction resistance together with optimization utility.

We added a direct binary-tree DP-FTRL-style comparator under the same CIFAR training protocol and nominal noise scale. Best-validation accuracy (%, three seeds, mean ±\pm SD):

Datasetσ\sigmaDP-SGDTree DP-FTRLTIDE
CIFAR-100.00387.69 ±\pm 0.3991.37 ±\pm 0.2091.16 ±\pm 0.17
CIFAR-1000.00362.86 ±\pm 0.4168.11 ±\pm 0.3468.57 ±\pm 0.50
CIFAR-100.03—87.60 ±\pm 0.2090.60 ±\pm 0.45
CIFAR-1000.03—60.12 ±\pm 0.0767.46 ±\pm 0.30

At σ=0.003\sigma=0.003, TIDE and tree DP-FTRL perform comparably; at σ=0.03\sigma=0.03, TIDE improves over tree DP-FTRL by 3.00/7.34 percentage points on CIFAR-10/CIFAR-100. The new Appendix section describes the tree mechanism and compares the reported three-seed training results. The β\beta ablation also examines the difference-noise endpoint, with β=0\beta=0 having the weakest defense in the reported multi-round ablation.

W2. Adaptive attacks

We would like to clarify that we have already investigated the robustness of our method against adaptive attacks in Appendix F, “Evaluation under Multi-Round and Adaptive Attacks”. This section provides additional experiments evaluating the proposed method under more challenging attack scenarios.

To further address any remaining concerns, we have expanded the adaptive multi-round evaluation to include covariance-aware GLS reconstruction from consecutive gradient releases of the same fixed four-image target batch. The attacker uses λ=1\lambda=1, Adam learning rate 0.060.06, and evaluates observation horizons T=10T=10 and T=20T=20. Each reported condition includes five attack evaluations (mean ±\pm SD):

DefenseT=10T=10 SSIMT=20T=20 SSIM
None0.6329 ±\pm 0.34610.6358 ±\pm 0.3476
DP-SGD (σ=0.003\sigma=0.003)0.1763 ±\pm 0.08630.4628 ±\pm 0.2065
TIDE (σ=0.03\sigma=0.03)0.0140 ±\pm 0.00440.0355 ±\pm 0.0132

The undefended positive controls establish attack efficacy. Increasing the observation horizon increases the observed reconstruction quality, while the TIDE configuration retains low SSIM at its evaluated noise level. Note TIDE typically requires a larger noise level, while DP-SGD's performance collapses under σ=0.03\sigma=0.03. The revised Appendix provides separate TGIAs-RO and GLS tables and describes the observations, attack objective, and optimization settings.

AuthorsOpenReview ↗

Response to Reviewer wteu

W3 & W4. Convergence assumptions and convergence speed

We thank the reviewer for this comment. We would like to clarify that the convergence rates of TIDE and DP-SGD have already been compared in both the Introduction and the theoretical analysis. Specifically, we theoretically establish that, with optimally tuned step sizes, TIDE achieves a faster convergence rate of O(T−2/3)O(T^{-2/3}), compared to O(T−1/2)O(T^{-1/2}) for DP-SGD. This result demonstrates the convergence advantage of TIDE over the standard DP-SGD baseline. We will further highlight this comparison in the revised manuscript to make the theoretical improvement more explicit.

Moreover, the fixed-learning-rate, inactive-clipping theorem isolates the effect of TIDE's injected correlated noise on the optimization trajectory. We supplemented it with a conditionally unbiased mini-batch extension. For g^t=∇L(wt)+ζt\widehat g_t=\nabla L(w_t)+\zeta_t, with E[ζt∣Ft]=0\mathbb E[\zeta_t\mid\mathcal F_t]=0 and E[∥ζt∥2∣Ft]≤V\mathbb E[\|\zeta_t\|^2\mid\mathcal F_t]\le V, the stationarity bounds receive additional O(lηV)O(l\eta V) contributions for both mechanisms (specifically 2lηV2l\eta V for TIDE and lηVl\eta V for DP-SGD under our proof constants). The injected-noise terms continue to exhibit their respective O(η2)O(\eta^2) and O(η)O(\eta) step-size dependence.

To align the training experiments directly with the theorem, we ran no clipping, fixed LR =0.3=0.3, zero warmup, 200 epochs, three clients, three seeds:

Datasetσ\sigmaDP-SGDTIDE
CIFAR-100.00384.25 ±\pm 0.4888.24 ±\pm 0.57
CIFAR-1000.00358.29 ±\pm 0.7166.09 ±\pm 0.23
CIFAR-100.0331.66 ±\pm 0.6687.16 ±\pm 0.54
CIFAR-1000.034.66 ±\pm 0.3860.67 ±\pm 0.37

These experiments provide a matched 200-epoch comparison under the theoretical regime. The convergence theorem itself gives a theoretical convergence-speed characterization through its average stationarity bound and dependence on the training horizon TT: the injected-noise contribution is O(η2)O(\eta^2) for TIDE and O(η)O(\eta) for independent-noise DP-SGD, leading to the corresponding optimized-step-size rate in the simplified model. The empirical results show that TIDE maintains higher accuracy under the same fixed learning rate and epoch budget. The main experiments additionally explore active clipping and scheduled learning rates, and the Cross-Device FL section studies partial participation and heterogeneous client partitions.

W5. Code availability

The implementation is included in the supplementary materials. The revised submission specifies the released batch-gradient interface, model initialization for inversion evaluations, training and reconstruction batch sizes, experimental seed/sample counts, and the expanded multi-round GLS configurations. The new DP-FTRL and FFHQ results are accompanied by their training protocols, allowing the experimental comparisons to be reproduced from the supplied materials.

Reviewer_QtGrOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes TIDE, a differential noise mechanism with error correction for defending shared gradients in federated learning against gradient inversion. Instead of injecting independent Gaussian noise at every iteration as in DPSGD, TIDE injects the first difference of an i.i.d. Gaussian sequence through an EMA error-correction state. This yields a telescoping cancellation property that bounds cumulative noise accumulation uniformly over training rounds, allowing larger per-step noise without dynamic retuning. The authors contribute: (1) a structured noise-defense mechanism with EMA error correction; (2) convergence and safety analyses for smooth nonconvex objectives, showing an O(η2)\mathcal{O}(\eta^2) residual stationarity floor versus O(η)\mathcal{O}(\eta) for DPSGD, a faster O(T−2/3)\mathcal{O}(T^{-2/3}) tuned rate versus O(T−1/2)\mathcal{O}(T^{-1/2}), and an improved single-round ℓ1\ell_1 inversion-resistance margin, plus multi-round sensitivity accounting; and (3) extensive experiments on CIFAR-10/100, Tiny-ImageNet, ViT-B/16, and scaled cross-device FL. Validation compares TIDE with six defense baselines under multiple inversion attacks, reporting competitive or better test accuracy, substantially reduced SSIM/LPIPS leakage, and tolerance of up to 30×30\times larger noise multipliers than conventional DP settings.

Critical strengths
  1. A structurally novel and theoretically consequential defense mechanism. TIDE’s core idea—injecting the first difference of an i.i.d. Gaussian sequence through an EMA error-correction state—is not a incremental tuning of DPSGD. The telescoping property bounds cumulative noise uniformly in TT, which directly enables the paper’s central theoretical separation: an O(η2)\mathcal{O}(\eta^2) residual stationarity floor and O(T−2/3)\mathcal{O}(T^{-2/3}) tuned rate versus DPSGD’s O(η)\mathcal{O}(\eta) floor and O(T−1/2)\mathcal{O}(T^{-1/2}) rate, while also yielding a larger single-round ℓ1\ell_1 inversion-resistance margin. This is consequential because it challenges the presumed privacy–utility trade-off and gives a principled reason why larger per-step noise need not proportionally harm optimization.

  2. Broad and convincing empirical validation. The evaluation spans CIFAR-10/100, Tiny-ImageNet, ViT-B/16 fine-tuning, and scaled cross-device FL, compares against six defense baselines, and tests multiple inversion attacks including multi-round/adaptive ones. The reported ability to tolerate up to 30×30\times larger noise multipliers while preserving competitive accuracy and substantially reducing SSIM/LPIPS leakage strongly supports the practical relevance of the theoretical claims. This breadth materially increases confidence that the mechanism is not merely a single-benchmark artifact.

Critical weaknesses
  1. The threat model and evaluation are centered on per-sample gradient releases, leaving batch-level and model-update inversion insufficiently addressed. In realistic FL, clients often release a batch-mean gradient or a model update after multiple local epochs, not per-sample gradients. TIDE’s telescoping/error-correction analysis and experiments largely operate at the per-sample release level, so it is unclear whether the bounded-accumulation argument and privacy margin transfer to aggregated batch gradients or multi-epoch updates.

  2. The defense is evaluated almost exclusively against gradient inversion, with no substantive assessment against membership inference or attribute inference. Even if TIDE suppresses reconstruction SSIM/LPIPS, it may leave other privacy leakage channels intact, and its temporally correlated noise could introduce exploitable multi-round signals for membership or property inference. This is consequential because the paper’s central claim is a stronger privacy–utility trade-off for federated learning generally.

Questions for authors

see weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer QtGr

We thank the reviewer for the insightful comments on the threat model and privacy evaluation. We have clarified the batch-level gradient release mechanism and expanded the discussion and evaluation of multi-round gradient observations. Our responses to the specific concerns are provided below.

W1. Batch-mean gradient releases and model-update observations

The revised Algorithm 1 explicitly identifies the communicated object as a single defended batch-mean gradient:

e_t=\beta e_{t-1}-(1-\beta)\bar\xi_t.$$ The per-example gradients in the earlier algorithmic presentation represented intermediate **client-side computations** for subsequent aggregation. The EMA correction state $e_t$ also stays within the client. All reported experimental noise-scale settings refer to the **batch-level Gaussian innovation parameter $\sigma$**; the sample-level notation satisfies $\sigma_p=\sqrt B\,\sigma$ under the equivalent independent per-example Gaussian construction. The inversion evaluation attacks the **released batch-mean gradient at initialization**, using a sigmoid-activated ResNet-18, **batch size four**, shared initialized weights, and **ten target batches (40 images)**. Reconstruction at training batch size 128 is substantially more challenging, so batch size four supports a controlled assessment of inversion resistance. At the optimization level, our aggregation proposition shows that a synchronous fixed-weight combination of client releases preserves the EMA correction recurrence: $$\bar e_t=\beta\bar e_{t-1}-(1-\beta)\bar\xi_t.$$ The empirical cross-device evaluation further studies partial participation and non-IID data, and the separate multi-round attack evaluations measure the effect of combining consecutive gradient observations. These additions clarify the batch-gradient FL interface, the relation of its local noise state to global updates, and the observation model used by the attacker. ### W2. Additional leakage objectives and temporal observations We have specified the paper's **empirical privacy evaluation in terms of gradient inversion reconstruction quality** under the stated batch-mean release protocol. The resulting SSIM/LPIPS measurements quantify image reconstruction, while the related conditional sensitivity analysis characterizes how a specified training sample's participation affects the correlated sequence of released gradients. To examine information available from **multiple temporally correlated releases**, we expanded the covariance-aware GLS evaluation to $T=10$ and $T=20$ **consecutive releases generated from the same fixed batch of four images**. The attack uses $\lambda=1$, Adam with learning rate $0.06$, and five attack evaluations per setting. Reconstruction SSIM (mean $\pm$ SD) is: | Defense | $T=10$ | $T=20$ | |---|---:|---:| | None | 0.6329 $\pm$ 0.3461 | 0.6358 $\pm$ 0.3476 | | DP-SGD, $\sigma=0.003$ | 0.1763 $\pm$ 0.0863 | 0.4628 $\pm$ 0.2065 | | **TIDE, $\sigma=0.03$** | **0.0140 $\pm$ 0.0044** | **0.0355 $\pm$ 0.0132** | The undefended control establishes an effective attack reference. The longer observation horizon increases reconstruction quality in both evaluated defended configurations; TIDE retains low SSIM at the reported noise scale. The revised paper presents this evaluation alongside TGIAs-RO in separate multi-round tables and states conclusions in terms of the corresponding reconstruction-attack protocols.
Reviewer_htnCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • The paper proposes TIDE, a structured differential noise injection method for gradient-inversion in federated learning. Rather than dynamically injecting noise, TIDE aims to improve privacy in a more effective manner without compromising model performance by exploiting a telescope cancellation with an exponential moving average error correction.
  • The authors provide a theoretical convergence analysis showing that TIDE achieves an O(η2)\mathcal O(\eta^2) residual stationary floor compared to O(η)\mathcal O(\eta) of DP-SGD and the convergence rate of O(T−2/3)\mathcal O(T^{-2/3}) to O(T−1/2)\mathcal O(T^{-1/2}).
  • For the evaluation, they conduct experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet using ResNet-18 and ViT-B/16, comparing TIDE against DP-related defense baselines. They also evaluate the effectiveness of the defense with representative gradient inversion attacks: GI, IG, iDLG, CI-Net, DGGI, TGIAs. The results show that TIDE can tolerate large noise while preserving the model accuracy and reducing the success of gradient inversion attacks.
Critical strengths

S1: The paper theoretically analyzes the convergence of the proposed TIDE and compares its residual stationarity floor scales and convergence rate with those of DPSGD.

S2: Experimentally evaluating privacy and utility in both cases—with and without gradient clipping—verified the effectiveness of TIDE.

S3: The authors extend their evaluation from single-round gradient inversion attacks to multi-round adversarial scenarios that exploit temporal correlations, even when TIDE's noise-mixing matrix MM is known. This verifies whether the proposed defense mechanism is effective even in more challenging adversarial settings.

Critical weaknesses

W1. Limited Novelty: While the proposed approach is effective, the paper does not sufficiently clarify its novelty compared to prior work on temporally correlated noise mechanisms for differentially private optimization. A more explicit discussion or comparison would strengthen the contribution.

W2. Convergence Analysis in Federated Learning: Although the paper theoretically shows that TIDE limits cumulative noise perturbations and improves optimization convergence, these guarantees are established at the optimization-iteration level. It remains unclear whether bounded cumulative noise at individual clients translates into global model convergence in federated learning, where local updates, partial client participation, and global aggregation may affect the telescoping cancellation mechanism.

Questions for authors

Q1. Clarification of Experimental Settings: Could the author provide a more detailed description of the gradient inversion attack settings in Table 1? Specifically, at which training rounds or epochs are the attacks performed? Additionally, please clarify how the target samples and observed gradients are selected, including whether per-sample or batch-aggregated gradients are used for reconstruction.

Q2. High-Resolution Evaluation: Could the authors evaluate TIDE on higher-resolution image datasets, such as FFHQ, to examine whether the reported privacy–utility trade-off generalizes beyond CIFAR and Tiny-ImageNet benchmarks?

3 comments
AuthorsOpenReview ↗

Response to Reviewer htnC

We thank the reviewer for the constructive feedback and recognition of the effectiveness of our approach. We have expanded the discussion of related work, clarified the convergence analysis in federated learning, and provided additional experimental details and high-resolution evaluations. We address each point below.

W1. Novelty and comparison with temporally correlated noise

We expanded the discussion of temporally correlated noise in both Related Work and the theory section. The first-difference construction of Anti-PGD (Orvieto et al., 2022) corresponds to the β=0\beta=0 special case of TIDE up to noise scaling and initialization. Koloskova et al. (2023) study convergence under linearly correlated noise and provide a general framework that recovers the same asymptotic order for the corresponding fixed-step analysis. These results establish the foundations of noise cancellation across optimization steps. Our contribution is the EMA-parameterized, client-local realization as a gradient-inversion defense, together with evaluations of reconstruction attacks and training robustness across datasets, model architectures, FL configurations, and observation horizons. The β\beta ablation further motivates considering the full EMA family: the first-difference case β=0\beta=0 exhibits the weakest reconstruction resistance in the reported multi-round ablation.

We also added a direct online binary-tree DP-FTRL baseline, using the same CIFAR training configurations and the same nominal noise-scale parameter. The three-seed best-validation accuracies (%, mean ±\pm sample SD) are:

Datasetσ\sigmaDP-SGDTree DP-FTRLTIDE
CIFAR-100.00387.69 ±\pm 0.3991.37 ±\pm 0.2091.16 ±\pm 0.17
CIFAR-1000.00362.86 ±\pm 0.4168.11 ±\pm 0.3468.57 ±\pm 0.50
CIFAR-100.03—87.60 ±\pm 0.2090.60 ±\pm 0.45
CIFAR-1000.03—60.12 ±\pm 0.0767.46 ±\pm 0.30

At σ=0.003\sigma=0.003, the two correlated-noise methods achieve similar utility. At σ=0.03\sigma=0.03, TIDE improves over tree DP-FTRL by 3.00 and 7.34 percentage points on CIFAR-10 and CIFAR-100, respectively. These comparisons directly place the proposed mechanism in the existing correlated-noise literature. The reported noise scales refer to the mechanisms' specified Gaussian noise parameters, and the results assess optimization utility under the stated training protocol.

W2. Convergence analysis in federated learning

The convergence theorem adopts fixed learning rates and inactive clipping to isolate the effect of temporally correlated injected noise on the optimization trajectory. We added an analysis of synchronous aggregation with fixed client weights. If each client uses

e_t^{(k)}=\beta e_{t-1}^{(k)}-(1-\beta)\xi_t^{(k)},$$ then their weighted aggregate obeys $$\bar y_t=\bar g_t+\bar\xi_t+\bar e_{t-1},\qquad \bar e_t=\beta\bar e_{t-1}-(1-\beta)\bar\xi_t.$$ Consequently, **the aggregated noise follows the same EMA recursion**. For independent client innovations, its variance is determined by the aggregation weights through $a^2\sum_k p_k^2\sigma_k^2$. This gives the same noise-cancellation structure at the global update level in the analyzed synchronous setting. The Cross-Device FL experiments complement this analysis with partial client participation and non-IID partitions. We additionally extended the analysis to **conditionally unbiased mini-batch gradients**, $\widehat g_t=\nabla L(w_t)+\zeta_t$, where $\mathbb E[\zeta_t\mid\mathcal F_t]=0$ and $\mathbb E[\|\zeta_t\|^2\mid\mathcal F_t]\le V$. With the constants in our proof, stochastic gradient variance contributes an additional **$2l\eta V$** in the TIDE stationarity bound and **$l\eta V$** in the DP-SGD bound. The injected-noise contributions retain their distinct $O(\eta^2)$ and $O(\eta)$ step-size dependence. This extends the interpretation of the simplified theorem to unbiased mini-batch sampling. We also conducted experiments under the precise **inactive-clipping/fixed-LR regime**: LR $=0.3$, zero warmup, 200 epochs, three clients, and three seeds. Best-validation accuracy (%, mean $\pm$ SD) is: | Dataset | $\sigma$ | DP-SGD | TIDE | |---|---:|---:|---:| | CIFAR-10 | 0.003 | 84.25 $\pm$ 0.48 | **88.24 $\pm$ 0.57** | | CIFAR-100 | 0.003 | 58.29 $\pm$ 0.71 | **66.09 $\pm$ 0.23** | | CIFAR-10 | 0.03 | 31.66 $\pm$ 0.66 | **87.16 $\pm$ 0.54** | | CIFAR-100 | 0.03 | 4.66 $\pm$ 0.38 | **60.67 $\pm$ 0.37** | These fixed-LR results are consistent with the predicted robustness to accumulated injected noise. The main theorem provides a theoretical convergence-speed characterization through the average stationarity bound and its dependence on the iteration horizon $T$, while the experimental comparisons report model-training utility under the stated optimizer settings.
AuthorsOpenReview ↗

Response to Reviewer htnC

Q1. Gradient inversion settings in Table 1

We clarified the attack checkpoint, released object, target selection, and batch sizes in Experiment Setup and the Appendix. The single-round inversion attacks are carried out at model initialization, using the same initial sigmoid-activated ResNet-18 weights across defenses. Each observed gradient corresponds to a batch-averaged gradient from four target images; ten fixed target batches are evaluated, giving 40 target images in total. The target batches and initial weights are shared across defenses for direct comparison. We use batch size four for reconstruction because joint image recovery becomes substantially more difficult at the training batch size of 128. The training evaluations and inversion evaluations consequently use distinct batch sizes for their respective objectives.

The communication interface releases one defended batch-mean gradient per step. Per-example gradient computation, when required for clipping and aggregation, remains client-internal; the EMA correction state is also maintained locally. The attack observes the released gradient together with the model parameters and employs the corresponding gradient-matching procedure. The paper reports the batch-level noise scale σ\sigma, with σp=B σ\sigma_p=\sqrt B\,\sigma for the equivalent independently perturbed per-example representation.

Q2. High-resolution FFHQ evaluation

We added experiments on 224×224224\times224 FFHQ-derived smile classification. A fixed pseudo-label manifest defines the smile labels and consistent train/validation splits; the 200-epoch training protocol uses batch size 64, RandomResizedCrop$(0.75,1.0)$, and random horizontal flips. Results are best validation accuracy (%), with mean ±\pm sample SD over three seeds for the defended methods:

Defenseσ=0.003\sigma=0.003σ=0.03\sigma=0.03
No defense (one seed)93.85—
DP-SGD94.03 ±\pm 0.2987.93 ±\pm 0.89
TIDE94.13 ±\pm 0.0394.18 ±\pm 0.19

At σ=0.003\sigma=0.003, TIDE and DP-SGD have comparable utility. Increasing the noise scale tenfold gives 94.18% validation accuracy for TIDE and 87.93% for DP-SGD, a 6.25-point difference. This supports the transfer of TIDE's training-utility robustness to higher-resolution facial images. We focus the FFHQ extension on training utility because full-resolution image reconstruction introduces a considerably more expensive attack-optimization problem and can reduce the discriminative value of cross-defense SSIM comparisons. Detailed protocol and results are now included in the FFHQ Appendix section.

Thanks for the authors' response. Most of my concerns have been resolved. I hope to keep my already positive assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.