acceptodds
Under review as a conference paper at ICLR 2027

ABG: An Adaptive Block-based Gradient Inversion Attack in Federated Learning

Abstract

Federated learning (FL) aims to protect training data by training models through gradient sharing without directly exchanging raw data. However, gradient inversion attacks (GIAs) demonstrate that shared gradients can still reveal sensitive client training data. In this paper, we proposes Adaptive Block-based Gradient inversion attack (ABG), an adaptive gradient inversion framework for high-fidelity image reconstruction. Unlike existing GIAs that rely on fixed reconstruction strategies and exhibit image-dependent reconstruction performance, ABG leverages the gradient mean to perform adaptive initialization and adaptively select the residual-layer weighting scheme. It further incorporates image encoding and decoding to reduce the pixel-level optimization space and performs block-wise gradient matching for ResNet architectures. Compared with the current SOTA approach, ABG improves PSNR by up to 5 dB while reducing reconstruction time by more than 12%. Extensive experiments across diverse settings validate the effectiveness and stability of the proposed adaptive strategy, highlighting the practical privacy risks of gradient leakage in FL.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Gradient inversion attack, data security and privacy, federated learning.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_fsASOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes a novel gradient inversion attack, which follows i) a gradient-mean guided adaptive initialization strategy, ii) block-wise gradient matching mechanism, iii) image encoding and decoding to reduce the optimization space. By adopting these techniques, it claims to improve the reconstruction stability and quality across different target images.

Critical strengths

The problem this paper studies is of significant importance in the relevant literature.

Critical weaknesses
  1. Related works are not discussed well. A recent work, SPEAR [1] proposes exact gradient inversion attack for a batch of images, leveraging low-rank structure of gradients. The paper does not cite this work and compare against this.

  2. Narrowing the search space by performing optimization in lower-dimensional space has been studied earlier [2]. This work should be discussed. An explicit explanation on whether the proposed image encoding, decoding mechanism performs better than this prior work should be included.

  3. The intuition of using a four-step gradient matching strategy across different blocks is not clear. For example, why are the gradients only from Block 4 matched in Step I, then gradients from Blocks 3 and 4 in Step II, later half of Block 2 in Step III? An explanation given is coarse-to-fine matching procedure, but even in that case, why this specific structure of block selection is adopted is not clearly explained or motivated. Why are gradients from Block 1 not used at all? Even though an ablation study in Fig. 8 is provided showing the impact of adding Block 1 gradient matching, the underlying cause has yet to be elucidated. Whether this phenomenon is specific to a particular dataset/architecture or there lies a valid underlying reason is unclear. Similarly, the intuition of using adaptive weighting of residual layers in different blocks is unclear. For example, when the gradient is less than \beta, why do we assign a weight of 0.1 to the residual layers in Block 4? These need to be well-explained.

  4. It appears that the proposed method is limited to ResNet like architectures with four blocks, as discussed throughout the paper. It is better to discuss its adaptability or potential extension to other architectures like attention layers in Transformer. Otherwise, the contribution seems much limited.

  5. The improvement in reconstruction over the baseline, TLG is marginal as observed in Figs. 2, 3.

[1] Dimitrov, D.I., Baader, M., Müller, M.N., & Vechev, M.T. (2024). SPEAR: Exact Gradient Inversion of Batches in Federated Learning. Neurips 2024.

[2] Jeon, J., Kim, J., Lee, K., Oh, S., & Ok, J. (2021). Gradient Inversion with Generative Image Prior. Neurips 2021.

Questions for authors

It is stated that x’ is initialized using the gradient mean. For a batch of images, is the entire batch initialized using the gradient mean, where the gradient is derived from a batch of images? Can you elaborate this further?

Reviewer_9DH3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates gradient inversion attacks (GIAs) in FL and propose ABG (Adaptive Block-based Gradient Inversion Attack). Authors focus on the image-dependant instability of existing GIA methods and propose and adaptive reconstruction framework. Specifically, ABG combines gradient-mean-guided initialization, image encoding and decoding, block-wise gradient matching, and adaptive residual-layer weighting. Authors evaluate ABG on quality and efficiency compared with optimization-based attacks.

Critical strengths
  • The motivation is important and clear.
  • The proposed framework is intuitive and relatively simple. The adaptive init and residual-layer weighting are interesting.
Critical weaknesses
  • The comparison with recent generative model-based gradient inversion attacks seems insufficient. Recent works have increasingly explored generative priors for gradient inversion, including GAN-based [1,2] and diffusion-based [3,4] methods. Some of these methods also focus on large batch reconstruction and robustness against gradient perturbation.
  • The experiments under defense mechanisms are not sufficiently comprehensive. In Sec 4.6, authors provide the reconstruction examples under 90% gradient pruning and Gaussian noise injection. However, the evaluation is mainly qualitative, and no quantitative comparison.
  • Moreover, consideration of various defense mechanisms should be included. Like [5-8], these methods employ different defense strategies, including perturbation, pruning and obfuscation. Since ABG relies on gradient statistics for adaptive initialization and residual-layer weighting, it is unclear whether its adaptive strategy remains effective when these statistics are modified by defense mechanisms.
  • More than any other minor points, I hope these issues will be addressed and the paper revised to improve its completeness and the coherence of the arguments.

[1] Li, Z., Zhang, J., Liu, L., & Liu, J. (2022, June). Auditing privacy defenses in federated learning via generative gradient leakage. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 10122-10132). IEEE.

[2] Fang, H., Chen, B., Wang, X., Wang, Z., & Xia, S. T. (2023, October). Gifd: A generative gradient inversion method with feature domain optimization. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 4944-4953). IEEE.

[3] Wu, L., Liu, Z., Pu, B., Wei, K., Cao, H., & Yao, S. (2025). DGGI: Deep generative gradient inversion with diffusion model. Information Fusion, 113, 102620.

[4] Liu, X., Cai, S., Zhou, Q., Guo, S., Li, R., & Lin, K. (2025, April). Mjölnir: Breaking the shield of perturbation-protected gradients via adaptive diffusion. In Proceedings of the AAAI conference on artificial intelligence (Vol. 39, No. 25, pp. 26308-26316).

[5] Sun, J., Li, A., Wang, B., Yang, H., Li, H., & Chen, Y. (2021, June). Soteria: Provable defense against privacy leakage in federated learning from representation perspective. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 9307-9315). IEEE.

[6] Xue, L., Hu, S., Zhao, R., Zhang, L. Y., Hu, S., Sun, L., & Yao, D. (2024, March). Revisiting gradient pruning: A dual realization for defending against gradient attacks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 6, pp. 6404-6412).

[7] Zhang, K., Cheng, S., Shen, G., Ribeiro, B., An, S., Chen, P. Y., ... & Li, N. (2025). Censor: Defense against gradient inversion via orthogonal subspace Bayesian sampling. arXiv preprint arXiv:2501.15718.

[8] Luo, C., Yau, D. K., & Song, Q. (2025). Svdefense: Effective defense against gradient inversion attacks via singular value decomposition. arXiv preprint arXiv:2510.03319.

Questions for authors

See the weaknesses.

Reviewer_fyoXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ABG, an adaptive block-based gradient inversion attack for reconstructing private training images in federated learning. The method introduces gradient-mean-guided initialization and an adaptive residual-layer weighting strategy to improve reconstruction stability across different images. It further combines low-resolution image optimization with progressive block-wise gradient matching and stage-specific regularization to improve reconstruction quality and efficiency. Experiments on ImageNet, CelebA, and WEB datasets demonstrate improvements over several existing gradient inversion methods, particularly TLG, in terms of reconstruction fidelity and runtime.

Critical strengths
  1. The paper addresses an important privacy concern in federated learning, and improving the stability and efficiency of gradient inversion attacks is a meaningful research direction.

  2. The proposed adaptive initialization and residual-layer weighting strategies are interesting and potentially useful for improving image-dependent reconstruction performance.

  3. The method achieves improvements in reconstruction quality and runtime over the evaluated baselines. The paper also provides experiments under different batch sizes, model architectures, and defense settings.

Critical weaknesses

1. Insufficient comparison with stronger and more diverse baselines.

The experiments mainly compare ABG with DLG, IG, HFG, and TLG. However, several relevant gradient inversion methods from different research directions are missing, such as:

  • DGGI (Deep Generative Gradient Inversion with Diffusion Model, Information Fusion 2025), which introduces diffusion priors for gradient inversion.
  • CI-Net (Generative Gradient Inversion via Over-Parameterized Networks in Federated Learning, ICCV 2023), which performs reconstruction using an over-parameterized generative network.
  • Cracking Federated Privacy: Initialization-Resilient Gradient Inversion with Fine-Grained Reconstruction (USENIX Security 2026), which introduces a coarse-to-fine reconstruction framework and directly addresses initialization sensitivity.

Although some generative methods involve additional assumptions or prior knowledge, comparisons with or discussions of these approaches would provide a more comprehensive evaluation of ABG's performance relative to existing research. In particular, the coarse-to-fine method appears closely related to the motivation of improving reconstruction stability.

2. Limited experimental scale and insufficient evaluation across architectures and datasets.

The main quantitative comparisons are conducted on only four ImageNet images, which may not be sufficient to establish the general effectiveness and stability of the proposed method. Although additional results on CelebA, WEB, ResNet18, and ResNet50 are provided, these evaluations remain limited in scope, with some settings supported mainly by qualitative examples.

Since improved reconstruction stability across different target images is a central motivation of this paper, a larger-scale quantitative evaluation would be more convincing. For example, reporting the mean, variance, and distribution of reconstruction metrics across randomly sampled images would help demonstrate whether the improvements are consistent rather than sample-dependent.

Furthermore, most experiments focus on ResNet34 and high-resolution datasets. Additional evaluations using smaller architectures (e.g., ResNet18) and standard low-resolution datasets (e.g., CIFAR-10) would be valuable, particularly because these settings are widely used in prior gradient inversion research and provide more comprehensive baseline results.

3. The overall attack framework contains multiple design choices, but the ablation studies and explanations are insufficient.

ABG combines adaptive initialization, image encoding and decoding, progressive gradient matching, residual-layer weighting, and stage-specific regularization. However, several important design choices are not sufficiently justified or independently evaluated.

  • Adaptive initialization: The proposed initialization uses 100 times the mean of non-negative gradient entries. The motivation for this particular transformation is not sufficiently explained. Is there an empirical relationship between this gradient statistic and the target image's pixel distribution, such as its mean intensity? A scatter plot or correlation analysis would help justify this design. Moreover, the current initialization ablation mainly compares against random initialization, without comparisons against fixed-color initialization or other improved initialization strategies.

  • Progressive gradient matching: The proposed method contains four matching stages, but the necessity of this particular schedule is not sufficiently validated. Figure 5 and Appendix E mainly investigate the two residual-layer weighting configurations (R4 and R4+R3), rather than the progressive matching schedule itself. A more systematic ablation comparing different numbers of stages, matching orders, and transition points would help establish the benefit of the proposed four-stage design.

  • Stage-specific regularization: Although the paper provides ablations of the regularization terms, the motivation for introducing total variation regularization only after the third transition point is not sufficiently supported. Comparisons with alternative regularization schedules would help clarify whether the proposed timing is important.

  • Residual-layer weighting: The selected residual layers are assigned a fixed weight of 0.1, but this choice lacks sufficient justification and sensitivity analysis. Although different selections of residual layers are compared, it remains unclear how sensitive reconstruction performance is to the weighting coefficient itself.

These components introduce considerable complexity into the overall framework. More comprehensive ablations would help determine which components are essential and whether a simpler configuration could achieve comparable performance.

4. Limited methodological novelty and insufficient explanation of the proposed design.

Several important components of ABG, including progressive layer-wise gradient matching, low-resolution image optimization, and regularization terms, are adapted from existing gradient inversion methods. Although the specific matching schedule differs from prior approaches, the paper provides limited analysis explaining why this particular design should be more effective.

The gradient-mean-guided initialization appears to be one of the more distinctive components. The adaptive residual-layer weighting strategy is also potentially interesting, but it essentially selects between two predefined weighting configurations using a threshold on a single gradient statistic. The underlying relationship between this statistic and the optimal weighting scheme is not sufficiently explained.

Overall, the method appears to be largely a combination of existing techniques with several heuristic modifications. Stronger empirical justification or additional analysis of the proposed adaptive mechanisms would help clarify the paper's methodological contributions.

Questions for authors
  1. Adaptive initialization: Why is the initialization specifically defined as 100 times the mean of non-negative gradient entries? Is this statistic correlated with the target image's pixel mean or other image characteristics? How does this initialization compare with fixed-color initialization and other existing initialization strategies?

  2. Progressive matching schedule: What motivates the four-stage matching schedule and the transition points at 3K, 5K, and 10K iterations? How would the performance change with fewer stages, different matching orders, or different transition points?

  3. Adaptive residual-layer weighting: Why is the reduced residual-layer weight fixed at 0.1, and how sensitive is the reconstruction performance to this value? Moreover, how is the threshold β = 0.49 determined, and does it generalize across architectures and datasets without further tuning?

  4. Regularization schedule: Why is total variation regularization introduced only after 10K iterations? Would applying it earlier, throughout optimization, or with a gradually changing coefficient produce different results?

  5. Experimental generalization: Could the authors provide more extensive quantitative evaluations on randomly sampled images, including variance or failure rates, and additional comparisons on standard settings such as ResNet18 with CIFAR-10?

Reviewer_YoYkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ABG, an iterative gradient inversion attack for ResNet models in federated learning. ABG initializes the image from the mean of the non-negative shared gradients, optimizes in a 121×121 low-resolution space, matches gradients block by block from Block4 to Block2, and uses a gradient-mean threshold (β = 0.49) to choose which residual layers receive a reduced weight. Experiments on a few ImageNet, CelebA and WEB images report higher PSNR and shorter runtime than DLG, IG, HFG and TLG.

Critical strengths

The paper targets a relevant privacy risk in federated learning, and improving reconstruction stability across target images is a reasonable goal. The method is simple, and pseudocode and code are provided.

Critical weaknesses
  1. The evaluation is far too small: the main comparison uses 4 images (Table 1), the ablations 2–3 (Tables 2 and 4), the batch-8 result one batch, and Fig. 6 one image without metrics. Mean and standard deviation over hundreds of images are needed to support the claims.
  2. The threshold β = 0.49 is tuned in Fig. 5 on the same kind of ImageNet images used for evaluation, and its selection accuracy on held-out images is never reported. It is also unclear how β relates to the gradient mean, which is stated to be on the order of 1e-3 (l. 154) but plotted as 0.1–0.6.
  3. The "adaptive initialization" is a uniform gray image whose intensity depends on one scalar (Algorithm 1), yet it is only compared with random initialization. A fixed gray initialization is needed to show that the gradient-derived value matters.
  4. The method is a set of hand-fixed, ResNet-specific heuristics (T1–T3, "latter half of Block2", weight 0.1, 121×121 resolution) with little insight into why they should transfer. Other architectures such as ViTs are not covered.
  5. Some claims are overstated: by Table 1 the runtime reduction over TLG averages 9.9% (6.3–12.1% per image), not "more than 12%". The DP result rests on one image without metrics or a privacy budget, and recent generative-prior attacks (e.g., GGL, GIFD) are not compared.
Questions for authors
  1. Can you report mean ± std of MSE, PSNR, LPIPS and SSIM over a large image set (e.g., 1,000 ImageNet validation images) for all methods?
  2. How does ABG compare with a fixed gray initialization, and how often does the β rule pick the better scheme on held-out images?
  3. How was β = 0.49 obtained, and how sensitive are the results to β and to T1–T3?
  4. Can you provide quantitative results under DP and pruning, with DP noise expressed via clipping norm and privacy budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.