acceptodds
Under review as a conference paper at ICLR 2027

TIGER: Inverting Multimodal Transformer Gradients via Embedding-Subspace Distance Optimization

Abstract

Federated learning (FL) allows multiple clients to jointly train a shared model by sending gradient updates to a central server while keeping their raw inputs local. Gradient inversion attacks, however, have shown that these updates can leak enough information to reconstruct client inputs. For transformers, though, existing attacks fall short in some of the following ways: they scale poorly to realistic model sizes, larger inputs, or non-causal attention; they require an enumerable vocabulary of token embeddings, preventing the recovery of continuous non-textual inputs; or they are brittle under numerical noise, e.g., from quantization or Differential Privacy (DP). In this paper, we introduce TIGER, a continuous gradient inversion attack that exploits the natural low-rank structure of attention gradients. Its novel objective aligns the features of dummy inputs across layers with the low-rank gradient subspaces, resulting in a single framework that recovers text from both causal and bidirectional attention models and, for the first time, multimodal inputs. On settings featuring bidirectional attention, TIGER substantially improves reconstruction quality over prior work and can recover images at roughly 720p resolution in the multimodal setting. On causal models, TIGER is significantly more robust than prior attacks, enabling accurate reconstructions even under quantization and DP-style defenses. Overall, TIGER uncovers further privacy vulnerabilities in LLMs, underscoring the need for stronger defenses in deployed FL systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
federated learning, gradient inversion, gradient leakage, llm, vllm, multimodal, privacy

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_aZZiOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a novel gradient inversion attack in an honest-but-curious setting, that leverages low-rank structure of attention layer gradients, to recover input data. The attack optimizes against gradient-induced subspaces rather than matching full gradients. The proposed attack can be applied to both image and text modalities, in contrast to state-of-the-art work, DAGER, while showing robustness against existing defense mechanisms. The paper extends the attack to both causal and bidirectional attention mechanisms, while handing different challenges such as preventing recovery of duplicate tokens and sequences. The effectiveness of the proposed attack has been demonstrated through extensive experiments.

Critical strengths
  1. The paper is well-motivated and clearly written. The related works discussed are sufficient.

  2. The paper addresses a critical limitation of state-of-the-art work, DAGER (Petrov et al., 2024). DAGER is an exact gradient inversion attack, that can only be applied to text recovery, since it relies on a predefined vocabulary. The proposed attack, TIGER can be applied to multiple modalities.

  3. TIGER is more robust than DAGER against the existing defense mechanisms like differential privacy and quantization as demonstrated in the experiments.

  4. It considers an honest-but-curious adversary model, hence does not maliciously modify the protocol like Fowl et al., 2022, Fowl et al., 2023, Sami et al., 2026. Hence, the proposed attack can be successfully executed without raising any suspicion across the users.

Critical weaknesses
  1. In Corollary 4.1, the condition for a global minima is restricted in the sense that the total number of tokens must be less than or equal to (Lmax−1)d/Lmax(L_{max}-1)d/L_{max}. This restricts recovery in practical settings, where the number of training tokens can be in the order of thousands.
Questions for authors
  1. In parameter-efficient fine-tuning settings like LoRA, adapter, the adversarial server has access to fine-tuning gradients only. For example, in LoRA, the server has access to gradients of low-rank matrices A, B only (MineGrad, Sami et al., 2026). How does corollary 4.1 change under such settings? How is the attack effectiveness being impacted?

  2. When we have same first token in multiple sequences, does n’t the deduplication loss negatively impact the recovery of the same first token in the later sequences? Can you explain this further?

  3. TIGER is an optimization-based attack whereas DAGER is an exact gradient inversion attack. In decoder architectures, how does total runtime for inversion compare against that of DAGER? This is particularly important for a resource-limited adversary.

I would consider raising the score if the above questions are clarified further.

5 comments

We thank the reviewer for their constructive feedback and inquiries. We appreciate their recognition of TIGER's clarity, multimodal generality beyond text-only attacks like DAGER, and robustness against defenses under an honest-but-curious threat model. We address each concern in detail below.

R3.1. (Q1) How does LoRA training impact the theoretical considerations of Corollary 4.1 and TIGER’s practical performance?

We thank the reviewer for this question. As detailed in R3 of our Common Response, it is possible to extend TIGER to LoRA fine-tuning both theoretically and experimentally. We show that LoRA gradients retain information about the hidden-state subspaces, although their rank is limited by the LoRA rank rr. Under the same rank assumptions as our original analysis, this yields a conservative recovery condition of T≤r(L−1)/LT\leq r(L-1)/L, analogous to the original condition involving the transformer dimension dd. We further show that combining query, key, and value LoRA gradients can increase the available rank to 3r3r, extending this condition to T≤3r(L−1)/LT\leq 3r(L-1)/L. Experimentally, we evaluate TIGER on multimodal inputs using LoRA ranks 64, 128, and 256. At rank 128, TIGER achieves 97.7% image-token recovery, compared to 98.2% with full gradients. Remarkably, even at rank 64, where the number of tokens exceeds the LoRA rank, TIGER still recovers 89.3% of image tokens, although text reconstruction degrades substantially. As explained in R3, this is possible because the observed low-rank gradient subspaces preserve disproportionately more information about image tokens than text tokens, allowing TIGER to recover images even when T>rT>r. By combining Q/K/V gradients at rank 64, image-token recovery increases to 98.4%, with 100 ROUGE-1/ROUGE-2 for the question. These results demonstrate that TIGER remains effective under LoRA fine-tuning, even beyond the regime where exact hidden-state span recovery is guaranteed, and that combining gradients from multiple projections can substantially mitigate the information loss introduced by low-rank adaptation. We refer the reviewer to R3 for the full theoretical analysis, experiments, and explanation of recovery when T>rT>r.

R3.2. (W1) Is the theoretical necessity requirement imposed by Corollary 4.1 realistic and can it be alleviated?

We thank the reviewer for raising this point. We first clarify that the condition in Corollary 4.1 guarantees that the ground-truth input is the unique global minimum, but it does not restrict TIGER from finding good reconstructions, even if partial. Empirically, TIGER can also remain effective beyond the regime covered by our theory: in our LoRA experiments (R3 of the Common Response), the analogous condition T≤rT\leq r is violated at rank 64, yet TIGER still recovers 89.3% of image tokens, because the observed gradient subspaces continue to carry a strong signal about the true hidden states. Finally, when the number of observed directions is the limiting factor, this limitation can be alleviated by combining gradients from multiple projections that share the same inputs. As in R3, one can fuse the Q/K/V gradients to triple the available rank and transform the condition into T<3r(Lmax−1)/LmaxT < 3r(L_{max}-1)/L_{max}, reaching >12,000 tokens for 7B+ models.

R3.3. (Q2) How is TIGER affected by a batch with multiple samples sharing a first token, given the presence of the deduplication loss?

In the current version of the paper, the first-token deduplication loss discourages recovering the same first token more than once. When multiple ground-truth sequences share that token, TIGER can still recover them, but may instead recover the same sentence multiple times rather than identifying all distinct sequences. This limitation directly affects reconstruction, particularly at larger batch sizes.

We have addressed this issue, as detailed in R1.2 of our response to reviewer y5UA. Specifically, for each of the first two token positions, we now recover up to BB candidate continuations for each prefix, rather than selecting only one. We extend the rotated-subspace deduplication loss to these positions to encourage distinct continuations and discard duplicate candidates. This allows multiple sequences sharing the same first token to be recovered through their different second tokens. On Rotten Tomatoes at B=8B=8, this improvement alone raises R-1/R-2 from 83.2/81.1 to 90.5/89.1. We will update the paper accordingly.

R3.4. (Q3) How does TIGER’s total runtime compare to DAGER’s?

We provide a detailed comparison in R4 of our Common Response. While DAGER is substantially faster on smaller batches, its runtime becomes highly unpredictable at B=8, with 2/10 batches exceeding our 8-hour timeout due to the combinatorial explosion of its discrete search space. In contrast, TIGER completes all batches, with runtime scaling approximately linearly with batch size (14.5 minutes at B=1 to 2.11 hours at B=8) and little variation across runs. Thus, TIGER trades off a higher typical runtime for substantially more predictable scaling, while maintaining comparable GPU memory usage. We also compare our runtime against optimization-based attacks such as LAMP, where TIGER achieves from 27 to 230x lower per-initialization iteration time across encoder and decoder models, while supporting substantially more parallel initializations by avoiding expensive second-order differentiation.

I thank the authors for providing detailed responses to my questions. Specially, I liked the solution to the deduplication loss concern - "allowing multiple sequences sharing the same first token to be recovered through their different second tokens." The improvements in the results are also impressive. Regarding LoRA fine-tuning as well, the detailed experimental results are promising. However, the authors show results for ranks 64 to 256. The LoRA paper (Hu et al., 2022) also reports results with smaller ranks as well (r=1,2,4,8). Even Sami et al., 2026 reports attack performance on smaller rank values. It would be nice to see how those small rank values impact the performance. I would appreciate if the authors could comment further on this/show some results. All the remaining concerns have been addressed.

Reviewer_uSqJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores the issue of gradient privacy leakage in federated multimodal large models. The authors propose a framework for continuous gradient inversion attacks, dubbed TIGER, which is capable of reconstructing a client's private training data. The attack specifically targets causal, bidirectional Transformer architectures and can recover both text and image information. Experimental results demonstrate high-quality reconstruction of text and image data, as well as improved robustness under the evaluated additive Gaussian noise and gradient quantization settings.

Critical strengths
  1. This paper proposes an innovative approach by reformulating the discrete subspace selection problem in large-scale models as a distance optimization problem involving multi-layered embedded subspaces. The design introduces a novel gradient reconstruction scheme, offering a meaningful methodological extension for future research in this field.
  2. The method proposed in this paper accounts for the attention-based dependencies between different modalities and leverages these dependencies to reconstruct private data, offering a significant methodological reference for subsequent multimodal gradient inversion attacks.
Critical weaknesses

One of the primary contributions of this paper concerns robustness within the context of differential privacy defense mechanisms. However, the current experiments only compare the impact of noise with varying standard deviations on reconstruction performance, failing to account for the standard differential privacy workflow in federated learning. While the existing experimental setup demonstrates a certain degree of system robustness against different levels of noise, it does not follow the standard procedure for implementing differential privacy in federated learning; consequently, it cannot provide insights into reconstruction performance under varying (ε, δ) privacy guarantees.

Questions for authors

Q1. One of the primary contributions of this paper is the robustness of its gradient inversion attack against differential privacy-style defenses. However, the current experimental setup merely involves injecting noise with varying standard deviations, rather than adhering to a rigorous differential privacy framework. The authors should further examine experimental conditions that reflect real-world differentially private federated learning, and evaluate the effectiveness of reconstruction attacks under different (ε, δ) privacy guarantees in practical application scenarios. Q2. Most existing research on gradient inversion attacks in federated learning assumes that the server has access to gradient information from a single client batch. In my view, this is a strong assumption; in practical federated learning, multiple epochs are typically performed locally, and aggregation at the batch level is highly inefficient and uncommon. I wonder if the authors have considered evaluating the reconstruction performance when using gradients aggregated over multiple batches or an entire epoch.

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer uSqJ

We appreciate the reviewer's careful reading of our paper and constructive suggestions. We are glad that they recognized TIGER's novel approach as a meaningful methodological contribution for future research, highlighted its potential to inform subsequent multimodal gradient inversion attacks, and appreciated the high-quality text and image reconstructions. Below, we provide further details and address the points raised in the review.

R2.1. (W1, Q1) How does TIGER perform under a full DP-SGD setting, including gradient clipping?

As detailed in R1 of our Common Response, we extend our evaluation to full DP-SGD in the encoder setting, including per-sample gradient norm clipping and Gaussian noise, and report the resulting privacy parameters and signal-to-noise ratios (SNR). Our additional experiments on BERT with batch size B=4B=4 show that TIGER remains robust to gradient norm clipping, with only minor differences in reconstruction quality compared to the corresponding unclipped settings. For example, at σ=10−4\sigma=10^{-4} (median SNR ≈5.8\approx 5.8), TIGER achieves R-1 scores of 76.4–79.0 under different clipping thresholds, compared to 80.7 without clipping. Even at σ=3⋅10−4\sigma=3\cdot10^{-4} (median SNR ≈1.9\approx 1.9), TIGER maintains R-1 scores above 60. Overall, gradient norm clipping provides limited additional protection against TIGER at the evaluated noise levels.

R2.2. (Q2) How does TIGER behave under widely deployed FL algorithms, such as FedAvg and Secure Aggregation?

TIGER remains applicable to both FedAvg and Secure Aggregation (SecAgg), as demonstrated by our theoretical analysis and additional experiments in R2 of our Common Response. We show that gradients aggregated over multiple local steps and clients retain the subspace structure exploited by TIGER, exactly for the first transformer block and approximately for deeper blocks under sufficiently small learning rates. Our experiments on BERT with batch size B=8B=8 demonstrate that TIGER maintains R-1 scores above 97% across 2–20 local epochs, 2–8 mini-batches per epoch, and 1–4 securely aggregated clients. Reconstruction quality decreases primarily at unusually large local learning rates, where later-layer hidden-state drift becomes significant. These results demonstrate that TIGER remains effective under more realistic federated learning configurations beyond single-batch gradient sharing.

Thank you for the detailed response. My concerns have been adequately addressed, and I will maintain my positive score.

Reviewer_y5UAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TIGER, an honest-but-curious gradient inversion attack on transformers. It builds on the observation, used by DAGER, that the query-projection weight gradient reveals the row span SlS^l of each layer's hidden states when T<dT<d.

DAGER uses this span as a discrete, thresholded membership test over vocabulary candidates. TIGER instead turns it into a continuous objective: it optimizes input embeddings so that their normalized hidden states in the first Lmax⁡L_{\max} layers lie in the observed subspaces.

The method has three variants:

  • Causal models: tokens are recovered sequentially. A first-token deduplication term and a calibrated "snap gap" stopping rule remove the need to know sequence lengths.
  • Bidirectional attention: tokens are optimized jointly. A backward span loss prevents collapse to repeated tokens, and continuous optimization alternates with greedy discrete reordering.
  • Multimodal inputs: the two modes are interleaved, with image pixels optimized through the known vision encoder under a TV prior.

On the theory side, a dimension-counting argument gives a necessary condition for local stability, (Lmax⁡−1)(d−T)≥T(L_{\max}-1)(d-T)\ge T, and a generic identifiability result is proved for MLPs.

Experiments cover Gemma-3, Llama-3.1, Gemma-4 (multimodal), BERT and EmbeddingGemma. TIGER improves over DAGER and LAMP on encoders, matches DAGER on clean causal models, stays robust under additive Gaussian gradient noise and BF16 where DAGER collapses, and recovers images with legible PII.

Critical strengths
  1. Simple and unifying idea. Replacing DAGER's hard span-membership test with a continuous span-distance objective is natural and well motivated. It removes the finite-vocabulary requirement, which makes this, to my knowledge, the first gradient inversion attack that recovers both text and continuous image inputs from multimodal LLMs. It also handles RoPE encoders, where DAGER is intractable.

  2. Robustness to gradient perturbation. Under additive noise and BF16 quantization, DAGER drops to near-zero ROUGE-2 while TIGER still recovers substantial content (e.g., R-1 72.6 at σ=10−3\sigma=10^{-3}, B=1B=1). This is a practically relevant finding: low-rank attacks that rely on exact arithmetic may understate the real leakage.

  3. Theory that informs design. The constraint-counting condition explains why a single probed layer cannot work and why returns diminish with Lmax⁡L_{\max}, which matches the ablation in Fig. 5. The interpretation of ℓback\ell_{\text{back}} as a chordal distance between subspaces of different dimensions (App. G) cleanly justifies optimizing with unobserved tokens.

Critical weaknesses
  1. The defense evaluation does not support the "DP-style" claims. The noise is i.i.d. Gaussian added to unclipped batch gradients. There is no per-sample clipping, no noise calibrated to sensitivity, and no reported (ε,δ)(\varepsilon,\delta), so this is not DP-SGD and the results say little about formally private training. Robustness also degrades quickly beyond B=1B=1: at σ=5×10−5\sigma=5\times10^{-5}, R-1 is already about 50/42/38 for B=2/4/8B=2/4/8. The upper noise level is justified by a single utility experiment (AdamW, lr 10−610^{-6}), where Adam's normalization makes "utility-preserving" noise scales hard to interpret. Other standard defenses, such as secure aggregation, multi-step FedAvg updates, and compression, are not evaluated.

  2. Limited scale, and no clean-setting gain on causal models. The attack is fundamentally bounded by T≲(1−1/Lmax⁡)dT \lesssim (1-1/L_{\max})d, and the experiments use short sequences (about 12–26 tokens) and small batches (at most 8 for decoders). On clean causal models TIGER is on par with or below DAGER, e.g., worse at every batch size on Rotten Tomatoes. The main advantage therefore rests on the perturbed setting, which brings weakness 1 back into focus. The multimodal evaluation is small (20 images) and relies partly on an LLM judge.

  3. Gap between theory and the claimed contributions. For transformers, only a necessary condition for local stability is shown. The sufficiency/identifiability result relies on rows propagating independently, which attention violates. There is no analysis of the optimization (non-convexity, which the method handles with 500 restarts and 10 stages) or of stability under noise, even though noise robustness is the paper's central empirical claim.

Questions for authors
  1. Can you evaluate under actual DP-SGD, with per-sample clipping and noise calibrated to the clipping norm, and report the corresponding ε\varepsilon? Equivalently, how does performance depend on the signal-to-noise ratio relative to the gradient norm, rather than on absolute σ\sigma?

  2. How does TIGER behave under realistic FL updates: multiple local steps (FedAvg), LoRA fine-tuning (where only the adapter gradients are shared), or secure aggregation over multiple clients?

  3. What is the wall-clock cost per batch compared with DAGER? With 500 initializations × 3000 steps per token in the causal mode, the efficiency claim in the introduction needs quantitative support in the main text.

  4. Since most causal failures stem from wrong first tokens (App. H.1), have you tried the hybrid you suggest, using DAGER for first tokens and TIGER for the rest? Does it close the gap with DAGER in the clean setting while keeping robustness under noise?

  5. The bidirectional and image attacks assume known sequence lengths or number of image tokens. How sensitive is recovery when these are misspecified? In the multimodal case, how do errors in the recovered text prefix propagate into the image reconstruction?

5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer y5UA

We thank the reviewer for their thoughtful and constructive feedback. We are glad that they found the core idea simple and unifying, the attack's robustness to gradient perturbations practically relevant, and our theoretical analysis informative for the method's design. We address their questions and concerns in detail below.

R1.1. (W1, Q1) How does TIGER perform under a full DP-SGD setting, including gradient clipping? Can you report SNR and evaluate under practical clipping values?

We find that TIGER is highly robust to gradient norm clipping and Gaussian noise under full DP-SGD, as detailed in R1 of our Common Response. To evaluate practical clipping thresholds, we set the clipping norm CC to the 75th, 90th, and 95th percentiles of per-sample gradient norms. Our experiments in the encoder setting on BERT B=4B=4 show that clipping has only a minor impact on reconstruction quality. At σ=10−4\sigma=10^{-4} (SNR ≈5.8\approx 5.8), TIGER achieves R-1 scores of 76.4–79.0 across these thresholds, compared to 80.7 without clipping. Even at σ=3⋅10−4\sigma=3\cdot10^{-4} (SNR ≈1.9\approx 1.9), TIGER maintains R-1 scores above 60. Overall, these results show that gradient norm clipping provides limited additional protection at the evaluated noise levels.

R1.2. (W2.1) Does TIGER always underperform against DAGER in non-defended decoder settings?

We revisited our evaluation against DAGER in the undefended decoder setting and made several relatively minor modifications to TIGER. Our updated experiments on Gemma-3-4B show that TIGER can outperform DAGER across all evaluated batch sizes, addressing the reviewer's concern about our initial results.

First, we fixed a numerical stability issue in our gradient computation during optimization, which had caused unstable gradients and reduced reconstruction accuracy by several percentage points. We also improved our gradient-rank estimation and used stronger deduplication for B=8B=8, setting λdedup=0.05\lambda_{\mathrm{dedup}}=0.05.

Further investigation identified two additional sources of reconstruction errors:

Sequence termination. TIGER occasionally truncated otherwise correctly recovered sequences too early. To address this, we extend the snap gap metric with a global termination procedure. Rather than truncating each sequence independently when it first exceeds the threshold, we truncate the sequences by minimizing the aggregate snap-gap score, subject to the total number of recovered tokens equalling the observed gradient rank. This reduces premature truncation by accounting for the total number of tokens in the batch.

First-token recovery. As the reviewer points out, TIGER can struggle when multiple sequences share the same first token. To address this, instead of recovering a single continuation for each partial sequence, we recover up to BB candidate continuations and discard duplicates. Sequences sharing a first token are then recovered as separate continuations of that token. We apply this only at the first two token positions, where such collisions occur, by extending the rotated-subspace loss with coefficient λ=0.005\lambda=0.005. This only affects our results at B=8B=8, as no repeated first tokens occur in our sampled batches at smaller batch sizes. Even then, such collisions are relatively infrequent, but resolving them substantially improves reconstruction accuracy, as entire sequences are otherwise missed.

Our updated Rotten Tomatoes results (R1/R2) are summarized below, alongside the original results from Table 3 of our paper.

MethodB=1B=2B=4B=8
DAGER98.1 / 98.095.0 / 94.695.4 / 94.672.9 / 71.9*
TIGER (original)94.4 / 94.189.4 / 89.087.4 / 86.650.4 / 45.6
TIGER (refined optimization, local termination)96.8 / 96.795.0 / 95.091.5 / 90.780.7 / 78.9
TIGER (+ global termination)99.4 / 99.499.5 / 99.497.4 / 96.783.2 / 81.1
TIGER (+ second-token recovery)99.4 / 99.499.5 / 99.497.4 / 96.790.5 / 89.1

For DAGER at B=8B=8, 2/10 samples exceeded the 8-hour per-batch timeout (4× TIGER's runtime) due to combinatorial explosion. As DAGER would not terminate within a reasonable timeframe, they were counted as zeros in the reported averages.

The improvements are strongest at B=8B=8, where TIGER's R1/R2 increases from 50.4/45.6 in the original paper to 90.5/89.1. Resolving the remaining first-token ambiguities alone improves reconstruction from 83.2/81.1 to 90.5/89.1 under global termination. For B≤4B\leq 4, no repeated first tokens occur in our sampled batches, so this modification does not affect the results.

Overall, these improvements demonstrate that TIGER's earlier underperformance was not inherent to the continuous optimization, and it can outperform DAGER even in the undefended setting, while retaining its advantages in perturbed settings. We will incorporate these improvements and updated results across all experiments into the next revision.

R1.3. (W2.2) Is the multimodal experiment sample size a significant issue?

We acknowledge that a larger multimodal evaluation would be desirable, but we believe our experiments have substantial scale and computational difficulty relative to prior gradient inversion work. TIGER reconstructs images that are several times larger than those considered in prior work (256x256) [1], using multimodal models that are themselves several times larger. To our knowledge, these settings are far beyond the practical capabilities of existing gradient leakage attacks, with the exception of DAGER, which is restricted to discrete text reconstruction and cannot recover continuous image inputs. Even with TIGER's substantially improved computational efficiency, reconstructing images at this scale requires considerable GPU resources, making evaluations over hundreds of images computationally prohibitive.

Importantly, our results are consistently strong rather than being driven by a small number of successful examples, with image reconstruction accuracy generally in the 90% range. Moreover, our evaluation includes 10 document images and 10 scenery images, covering two distinct types of sensitive information: private text and identifiable faces.

Thus, while we recognize the value of a larger sample size, our evaluation already goes substantially beyond prior gradient inversion work in model scale and image size. We believe the consistently high reconstruction quality across these challenging examples provides compelling evidence of the practical privacy risks demonstrated by TIGER.

R1.4. (W2.3) Is the conducted LLM-as-judge evaluation in the multimodal setting reliable?

We appreciate the reviewer's concern regarding the reliability of LLM-based evaluation, particularly when assessing sensitive information in reconstructed images. We would like to point out that, in addition to our LLM-as-a-judge evaluation, we have also reported in our paper standard reconstruction metrics used in prior gradient leakage work, including PSNR and SSIM for images and ROUGE-1/ROUGE-2 for reconstructed questions and answers. However, in our view, these metrics do not fully capture the privacy implications of our attack, as not all parts of an image contain equally sensitive information. For instance, they treat reconstruction errors in a bank card number and an ordinary background object equally, despite their vastly different privacy implications. To this end, we introduce an additional privacy-focused evaluation, which, to our knowledge, has not been systematically explored in prior gradient inversion work.

Specifically, we consider privacy leakage of text and faces within the images. Our private-text evaluation closely resembles a realistic PII extraction attack: the LLM is given only the reconstructed image and asked to extract the private information it contains, which is then compared against the ground-truth information extracted from the original image. For faces, the evaluation is necessarily more subjective and may be influenced by LLM biases, for which we employ five-sample majority voting. Furthermore, as discussed in the paper, our manual inspection of a large number of face judgments suggested that the LLM judge occasionally underestimates facial reconstruction quality, rather than overestimating it.

To assess this more systematically, we conducted an additional human study in which two independent annotators evaluated both the textual and face reconstructions using the same criteria as our LLM-as-a-judge. The results are summarized below, comparing their average scores against those obtained using Gemini 3.1 Pro (as in the original paper):

NTGemini DocHuman DocGemini FaceHuman Face
7073.875.190.899.5
28066.264.896.7100.0

The average human scores for private-text recovery are very close to those obtained using the LLM judge, differing by only 1.3 and 1.4 percentage points for the two settings. For faces, human annotators also consistently assign somewhat higher scores. Overall, these results support the reliability of our LLM-as-a-judge evaluation, while reinforcing our central finding that TIGER can recover substantial sensitive information from reconstructed images. We will include the human evaluation results and their comparison with the LLM judge in the revised paper.

R1.5. (W3.1) What are the theoretical implications for transformers and their relevance?

We would like to clarify the role of our theory. As TIGER is primarily an attack paper, its central contributions are algorithmic and empirical. Our analysis in App. F supports these contributions by explaining when and why subspace-based inversion can work. It is not intended as a complete theory of transformer invertibility.

We emphasize that prior optimization-based gradient inversion attacks on transformers (e.g. LAMP) provide no recovery guarantees, while DAGER's guarantees rely on a finite vocabulary and do not apply to continuous inputs such as images. A sufficiency result for full transformers would require proving that deep attention networks with their actual (non-injective) activations are generically injective on continuous inputs, which is beyond our scope.

Within this scope, our analysis has direct implications for transformers. (1) Theorem F.1 places no restriction on the network blocks, including rows propagating jointly, so Corollary 4.1 applies to transformers exactly. The limit predicts that a single probed layer cannot work and that returns diminish with LmaxL_{max}, and our ablation in Fig. 5 confirms this. (2) Theorem F.2 shows that, when rows propagate independently, the same constraint count is also sufficient for identifiability, which attention violates by coupling the rows. However, our causal attack recovers tokens one at a time, and only a single row is optimized. The coupling between unknown rows therefore does not arise, which brings the causal attack much closer to the setting of Theorem F.2. Extending the genericity argument from MLP layers to attention blocks is a natural, but likely challenging, direction for future work.

R1.6. (W3.2) Why is there no analysis of the optimization, namely its non-convexity and its robustness under noise?

We agree that these are interesting theoretical questions, but we respectfully note that they lie well beyond the scope of a gradient inversion paper and more broadly remain open in ML Theory. In particular, convergence guarantees for non-convex optimization through deep transformers are currently out of reach even for standard training. Existing results are restricted to simplified settings such as shallow or linear-attention models [2,3,4], and no existing optimization-based gradient inversion attack on realistic architectures provides such an analysis. In practice, we found we could not avoid non-convexity in the encoder setting through multiple initialisations alone. We instead address it by additionally interleaving continuous optimization and discrete reordering, rising R1/R2 from 60.9/19.0 to 94.0/84.3 after 5 stages and 99.6/97.9 after 10.

On robustness under noise, TIGER compares subspaces rather than testing exact membership, and classical perturbation theory guarantees that subspaces move only gradually under noise. If the observed gradient is G^=G+E\hat{G} = G + E, where the clean gradient GG has rank rr and EE is the noise, [5] gives us

∥sin⁡Θ(U^r,Ur)∥2≤2∥E∥2σr(G),\|\sin\Theta(\hat{U}_r, U_r)\|_2 \le \frac{2\|E\|_2}{\sigma_r(G)}, where UrU_r and U^r\hat{U}_r span the top-rr singular subspaces of GG and G^\hat{G}, and σr(G)\sigma_r(G) is the smallest nonzero singular value of GG. The subspace TIGER matches against thus shifts smoothly, causing gradual instead of sharp performance decline, as confirmed by our DP-SGD experiments (R1 of our Common Response). We will add this discussion to the revised paper.

R1.7. (Q2.1) How does TIGER perform when considering multiple local model updates or those over multiple clients?

TIGER remains effective under both multiple local model updates (FedAvg) and aggregation across multiple clients (Secure Aggregation, SecAgg). As detailed in R2 of our Common Response, the aggregated weight updates retain the gradient-induced subspace information exploited by TIGER. For the first transformer block, this relationship holds exactly, while for deeper blocks it holds approximately when the model parameters change only moderately between local updates. Under SecAgg, the sum of client updates can similarly be treated as a larger effective batch.

We evaluate these settings on BERT B=8 using SST-2, varying the number of local epochs (2–20), mini-batches (1–8), and aggregated clients (1–4). TIGER maintains R1 of at least 97.3 across all these settings, demonstrating that reconstruction remains highly effective even after multiple local updates or aggregation across clients. Performance degrades primarily at large learning rates, dropping to 78.6 R1 at η=10−3\eta=10^{-3}, where changes in hidden states across local steps weaken the subspace approximation.

These results show that TIGER is not restricted to single-batch gradient observations and remains effective under many realistic federated learning setups. We point the reviewer to R2 for the theoretical explanation, and full experimental setup and results.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Common Rebuttal Response

We thank the reviewers for their valuable feedback, and appreciate them finding our work novel and well-motivated [y5UA, uSqJ, aZZi], our continuous reformulation simple and natural [y5UA, uSqJ], as well as appreciating the impact of the results in discovering multimodal vulnerabilities [y5UA, uSqJ, aZZi], and robustifying text attacks to perturbations [y5UA, aZZi]. Below we address shared questions between reviewers.

R1. How does TIGER perform under a full DP-SGD setting, including gradient clipping? [y5UA, uSqJ]

\newcommand{\E}[1]{10^{-#1}}\newcommand{\T}[1]{3\cdot\E{#1}}Beyond additive Gaussian noise, TIGER also handles the gradient clipping of full DP-SGD. We extend our results to the encoder setting and performed full DP-SGD attacks on the same setting. We vary the noise σ∈[\E5,\E3]\sigma \in [\E5, \E3] and clip per-sample gradient norms clipping to to CC, set to the 95th, 90th or 75th percentile of per-sentence gradient norms. We report results with 20 optimization stages on SST-2, BERT, B=4B=4, with the gradient signal-to-noise ratio (SNR) and the resulting DP ε\varepsilon at $ \delta=10^{-5} $:

σ\sigmaCCClipped (/40)R-1R-2Tokens (/454)SNR median [10%, 90%]ε\varepsilon
00none0100.0100.0454∞\infty∞\infty
\E5\E5none099.497.241666.5 [44.3, 90.1]∞\infty
\T5\T5none096.290.835722.2 [14.8, 30.0]∞\infty
\E4\E4none080.770.62106.6 [4.4, 9.0]∞\infty
\E4\E421.20276.463.71755.8 [4.4, 9.0]1.4×1091.4\times10^{9}
10−410^{-4}19.91477.664.81685.8 [4.4, 9.0]1.2×1091.2\times10^{9}
\E4\E418.421079.066.51925.7 [4.2, 8.7]1.1×1091.1\times10^{9}
\T4\T4none062.343.51142.2 [1.5, 3.0]∞\infty
\T4\T421.20263.344.71221.9 [1.5, 3.0]1.6×1081.6\times10^{8}
\T4\T419.91461.241.51151.9 [1.5, 3.0]1.4×1081.4\times10^{8}
\T4\T418.421062.844.21221.9 [1.4, 2.9]1.2×1081.2\times10^{8}
\E3\E3none050.724.1780.7 [0.4, 0.9]∞\infty

We observe that clipping barely affects TIGER, so our results carry over to full DP-SGD, even when the noise is comparable to the gradient. At these noise scales the formal guarantee is weak (ϵ∘108\epsilon \circ 10^8), but given the high SNR, we expect the utility-privacy tradeoff to be significant. We will include these results in the next revision of the paper.

R2. How does TIGER behave under widely deployed FL algorithms, such as FedAvg and Secure Aggregation? [y5UA, uSqJ]

TIGER can be applied directly to both settings. The weight update the server receives is simply the sum of all local updates ΔWQl=−η∑t(Zl,(t))⊤∂L∂Ql,(t)\Delta W_Q^l = -\eta \sum_{t} (Z^{l,(t)})^\top \frac{\partial \mathcal{L}}{\partial Q^{l,(t)}}, where η\eta is the local learning rate and Zl,(t)Z^{l,(t)} are the inputs to block ll at local step tt. Following the FedAvg analysis of DAGER [1, App. B.3], for the first block, Z1,(t)=Z1Z^{1,(t)} = Z^1 does not change between steps, so Theorem 3.1 holds for the set of all inputs. For deeper blocks, Zl,(t)Z^{l,(t)} changes as the weights are trained, but remains approximately constant for reasonable learning rates and numbers of local steps, so the column span of ΔWQl\Delta W_Q^l approximately matches Sl\mathcal{S}^l. Splitting the data into mini-batches does not change these spans. Under Secure Aggregation (SecAgg), the server observes the sum of these updates over CC clients. The same argument as for FedAvg holds for the first layer, while further layers behave similarly under reasonable learning rates, despite the disjoint optimisation over multiple clients.

We evaluate on BERT, SST-2, B=8B=8 (8/C8/C per client), with defaults E=10E=10 local epochs, η=\E4\eta = \E4, 8 mini-batches, and one client, varying one parameter at a time:

Varied parameterValueR-1R-2
Standard FedSGD100.0100.0
Local epochs EE298.784.2
599.284.80
10 (default)97.9783.03
2097.2881.07
Learning rate η\eta10−510^{-5}98.7884.06
5⋅10−55\cdot10^{-5}97.5081.72
5⋅10−45\cdot10^{-4}92.6177.29
10−310^{-3}78.6364.60
Mini-batches per epoch498.6583.24
297.9582.61
198.6284.05
SecAgg clients CC298.7684.31
498.1683.33

TIGER is essentially unaffected by local epochs, mini-batches, or SecAgg clients (R-1 $ \geq $ 97.3). It degrades only at large learning rates (78.6 R-1 at η=\E3\eta = \E3), where hidden states drift between local steps and the approximation above breaks. As DAGER notes, such rates are typically too high for FL convergence, and far exceed those used to fine-tune BERT (22-5⋅\E55\cdot\E5 [2]). We will include these results in the next revision.

R3. What are the theoretical and practical implications of LoRA training on the TIGER attack? [y5UA, aZZi]

We thank the reviewers for this question. TIGER remains applicable to LoRA fine-tuning, although the lower rank of the observed gradients introduces additional constraints on reconstruction.

Theoretical implications. Following the notation of Section 3.3, consider a query projection with LoRA adaptation WQl=WQ,0l+sAlBlW_Q^l = W_{Q,0}^l + sA^lB^l, where Al∈Rd×rA^l \in \mathbb{R}^{d\times r}, Bl∈Rr×dB^l \in \mathbb{R}^{r\times d}, rr is the LoRA rank, and ss is the scaling factor. The gradient with respect to the first LoRA factor is

=s(Z^l)^\top\frac{\partial\mathcal L}{\partial Q^l}(B^l)^\top.$$ Here, $\frac{\partial\mathcal L}{\partial Q^l}(B^l)^\top\in\mathbb{R}^{T\times r}$ is the projected backpropagated query gradient. As in Theorem 3.1, the factorization implies that the column span of $\frac{\partial\mathcal L}{\partial A^l}\in\mathbb{R}^{d\times r}$ is contained in the span of $Z^l$. When $T\le r$ and the projected gradient has full row rank $T$, these spans coincide. Combining this with the stability condition of Corollary 4.1, the conservative bound $T\le r(L-1)/L$ ensures that both conditions hold, under the same rank assumptions. **Fusing Q/K/V gradients.** Since the query, key, and value projections share the same input hidden states $Z^l$, we can concatenate their LoRA gradients: $$G_{\mathrm{QKV}}^l = \left[ \frac{\partial\mathcal L}{\partial A_Q^l} \middle| \frac{\partial\mathcal L}{\partial A_K^l} \middle| \frac{\partial\mathcal L}{\partial A_V^l} \right].$$ While each gradient has rank at most $r$, their concatenation can have rank up to $\min(T,3r)$. Thus, combining Q/K/V gradients can recover additional independent directions, extending the stability bound to $T\le 3r(L-1)/L$. **Experimental results.** We evaluate TIGER under LoRA fine-tuning in the paper's multimodal setting, on 6 (3 document and 3 scenery) 64-token images, each accompanied by approximately 15–30 text tokens depending on the corresponding question, producing an input of $64 < T < 128$ tokens for all examples. We evaluate a query-only LoRA attack (as in the paper) with $r\in\{64,128,256\}$, as well as $r=64$ applied jointly to the Q,K,V gradients using the fusion procedure above. The results are shown below: |Setting|Q R-1|Q R-2|Tok. acc.|PSNR|SSIM|PII doc.|PII face|A R-1|A R-2| |-|-|-|-|-|-|-|-|-|-| |Q-only, r=256|90.0|87.5|93.5|20.33|0.701|83.7|88.7|48.3|39.1| |Q-only, r=128|100.0|100.0|97.7|20.74|0.721|86.1|88.7|47.7|38.5| |Q-only, r=64|10.2|0.0|89.3|20.04|0.679|89.8|87.1|1.9|0.0| |Fused Q/K/V, r=64|100.0|100.0|98.4|20.70|0.723|76.4|88.3|46.6|39.1| |Full gradients|89.4|87.0|98.2|20.60|0.717|88.2|90.3|88.8|45.7| The results demonstrate that TIGER remains effective under LoRA fine-tuning, achieving near-perfect image reconstruction at ranks 128 and 256, comparable to full-gradient access. Furthermore, combining the query, key, and value gradients at rank 64 achieves 98.4% image-token accuracy and 100 ROUGE-1/2 for the question, consistent with the additional independent constraints provided by the three projections. **Recovery beyond the LoRA rank.** For $T>r$, the observed LoRA gradient has rank at most $r$, so the complete hidden-state span can no longer be recovered from a single gradient. Surprisingly, TIGER can still recover 89.3% of image tokens at rank 64, although text reconstruction degrades substantially. To understand this behavior, we examine how well the ground-truth hidden states align with the observed rank-64 query LoRA gradient subspaces. Specifically, we measure the normalized projection magnitude $\|P^lz\|_2/\|z\|_2$ for question, image, and answer tokens. |Layers|Question|Image|Answer| |-|-|-|-| |0–2|0.82 / 0.83|0.97 / 0.98|0.65 / 0.64| |3–5|0.74 / 0.73|0.98 / 0.99|0.59 / 0.60| |6–8|0.86 / 0.86|0.98 / 0.99|0.72 / 0.73| |9–11|0.94 / 0.97|0.95 / 0.96|0.81 / 0.82| |12–14|0.97 / 0.99|0.94 / 0.95|0.81 / 0.90| We observe that the gradients preserve substantially more information about image tokens, whose projection magnitudes remain around 0.94–0.99 across all evaluated layers, compared to question and answer tokens, particularly in earlier layers. Importantly, even though the true inputs no longer achieve zero subspace-distance loss, the observed subspaces contain directions from the true hidden-state spans. Consequently, the distance objective retains a signal about the ground-truth inputs, and enforcing alignment across multiple layers can still distinguish correct from incorrect candidates. Overall, these results show that **TIGER remains effective under LoRA fine-tuning even beyond the exact span-recovery regime**, with Q/K/V fusion further mitigating the information loss from low-rank gradients.

R4. Can you compare TIGER’s runtime/memory to that of the baselines? [y5UA, aZZi]

To provide a quantitative comparison, we benchmarked TIGER's runtime and memory usage against LAMP and DAGER on a single NVIDIA H200 GPU. We compare against: (1) optimization-based gradient-matching attacks, such as LAMP, TAG[3], and DLG[4], which reconstruct inputs by iteratively matching model gradients and require expensive second-order differentiation; and (2) DAGER, which exploits discrete combinatorial search with gradient-induced subspaces filtering.

Comparison with gradient-matching attacks. We benchmark both attacks on RottenTomatoes for B∈{1,2,4,8}B\in\{1,2,4,8\}. For each attack, we use the maximum number of initializations that can be optimized in parallel within GPU memory constraints. We report the average runtime per initialization per iteration per optimized token, measured after 100 warmup iterations, together with the maximum number of concurrent initializations. For the encoder model (EmbeddingGemma-300M, shortened to EmbGemma), TIGER likewise optimizes all tokens jointly, while for the causal decoder (Gemma-3-4B), TIGER recovers tokens sequentially and optimizes a single token per iteration.

ModelBLAMP max. initsLAMP µs/init/iter/tokenTIGER max. initsTIGER µs/init/iter/tokenSpeedup
Gemma-3-4B146,01410,20826.2230×
Gemma-3-4B234,0389,70126.7151×
Gemma-3-4B432,87213,72826.5108×
Gemma-3-4B832,3319,71427.684×
EmbGemma1250152.311,5842.4562.2×
EmbGemma2203114.35,0812.3947.7×
EmbGemma414395.52,4422.3840.0×
EmbGemma88188.01,1443.2227.3×

TIGER achieves 84–230× lower per-token runtime on Gemma-3-4B, while supporting thousands of parallel initializations compared to only 3–4 for LAMP. On EmbGemma, TIGER is 27–62× faster, while accommodating 14–46× more parallel initializations. Although LAMP spreads its per-iteration cost over more positions as B grows, its total runtime per initialization still increases by 3.2× on Gemma-3-4B and 4.8× on EmbGemma from B=1 to B=8, while the number of initializations fitting in memory decreases. This gap arises because gradient-matching attacks (LAMP, TAG, DLG) require costly second-order differentiation, whereas TIGER's subspace-distance objective needs only first-order differentiation through a few layers. As the problem is highly non-convex across both types of attacks, reliable recovery requires many initializations, which gradient-matching attacks cannot parallelize efficiently at scale, but TIGER can.

Comparison with DAGER. To compare against DAGER, we measure end-to-end reconstruction time across 10 batches, capping runtime at 8 hours per batch. We report the 10th percentile, median, and 90th percentile of runtime, along with peak GPU memory usage.

BTIGER runtime (p10 / median / p90)DAGER runtime (p10 / median / p90)Peak GPU memory
114.1 / 14.5 / 16.0 min4 / 4 / 4 s34.0 GiB
228.5 / 30.2 / 32.0 min4 / 4 / 5 s34.0 GiB
459.3 / 60.8 / 63.9 min6 / 8 / 14 s34.1 GiB
82.06 / 2.11 / 2.18 h2.3 min / 26.4 min / ~32 h*34.3 GiB

While DAGER is substantially faster when its discrete reconstruction succeeds, its runtime becomes unpredictable as batch size grows. At B=8B=8 it ranges from minutes to tens of hours, with 2/10 batches exceeding our 8-hour cap. For these 2 entries, we estimated the amount of time DAGER would take to converge if it were to complete all discrete checks. On the other hand, TIGER completes every batch, scaling roughly linearly with low variance, and achieving comparable peak memory. TIGER thus offers more predictable scaling, and is further able to handle continuous inputs and perturbed gradients, where DAGER is inapplicable or fails.

References

[1] Petrov, Ivo, et al. "Dager: Exact gradient inversion for large language models."

[2] Sun, Chi, et al. "How to fine-tune bert for text classification?."

[3] Deng, Jieren, et al. "Tag: Gradient attack on transformer-based language models."

[4] Zhu, Ligeng, Zhijian Liu, and Song Han. "Deep leakage from gradients."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.