- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper first analyzes the security issues of SLIM under multi-query inference and proposes the solution GateFlow based on the problems encountered by SLIM. I also agree with one observation in the paper: although some intermediate states in a single inference are not directly exposed, attackers can still combine information seen in multiple inferences and then use this information to recover the authorization transformations.
The authors formulate the problems encountered by SLIM as a linear recovery problem and verify the feasibility of transformation recovery on GPT-2-medium and Llama2-7B. Subsequently, the proposed GateFlow advances the key protection to the first-layer attention layer. By calculating attention weights within the TEE and outsourcing encrypted linear computation, it not only hides the relevant intermediate results but also preserves the ability for subsequent decoder blocks to continue execution.
Critical strengths
First, I really liked the presentation of this paper; it read very smoothly.
-
Excellent observations. The paper focuses on how information generated from multiple inferences is combined, which is actually quite common in current real-world inference scenarios. Furthermore, the paper reorganizes the two secret transformations into unknowns that can be jointly solved, which I think is the most valuable aspect of the paper.
-
The attack mechanism is well-done. GPT-2-medium uses 16–24 prompts, and Llama2-7B uses 53–88 prompts to achieve the corresponding recovery conditions, and the recovered matrix is quite close to the original matrix.
-
The defense design balances security and efficiency. GateFlow concentrates additional security processing in the first decoder block, avoiding the need for TEE at each layer. Experiments also report a latency advantage compared to layer-by-layer interaction schemes.
Critical weaknesses
-
I appreciate GateFlow's design of hiding attention weights and outputs. However, I'm a little curious: does the attacker not need to retrieve the original Q/K or each secret matrix, but only need a computational method that produces the same result? Appendix G provides a concrete example. Even with the output transformation, the authorization process can still be rearranged as:
where the input , the obfuscated output , and the authorization result are all visible to the GPU. If these transformations remain invariant across queries, an attacker can also accumulate multiple sets of observations and jointly estimate the unknown matrices. I hope the authors can answer whether this joint recovery path holds true, and what conditions the defense guarantees depend on.
-
In evaluating the effectiveness of GateFlow, Table 2 marks the attack method as GAME, but the article does not explicitly state whether this attack uses the recovery method proposed in Section 4. Therefore, I currently find it difficult to determine whether these experiments directly test GateFlow's defensive capabilities.
Questions for authors
-
Regarding the output projection protection in Appendix G, can you explain how it prevents joint recovery under multiple queries? When the secret transformation is reused, can an attacker utilize the information mentioned in Weakness 1?
-
Does the defense experiment in Table 2 include the multi-query recovery method proposed in Section 4? Please explain which observations the attacker actually used and how the recovery was performed.
-
How much performance benefit does GateFlow's cryptographic outsourcing bring compared to executing the first-layer attention directly within the TEE? I want to know how much of GateFlow's efficiency gain comes from reducing layer-by-layer interactions and how much comes from the cryptographic outsourcing design introduced in this paper.
I am mainly concerned with Weakness 1 and Question 1, which are also the main reasons affecting my score. If the authors can address my main concerns, I will increase my score.
10 comments
Comment:
Does the defense experiment in Table 2 include the multi-query recovery method proposed in Section 4? Please explain which observations the attacker actually used and how the recovery was performed.
Response:
We thank you for this insightful observation. We clarify that the original defense evaluation primarily considered the existing GAME model recovery attack, rather than directly applying our proposed multi-query transformation recovery attack to GateFlow. To address this concern, we have added a dedicated Transformation Recovery Evaluation, including the original attack against SLIM, an adapted multi-query attack against GateFlow, and an additional oracle setting. The corresponding theoretical analysis and experimental results are provided in Appendix D and Section 5 (Table 2), respectively.
1. Direct Recovery Analysis.
We first theoretically analyze why the original SLIM recovery attack cannot be directly applied to GateFlow. Specifically, the GPU can observe the input representation , obfuscated output , and authorization result , as well as the transformed query/key representations and . However, the plaintext attention weights and secret transformations remain unavailable. For equal-length queries, the authorization relationship can be expressed as
where . Unlike SLIM, the second regression term depends on hidden attention weights and cannot be directly constructed from the plaintext observations used by the original attack. Consequently, the complete regression matrix required by the original closed-form linear recovery method is unavailable, preventing its direct application to GateFlow.
2. Adapted Multi-query Recovery Attack.
To provide a more direct security evaluation, we further develop an adapted attack that preserves the original multi-query recovery objective while jointly estimating the secret transformations and hidden attention weights from GPU-visible observations. Specifically, the attacker uses and solves the following optimization problem.
We first define the estimated attention-dependent term:
\widehat S_iM_i'-\widehat S_1M_1'.$$ The corresponding reconstruction residual is $$\widehat E_i= \Delta h_i'-\Delta x_i\widehat\Pi -\widehat R_i\widehat\Pi_o^\top\widehat\Pi.$$ The attacker jointly optimizes the secret transformations and attention estimates by minimizing $$\min_{\widehat\Pi,\widehat\Pi_o,\\{\widehat S_i\\}} \sum_{i=2}^{N} \operatorname{tr}(\widehat E_i^\top\widehat E_i).$$ The estimated transformations satisfy orthogonality constraints, while the attention estimates satisfy nonnegativity, row normalization, and causal masking. This formulation does not provide the attacker with the true attention weights. We additionally introduce a **GateFlow / Oracle** setting, in which the attacker is explicitly given the true attention weights $S_i$. This enables direct linear recovery when the resulting regression matrix has full column rank, allowing us to examine the importance of the hidden attention information. **3. Experimental Results.** We evaluate all three attacks under the same query budgets for each model. The results are summarized in Table 2. **Table 2: Multi-query transformation recovery.** | Model | Scheme / Attack | Queries | $\Pi$ Rel. Error ↓ | $\Pi_{\mathrm{aux}}$ Rel. Error ↓ | $\Pi/\Pi_{\mathrm{aux}}$ Cos. ↑ | Fit Residual ↓ | |:---|:---|---:|---:|---:|:---:|---:| | GPT-2-medium | SLIM / Original | 80 | 7.57e-5 | 1.08e-4 | ≈1 / ≈1 | 7.03e-5 | | GPT-2-medium | GateFlow / Adapted | 80 | 1.221011 | 1.253892 | 0.254566 / 0.213877 | 0.780838 | | GPT-2-medium | GateFlow / Oracle | 80 | 8.57e-5 | 1.78e-4 | ≈1 / ≈1 | 4.98e-4 | | Llama-2-7B | SLIM / Original | 320 | 1.33e-4 | 1.89e-4 | ≈1 / ≈1 | 1.20e-4 | | Llama-2-7B | GateFlow / Adapted | 320 | 1.207457 | 1.341499 | 0.271024 / 0.100191 | 0.872631 | | Llama-2-7B | GateFlow / Oracle | 320 | 1.28e-4 | 9.49e-5 | ≈1 / ≈1 | 3.73e-5 | As shown above, both the original attack against SLIM and the oracle attack against GateFlow achieve near-exact transformation recovery, with relative errors below $2\times10^{-4}$. In contrast, the adapted attack against GateFlow yields substantially higher transformation errors (above 1.2) and fitting residuals of 0.781 and 0.873, indicating that it fails to accurately recover the secret transformations or fit the observed authorization relationships under the evaluated query and optimization budgets. Importantly, the oracle results demonstrate that accurate recovery becomes possible when the true attention weights are provided, highlighting the significance of protecting attention-dependent intermediate representations.Comment:
How much performance benefit does GateFlow's cryptographic outsourcing bring compared to executing the first-layer attention directly within the TEE?
Response:
We thank you for this valuable suggestion. To quantitatively evaluate the performance benefits of cryptographic attention outsourcing, we have conducted additional experiments on Gemma3-12B, with input lengths ranging from 512 to 4096 tokens. We compare GateFlow against two baselines:
- All-TEE: Executes the entire first-block attention computation inside the TEE rather than outsourcing encrypted attention operations to the GPU.
- Interactive: Retains GateFlow's first-block protection but requires repeated TEE–GPU interactions to recover obfuscated representations after GPU-side computations in subsequent decoder blocks.
The detailed results are presented below and have been incorporated into Table 6 of the revised manuscript.
Table 6: TTFT, throughput, and communication breakdown under different input lengths.
| Input Length | Scheme | TTFT (ms) ↓ | Throughput (tokens/s) ↑ | TEE Comp. (ms) ↓ | Data Transfer (ms) ↓ | Comm. Volume (MB) ↓ |
|---|---|---|---|---|---|---|
| 512 | GateFlow | 457.357 | 18.975 | 305.705 | 56.243 | 142.61 |
| Interactive | 13912.690 | 6.227 | 4169.389 | 8884.964 | 4,627.37 | |
| All-TEE | 1148.436 | 17.621 | 1088.707 | 2.868 | 3.93 | |
| 1024 | GateFlow | 778.949 | 18.079 | 538.105 | 61.978 | 159.38 |
| Interactive | 25352.070 | 5.280 | 8003.159 | 16232.378 | 9,128.90 | |
| All-TEE | 2774.187 | 14.569 | 2669.368 | 3.808 | 7.86 | |
| 2048 | GateFlow | 1831.565 | 15.504 | 1376.859 | 94.834 | 192.94 |
| Interactive | 63009.520 | 3.792 | 14710.100 | 45570.250 | 18,131.98 | |
| All-TEE | 5898.625 | 13.178 | 5684.792 | 6.151 | 15.73 | |
| 4096 | GateFlow | 5513.828 | 11.757 | 4218.834 | 220.994 | 520.09 |
| Interactive | 121675.855 | 2.644 | 29512.384 | 87245.069 | 36,398.17 | |
| All-TEE | 13175.584 | 10.798 | 12683.039 | 23.985 | 31.46 |
1. Comparison with All-TEE.
As shown above, GateFlow consistently achieves lower TTFT and higher throughput than All-TEE across all evaluated input lengths. At 4096 tokens, GateFlow reduces TTFT from 13.18 s to 5.51 s (2.39× speedup), while reducing TEE computation latency from 12.68 s to 4.22 s (3.01× reduction). Although encrypted attention outsourcing introduces additional communication, the data-transfer latency accounts for only 4.0% of total TTFT, demonstrating that the reduction in TEE computation substantially outweighs the additional communication overhead.
2. Comparison with Interactive.
GateFlow also significantly outperforms Interactive by avoiding repeated TEE–GPU interactions in subsequent decoder blocks. At 4096 tokens, GateFlow achieves a 22.1× TTFT speedup (5.51 s vs. 121.68 s) and 4.45× higher throughput (11.757 vs. 2.644 tokens/s). In particular, the data-transfer latency decreases from 87.25 s to only 0.22 s, highlighting the effectiveness of transformation propagation in reducing communication overhead.
Thank you for your thoughtful review and positive assessment of our work! We appreciate your recognition of our multi-query transformation recovery analysis, attack methodology, and GateFlow's security–efficiency design. We have carefully considered all the concerns you raised and addressed them through additional theoretical analyses and experiments. Our detailed responses are provided below.
Comment:
Response:
We thank you for this insightful observation. We have carefully investigated the potential joint recovery attack and introduced an effective protocol-level mitigation based on fixed-BOS initialization, which prevents attackers from accumulating independent linear constraints through the initial-token degeneracy. The corresponding analysis and security enhancement have been added to Appendix G.
1. Initial-Position Degeneracy and Joint Recovery.
In general, GateFlow's authorization process satisfies
hi′=(xi+SiMi′Πo⊤+bo)Π,
where the attention weights Si are not directly exposed to the GPU. However, for the first token of a new inference session, only one key is available, resulting in the deterministic attention weight S=[1]. Consequently, the authorization relationship degenerates to
h′=xΠ+M′(Πo⊤Π)+boΠ.
As you correctly observe, repeated sessions with different initial tokens may expose sufficient independent constraints to jointly estimate the unknown transformations, even without recovering the original Q/K representations.
2. Fixed-BOS Initialization.
To prevent an adversary from accumulating diverse initial-position observations, we introduce a simple protocol-level mitigation for models supporting a beginning-of-sequence (BOS) token. Every new session must start with the same BOS token at a fixed initial position:
[BOS,t1,…,tL].
This is consistent with the bos=True prompt encoding used in the Llama reference implementation. The trusted request handler must enforce the BOS token, its position, and the associated attention-mask and cache initialization, rather than relying solely on untrusted preprocessing.
Under fixed model parameters and transformations, the deterministic initial-position computation yields the same noiseless observations across sessions:
$ x_{i,1}=x_{\mathrm{BOS}}, $
Mi,1=MBOS,
hi,1=hBOS.
Therefore, repeatedly collecting the first-position observations only duplicates the same row of the linear system, rather than providing additional independent constraints. Meanwhile, user-controlled tokens occur after the BOS token and are processed with preceding context. This mitigation prevents rank accumulation through varying the initial token and addresses the identified joint recovery vulnerability.