acceptodds
Under review as a conference paper at ICLR 2027

Beyond Static Masks: Attacking and Defending TEE-Assisted LLM Inference

Abstract

TEE–GPU collaborative inference enables efficient deployment of proprietary large language models (LLMs) by combining trusted CPU execution with untrusted GPU acceleration. Recent approaches, such as SLIM, reduce TEE–GPU interaction through transformed-domain propagation across decoder blocks. However, we reveal that this propagation mechanism introduces a transformation identifiability vulnerability under multi-query inference. By exploiting reusable linear relationships exposed during authorization, an adversary can recover equivalent secret transformations and compromise the protection without accessing the original secrets. To address this issue, we propose GateFlow, a secure authorization propagation framework that redesigns the initial decoder execution while preserving efficient transformed-domain propagation. GateFlow prevents recoverable intermediate representations during authorization and enables subsequent decoder blocks to inherit the propagation mechanism without additional TEE involvement. Extensive experiments validate the effectiveness of the proposed attack and demonstrate that GateFlow effectively mitigates transformation recovery attacks with practical inference overhead.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model, Trusted Execution Environment, Model Obfuscation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_U8x5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper first analyzes the security issues of SLIM under multi-query inference and proposes the solution GateFlow based on the problems encountered by SLIM. I also agree with one observation in the paper: although some intermediate states in a single inference are not directly exposed, attackers can still combine information seen in multiple inferences and then use this information to recover the authorization transformations.

The authors formulate the problems encountered by SLIM as a linear recovery problem and verify the feasibility of transformation recovery on GPT-2-medium and Llama2-7B. Subsequently, the proposed GateFlow advances the key protection to the first-layer attention layer. By calculating attention weights within the TEE and outsourcing encrypted linear computation, it not only hides the relevant intermediate results but also preserves the ability for subsequent decoder blocks to continue execution.

Critical strengths

First, I really liked the presentation of this paper; it read very smoothly.

  1. Excellent observations. The paper focuses on how information generated from multiple inferences is combined, which is actually quite common in current real-world inference scenarios. Furthermore, the paper reorganizes the two secret transformations into unknowns that can be jointly solved, which I think is the most valuable aspect of the paper.

  2. The attack mechanism is well-done. GPT-2-medium uses 16–24 prompts, and Llama2-7B uses 53–88 prompts to achieve the corresponding recovery conditions, and the recovered matrix is quite close to the original matrix.

  3. The defense design balances security and efficiency. GateFlow concentrates additional security processing in the first decoder block, avoiding the need for TEE at each layer. Experiments also report a latency advantage compared to layer-by-layer interaction schemes.

Critical weaknesses
  1. I appreciate GateFlow's design of hiding attention weights and outputs. However, I'm a little curious: does the attacker not need to retrieve the original Q/K or each secret matrix, but only need a computational method that produces the same result? Appendix G provides a concrete example. Even with the output transformation, the authorization process can still be rearranged as:

    h′=xΠ+M′(Πo⊤Π)+boΠ.h' = x\Pi + M'(\Pi_o^\top\Pi) + b_o\Pi.

    where the input xx, the obfuscated output M′M', and the authorization result h′h' are all visible to the GPU. If these transformations remain invariant across queries, an attacker can also accumulate multiple sets of observations and jointly estimate the unknown matrices. I hope the authors can answer whether this joint recovery path holds true, and what conditions the defense guarantees depend on.

  2. In evaluating the effectiveness of GateFlow, Table 2 marks the attack method as GAME, but the article does not explicitly state whether this attack uses the recovery method proposed in Section 4. Therefore, I currently find it difficult to determine whether these experiments directly test GateFlow's defensive capabilities.

Questions for authors
  1. Regarding the output projection protection in Appendix G, can you explain how it prevents joint recovery under multiple queries? When the secret transformation is reused, can an attacker utilize the information mentioned in Weakness 1?

  2. Does the defense experiment in Table 2 include the multi-query recovery method proposed in Section 4? Please explain which observations the attacker actually used and how the recovery was performed.

  3. How much performance benefit does GateFlow's cryptographic outsourcing bring compared to executing the first-layer attention directly within the TEE? I want to know how much of GateFlow's efficiency gain comes from reducing layer-by-layer interactions and how much comes from the cryptographic outsourcing design introduced in this paper.

I am mainly concerned with Weakness 1 and Question 1, which are also the main reasons affecting my score. If the authors can address my main concerns, I will increase my score.

10 comments

Thank you for your thoughtful review and positive assessment of our work! We appreciate your recognition of our multi-query transformation recovery analysis, attack methodology, and GateFlow's security–efficiency design. We have carefully considered all the concerns you raised and addressed them through additional theoretical analyses and experiments. Our detailed responses are provided below.

Comment:

Regarding the output projection protection in Appendix G, can you explain how it prevents joint recovery under multiple queries? When the secret transformation is reused, can an attacker utilize the information mentioned in Weakness 1?

Response:

We thank you for this insightful observation. We have carefully investigated the potential joint recovery attack and introduced an effective protocol-level mitigation based on fixed-BOS initialization, which prevents attackers from accumulating independent linear constraints through the initial-token degeneracy. The corresponding analysis and security enhancement have been added to Appendix G.

1. Initial-Position Degeneracy and Joint Recovery.

In general, GateFlow's authorization process satisfies

hi′=(xi+SiMi′Πo⊤+bo)Π,h_i'=(x_i+S_iM_i'\Pi_o^\top+b_o)\Pi,

where the attention weights SiS_i are not directly exposed to the GPU. However, for the first token of a new inference session, only one key is available, resulting in the deterministic attention weight S=[1]S=[1]. Consequently, the authorization relationship degenerates to

h′=xΠ+M′(Πo⊤Π)+boΠ.h'=x\Pi+M'(\Pi_o^\top\Pi)+b_o\Pi.

As you correctly observe, repeated sessions with different initial tokens may expose sufficient independent constraints to jointly estimate the unknown transformations, even without recovering the original Q/K representations.

2. Fixed-BOS Initialization.

To prevent an adversary from accumulating diverse initial-position observations, we introduce a simple protocol-level mitigation for models supporting a beginning-of-sequence (BOS) token. Every new session must start with the same BOS token at a fixed initial position:

[BOS,t1,…,tL].[\mathrm{BOS},t_1,\ldots,t_L].

This is consistent with the bos=True\texttt{bos=True} prompt encoding used in the Llama reference implementation. The trusted request handler must enforce the BOS token, its position, and the associated attention-mask and cache initialization, rather than relying solely on untrusted preprocessing.

Under fixed model parameters and transformations, the deterministic initial-position computation yields the same noiseless observations across sessions:

$ x_{i,1}=x_{\mathrm{BOS}}, $

Mi,1=MBOS,M_{i,1}=M_{\mathrm{BOS}},

hi,1=hBOS.h_{i,1}=h_{\mathrm{BOS}}.

Therefore, repeatedly collecting the first-position observations only duplicates the same row of the linear system, rather than providing additional independent constraints. Meanwhile, user-controlled tokens occur after the BOS token and are processed with preceding context. This mitigation prevents rank accumulation through varying the initial token and addresses the identified joint recovery vulnerability.

Comment:

Does the defense experiment in Table 2 include the multi-query recovery method proposed in Section 4? Please explain which observations the attacker actually used and how the recovery was performed.

Response:

We thank you for this insightful observation. We clarify that the original defense evaluation primarily considered the existing GAME model recovery attack, rather than directly applying our proposed multi-query transformation recovery attack to GateFlow. To address this concern, we have added a dedicated Transformation Recovery Evaluation, including the original attack against SLIM, an adapted multi-query attack against GateFlow, and an additional oracle setting. The corresponding theoretical analysis and experimental results are provided in Appendix D and Section 5 (Table 2), respectively.

1. Direct Recovery Analysis.

We first theoretically analyze why the original SLIM recovery attack cannot be directly applied to GateFlow. Specifically, the GPU can observe the input representation xix_i, obfuscated output Mi′M_i', and authorization result hi′h_i', as well as the transformed query/key representations Qi′Q_i' and Ki′K_i'. However, the plaintext attention weights SiS_i and secret transformations remain unavailable. For equal-length queries, the authorization relationship can be expressed as

Δhi′=ΔxiΠ+ΔRiΠo⊤Π,\Delta h_i'=\Delta x_i\Pi+\Delta R_i\Pi_o^\top\Pi,

where ΔRi=SiMi′−S1M1′\Delta R_i=S_iM_i'-S_1M_1'. Unlike SLIM, the second regression term ΔRi\Delta R_i depends on hidden attention weights and cannot be directly constructed from the plaintext observations used by the original attack. Consequently, the complete regression matrix required by the original closed-form linear recovery method is unavailable, preventing its direct application to GateFlow.

2. Adapted Multi-query Recovery Attack.

To provide a more direct security evaluation, we further develop an adapted attack that preserves the original multi-query recovery objective while jointly estimating the secret transformations and hidden attention weights from GPU-visible observations. Specifically, the attacker uses xi,Mi′,hi′i=1N\\{x_i,M_i',h_i'\\}_{i=1}^{N} and solves the following optimization problem.

We first define the estimated attention-dependent term:

\widehat S_iM_i'-\widehat S_1M_1'.$$ The corresponding reconstruction residual is $$\widehat E_i= \Delta h_i'-\Delta x_i\widehat\Pi -\widehat R_i\widehat\Pi_o^\top\widehat\Pi.$$ The attacker jointly optimizes the secret transformations and attention estimates by minimizing $$\min_{\widehat\Pi,\widehat\Pi_o,\\{\widehat S_i\\}} \sum_{i=2}^{N} \operatorname{tr}(\widehat E_i^\top\widehat E_i).$$ The estimated transformations satisfy orthogonality constraints, while the attention estimates satisfy nonnegativity, row normalization, and causal masking. This formulation does not provide the attacker with the true attention weights. We additionally introduce a **GateFlow / Oracle** setting, in which the attacker is explicitly given the true attention weights $S_i$. This enables direct linear recovery when the resulting regression matrix has full column rank, allowing us to examine the importance of the hidden attention information. **3. Experimental Results.** We evaluate all three attacks under the same query budgets for each model. The results are summarized in Table 2. **Table 2: Multi-query transformation recovery.** | Model | Scheme / Attack | Queries | $\Pi$ Rel. Error ↓ | $\Pi_{\mathrm{aux}}$ Rel. Error ↓ | $\Pi/\Pi_{\mathrm{aux}}$ Cos. ↑ | Fit Residual ↓ | |:---|:---|---:|---:|---:|:---:|---:| | GPT-2-medium | SLIM / Original | 80 | 7.57e-5 | 1.08e-4 | ≈1 / ≈1 | 7.03e-5 | | GPT-2-medium | GateFlow / Adapted | 80 | 1.221011 | 1.253892 | 0.254566 / 0.213877 | 0.780838 | | GPT-2-medium | GateFlow / Oracle | 80 | 8.57e-5 | 1.78e-4 | ≈1 / ≈1 | 4.98e-4 | | Llama-2-7B | SLIM / Original | 320 | 1.33e-4 | 1.89e-4 | ≈1 / ≈1 | 1.20e-4 | | Llama-2-7B | GateFlow / Adapted | 320 | 1.207457 | 1.341499 | 0.271024 / 0.100191 | 0.872631 | | Llama-2-7B | GateFlow / Oracle | 320 | 1.28e-4 | 9.49e-5 | ≈1 / ≈1 | 3.73e-5 | As shown above, both the original attack against SLIM and the oracle attack against GateFlow achieve near-exact transformation recovery, with relative errors below $2\times10^{-4}$. In contrast, the adapted attack against GateFlow yields substantially higher transformation errors (above 1.2) and fitting residuals of 0.781 and 0.873, indicating that it fails to accurately recover the secret transformations or fit the observed authorization relationships under the evaluated query and optimization budgets. Importantly, the oracle results demonstrate that accurate recovery becomes possible when the true attention weights are provided, highlighting the significance of protecting attention-dependent intermediate representations.

Comment:

How much performance benefit does GateFlow's cryptographic outsourcing bring compared to executing the first-layer attention directly within the TEE?

Response:

We thank you for this valuable suggestion. To quantitatively evaluate the performance benefits of cryptographic attention outsourcing, we have conducted additional experiments on Gemma3-12B, with input lengths ranging from 512 to 4096 tokens. We compare GateFlow against two baselines:

  1. All-TEE: Executes the entire first-block attention computation inside the TEE rather than outsourcing encrypted attention operations to the GPU.
  2. Interactive: Retains GateFlow's first-block protection but requires repeated TEE–GPU interactions to recover obfuscated representations after GPU-side computations in subsequent decoder blocks.

The detailed results are presented below and have been incorporated into Table 6 of the revised manuscript.

Table 6: TTFT, throughput, and communication breakdown under different input lengths.

Input LengthSchemeTTFT (ms) ↓Throughput (tokens/s) ↑TEE Comp. (ms) ↓Data Transfer (ms) ↓Comm. Volume (MB) ↓
512GateFlow457.35718.975305.70556.243142.61
Interactive13912.6906.2274169.3898884.9644,627.37
All-TEE1148.43617.6211088.7072.8683.93
1024GateFlow778.94918.079538.10561.978159.38
Interactive25352.0705.2808003.15916232.3789,128.90
All-TEE2774.18714.5692669.3683.8087.86
2048GateFlow1831.56515.5041376.85994.834192.94
Interactive63009.5203.79214710.10045570.25018,131.98
All-TEE5898.62513.1785684.7926.15115.73
4096GateFlow5513.82811.7574218.834220.994520.09
Interactive121675.8552.64429512.38487245.06936,398.17
All-TEE13175.58410.79812683.03923.98531.46

1. Comparison with All-TEE.

As shown above, GateFlow consistently achieves lower TTFT and higher throughput than All-TEE across all evaluated input lengths. At 4096 tokens, GateFlow reduces TTFT from 13.18 s to 5.51 s (2.39× speedup), while reducing TEE computation latency from 12.68 s to 4.22 s (3.01× reduction). Although encrypted attention outsourcing introduces additional communication, the data-transfer latency accounts for only 4.0% of total TTFT, demonstrating that the reduction in TEE computation substantially outweighs the additional communication overhead.

2. Comparison with Interactive.

GateFlow also significantly outperforms Interactive by avoiding repeated TEE–GPU interactions in subsequent decoder blocks. At 4096 tokens, GateFlow achieves a 22.1× TTFT speedup (5.51 s vs. 121.68 s) and 4.45× higher throughput (11.757 vs. 2.644 tokens/s). In particular, the data-transfer latency decreases from 87.25 s to only 0.22 s, highlighting the effectiveness of transformation propagation in reducing communication overhead.

Reviewer_jZCROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies security vulnerabilities in TEE–GPU collaborative LLM inference, especially SLIM. The authors found that repeated inference observations can expose linear relationships that allow attackers to recover secret transformations. To address this vulnerability, they propose GateFlow, which combines secure attention authorization and RLWE-based encryption while preserving transformed-domain propagation across decoder blocks.

Critical strengths
  1. The paper is easy to follow. The proposed transformation-recovery attack is formulated as a linear inverse problem, and experiments show that the secret transformations can be recovered accurately from a small number of inference queries.

  2. GateFlow adds a secure attention-authorization step in the first decoder block while keeping SLIM’s transformed-domain propagation, so later layers run without further TEE involvement.

Critical weaknesses
  1. The proposed attack specifically targets reusable linear relationships in SLIM's authorization mechanism. It is unclear whether GateFlow can be applied beyond SLIM. Additionally, the security analysis primarily focuses on the difficulty of uniquely recovering secret transformations, which does not necessarily prevent functional model extraction.

  2. In Table 2, SLIM already performs close to the random baseline on several tasks, making it difficult to assess the additional security benefits provided by GateFlow. Moreover, Table 4 reports high logit similarity and low numerical error but does not directly evaluate downstream task accuracy. The authors should conduct a comparative evaluation of SLIM and GateFlow under the proposed attack, reporting task accuracy, token-level agreement, and generation quality to establish whether the improved security guarantees can be achieved without sacrificing model utility.

Questions for authors
  1. The attack appears specific to SLIM's reusable orthogonal transformations. Does it apply to other TEE–GPU schemes, such as GroupCover or ArrowCloak?

  2. Does GateFlow preserve downstream task accuracy? Can the authors compare the original model, SLIM, and GateFlow on standard benchmarks?

  3. Can the authors provide concrete RLWE parameters and clarify the correctness guarantees for encrypted attention computation, including plaintext growth and decryption noise?

  4. What is the percentage increase in TTFT and decrease in throughput compared with SLIM? How does this overhead scale with longer input contexts, output lengths, and larger models?

5 comments

Thank you for your thoughtful review and positive assessment!We appreciate your recognition of our attack analysis and GateFlow's security design. We have addressed your concerns through additional theoretical analyses and experiments, as detailed below.

Comment:

The attack appears specific to SLIM's reusable orthogonal transformations. Does it apply to other TEE–GPU schemes, such as GroupCover or ArrowCloak?

Response: We thank you for this important question. Our proposed attack specifically targets SLIM (ICML 2026), a recent state-of-the-art TEE–GPU inference framework, rather than all existing schemes. Unlike GroupCover and ArrowCloak, which rely on layer-wise TEE–GPU interactions to recover obfuscated computation results, SLIM introduces a distinctive transformation propagation mechanism that allows orthogonally transformed representations to propagate across consecutive decoder blocks without repeated TEE interactions. However, its reusable transformations also introduce exploitable linear relationships between GPU-visible inputs and authorization outputs across multiple queries. Our attack specifically exploits these relationships to recover the secret transformations.

In contrast, GroupCover and ArrowCloak have already been attacked using fundamentally different approaches. BiShield-TEE (NeurIPS 2026) exploits structural leakage in unilateral weight obfuscation, recovering GroupCover's group structure through clustering and ArrowCloak's shared masking direction through principal component analysis (PCA). Furthermore, Understanding the Security Boundary of Obfuscation-based On-Device LLM Protection (CCS 2026) proposes Collapse, a primitive-guided attack that exploits vulnerabilities in weight-obfuscation primitives, including those used by ArrowCloak.

Unlike these weight-reconstruction attacks, our approach focuses on recovering reusable secret transformations from multi-query authorization observations. Therefore, our work identifies a distinct security vulnerability arising from SLIM's efficient transformation propagation mechanism, rather than claiming a universal attack against all TEE–GPU schemes.

Comment:

The security analysis primarily focuses on the difficulty of uniquely recovering secret transformations, which does not necessarily prevent functional model extraction. Does GateFlow preserve downstream task accuracy? Can the authors compare the original model, SLIM, and GateFlow on standard benchmarks?

Response: We thank you for this important observation. To further evaluate whether attackers can recover functionally equivalent models without recovering the original secret transformations, we have added additional model-stealing experiments (see Table 4).

Experimental Setup. Following prior work, we construct a recovery dataset by querying the target model with 100 prompts (1% of the dataset) and collecting the corresponding responses. We evaluate several attack strategies:

  1. Pretrained FT: Fine-tunes a public pretrained model using the collected data.
  2. ArrowMatch-based: Attempts to recover model weights by matching pretrained and obfuscated weights.
  3. Obfuscated-weight FT: Directly fine-tunes GateFlow's obfuscated model.
  4. TEE-Function Recovery: Learns a surrogate authorization function from GPU-visible observations to replace the protected TEE computation. We additionally include a randomly selected model as a baseline.

Evaluation Metrics. We evaluate functional recovery on Alpaca using three metrics:

  • Task Accuracy: Measures next-token prediction accuracy against Alpaca reference answers under teacher forcing.
  • Token Agreement: Measures token-level agreement between independently generated responses from the recovered and target models.
  • ROUGE-L F1: Measures response similarity based on the longest common subsequence. Table 4: Functional model recovery on Alpaca.
SchemeTask Acc. (%) ↑Token Agr. (%) ↑ROUGE-L F1 ↑
Victim62.78100.001.00
SLIM62.7779.290.88
GateFlow62.7899.641.00
Pretrained FT59.783.690.32
ArrowMatch-based0.300.100.03
Obfuscated-weight FT0.460.100.01
TEE-Function Recovery0.340.0090.01
Random Model0.020.0020.006

As shown in Table 4, the three attacks targeting GateFlow's exposed representations or obfuscated weights achieve at most 0.46% task accuracy, substantially below pretrained-model fine-tuning (59.78%). Meanwhile, GateFlow preserves the original model's functionality, achieving 62.78% task accuracy and 99.64% token agreement. These results provide additional empirical evidence that the evaluated attacks fail to recover functionally equivalent models, extending our security evaluation beyond exact transformation recovery.

Comment:

In Table 2, SLIM already performs close to the random baseline on several tasks, making it difficult to assess the additional security benefits provided by GateFlow.

Response:

We thank you for this valuable suggestion. To better distinguish GateFlow's security benefits over SLIM, we have added a dedicated transformation recovery evaluation. We first examine whether the original SLIM recovery attack can be directly applied to GateFlow, then develop an adapted multi-query attack using joint optimization. Finally, an oracle-based ablation reveals the true attention weights SS to isolate their contribution to security (see Table 2 and Appendix D).

1. Direct Linear Recovery Analysis.

The original SLIM attack requires a fully observable linear system. GateFlow's authorization process satisfies

hi′=(xi+SiMi′Πo⊤+bo)Π.h_i'=(x_i+S_iM_i'\Pi_o^\top+b_o)\Pi.

Here, xix_i, Mi′M_i', and hi′h_i' are GPU-visible, whereas SiS_i and the secret transformations are hidden. Subtracting observations from equal-length queries gives

Δhi′=ΔxiΠ+(SiMi′−S1M1′)Πo⊤Π.\Delta h_i'=\Delta x_i\Pi+(S_iM_i'-S_1M_1')\Pi_o^\top\Pi.

Because the attention-dependent term is unknown, the attacker cannot construct the complete regression matrix required for SLIM's original closed-form solver.

2. Adapted Multi-query Recovery Attack.

We therefore develop an adapted attack that jointly estimates the secret transformations and hidden attention weights from GPU-visible observations xi,Mi′,hi′i=1N\\{x_i,M_i',h_i'\\}_{i=1}^{N}. To express its objective, define

Ri=S^iMi′−S^1M1′.R_i=\widehat{S}_iM_i'-\widehat{S}_1M_1'.

Ei=Δhi′−ΔxiΠ^−RiΠ^o⊤Π^.E_i=\Delta h_i'-\Delta x_i\widehat{\Pi}-R_i\widehat{\Pi}_o^\top\widehat{\Pi}.

We jointly minimize the squared reconstruction error over the estimated transformations and attention weights:

L=∑i=2Ntr⁡(Ei⊤Ei).\mathcal{L}=\sum_{i=2}^{N}\operatorname{tr}(E_i^\top E_i).

The transformations are constrained to be orthogonal, while the estimated attention matrices satisfy nonnegativity, row normalization, and causal masking.

3. Oracle-based Ablation Study.

In an oracle setting, we additionally provide the attacker with the true attention weights SiS_i, enabling direct linear recovery when the regression matrix has full column rank. This isolates the security contribution of hidden attention weights.

4. Experimental Results.

We evaluate the original SLIM attack, the adapted GateFlow attack, and the oracle attack under identical query budgets for each model.

Table 2: Multi-query transformation recovery.

ModelScheme / AttackQueriesΠ\Pi Rel. Error ↓Πaux\Pi_{\mathrm{aux}} Rel. Error ↓Cos. (Π\Pi / Πaux\Pi_{\mathrm{aux}}) ↑Fit Residual ↓
GPT-2-mediumSLIM / Original807.57e-51.08e-4≈1 / ≈17.03e-5
GPT-2-mediumGateFlow / Adapted801.2210111.2538920.254566 / 0.2138770.780838
GPT-2-mediumGateFlow / Oracle808.57e-51.78e-4≈1 / ≈14.98e-4
Llama-2-7BSLIM / Original3201.33e-41.89e-4≈1 / ≈11.20e-4
Llama-2-7BGateFlow / Adapted3201.2074571.3414990.271024 / 0.1001910.872631
Llama-2-7BGateFlow / Oracle3201.28e-49.49e-5≈1 / ≈13.73e-5

As shown above, the original attack recovers SLIM's transformations with relative errors below 2×10−42\times10^{-4}. By contrast, GateFlow's adapted attack yields errors above 1.2 and fitting residuals of 0.781 and 0.873 on GPT-2-medium and Llama-2-7B, respectively, indicating that accurate recovery is not achieved under the evaluated settings.

More importantly, revealing the true attention weights in the oracle setting restores near-exact recovery, with errors again below 2×10−42\times10^{-4}. This contrast provides direct experimental evidence that protecting attention-dependent intermediate representations plays a critical role in resisting the evaluated transformation recovery attack.

Comment:

Can the authors provide concrete RLWE parameters and clarify the correctness guarantees for encrypted attention computation, including plaintext growth and decryption noise?

Response:

We thank you for this valuable suggestion. The original manuscript already provides a theoretical analysis of Coef-RLWE correctness and decryption noise in Appendices E.3 and E.4. To further strengthen the correctness guarantees and reproducibility, we have specified concrete RLWE parameters and validated the derived correctness conditions, including plaintext growth and accumulated decryption noise, using actual inference data (see Appendix E.5).

1. Concrete RLWE Parameters.

We instantiate Coef-RLWE with polynomial degree N=2048N=2048 and ciphertext modulus q=167772161×469762049q=167772161\times469762049. The secret coefficients follow a ternary distribution −1,0,1\\{-1,0,1\\}, while the encryption noise follows a centered binomial distribution with η=3\eta=3. We adopt separate fixed-point scales αS=65536\alpha_S=65536 and αM=1024\alpha_M=1024, with an encryption message interval Γ=224\Gamma=2^{24}.

2. Correctness Verification.

We verify two essential conditions: bounded decryption noise for correct integer recovery and sufficient plaintext range to prevent modular wrap-around.

For decryption noise, the accumulated error satisfies

\leq \sum_k |P_{kj}|\Vert e_k\Vert_\infty.$$ Across 28 forward evaluations, the maximum absolute column sum is $1{,}810{,}835$. Since $\Vert e_k\Vert_\infty\leq3$, we obtain $$\Vert e_j^{\mathrm{out}}\Vert_\infty \leq 5{,}432{,}505 < 8{,}388{,}608 = \Gamma/2.$$ The maximum observed noise magnitude is only **421,840**, remaining well below the derived bound. For plaintext growth, the maximum observed encoded output magnitude is $B_m=664{,}675{,}456$. Substituting the measured bounds gives $$\Gamma B_m+\Vert e^{\mathrm{out}}\Vert_\infty \leq 1.11514\times10^{16} < 3.94065\times10^{16}=q/2.$$ Therefore, **both the decryption-noise and modular wrap-around conditions are satisfied for the evaluated inputs**, ensuring correct recovery of the encoded integer results. **3. Experimental Validation.** We further validate the complete encode–encrypt–evaluate–decrypt–decode pipeline on **GPT-2-medium, Llama-2-7B, and Qwen3-4B**, covering different prefill lengths and KV-cache-based decoding. The detailed results are presented below and incorporated into **Table 9** of the revised manuscript. **Table 9: Correctness validation of Coef-RLWE.** | Model | Scenario | Max Abs. Error ↓ | Rel. Frobenius Error ↓ | Integer Failures | |---|---|---:|---:|---:| | GPT-2-medium | Prefill-64 | 4.881e-4 | 0.0294% | 0/3 | | GPT-2-medium | Prefill-512 | 4.945e-4 | 0.0524% | 0/3 | | GPT-2-medium | Decode-1024 | 5.319e-4 | 0.0225% | 0/3 | | Llama-2-7B | Prefill-64 | 4.882e-4 | 0.0293% | 0/3 | | Llama-2-7B | Prefill-512 | 5.362e-4 | 0.0517% | 0/3 | | Llama-2-7B | Decode-1024 | 6.438e-4 | 0.0318% | 0/3 | | Qwen3-4B | Prefill-64 | 4.897e-4 | 0.0290% | 0/3 | | Qwen3-4B | Prefill-512 | 5.029e-4 | 0.0516% | 0/3 | As shown above, **no integer-recovery failures are observed across any evaluated setting**, while the maximum absolute numerical error remains below $6.44\times10^{-4}$ and the relative Frobenius error does not exceed **0.0524%**. These results validate the correctness of the selected parameters for the evaluated encrypted attention computations.
Reviewer_gRQXOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the security vulnerabilities of transformed-domain propagation in TEE–GPU collaborative LLM inference schemes such as SLIM, demonstrating that exposed linear relationships during authorization enable an untrusted GPU adversary to recover equivalent secret transformation matrices across multi-query inference. To resolve this vulnerability, this paper introduce GateFlow, which redesigns the first-decoder authorization stage using a customized, lightweight Ring-LWE homomorphic encryption mechanism to eliminate observable linear correlations while preserving low-communication, transformed-domain propagation across subsequent layers.

Critical strengths
  1. The paper exposes a critical algebraic identifiability vulnerability in orthogonal transformation-based collaborative inference, formally demonstrating that an adversary on an untrusted GPU can reconstruct full-rank equivalent transformation matrices from as few as dozens of passive inference observations, which challenges foundational security assumptions in state-of-the-art obfuscation frameworks.

  2. The proposed GateFlow framework introduces an architecturally elegant defense that confines trusted execution and encryption overhead strictly to the first decoder block, which effectively breaks linear invertibility while fully preserving zero-communication, transformed-domain GPU execution across all subsequent deep layers without incurring depth-proportional latency explosions.

  3. The tailored Coef-RLWE encryption scheme targets solely ciphertext-plaintext matrix multiplication via compact polynomial encoding, which avoids the heavy computational overhead of generic homomorphic primitives and achieves provable cryptographic confidentiality with practical end-to-end inference efficiency.

Critical weaknesses
  1. In Section 4.2 (Steps 2–3), computing the attention score matrix S=Softmax(QK⊤/dk)S = \text{Softmax}(QK^\top / \sqrt{d_k}) inside the CPU TEE and encrypting it column-wise using the Coef-RLWE scheme introduces potential computational and communication scalability concerns for long-context workloads. While the overhead is mitigated during incremental auto-regressive decoding where the query length is single-token (L=1L=1), modern LLM deployment scenarios frequently involve extensive prefill phases with input sequences spanning thousands of tokens. In such extended contexts, the quadratic memory and execution burden of softmax operations within resource-constrained enclave memory, combined with transferring LL large polynomial ciphertexts across the PCIe bus, may impose noticeable tail latency on time-to-first-token. Could the authors report micro-benchmark latency and communication bandwidth scaling across varying prefill context lengths (e.g., from 512 up to 4096 tokens) to quantify the precise operational boundary where TEE-side attention outsourcing becomes a bottleneck?

  2. In Section 4.2 (Step 1), the first-decoder authorization pipeline explicitly leaves the value projection weights unperturbed on the untrusted GPU (Wv′=WvW_v' = W_v), resulting in the raw parameter matrix WvW_v remaining completely visible in GPU memory in cleartext. While outsourcing encrypted ciphertext-plaintext multiplication Enc(S)⋅VWo\text{Enc}(S) \cdot VW_o justifies this architectural design choice, leaving foundational projection weights unmasked technically creates an asymmetrical confidentiality guarantee across the first transformer layer. Although single-layer weight exposure is insufficient to reconstruct functional substitute models across deep architectures, the paper does not formally characterize the theoretical leakage boundary of this partial exposure. Could the authors include a formal security clarification explaining why cleartext retention of WvW_v in the first block preserves provable model confidentiality against adversaries attempting targeted functional inversion on early token representations?

Questions for authors

See Weakness

2 comments

Thank you for your thoughtful review and positive assessment of our work! We appreciate your recognition of the transformation recovery analysis, GateFlow's security architecture, and the tailored Coef-RLWE encryption scheme. We have carefully considered all the questions you raised and conducted additional theoretical analyses and experiments to address them. Our detailed responses are provided below.

Comment:

Could the authors report micro-benchmark latency and communication bandwidth scaling across varying prefill context lengths (e.g., from 512 up to 4096 tokens) to quantify the precise operational boundary where TEE-side attention outsourcing becomes a bottleneck?

Response:

We thank you for this valuable suggestion. We extend our evaluation to a new and larger model, Gemma3-12B, with input lengths ranging from 512 to 4096 tokens. We further compare GateFlow with an All-TEE baseline that performs the entire first-block attention computation inside the TEE. We measure TTFT, TEE computation latency, data-transfer latency, and communication volume to quantify how computation and communication overhead scale with context length. The detailed results are presented below (see Table 6).

Table 6: TTFT and communication breakdown under different input lengths.

Input LengthSchemeTTFT (ms) ↓TEE Comp. (ms) ↓Transfer (ms) ↓Comm. Volume (MB)
512GateFlow457.36305.7156.24142.61
All-TEE1148.441088.712.873.93
1024GateFlow778.95538.1161.98159.38
All-TEE2774.192669.373.817.86
2048GateFlow1831.571376.8694.83192.94
All-TEE5898.635684.796.1515.73
4096GateFlow5513.834218.83220.99520.09
All-TEE13175.5812683.0423.9931.46

The results demonstrate that GateFlow consistently outperforms All-TEE across all evaluated context lengths, confirming the effectiveness of secure attention outsourcing even for larger models and extended prefill workloads. Notably, at 4096 tokens, GateFlow achieves a 2.39× speedup in TTFT (13.18 s vs. 5.51 s) and approximately 3.0× lower TEE computation latency. Although the encrypted communication volume increases with context length, the data-transfer latency accounts for only 4.0% of the total TTFT, whereas TEE computation remains the dominant overhead.

Comment:

Could the authors include a formal security clarification explaining why cleartext retention of the value projection in the first block preserves model confidentiality against adversaries attempting targeted functional inversion on early token representations?

Response:

We thank you for this valuable suggestion. We have added a formal security analysis to clarify the information leakage associated with the plaintext first-block value projection WvW_v. First, we demonstrate that exposing WvW_v alone does not reveal the complete authorization function, which also depends on hidden attention weights and secret transformations. Then, we provide a formal information-theoretic analysis showing that, under sufficiently informative multi-query observations, explicitly revealing WvW_v introduces zero additional information about the remaining model parameters beyond the GPU-visible observations. Furthermore, we derive a general upper bound on the incremental information leakage without requiring this condition. The detailed analysis and proofs are provided in Appendix D.

Leakage Analysis of the First-Block Value Projection.

We treat the first-block value projection WvW_v as explicitly disclosed model information. Inputs are adversary-controlled and are not considered confidential. The output projection remains transformed as Wo′=WoΠoW_o'=W_o\Pi_o, and the GPU computes

Vi=xˉiWv,Mi′=ViWo′.V_i=\bar{x}_iW_v,\qquad M_i'=V_iW_o'.

The corresponding authorization output is

hi′=(xi+SiMi′Πo⊤+1bo)Π.h_i'=\left(x_i+S_iM_i'\Pi_o^{\top}+\mathbf{1}b_o\right)\Pi.

Thus, knowledge of WvW_v provides the value-projection mapping but does not, by itself, specify the complete authorization function. In particular, holding the other terms fixed, two candidate attention matrices satisfy

M_i'\Pi_o^{\top}\Pi.$$ Since the transformations are invertible, the two authorization outputs differ whenever $$\left(S_i^{(a)}-S_i^{(b)}\right)M_i'\neq 0.$$ Therefore, identical public value-branch information can correspond to different authorization outputs. **Proposition (Incremental Leakage Bound).** Let $\Theta_{\mathrm{rest}}$ denote the remaining model parameters, and let $\mathcal Z_N$ contain the GPU-visible observations from $N$ queries, excluding the explicitly stored $W_v$. In particular, $\mathcal Z_N$ includes the normalized inputs $\\{\bar{x}_i\\}$, plaintext value representations $\\{V_i\\}$, the transformed output projection $W_o'=W_o\Pi_o$, and the other protocol observations. Suppose $W_v\in\mathbb R^{d\times d_v}$ uses $b$ bits per entry, where $d$ and $d_v$ are the input and value dimensions. For any joint distribution of the model parameters, the incremental leakage satisfies $$I(\Theta_{\mathrm{rest}};W_v\mid\mathcal Z_N) \leq H(W_v\mid\mathcal Z_N) \leq bdd_v,$$ where $I$ and $H$ denote conditional mutual information and entropy. **Proof.** Stack the observed input--value pairs along the token dimension: $$\mathcal X=[\bar{x}_1;\ldots;\bar{x}_N], \qquad \mathcal V=[V_1;\ldots;V_N].$$ Since $V_i=\bar{x}_iW_v$, these observations satisfy $$\mathcal V=\mathcal XW_v.$$ If $\operatorname{rank}(\mathcal X)=d$, a simulator given only $\mathcal Z_N$ reconstructs $$\widetilde W_v=\mathcal X^{\dagger}\mathcal V=W_v,$$ where $\mathcal X^{\dagger}$ is the Moore--Penrose pseudoinverse. The simulator can therefore reproduce the augmented view $(\mathcal Z_N,W_v)$ without being given $W_v$ separately. Consequently, $W_v$ is a deterministic function of $\mathcal Z_N$, implying $$H(W_v\mid\mathcal Z_N)=0$$ and hence $$I(\Theta_{\mathrm{rest}};W_v\mid\mathcal Z_N)=0.$$ This conclusion requires no assumption of independence between $W_v$ and the remaining parameters. Without the full-rank condition, the general bound follows from $$I(\Theta_{\mathrm{rest}};W_v\mid\mathcal Z_N)=H(W_v\mid\mathcal Z_N)-H(W_v\mid\Theta_{\mathrm{rest}},\mathcal Z_N).$$ Using nonnegative conditional entropy and the finite representation length of $W_v$, we obtain $$I(\Theta_{\mathrm{rest}};W_v\mid\mathcal Z_N) \leq H(W_v\mid\mathcal Z_N) \leq H(W_v) \leq bdd_v.$$ $\square$ Thus, once the plaintext input--value observations uniquely identify the projection, explicitly retaining $W_v$ adds no information beyond that transcript.
Reviewer_jDSMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the security of propagation based TEE-GPU collaborative LLM inference. The authors identify a transformation identifiability vulnerability in SLIM and propose a multi query transformation recovery attack. Through exploiting linear relationship in GPU visible intermediate representations and authorization output, the attack recover the hidden transformation through a linear inverse problem. The author then introduces Gateflow, which redesign the initial decoder authorization process using protected query/key transformations and RLWE based encrypted attention computation. The proposed framework aims to prevent transformation recovery while preserving efficient transformerd domain propagation in decoder blocks.

Critical strengths
  1. The paper addresses an important problem in efficient and secure LLM deployment. The tension between protecting proprietary model parameters and minimizing trusted computation is well motivated.

  2. The transformation recovery attack is clearly formulated and experimentally supported. The ability to recover hidden transformations with relatively few prompts highlights a concrete limitation of existing propagation-based protection mechanisms.

  3. The proposed defense has an intuitive architectural motivation. Preserving transformed-domain propagation while strengthening the initial authorization step is an interesting approach to balancing security and inference efficiency.

Critical weaknesses
  1. Limited direct validation against the proposed transformation recovery attack.

My primary concern is the gap between the attack introduced in section 4 of the paper and the defense evaluation in section 5. The author demonstrates accurate transformation recovery against SLIM, but the subsequent evaluation of GateFlow mostly considers the existing GAME model recovery attack. Furthermore, table 2 shows that SLIM already achieves lower attacker performance under GAME across most evaluated settings. So its a bit unclear how much additional protection GateFlow provides against the specific vulnerability. what would be more convincing is to directly apply the proposed multi query transformation recovery attack, or an appropriately adapted version, to GateFlow.

  1. Security guarantees are not fully aligned with the stated threat model. The threat model defines success in terms of recovering a functionally equivalent substitute model. However, the security analysis primarily establishes that the original query/key representations and secret transformations cannot be uniquely identified from GPU-visible observations. Non-identifiability of the original coordinate representation does not necessarily rule out reconstruction of a functionally equivalent model. I would appreciate a more explicit discussion of this distinction and a stronger connection between the theoretical analysis and the stated model-stealing objective. In particular, the analysis would benefit from considering whether repeated chosen-query observations could reveal equivalent transformations or functional relationships without recovering the exact original secrets.
Questions for authors
  1. Can we directly evaluate the proposed transformation recovery attack against Gateflow?
  2. How are the post-RoPe orthogonal transformation implemented without exposing secrete matrices to the GPU?
  3. Can the authors clarify whether the authorization overhead is independent of the decoder depth?
5 comments

Thank you for your thoughtful review and the time you dedicated to our work! We appreciate your recognition of the importance of secure and efficient LLM deployment, our transformation recovery attack, and the architectural motivation of our proposed defense. Below, we provide clarifications and additional experiments that address the concerns you raised.

Comment:

My primary concern is the gap between the attack introduced in Section 4 of the paper and the defense evaluation in Section 5. Can we directly evaluate the proposed transformation recovery attack against GateFlow?

Response:

We sincerely thank you for this valuable suggestion. We first attempt to directly apply the proposed recovery attack to GateFlow and find that the required linear system cannot be constructed from the available observations. We then develop an adapted attack based on joint optimization, which achieves poor recovery accuracy. Furthermore, we introduce an oracle-based ablation study, where revealing the true attention weights SS enables near-exact recovery. These results demonstrate GateFlow's effectiveness against the evaluated transformation recovery attacks (see Table 2 and Appendix D).

1. Direct Linear Recovery Analysis.

The original SLIM attack relies on constructing a fully observable linear system to recover the secret transformations. However, GateFlow's authorization process satisfies

hi′=(xi+SiMi′Πo⊤+bo)Π,h_i'=(x_i+S_iM_i'\Pi_o^\top+b_o)\Pi,

where xix_i, Mi′M_i', and hi′h_i' are GPU-visible, whereas SiS_i and the secret transformations remain hidden. By subtracting observations from equal-length queries, we obtain

(S_iM_i'-S_1M_1')\Pi_o^\top\Pi.$$ Unlike SLIM, the attention-dependent term $S_iM_i'-S_1M_1'$ is not directly observable. Consequently, the attacker cannot construct the complete regression matrix required by the original closed-form linear recovery method, preventing its direct application to GateFlow. **2. Adapted Multi-query Recovery Attack.** We develop an adapted attack that jointly estimates the hidden attention weights and secret transformations from GPU-visible observations. Let $$A=\widehat{\Pi}, \qquad B=\widehat{\Pi}_o^{\top}\widehat{\Pi}.$$ For each query, we define the reconstruction residual as $$E_i=\Delta h_i'-\Delta x_i A-(\widehat{S}_iM_i'-\widehat{S}_1M_1')B.$$ The attacker minimizes the following objective over the estimated transformations and attention weights: $$\mathcal{L}=\sum_{i=2}^{N}\operatorname{tr}(E_i^{\top}E_i).$$ The transformations are constrained to be orthogonal, while the estimated attention matrices satisfy nonnegativity, row normalization, and causal masking. **3. Oracle-based Ablation Study.** To further understand the effectiveness of our defense, we additionally introduce an oracle setting in which the attacker is given the true attention weights $S_i$. This allows direct linear recovery when the regression matrix has full column rank, isolating the contribution of attention-weight confidentiality. **4. Experimental Results.** We evaluate the original SLIM attack, the adapted GateFlow attack, and the oracle attack under identical query budgets for each model. The results are summarized in Table 2. **Table 2: Multi-query transformation recovery.** | Model | Scheme / Attack | Queries | $\Pi$ Rel. Error ↓ | $\Pi_{\mathrm{aux}}$ Rel. Error ↓ | $\Pi/\Pi_{\mathrm{aux}}$ Cos. ↑ | Fit Residual ↓ | |:---|:---|---:|---:|---:|:---:|---:| | GPT-2-medium | SLIM / Original | 80 | 7.57e-5 | 1.08e-4 | ≈1 / ≈1 | 7.03e-5 | | GPT-2-medium | GateFlow / Adapted | 80 | 1.221011 | 1.253892 | 0.254566 / 0.213877 | 0.780838 | | GPT-2-medium | GateFlow / Oracle | 80 | 8.57e-5 | 1.78e-4 | ≈1 / ≈1 | 4.98e-4 | | Llama-2-7B | SLIM / Original | 320 | 1.33e-4 | 1.89e-4 | ≈1 / ≈1 | 1.20e-4 | | Llama-2-7B | GateFlow / Adapted | 320 | 1.207457 | 1.341499 | 0.271024 / 0.100191 | 0.872631 | | Llama-2-7B | GateFlow / Oracle | 320 | 1.28e-4 | 9.49e-5 | ≈1 / ≈1 | 3.73e-5 | As shown above, the original attack against SLIM achieves nearly exact transformation recovery, with relative errors below $2\times10^{-4}$. In contrast, the adapted attack against GateFlow yields relative errors above 1.2, with fitting residuals of 0.781 and 0.873 on GPT-2-medium and Llama-2-7B, respectively. These results demonstrate that the adapted attack fails to accurately recover the secret transformations under the evaluated settings. More importantly, when the true attention weights are revealed in the oracle setting, the recovery errors immediately decrease to below $2\times10^{-4}$, achieving accuracy comparable to the original SLIM attack. This significant contrast provides direct experimental evidence that protecting attention-dependent intermediate representations plays a critical role in resisting the evaluated transformation recovery attacks.

Comment:

I would appreciate a more explicit discussion of this distinction and a stronger connection between the theoretical analysis and the stated model-stealing objective. In particular, the analysis would benefit from considering whether repeated chosen-query observations could reveal equivalent transformations or functional relationships without recovering the exact original secrets.

Response:

We thank you for this insightful suggestion. We agree that recovering the exact secret transformations is not necessary for model stealing, as an attacker may instead reconstruct functionally equivalent computations. To address this concern, we have added a Functional Model Recovery Evaluation that directly examines whether attackers can recover GateFlow's functionality without obtaining the original secrets (see Table 4).

Specifically, we evaluate several recovery strategies, including pretrained-model fine-tuning, ArrowMatch-based recovery, direct fine-tuning of obfuscated weights, and TEE-function recovery, where the attacker attempts to reconstruct the protected authorization function using GPU-visible observations. These experiments complement our transformation recovery analysis by evaluating the practical model-stealing objective rather than relying solely on secret-matrix recovery errors.

Table 4: Functional model recovery on Alpaca.

SchemeTask Acc. (%) ↑Token Agr. (%) ↑ROUGE-L F1 ↑
Victim62.78100.001.00
SLIM62.7779.290.88
GateFlow62.7899.641.00
Pretrained FT59.783.690.32
ArrowMatch-based0.300.100.03
Obfuscated-weight FT0.460.100.01
TEE-Function Recovery0.340.0090.01
Random Model0.020.0020.006

As shown above, all three attacks exploiting GateFlow's protected representations or obfuscated weights achieve at most 0.46% task accuracy, approaching the random baseline and remaining substantially below pretrained-model fine-tuning (59.78%). These results indicate that the evaluated attacks fail to reconstruct functionally equivalent models from GateFlow's exposed information, providing additional empirical evidence of resistance to functional model recovery beyond exact transformation recovery.

Comment:

How are the post-RoPE orthogonal transformations implemented without exposing secret matrices to the GPU?

Response:

We thank you for this valuable suggestion. Since RoPE is position-dependent and does not generally commute with arbitrary orthogonal transformations, GateFlow handles the first decoder block and subsequent blocks differently to ensure that RoPE is always applied to query and key representations in their original coordinate system, without exposing the secret transformation matrices to the GPU (see Appendix C.3).

Compatibility with Rotary Position Embedding.

First Decoder Block. The GPU computes the obfuscated query and key representations Q′=QΠqQ'=Q\Pi_q and K′=KΠkK'=K\Pi_k, where Πq\Pi_q and Πk\Pi_k are secret orthogonal matrices maintained by the TEE. After receiving Q′Q' and K′K', the TEE first recovers the original representations:

Q=Q′Πq⊤,K=K′Πk⊤.Q=Q'\Pi_q^\top,\qquad K=K'\Pi_k^\top.

The TEE then directly applies RoPE to the recovered representations and computes the attention weights:

Qr=RoPE(Q),Kr=RoPE(K).Q_r=\mathrm{RoPE}(Q),\qquad K_r=\mathrm{RoPE}(K).

\frac{Q_rK_r^\top}{\sqrt{d_k}}+M_{\mathrm{causal}} \right).$$ Therefore, the secret transformations and the first-block RoPE computation remain entirely inside the TEE. Since $Q'$ and $K'$ are already transferred to the TEE for attention computation, no additional TEE--GPU communication is introduced. **Subsequent Decoder Blocks.** For subsequent decoder blocks, the hidden representation remains in the transformed domain, i.e., $x'=x\Pi$. To enable standard RoPE execution on the GPU without revealing $\Pi$, we precompute the transformed query and key projection weights: $$W_q'=D^{-1}\Pi^\top DW_q,\qquad W_k'=D^{-1}\Pi^\top DW_k,$$ where $D$ denotes the RMSNorm scaling matrix. Using the RMSNorm transformation property, $$\mathrm{RMSNorm}(x\Pi)= \mathrm{RMSNorm}(x)D^{-1}\Pi D.$$ The GPU can directly compute the original query and key representations through the transformed projection weights: $$Q'=\mathrm{RMSNorm}(x\Pi)W_q'=Q,$$ $$K'=\mathrm{RMSNorm}(x\Pi)W_k'=K.$$ Thus, although the hidden representation remains transformed, the query and key projections automatically produce their original representations. The GPU can then directly perform the standard RoPE and attention operations: $$Q_r=\mathrm{RoPE}(Q'),\qquad K_r=\mathrm{RoPE}(K').$$ $$S=\mathrm{Softmax}\left(\frac{Q_rK_r^\top}{\sqrt{d_k}}+M_{\mathrm{causal}} \right).$$ In this case, RoPE is executed entirely on the GPU using the pre-transformed projection weights, without requiring access to the secret matrix $\Pi$. --- **Comment:** > Can the authors clarify whether the authorization overhead is independent of the decoder depth? **Response:** We thank you for this important question. Yes, GateFlow's authorization overhead is independent of the decoder depth, as authorization is performed only in the first decoder block. Specifically, the TEE securely generates the transformed representation $$h'=(x+o+f)\Pi,$$ which enables subsequent decoder blocks to operate directly in the orthogonally transformed domain. Unlike interactive TEE--GPU schemes, GateFlow does not require repeated authorization or the recovery of intermediate representations in subsequent blocks. Therefore, the authorization overhead is incurred only once and does not increase with the number of decoder blocks, allowing GateFlow to maintain low-interaction inference as model depth increases.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for your valuable feedback and constructive suggestions. We are encouraged by the positive feedback from reviewers, particularly their acknowledgment that our responses have addressed their concerns, with two reviewers raising their scores. We have also conducted substantial additional experiments and theoretical analyses to further strengthen our work. Below, we summarize the reviewers' feedback and our key revisions to facilitate the final assessment.

ReviewerInitial ScoreUpdated ScoreNotes (Quotes from Discussions)
jDSM23"Your response directly addresses my concerns and questions. I have revised my original score to a 3."
gRQX33-
jZCR22-
U8x523"It resolves my main concern. I’ll keep my overall rating at 3."

1. Enhanced Security and Privacy Analysis.

Addressed Concerns: Reviewer [jDSM] (Weakness [1,2] Question[1]); Reviewer [gRQX] (Weekness [2]); Reviewer [jZCR] (Weakness [1,2]); Reviewer [U8x5] (Weakness [1,2], Question [1,2]).

We strengthened GateFlow's security evaluation by theoretically analyzing the limitations of direct transformation recovery and introducing an adaptive optimization-based attack. We also evaluated additional model-stealing and functional-equivalence attacks, including ArrowMatch-based recovery, obfuscated-model fine-tuning, and TEE authorization attacks. Furthermore, we provided a privacy analysis of first-layer WVW_V disclosure and introduced Fixed-BOS Initialization with corresponding security analysis. The results further demonstrate GateFlow's robustness against these threats.

2. Cryptographic Correctness and Model Compatibility.

Addressed Concerns: Reviewer [jDSM] (Question [2,3]); Reviewer [jZCR] (Question [1,3]).

We clarified the RLWE encryption parameters and theoretically analyzed plaintext growth and decryption noise to establish decryption correctness. We also detailed GateFlow's compatibility with RoPE, including the different mechanisms adopted in the first and subsequent layers. Furthermore, we clarified that the authorization mechanism operates only in the first decoder layer, making its overhead independent of the decoder depth. We additionally discussed the applicability and limitations of our proposed attack to other TEE-GPU schemes, including GroupCover and ArrowCloak.

3. Extended Performance Evaluation and Ablation Studies.

Addressed Concerns: Reviewer [gRQX] (Weakness [1]); Reviewer [jZCR] (Question [2,4]); Reviewer [U8x5] (Question [3]).

We extended our experiments to longer inputs and larger models, evaluating TTFT, throughput, TEE computation, data transfer latency, and communication overhead. Additional ablations, including first-layer execution entirely within the TEE and interactions across all layers, further demonstrate GateFlow's efficiency advantages. We also evaluated Fixed-BOS Initialization, showing negligible additional performance overhead.

Finally, we sincerely thank the Area Chair and all reviewers for their valuable time and constructive feedback. Although some reviewers have not yet responded, we believe that our detailed responses, additional experiments, and theoretical analyses have thoroughly addressed their concerns. We respectfully invite the Area Chair to review our responses and revisions when making the final assessment. We greatly appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.