- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper shows that in TEE-partitioned LLM inference, where an untrusted GPU runs the linear projections, the exposed intermediate states can leak the prompt. Its Prompt-State Matching Attack lets a model-aware provider probe every vocabulary token on its own copy of the model and match direct, invariant, or joint-state fingerprints against the victim’s exposed states. This recovered 100% of tokens from unprotected Q/K/V/S/A/O at first, middle, and last layers across four LLMs, and 99.81–100% under masking, permutation, and orthogonal encodings. As a defense, StateVeil masks quantized activations with fresh sparse Dual-LPN masks over a finite field, using a shared public matrix. The TEE unmasks the exact integer products the GPU returns.
Critical strengths
-
The attack analysis is systematic and shows that common transformations don’t protect exposed states。
-
The proposed framework is evaluated against its own attack, with carefully claims.
Critical weaknesses
-
The attack is largely a re-application of known vocabulary matching under assumptions that favor the attacker, and parts of it are not measured. The 100% recovery numbers come from an idealized setting: FP32, deterministic exact computation, and a probe model identical to the victim’s. They also assume the adversary can see the exposed states and position-aligned heads. So the paper doesn’t show that these attacks work against the actual deployed systems it names (Talaria, KV-Cloak, STIP) in realistic conditions.
-
StateVeil’s security evidence is thinner than the abstract suggests.The privacy theorem is conditional on an exact-weight Dual-LPN assumption. The cost estimates (about 2^162–2^168) are for classical attacks on a single instance. The usable guarantee depends on a heuristic exposure cap (about 2^24 token positions per master matrix). The paper itself says this is “not a proven security limit.” In practice that is roughly 47k requests of 256+100 tokens before the matrix must be rotated, which is a real operational cost that is barely evaluated. The empirical defense test is a single matcher (Euclidean direct matching) at 0% recovery. That shows only that this specific attack fails, and it has no stronger or learned adversaries.
-
Limited novelty. The attack applies the existing vocabulary-matching recipe [1,2] to more tensors (Q, K, V, S, A, O). The 100% direct-matching result follows from determinism: an unmasked state matched against an identical probe model should match exactly. The “invariant” and “joint” strategies reuse known tricks, such as sorting to remove permutations and paired Q/K encodings from KV-Cloak and STIP. The joint result is derived analytically (Eq. 8), not measured. StateVeil combines Slalom/Talaria-style additive masking, int8 quantization with modular arithmetic, and an off-the-shelf Dual-LPN assumption. Its new elements are a shared public master matrix and precomputed products, which are engineering choices. The privacy theorem is a conditional reduction to an existing assumption. The paper never states precisely what is new relative to these baselines.
[1] Rahul Krishna Thomas, Louai Zahran, Erica Choi, Akilesh Potti, Micah Goldblum, and Arka Pal. Hidden no more: Attacking and defending private third-party LLM inference. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 59434–59469. PMLR, 2025. [2] Zhifan Luo, Shuo Shao, Su Zhang, Lijing Zhou, Yuke Hu, Chenxu Zhao, Zhihao Liu, and Zhan Qin. Shadow in the cache: Unveiling and mitigating privacy risks of KV-cache in LLM inference. In Network and Distributed System Security Symposium. Internet Society, 2026.
Questions for authors
see weakness
3 comments
2. Conditional privacy, exposure budget, and stronger attack evaluations
We thank Reviewer Esmk for highlighting the distinction between the formal guarantee, concrete estimates, operational assumptions, and empirical evidence.
Formal guarantee and concrete estimates. StateVeil’s privacy theorem is a conditional reduction under the stated exact-weight Dual-LPN assumption. Its analysis accounts for jointly exposed values and repeated observations under the shared public matrix. The approximately – estimates concern the analyzed classical attacks on a single instance; they do not establish an equivalent security level for the complete service under arbitrary repeated observations.
Exposure budget and rotation. The approximately -token-position cap is a heuristic operational budget, not a proven security limit. Exposure accounting must include observations across projections sharing the same matrix.
We measured the preparation time for the master matrix and projection-specific recovery tables using one NVIDIA L40S GPU and 16 CPU threads. Preparation takes 48 seconds for Llama-3.2-1B and 424 seconds for Llama-3.1-8B. Under the illustrative budget of approximately 47k requests per matrix, with 256 input and 100 output tokens per request, the amortized preparation overhead is small, approximately 1.0 ms and 9.0 ms per request, respectively. This budget remains a heuristic operating choice under the stated exposure accounting. Preparation costs are reported separately from online inference latency.
Additional learned and optimization-based adversaries. We agree that failure of Euclidean direct matching alone is insufficient empirical coverage. We have added three complementary attack families:
- TokenInfer [1]: classification-based attack. A learned classifier maps exposed intermediate representations to token identities.
- TokenInv [2]: optimization-based attack. The attacker optimizes continuous input embeddings and recovers discrete tokens using activation consistency and semantic information.
- SentenceInv [3]: generation-based attack. A learned autoregressive decoder reconstructs text from exposed representations, targeting sentence-level content and semantics.
We compare the unprotected model, Talaria, and StateVeil using Llama-3.2-1B on LMSYS-Chat data.
We report Token Reconstruction Accuracy (TRA), the proportion of original tokens correctly recovered, and Cosine Semantic Similarity (CSS), the cosine similarity between embeddings of reconstructed and original text. Lower values indicate less successful reconstruction.
| Attack | Original TRA | Original CSS | Talaria TRA | Talaria CSS | StateVeil TRA | StateVeil CSS |
|---|---|---|---|---|---|---|
| State Matching | 100.00% | — | 100.00% | — | 0.00% | — |
| TokenInfer | 90.10% | 84.25% | 2.89% | 8.76% | 2.86% | 7.08% |
| TokenInv | 100.00% | 100.00% | 0.58% | 12.19% | 0.00% | 7.56% |
| SentenceInv | 29.70% | 72.79% | 5.78% | 16.64% | 5.59% | 17.42% |
Bold indicates the lower value between the two defenses. “—” indicates an unreported metric.
Both defenses substantially reduce reconstruction under the additional attacks. StateVeil achieves slightly lower TRA, while SentenceInv yields lower CSS against Talaria. These small differences do not establish a consistent advantage across all metrics. The clearest distinction is State Matching: it achieves 100% token recovery against the evaluated Talaria configuration and 0% against StateVeil, identifying a leakage mode not captured effectively by the other attacks.
These results extend the defense evaluation beyond a single Euclidean matcher to include learned classification, optimization, and generative reconstruction. They provide broader empirical evidence under the tested settings, while the general privacy guarantee remains conditional on the stated assumptions and exposure model.
[1] Xinjian Luo, Ting Yu, and Xiaokui Xiao. Prompt Inference Attack on Distributed Large Language Model Inference Frameworks. ACM CCS, 2025.
[2] Wenjie Qu et al. Prompt Inversion Attack Against Collaborative Inference of Large Language Models. IEEE Symposium on Security and Privacy, 2025.
[3] Kang Gu et al. Towards Sentence Level Inference Attack Against Pre-trained Language Models. Proceedings on Privacy Enhancing Technologies, 2023(3), pp. 62–78.
We thank Reviewer Esmk again for the suggestions that motivated this broader evaluation and clearer separation of security claims.
3. Clarifying the contributions relative to prior work
We thank Reviewer Esmk for requesting a more precise novelty statement. We acknowledge the established foundations of vocabulary-based reconstruction, transformation invariants, additive masking, and Dual-LPN. Our contributions concern their application and analysis at specified outsourcing boundaries and the construction of StateVeil for protected LLM linear execution.
Prompt-State Matching Attack: analysis of specified TEE-boundary observations.
Thomas et al. demonstrate vocabulary-based reconstruction from hidden states, including transformed hidden states, and Luo et al. investigate reconstruction from exposed KV caches. Our contribution is to characterize what remains identifiable from the particular values exposed by outsourced operations, including relationships between simultaneously observable states.
This analysis considers three cases: direct fingerprints from exposed attention states, invariant fingerprints from token-aligned encodings, and joint fingerprints derived from paired observations. For example, individual Q/K transformations can obscure their coordinates while preserving a joint score fingerprint that remains comparable across independently instantiated transformations.
Determinism explains why the correct candidate reproduces an unprotected state. Unique token recovery additionally requires sufficient discrimination between candidate fingerprints: deterministic functions can still produce collisions. We treat the direct-matching results as a baseline and examine which states and preserved relationships retain this discrimination. The additional BF16 experiments demonstrate that recovery is not uniformly perfect across states and models.
StateVeil: fresh sparse masking with reusable recovery tables.
For a quantized linear operation, StateVeil uses a public matrix and a projection-specific table . For each protected input row, it samples fresh exact-weight sparse coefficients , sends , and subtracts from the returned product. The sparse coefficients allow both masking and recovery to use sparse combinations of table rows.
Compared with Slalom. Slalom adds a fresh pseudorandom mask to each input and removes its effect using a precomputed mask–weight product. It provides cryptographic privacy and computation integrity, but each recovery record is used only once, requiring continued preprocessing and loading during sustained inference. StateVeil generates fresh masks and corresponding recovery terms from reusable tables. Within the analyzed exposure budget, this avoids replenishing per-invocation recovery records while keeping online masking and recovery sparse. Its privacy guarantee relies on the exact-weight Dual-LPN assumption and accounts for repeated observations under the shared matrix.
Compared with Talaria. Talaria combines a reusable public low-rank basis with fresh private coefficients and uses precomputed basis–weight products for recovery. This low-rank masking leaves some input features unchanged, which our State Matching attack exploits in the evaluated configuration. StateVeil uses a full-column-rank public matrix and fresh sparse finite-field coefficients, removing that particular public-nullspace leakage. Its broader privacy guarantee is supplied by the conditional reduction.
The specific contribution combines a shared master matrix across projection widths, projection-specific recovery tables, fresh sparse mask generation, exact quantized execution, and an analysis of jointly exposed values and repeated observations. Our TDX prototype demonstrates this construction in an inference system, achieving end-to-end latency comparable to our Talaria reimplementation.
We thank Reviewer Esmk again for helping us make these technical distinctions and the scope of the contribution more explicit.
1. Attack assumptions and additional experimental evidence
We thank Reviewer Esmk for the detailed comments on the attack assumptions and experimental coverage.
Our intended adversary is the model-aware cloud provider, which knows the model weights, tokenizer, and inference implementation. Access to an identical probe model follows from this threat model. The victim’s TEE secrets remain unavailable, and each matching construction requires fingerprints that can be compared without those secrets. Our conclusions are scoped to the states observable under the specified outsourcing configuration.
Additional FP16/BF16 and MoE experiments. We have expanded the evaluation to include FP16/BF16 experiments on Llama-3.2-1B, Llama-3.1-8B, Qwen2.5-7B, Qwen3-8B, and OLMoE-1B-7B-0924, together with FP32 experiments on OLMoE.
Direct matching covers the first, middle, and last layers and all six states, Q,K,V,S,A,O. Invariant matching is evaluated at layer 0 using:
Gram and Projection are evaluated on Q,K,V,A,O, with S not evaluated; Sorting covers all six states.
Under FP16, every evaluated direct-matching and invariant-matching configuration achieves 100% token recovery across all five models. In the additional FP32 OLMoE experiments, direct matching achieves 100% for all six states at layers 0, 8, and 15, and every evaluated invariant fingerprint achieves 100% at layer 0.
BF16 direct-matching results:
Under BF16, direct matching of Q,K,V maintains 97.9%–100% recovery across all evaluated models and layers. However, recovery drops substantially for some S,A,O states. For example, the corresponding rates at the last layer of Qwen2.5-7B are 37.3%, 7.6%, and 57.6%. These results demonstrate that recovery depends on numerical precision, model, layer, and exposed state.
BF16 invariant-matching results: The following table lists configurations with recovery below 100%. Every tested state in all remaining evaluated configurations achieves 100%. All experiments use layer 0; “—” denotes a state that was not evaluated.
Additional joint-state matching experiments. We also test the specified KV-Cloak and STIP Q/K transformations offline.
For KV-Cloak’s operator-fused transformation, Q=QM−T and K=KM. Their joint product preserves QK⊤. For STIP’s evaluated feature-permutation configuration, Q=QΠ and K=KΠ, giving the same joint-product invariance. We compute these fingerprints from transformed layer-0 state pairs and use them for vocabulary-based reconstruction:
These experiments add measured evidence for the specified joint fingerprints. Our named-system conclusions refer to these evaluated transformation configurations and our Talaria reimplementation.
We thank Reviewer Esmk for prompting these additional evaluations and a clearer statement of their scope.