- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces SHIM, a hybrid framework combines FHE and secret sharing schemes for privacy-preserving LLMs inference. Specifically, SHIM uses a mask to maintained KV cache and non-linear function in plaintext. It claims KV cache storage reduces memory from 198 GB to 3 GB, and 15.7x speedup over NEXUS.
Critical strengths
- This paper identifies the latency and memory bottlenecks brought by non-linear function evaluation and bootstrapping, which are two heavy exponents during inference and valuable to solve for practical secure LLM inference.
Critical weaknesses
-
Security and performance concerns. The paper use masked plaintext as a one-time pad for security, however, this design is problematic under proposed and parameters, where under additive mask. The secret value is close to masked value which is critical security issue. The multiplicative mask has the same problem. In addition, the integration of secret sharing require server transfer CKKS ciphertexts, and this latency and communication cost are not reflected in Table 1, since reported Activation latency (0.002 s) is less than WAN latency setting (0.04 s) and one CKKS ciphertext under is at least 1 MB, 70 MB communication for every non-linear function in LLaMa-3.1-8B seems unrealistic.
-
Soundness concerns. Comparison seems unfair as some of methods are outdated such as NEXUS where 15.7x speedup comes from. Missing SOTA baselines such as BumbleBee(NDSS '25)[1], MOAI(ICLR '26)[2], Cachemir(S&P '27)[3], ROSETTA(CCS '26)[4] undermine validity of the evaluation.
[1] Lu, Wen-jie, et al. "Bumblebee: Secure two-party inference framework for large transformers." Cryptology ePrint Archive (2023).
[2] Zhang, Linru, et al. "MOAI: Module-optimizing architecture for non-interactive secure transformer inference." International Conference on Learning Representations. Vol. 2026. 2026.
[3] Yu, Ye, et al. "Cachemir: Fully homomorphic encrypted inference of generative large language model with kv cache." arXiv preprint arXiv:2602.11470 (2026).
[4] Yu, Jiangrui, et al. "ROSETTA: Efficient and Accurate Privacy-Preserving LLM Decoding via Hybrid CKKS/TFHE Evaluation." arXiv preprint arXiv:2609.16915 (2026).
Questions for authors
-
How do the authors justify the privacy guarantees of mask plaintexts under the actual parameters? Could the authors provide a indistinguishability or leakage analysis, since masked values are extremely close to original values.
-
Does the reported activation latency is correct and provide the runtime breakdown including the communication latency. Also, there should be a detailed presentation of communication volume. Could the authors provide a breakdown of communication rounds, transferred bytes, and latency, and clarify how the three-server WAN environment was implemented?
-
Why are more recent published secure LLM inference work(Bumblebee, MOAI, Cachemir, ROSETTA) are not included in comparison? Also, could author provides a fair comparison under the same security and deployment conditions or illustrate the potential unfair conditions.