- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Private LLM applications are time-consuming to deploy and involve a large design space, creating a need for a tool that can estimate the costs of different parameter configurations. This paper presents CipherScope to address this need. CipherScope essentially counts the homomorphic operations and identifies their parameters, then profiles the cost of each operation. The total cost is estimated by summing the costs of all operations. The results show that the estimated costs are close to the actual costs.
Critical strengths
- This paper addresses an important need in the community by providing a tool to estimate the costs of different methods for private LLM applications.
- The method is reasonable: it counts homomorphic operations and identifies their parameters, profiles the cost of each operation, and estimates the total cost by summing these costs.
Critical weaknesses
- The main results are obtained on CPUs. However, GPUs are more important for private LLM applications. I am not sure whether the CPU results can be generalized to GPUs, as the costs of individual operations may be more difficult to calibrate on GPUs.
- Related to the above point, if private LLM applications are primarily run on GPUs, the actual development time for LLaMA-7B is around 10 minutes or even less. I therefore wonder how much benefit this tool would provide for GPU-based development.
Questions for authors
- Are there plans to extend CipherScope to support GPU-based development?
- The makespan reported in Table 3 is much shorter than I expected, as I would expect CPU-based FHE execution to be much slower. It appears that you use existing implementations of these private LLM applications. Could you provide more details?
3 comments
Comment Part II
3. Why a System Simulator/Predictor is Essential Even with -Min GPU Runs (Weakness 2)
Just as plaintext LLM serving relies on simulators (e.g., Vidur [3]) despite sub-second GPU execution, a single 10-minute GPU run cannot replace CipherScope’s simulation and prediction for three reasons:
- Simulating Distributed Network & Resource Regimes Without Cluster Re-deployment: A direct GPU run only measures one fixed physical setup. However, as GPUs shrink local compute, end-to-end latency becomes dominated by sequential WAN rounds, offline preprocessing waiting, and discrete HE slot/key batching steps. In our network study, an execution taking just 1.4 s of compute on LAN balloons by to 26.4 s under a 5 ms WAN delay. CipherScope simulates these distributed regimes within 1.2%–4.3% error purely from protocol structure, without occupying multi-node GPU clusters across network topologies.
- Simulating Stateful Multi-Tenant Queueing & Contention: Timing one isolated 10-minute GPU session fails to capture cross-request preprocessing contention, staggered arrivals, and network-compute interleaving. Across our 57 capacity-admission tasks, extrapolating from a solo run violates latency SLAs in 54% of cases, whereas CipherScope’s contention simulator resolves 42/57 configurations with zero target runs.
- Pre-Implementation Design Space Exploration: While running one implemented setup takes 10 minutes, sweeping across HE/MPC operator assignments batch sizes key groups WAN delays concurrency explodes combinatorially. Benchmarking just 100 candidates requires >16 hours of dedicated multi-node, multi-GPU cluster time, plus manually re-deriving ring/modulus chains and recompiling for every hybrid HE/MPC cut. CipherScope simulates and prunes the entire space in seconds before target implementation.
We will add the third-party BertLayer and 1,024-step dynamic KV validations, expand the GPU generalization analysis, and make the MPC vs. FHE workload specifications in Table 3 and Appendix B.1 much more explicit to prevent any potential misunderstanding for readers. We hope these clarifications address your concerns and support the positive assessment of our work.
Reference.
- [1] Li, Zhengyi, et al. "Nimbus: Secure and efficient two-party inference for transformers." Advances in Neural Information Processing Systems 37 (2024): 21572-21600.
- [2] Pang, Qi, et al. "Bolt: Privacy-preserving, accurate and efficient inference for transformers." 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.
- [3] Agrawal, Amey, et al. "Vidur: A large-scale simulation framework for llm inference." Proceedings of Machine Learning and Systems 6 (2024): 351-366.
The authors have addressed most of my concerns. However, I will maintain my original rating of Weak Accept.
Comment Part I
We thank the reviewer for the constructive feedback. (1) CipherScope decouples hardware-invariant structural counts from in-context primitive calibration, providing a direct path to GPU backends; (2) Table 3 is fast because it evaluates LAN-based MPC (534–1,935 s), whereas CPU FHE is evaluated in Table 4 and §5; and (3) even when GPUs shrink single-run compute to ≤10 min, end-to-end serving remains governed by WAN round stalls (19× latency inflation), multi-tenant contention, and combinatorial exploration costs.
1. Extending CipherScope to GPUs & Calibrating GPU Costs (Q1 & Weakness 1)
Yes. CipherScope’s decoupled architecture separates protocol-structural work from backend-specific kernel timing, enabling direct generalization to GPU MPC (e.g., SIGMA/CrypTen) and GPU CKKS (e.g., Phantom):
2. Why Table 3 Makespan is Short & Implementation Details (Q2)
Why Table 3 is fast: It evaluates MPC over a 0.05 ms LAN, whereas FHE is in Table 4 and §5. Your intuition that CPU FHE is much slower is completely correct: in our OpenFHE evaluations, an all-HE BERT-tiny execution takes 298 s (78% in bootstrapping), and a single HE block pass in Table 4 takes 3.8–4.1 s.
In contrast, Table 3 evaluates pure MPC in MP-SPDZ on a 0.05 ms LAN: replicated 3PC for GPT-2-width (12 layers) and LLaMA-width (2 layers), and dishonest-majority 2PC with OT triples for BERT-tiny (2 layers). Across 4 concurrent requests (111 token positions over 11 scheduling steps), measured makespans are 1,796 s, 1,935 s, and 534 s, respectively.
Implementation Details & Validation on Third-Party Native Code:
MP-SPDZ BertLayer, 3PC LAN)