acceptodds
Under review as a conference paper at ICLR 2027

Know the Cost Before You Deploy: Simulating HE/MPC Private LLM Serving with CipherScope

Abstract

Homomorphic encryption (HE) and secure multi-party computation (MPC) let a server run a large language model (LLM) on a user's prompt without ever seeing it. However, deploying such private LLM serving requires choosing among many cryptographic and hardware configurations, and trying each one is costly: a single encrypted inference can take minutes to hours, and cost depends on how operators are split between HE and MPC, how the conversation grows during generation, and how many users share the system. We present **CipherScope**, a simulator that predicts the latency, throughput, communication, and protected-memory requirements of a specified private LLM deployment without running its cryptography, so that configurations can be compared before any is deployed. On real MPC executions of transformer models from BERT-tiny up to LLaMA-7B dimensions, CipherScope predicts communication within % and the latency of complete BERT forward passes within approximately %; the 32-layer LLaMA-dimension run validates communication only. It predicts a four-user text-generation schedule replay within % at GPT-2 dimensions and captures effects that simpler estimators miss, such as time spent waiting for precomputed cryptographic material. Harder cases, such as all-HE models and LLaMA-specific layers, show larger timing errors of %, which we report. In-allocation calibration takes about three minutes in our 2PC test; once calibrated, evaluated full predictions take under a millisecond, to times faster than running the deployment.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Secure inference, Homomorphic encryption, Secure multi-party computation, LLM serving, Performance simulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zUeUOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Private LLM applications are time-consuming to deploy and involve a large design space, creating a need for a tool that can estimate the costs of different parameter configurations. This paper presents CipherScope to address this need. CipherScope essentially counts the homomorphic operations and identifies their parameters, then profiles the cost of each operation. The total cost is estimated by summing the costs of all operations. The results show that the estimated costs are close to the actual costs.

Critical strengths
  1. This paper addresses an important need in the community by providing a tool to estimate the costs of different methods for private LLM applications.
  2. The method is reasonable: it counts homomorphic operations and identifies their parameters, profiles the cost of each operation, and estimates the total cost by summing these costs.
Critical weaknesses
  1. The main results are obtained on CPUs. However, GPUs are more important for private LLM applications. I am not sure whether the CPU results can be generalized to GPUs, as the costs of individual operations may be more difficult to calibrate on GPUs.
  2. Related to the above point, if private LLM applications are primarily run on GPUs, the actual development time for LLaMA-7B is around 10 minutes or even less. I therefore wonder how much benefit this tool would provide for GPU-based development.
Questions for authors
  1. Are there plans to extend CipherScope to support GPU-based development?
  2. The makespan reported in Table 3 is much shorter than I expected, as I would expect CPU-based FHE execution to be much slower. It appears that you use existing implementations of these private LLM applications. Could you provide more details?
3 comments
AuthorsOpenReview ↗

Comment Part I

We thank the reviewer for the constructive feedback. (1) CipherScope decouples hardware-invariant structural counts from in-context primitive calibration, providing a direct path to GPU backends; (2) Table 3 is fast because it evaluates LAN-based MPC (534–1,935 s), whereas CPU FHE is evaluated in Table 4 and §5; and (3) even when GPUs shrink single-run compute to ≤10\le 10 min, end-to-end serving remains governed by WAN round stalls (19×19\times latency inflation), multi-tenant contention, and combinatorial exploration costs.


1. Extending CipherScope to GPUs & Calibrating GPU Costs (Q1 & Weakness 1)

Yes. CipherScope’s decoupled architecture separates protocol-structural work from backend-specific kernel timing, enabling direct generalization to GPU MPC (e.g., SIGMA/CrypTen) and GPU CKKS (e.g., Phantom):

  • Hardware-Invariant Structural Counts (Transfers 100% to GPUs): For a given protocol, parameter set, graph, and packing, logical work is hardware-independent. In our validations, structural communication is deterministic across runs and predicted within <0.2%.
  • In-Context GPU Calibration: While isolated GPU kernels exhibit launch, occupancy, and transfer overheads, CipherScope already uses in-context microbenchmarking (profiling primitives inside representative operator contexts rather than in isolation, which eliminated 17–32% cache/context skew in §5.3). Applying this same pre-registered calibration procedure to GPU backends captures sustained SM occupancy, HBM bandwidth saturation on large LLM shapes, and compute–communication overlap.

2. Why Table 3 Makespan is Short & Implementation Details (Q2)

Why Table 3 is fast: It evaluates MPC over a 0.05 ms LAN, whereas FHE is in Table 4 and §5. Your intuition that CPU FHE is much slower is completely correct: in our OpenFHE evaluations, an all-HE BERT-tiny execution takes 298 s (78% in bootstrapping), and a single HE block pass in Table 4 takes 3.8–4.1 s.

In contrast, Table 3 evaluates pure MPC in MP-SPDZ on a 0.05 ms LAN: replicated 3PC for GPT-2-width (12 layers) and LLaMA-width (2 layers), and dishonest-majority 2PC with OT triples for BERT-tiny (2 layers). Across 4 concurrent requests (111 token positions over 11 scheduling steps), measured makespans are 1,796 s, 1,935 s, and 534 s, respectively.

Implementation Details & Validation on Third-Party Native Code:

  • Parameterized Serving Replay & Native Operators: To isolate stateful KV-cache growth and multi-request scheduling in Table 3, our MP-SPDZ harness steps through pre-allocated secret-shared KV matrices at LLaMA/GPT-2 hidden dimensions (following standard reduced-depth/per-block practices in Nimbus [1], BOLT [2], PUMA, and Garimella et al.). We also benchmarked full native LLaMA operator blocks (718 s vs. 458 s for the streamlined projection block) and a continuous 1,024-step autoregressive generation with KV cache growing dynamically from 32 to 1,055 tokens (measured 704.7 s vs. pre-registered prediction of 728.1 s, +3.3% error).
  • Generalization to Unmodified Third-Party Code: Beyond our harness, we evaluated MP-SPDZ’s official native BertLayer. Using 7 non-target calibration shapes, pre-registered predictions match measurements within 5.0%–7.7%:
Unmodified Third-Party Target (MP-SPDZ BertLayer, 3PC LAN)Measured LatencyCipherScope (Pre-Registered)Relative ErrorComm. Bytes Error
BERT-base-width, 2-layer stack (64 tokens, full MHA + GELU)61.9±0.7 s61.9 \pm 0.7\text{ s}65.0 s65.0\text{ s}+5.0%< 0.2%
BERT-mini-width, 4-layer stack (batch 2, full MHA + GELU)72.5±0.4 s72.5 \pm 0.4\text{ s}78.1 s78.1\text{ s}+7.7%< 0.2%
AuthorsOpenReview ↗

Comment Part II

3. Why a System Simulator/Predictor is Essential Even with ≤10\le 10-Min GPU Runs (Weakness 2)

Just as plaintext LLM serving relies on simulators (e.g., Vidur [3]) despite sub-second GPU execution, a single 10-minute GPU run cannot replace CipherScope’s simulation and prediction for three reasons:

  1. Simulating Distributed Network & Resource Regimes Without Cluster Re-deployment: A direct GPU run only measures one fixed physical setup. However, as GPUs shrink local compute, end-to-end latency becomes dominated by sequential WAN rounds, offline preprocessing waiting, and discrete HE slot/key batching steps. In our network study, an execution taking just 1.4 s of compute on LAN balloons by 19×19\times to 26.4 s under a 5 ms WAN delay. CipherScope simulates these distributed regimes within 1.2%–4.3% error purely from protocol structure, without occupying multi-node GPU clusters across network topologies.
  2. Simulating Stateful Multi-Tenant Queueing & Contention: Timing one isolated 10-minute GPU session fails to capture cross-request preprocessing contention, staggered arrivals, and network-compute interleaving. Across our 57 capacity-admission tasks, extrapolating from a solo run violates latency SLAs in 54% of cases, whereas CipherScope’s contention simulator resolves 42/57 configurations with zero target runs.
  3. Pre-Implementation Design Space Exploration: While running one implemented setup takes 10 minutes, sweeping across HE/MPC operator assignments ×\times batch sizes ×\times key groups ×\times WAN delays ×\times concurrency explodes combinatorially. Benchmarking just 100 candidates requires >16 hours of dedicated multi-node, multi-GPU cluster time, plus manually re-deriving ring/modulus chains and recompiling for every hybrid HE/MPC cut. CipherScope simulates and prunes the entire space in seconds before target implementation.

We will add the third-party BertLayer and 1,024-step dynamic KV validations, expand the GPU generalization analysis, and make the MPC vs. FHE workload specifications in Table 3 and Appendix B.1 much more explicit to prevent any potential misunderstanding for readers. We hope these clarifications address your concerns and support the positive assessment of our work.

Reference.

  • [1] Li, Zhengyi, et al. "Nimbus: Secure and efficient two-party inference for transformers." Advances in Neural Information Processing Systems 37 (2024): 21572-21600.
  • [2] Pang, Qi, et al. "Bolt: Privacy-preserving, accurate and efficient inference for transformers." 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.
  • [3] Agrawal, Amey, et al. "Vidur: A large-scale simulation framework for llm inference." Proceedings of Machine Learning and Systems 6 (2024): 351-366.

The authors have addressed most of my concerns. However, I will maintain my original rating of Weak Accept.

Reviewer_ik6cOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CipherScope is a performance simulator for private LLM inference using HE, MPC, and hybrid assignments. It combines assignment-dependent cryptographic costs, stateful serving simulation, and protocol- and key-dependent batching. The evaluation demonstrates accurate MPC communication prediction and useful timing accuracy within several calibrated settings. The main concerns are the positioning of its novelty, the absence of strong compositional baselines, and incomplete validation of interactions central to deployment planning.

Critical strengths
  • The problem is practically important: executing every candidate private-inference configuration is expensive.

  • The framework captures meaningful dependencies among HE segment parameters, operator placement, growing KV state, preprocessing supply, and batching compatibility.

  • Several MPC results are convincing, including approximately 1% or lower communication error and 2.1% makespan error for the four-user GPT-2 replay.

  • The paper carefully distinguishes direct measurements, schedule replays, composed references, and projections. Held-out workloads and predictions recorded before execution strengthen the evaluation.

Critical weaknesses
  • The novelty boundary needs more articulation. CostCO and CryptOracle already provide calibrated cryptographic cost models; SING explicitly supports nonlinear MPC assignment costs; Garimella et al. (ASPLOS 2023) model private-serving arrivals, finite preprocessing storage, and offline waiting; and Vidur models LLM scheduling and KV state. EncFormer also provides a calibrated HE/MPC boundary cost model. CipherScope’s plausible novelty lies in coupling these concerns, particularly HE segment repricing with stateful LLM serving. The paper should distinguish this contribution more precisely, including qualifying §2.2’s broad characterization of prior assignment models as decomposable.

  • The baselines do not establish the incremental value of that coupling. Table 5 demonstrates failures of simplified assumptions, such as ignoring offline waiting or assuming idealized batching. A stronger comparison would combine a conventional serving simulator with calibrated batch/context costs and explicit preprocessing-buffer accounting, using the same measurements and scheduling policy. A backend-informed static HE evaluator would also help isolate the benefit of CipherScope’s assignment evaluation. Accuracy and configuration-selection regret should be compared as the proposed mechanisms are added incrementally.

Questions for authors
  • Can configuration selection be evaluated against the stronger baselines on closely competing candidates whose ranking changes with network, memory, or preprocessing constraints? Selection regret, latency-target violations, and uncertainty would be more informative than predominantly unambiguous protocol choices.
4 comments
AuthorsOpenReview ↗

Comment Part I

We thank the reviewer for the insightful feedback. CipherScope’s novelty lies in jointly modeling cross-layer couplings between cryptographic compilation (HE segment repricing, SIMD slot/key batching) and stateful LLM serving (KV growth, offline buffer depletion, multi-tenant contention). Adding these coupled mechanisms to a calibrated simulator reduces prediction errors from 34%–1,302% down to 4.4%–21%, and achieves 42/57 optimal capacity selections on close candidates without running any target configuration, which outperforms proportional-sharing simulators and matching an oracle contention model that measures every candidate at scale.


1. Incremental Value of Coupling & Stronger Baselines (Weakness 2 & Question)

(a) Incremental Mechanism Addition: Starting from the stronger baseline, a FIFO serving simulator calibrated with per-request/context costs, we add CipherScope’s coupled mechanisms one by one on the same measurements (Error=(predicted−measured)/measured\text{Error} = (\text{predicted} - \text{measured}) / \text{measured}):

Incremental Mechanism Added to SimulatorTarget Workload & Coupling TestedStronger Baseline (Without Mechanism)With Coupled Mechanism (CipherScope)
+ Explicit Preprocessing Buffer & Waiting2PC 4-user replay makespan (stateful KV + offline depletion)−93% (ignores offline generation bottleneck)+4.4%
+ Slot- & Key-Aware HE BatchingHE 43 / 64 shared-key reqs; 8 independent-key usersLinear scaling: +818% / +1,302% Empirical batch curve (B=1,8B=1, 8): −75% / −73% (shared); −88% (multi-key)−21% / −19% (shared); +15% (multi-key, exact passes)
+ Whole-Assignment HE Repricing (vs. Static Evaluator)Unchanged neighbor ops after moving LayerNorm to HE (+51% actual slowdown, 37.5 s)−34% (backend-informed static evaluator retaining pre-move ring/modulus)−21% (dynamically re-derives global segment parameters)
+ Multi-Tenant Preprocessing & WAN Contention57 close capacity decisions across CPU cores & WAN delays26/57 (unadjusted solo) 35/57 (proportional core-sharing simulator)42/57 (zero target-scale runs needed)
  • Why an empirical batch curve fails (Row 2): Fitting a simulator to measured batch sizes (B=1,8B=1, 8) leaves −73% to −88% error because HE cost steps discretely at SIMD slot boundaries and cannot batch across isolated user keys.
  • Why a backend-informed static HE evaluator fails (Row 3): Pre-calibrated static operator costs miss the +51% slowdown on unchanged neighbors when moving LayerNorm forces a deeper circuit and larger ring/modulus parameters. Dynamically re-deriving segment parameters for each candidate is CipherScope’s assignment evaluator.

(b) Selection Regret, Violations, and Uncertainty on Close Candidates (Question): We evaluated three pre-registered studies (57 tasks) where optimal tenant capacity under a 1.75×1.75\times slowdown SLA shifts with CPU cores, arrival staggering, and WAN latency:

  • Compute & Preprocessing Contention (48 tasks): Concurrent 2PC tenants sharing 1–4 cores/party under simultaneous or staggered arrivals (e.g., at 1 core/party staggered, 2 tenants meet the SLA at 1.51×1.51\times, while 3 violate it at 2.14×2.14\times).
  • WAN-Induced Ranking Shifts (9 tasks): On MP-SPDZ’s native BertLayer (3PC), communication waits free local CPU cores for other tenants' preprocessing, shifting maximum feasible capacity from 1 tenant (0 ms) →\rightarrow 6 tenants (1 ms) →\rightarrow 8 tenants (5 ms).
Evaluation MethodTarget Measurements Needed1–2 Cores (24)3–4 Cores (24)WAN Sweep (9)Total (57): Optimal (0 Regret) / SLA Violation / Under-Utilized Regret
Unadjusted Solo Timing1 solo run per candidate8 / 16 / 014 / 10 / 04 / 5 / 026 / 31 / 0
Proportional Core-Sharing Simulator Solo×max⁡(1,n/C)\text{Solo} \times \max(1, n/C)1 solo run per candidate18 / 2 / 414 / 8 / 23 / 0 / 635 / 10 / 12
Measured Solo + Our Contention Model1 solo run per candidate20 / 4 / 014 / 10 / 09 / 0 / 043 / 14 / 0
CipherScope (Pre-Registered)Microbenchmarks only20 / 4 / 015 / 9 / 07 / 2 / 042 / 15 / 0
  • Selection Regret & Violations: Without running any target candidate, CipherScope achieves zero selection regret in 42/57 tasks, beating the Proportional Simulator and matching the Measured-Solo + Contention oracle.
  • Error Magnitude & Uncertainty: Across the 15 boundary violations, predictions admit just 1 extra tenant, exceeding the 1.75×1.75\times SLA by 0.6%–37.7%. We will report multi-run variance ranges and SLA-threshold sensitivity curves in §5.
  • Memory Constraints: Memory growth follows deterministic structural counts calibrated from our runs: per-user HE evaluation keys add ~47 MB/user, and secret-shared KV entries add ~17 B/value. Because these footprints scale linearly with structural counts, memory-capped admission is bounded directly by these calibrated slopes.
AuthorsOpenReview ↗

Comment Part II

2. Articulating the Novelty Boundary & Qualifying §2.2 (Weakness 1)

While prior works model either single-execution cryptographic costs or simplified serving, CipherScope is the first to formulate and capture their cross-layer non-linear couplings:

Prior WorkScope of ModelingHow CipherScope Extends This Scope (Coupled Modeling)
CostCO [1]Single-execution MPC program costCouples MPC with HE parameter selection and stateful multi-request LLM serving
CryptOracle [2]Isolated single-execution HE primitive costsIn-context calibration: Isolated HE rotations cost 17–32% less than inside MatMul (§5.3); adds serving & assignment
SING [3]Single-execution nonlinear MPC assignmentGlobal HE segment repricing (depth changes alter neighbor ring/modulus parameters) coupled with multi-tenant serving
EncFormer [4]Static CKKS–MPC boundary in one transformerEvaluates hybrid assignments under dynamic LLM serving (KV growth, key-group batching, preprocessing contention)
Garimella et al. [5]Private CNN serving (arrivals, finite offline buffers)Stateful LLM mechanics (KV-cache scaling, continuous batching) coupled with HE slot/key batching and segment repricing
Vidur [6]Plaintext LLM serving (profiled ops, KV state)Cryptographic resource mechanics: WAN round stalls, offline depletion, CKKS level consumption, and multi-key isolation

Generalization to Third-Party Code: Using 7 non-target microbenchmark shapes, pre-registered CipherScope predictions on MP-SPDZ’s unmodified native BertLayer achieve +5.0% and +7.7% timing error and <0.2% communication error.

We will add the novelty table, the incremental ablation, and the 57-task selection regret study in the revision. We will also revise §1 and §2.2 to credit nonlinear MPC models and state our novelty boundary, specifically CKKS depth-parameter propagation and stateful serving coupling, and make it more explicit to prevent any potential misunderstanding for readers. We hope these results and clarifications fully address your concerns and support a more positive assessment of our work.

Reference.

  • [1] Fang, Vivian, et al. "CostCO: An automatic cost modeling framework for secure multi-party computation." 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). IEEE, 2022.
  • [2] Brynds, Cory, et al. "Cryptoracle: A modular framework to characterize fully homomorphic encryption." arXiv preprint arXiv:2510.03565 (2025).
  • [3] Blüml, Jannis, et al. "{SING}: Improving the Efficiency of {MPC} Protocol Assignment using Graph Neural Networks." 35th USENIX Security Symposium (USENIX Security 26). 2026.
  • [4] Zhu, Yufan, et al. "EncFormer: Secure and Efficient Transformer Inference over Encrypted Data." IEEE Transactions on Dependable and Secure Computing (2026).
  • [5] Garimella, Karthik, et al. "Characterizing and optimizing end-to-end systems for private inference." Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. 2023.
  • [6] Agrawal, Amey, et al. "Vidur: A large-scale simulation framework for llm inference." Proceedings of Machine Learning and Systems 6 (2024): 351-366.

Thanks for your response. I now agree that the work has considerable novelty and sees the value of coupling. I will increase my score accordingly.

I still have one question, though: The MPC and HE solutions are evolving quickly, and so are LLMs. How do you make sure your simulator keeps up with these fast changes? Does that mean you need to maintain and update it often?

AuthorsOpenReview ↗

Follow-up on Evolving MPC/HE Solutions and LLMs

We sincerely thank you for the positive feedback! Regarding your follow-up question on how CipherScope keeps up with fast-evolving MPC/HE solutions and LLMs:

Keeping up does not require frequent maintenance or simulator redesigns. Instead of hard-coding specific models or protocols, CipherScope relies on a fixed interface: the primitive cost signature (bytes, rounds, offline material, compute time, and HE levels/keys). Upper layers (operators, models, serving) only consume these signatures, while lower layers (protocols, kernels, hardware) simply produce them. As a result, new advances plug in locally with zero to minimal code changes:

  1. New Kernels, Approximations & Hardware →\rightarrow Recalibration Only (Zero Code Changes): Structural counts follow from algorithm definitions, while execution times are fitted via in-context microbenchmarks. Faster kernels, new nonlinear approximations, or new hardware only require a few minutes of recalibration without touching the simulator.
  2. New LLM Architectures →\rightarrow Shape-Parameterized Composition (Zero to 1-Line Update): Operators are defined as shape-parameterized compositions of primitives (MatMul, polynomials, comparisons, rotations). Variants like GQA or SwiGLU simply recombine existing primitives with new dimensions and are priced automatically across any batch or context length.
  3. New MPC/HE Protocols & Serving Policies →\rightarrow One-Time Plug-in: A new cryptographic scheme or primitive enters CipherScope by declaring its structural equations once. Because the serving engine advances queues, KV cache, and shared resources purely from step-level signatures without inspecting the underlying protocol, any new primitive automatically inherits KV scaling, slot/key batching, offline waits, and multi-tenant contention.

In short, adopting a new technique is a repeatable workflow: declare or compose →\rightarrow run minute-level microbenchmarks →\rightarrow verify on a held-out target, rather than a simulator rewrite.

We will incorporate this extensibility and maintenance discussion into the revised paper. Thank you again for your time and supportive engagement.

Reviewer_7wusOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work presents a simulator for private LLM serving under MPC, HE (CKKS), and hybrid HE/MPC setups. It predicts latency, throughput, communication, preprocessing demand, and memory without running the cryptography. The key ideas are, moving an operator into HE changes the parameters and the cost of its whole HE segment, decoding draws on a finite pool of preprocessing material and can stall waiting for it, how much batching helps depends on the protocol and the key setup. The authors validate it against MP-SPDZ and OpenFHE runs on CPUs.

Critical strengths

Solid engineering and careful, honest evaluation.

Interestingly, predictions are registered before the runs. And failures are reported.

Critical weaknesses

I’m not convinced the problem is significant enough. Since transformer layers repeat, timing one layer and extrapolating is cheap. It seems unlikely in a research context that timing measurement is the only need to run a setup (you'd want to test other things) so it's hard for me to imagine a context in which avoiding direct time measurements is actually valuable.

Questions for authors

Is there a concrete decision where prediction, rather than timing one layer or one decoding step, changed the answer.

How does one-layer extrapolation compare with CipherScope across your workloads?

1 comment

We thank the reviewer for the feedback. Timing one layer/step and multiplying works for isolated, single-request execution under a fixed setup. However, it fails in stateful private LLM serving and protocol design, which depend on non-additive axes. For instance, batching, key isolation, KV growth, HE/MPC assignment, WAN delay, and contention. And in these scenarios, one layer extrapolation can cause up to 1,920% error and SLA violations in 54% of capacity decisions.

1. Workload Comparison & Concrete Decisions Changed

A. Non-Additive Axes (Where Unit Timing Fails): Changing a deployment/protocol knob alters the cryptographic or resource regime. CipherScope predictions were pre-registered using only microbenchmarks (Error=(predicted−measured)/measured\text{Error} = (\text{predicted} - \text{measured}) / \text{measured}):

Design AxisConfiguration DependencyTarget & Ground TruthUnit-Timing Error / DecisionCipherScope Error / Decision
HE BatchingShared SIMD slots; cost grows in discrete pass stepsPass time: 21 / 64 shared-key reqs+1,920% / +1,302%+19% / −19% (exact passes)
HE KeysSeparate keys prevent SIMD slot sharing8 independent-key users−88%+15%
KV GrowthStep 1,024 costs 1.8×1.8\times step 11,024 decode steps (704.7 s)−25%+3.3%
AssignmentMoving LayerNorm to HE deepens circuit, enlarging ring/modulus for neighborsUnchanged neighbor slowdown (+51%)−33.8% (predicts 0% impact)−20% (captures shift)
WAN DelayPer-round trip time dominates computeSolo latency at 5 ms WAN (from LAN)−95%+1.2%
Co-locationConcurrent sessions contend for preprocessing coresMax tenants ≤1.75×\le 1.75\times SLA (1 core)Picks 3 (2.14×, Violation)Picks 2 (1.51×, Feasible)
Co-loc + WANNetwork waits leave cores idle, shifting contentionMax tenants ≤1.75×\le 1.75\times (LAN / 5 ms)Picks 8 (7.9×, Violation) / 8Picks 1 / 8 (both Feasible)

B. Concrete Decisions Changed:

  • Capacity Admission (Rows 6–7): Solo-unit timing ignores preprocessing contention and network-compute interleaving, picking 3 or 8 tenants (2.14×,7.9×2.14\times, 7.9\times SLA violations) where CipherScope selects feasible limits (2 and 1/8). Across 57 registered capacity decisions, unit timing violates the SLA in 31/57 cases (54%), whereas CipherScope succeeds in 42/57.
  • HE/MPC Assignment (Row 4): Moving LayerNorm to HE forces a segment-wide ring/modulus increase, slowing unchanged neighbors by +51%. Unit timing of existing ops misses this (0% predicted impact); catching it via measurement requires manually re-implementing and recompiling the new cryptographic layout first.

C. Homogeneous Scaling (Where Unit Timing Works): When all system/cryptographic parameters are fixed and identical layers or steps are stacked, MPC cost is additive (§3.4) and both agree:

  • Depth (LLaMA-7B width, 1 → 4 / 8 layers): Unit timing −1.1% / −14.3% vs. CipherScope +0.2% / −13.2%.
  • Fixed-step replay (4-user decode, GPT-2 / LLaMA / 2PC): Both within ~2–4% on GPT-2 and 2PC.

2. Why Prediction Matters in Research (Weakness)

Even when running a setup to check functional correctness, prediction avoids two bottlenecks:

  1. Decoupling Correctness from Combinatorial Sweeps: Correctness is checked once at small scale (1 layer, LAN, batch 1), whereas system evaluation sweeps assignments ×\times batches ×\times networks ×\times concurrency. Our network/concurrency studies require 15–57 target-scale runs for direct timing, which CipherScope prices beforehand using just 7 microbenchmark probes and 1 network probe.
  2. Avoiding Re-implementation for Suboptimal Cuts: Shifting HE/MPC boundaries alters packing and modulus chains. Without prediction, researchers must implement, tune, and compile dozens of cryptographic variants just to time one layer of each.

We will add the unit-timing baseline and 57-case study to §5, and state the distinction between unit scaling and stateful serving more explicitly in §1 to prevent potential reader misunderstandings. We hope these results address your concerns and support a positive assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.