acceptodds
Under review as a conference paper at ICLR 2027

Scalable Multi-User Secure Transformer Inference with SSE-Enabled Threshold CKKS

Abstract

Homomorphic encryption (HE) enables privacy-preserving transformer inference on encrypted inputs, but existing systems largely assume a single cryptographic owner. Extending HE inference to independent users therefore requires reusable collective cryptographic infrastructure. For transformer inference, we further employ sparse-secret encapsulation (SSE) to make CKKS bootstrapping more level-efficient. However, SSE introduces a fixed-weight sparse secret in addition to the dense secret used for regular computation, making distributed setup substantially more challenging. We develop an SSE-enabled threshold-CKKS framework centered on a reusable two-server-assisted setup. It generates dense and sparse collective secrets while preserving their prescribed distributions and structures, realizes both first in the Boolean-sharing domain, and delays RNS conversion until distributed CKKS and SSE key generation. Built on this infrastructure, ciphertexts from multiple independent users are consolidated through interleaved packing for shared Transformer evaluation, followed by per-user extraction and synchronized threshold decryption. With 16 users, shared inference reduces service time by with only interface overhead, while maintaining task accuracy within percentage points of plaintext inference and producing identical predictions to single-user encrypted inference.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Secure Transformer Inference, Threshold Homomorphic Encryption, Sparse-secret Encapsulation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_VegEOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a threshold CKKS-based secure transformer inference framework for multi-user setting. While one can straightforwardly achieve multi-user setting via threshold CKKS, the distributed setup for ternary/sparse secret keys remains a core bottleneck. This paper’s solution is to employ two, non-colluding auxiliary servers and run a generic two-party computation protocol (GMW in the paper) to evaluate the Boolean circuit corresponding to ternary key sampling. After then, both parties lift the Boolean key shares to an RNS domain where CKKS ciphertexts live in. To achieve multi-user inference, they extend interleaved batching through a careful packing of ciphertexts with designated payload placements per party. The authors provided an implementation of their framework on anonymous GitHub/supplementary material and presented benchmark results on various numbers of parties.

Critical strengths
  • The MPC-based sparse ternary key sampling looks interesting—though I did not fully verify the novelty of the method—and could be reused for other privacy-preserving computation tasks.
Critical weaknesses
  • I think the paper’s contribution lies beyond the scope of ICLR. From my understanding, the key contribution of the paper is about efficiently incorporating the SSE technique to threshold CKKS during collaborative key setup phase. No techniques specialized for “transformer” architecture were introduced, unlike other STI papers that has published/presented in ML venues. This paper’s contribution can be stated even without introducing the STI scenario, and I personally think this paper should have been submitted to security or applied cryptography venues.

  • The paper introduces somewhat strong assumption to solve the distributed key generation problem: Two non-colluding servers for key generation. However, the authors did not justify why such a setting is reasonable in practice. This gives me the impression that the authors sidestep the problem by introducing a strong assumption rather than addressing it directly. Of course, I don’t deny the GMW-based key generation is one of the possible approaches. But the rationale for adopting this approach should be more carefully treated and justified as it deviates the problem setting. After all, this approach makes the comparison with Min et al. (Table 5) unfair in terms of the underlying assumption.

Questions for authors
  • I think the core technical parts of the paper are the distributed dense/sparse secret generation (Section 4.1, 4.2) but I could not assess how novel they are due to the lack of detailed comparison on sampling methods from prior works, e.g., Min et al. Could you compare your method with Min et al.’s idea, and if possible, could you discuss about what happens if you adapt Min et al.’s approach to your GMW-based framework?

  • After reading Table 4, I have doubts about the practicality of the proposed protocol. For n×nn\times n matrix multiplication, nearly 2n2 \sqrt{n} rotation keys (e.g., Galois keys in the paper) should be prepared, i.e., at least tens or more keys should be prepared for computational efficiency of the algorithm (e.g., well-known Halevi-Shoup's algorithm with BSGS optimziation). However, even for a single rotation key, Table 4 says that the communication cost for this takes nearly 2.5GB. Hence, if we wish to multiply two 512×512512 \times 512 matrices, which is a moderate size in typical ML applications, at least 46 rotation keys should be generated, eventually yielding ≥100\ge 100GB of communication during setup. I don't think such an amount of communication is practical. Could you comment on this overhead, or any mitigations to circumvent this issue?

  • I’m not fully convinced the claimed efficiency gain reported in Table 8 is reasonable. In my opinion, there would be a trade-off between the number of tokens handled by a single ciphertext and the number of users involved. Hence, such a gain would disappear when the user’s input is long enough; so, in this case it is necessary to use all the available slots of a single ciphertext. Could you comment on how the efficiency gain observed in Table 8 is affected in such cases?

8 comments
AuthorsOpenReview ↗

Response to Reviewer VegE (W1; W2)

We thank the reviewer for these constructive comments and insightful questions. We address each concern below.

W1: We sincerely thank the reviewer for recognizing our multiparty threshold CKKS construction, and we agree that this construction has broader applicability. However, as discussed in the Introduction, our work focuses on efficient secure Transformer inference (STI) in multi-user settings, for which establishing a multi-user cryptographic infrastructure is essential. Beyond this cryptographic construction, we develop a multi-user STI framework incorporating user-input packing and synchronized threshold decryption, both of which directly support multi-user STI. Furthermore, our inference framework builds upon the interleaved batching (IB) technique introduced in MOAI, which is specifically designed to exploit the computational characteristics of Transformer architectures. Thus, our framework inherently leverages Transformer-specific properties through IB, even though these optimizations are not explicitly introduced as new architectural techniques. We therefore believe that our contributions are well aligned with the scope of ICLR.

W2: Introducing two non-colluding auxiliary servers represents a deliberate tradeoff between security and practicality. As stated in the Introduction, “Our key insight is to decouple distributed ownership of the common cryptographic infrastructure from its expensive construction.” The primary benefit is to decouple the communication-intensive secret and key generation procedures from the number of participants. By performing these procedures through communication between two auxiliary servers, our approach substantially reduces communication overhead and limits noise accumulation during key generation.

This design choice is particularly important for distributed key generation. For example, Min et al.'s approach requires all participants to contribute secret shares and noise during CKKS secret and key generation. As the reviewer noted in Q2, generating even a single key incurs substantial communication overhead under our two-server setting. In contrast, the communication cost of Min et al.'s approach scales with the number of participants, reaching approximately 40 GB per key for 16 participants.

Regarding noise accumulation, as shown in Algorithms 5–7, our construction introduces approximately twice the noise contribution of a conventional single-party construction, whereas the number of noise contributions in Min et al.'s approach increases with the number of participants. Thus, the two-server design provides a practical means of controlling both communication overhead and key-generation noise while preserving distributed key ownership.

AuthorsOpenReview ↗

Response to Reviewer VegE (Q1)

Q1: Our approach differs from Min et al.'s Distributed Key Generation for Efficient Threshold-CKKS in four main aspects.

1. System and security assumptions. Min et al. consider a multiparty threshold-CKKS setting that tolerates the corruption of up to M−1M-1 participants. Their construction first establishes a temporary multiparty threshold-CKKS instance to support subsequent homomorphic secret generation. In contrast, our framework introduces two non-colluding auxiliary servers for setup and a separate inference server for multi-user secure Transformer inference (STI). The auxiliary servers construct the shared cryptographic infrastructure and subsequently distribute secret shares to the users, separating key construction from distributed key ownership.

2. Cryptographic primitives. Min et al. rely on CKKS-based homomorphic evaluation to generate secret keys, whereas our construction employs secure multiparty computation (MPC) over Boolean shares. This distinction allows our framework to perform secret sampling and selection directly in the Boolean domain, avoiding the homomorphic evaluation and bootstrapping required by CKKS-based secret generation.

3. Key-generation overhead and inference support. In the fully distributed construction of Min et al., participants contribute to the initial threshold-CKKS setup, introducing communication and noise contributions that depend on the number of participants. For independently sampled noise terms of equal variance, the aggregate noise standard deviation grows proportionally to M\sqrt{M}. In contrast, the main key-generation communication in our framework occurs between two auxiliary servers, making this component independent of the number of users and limiting the corresponding noise contributions to two parties. Beyond distributed key generation, our framework provides user-input packing, multi-user ciphertext processing, and synchronized threshold decryption to support practical multi-user STI.

4. Secret-generation mechanisms. The two approaches employ fundamentally different procedures for generating dense and sparse secrets.

Dense-secret generation. Min et al. employ a temporary threshold-CKKS instance and use small-integer bootstrapping (SI-BTS) and bit extraction to obtain encrypted random values modulo 3, from which ternary dense-secret coefficients are generated. In contrast, our framework uses Boolean rejection sampling: each coefficient is sampled from two random bits, the combination (1,1)(1,1) is rejected, and the remaining combinations are mapped to {0,+1,−1}\{0,+1,-1\}, yielding the target ternary distribution.

Sparse-secret generation. Min et al. use SI-BTS and bit extraction to generate encrypted random bits, followed by OneHotVecGen to construct random one-hot vectors. They then compute prefix sums of these vectors, eliminate duplicate positions, and use Hamming-weight detection and an Integer Indicator Function to select a vector reaching the prescribed weight. After sign assignment, ShGen converts the resulting sparse secret into additive shares held by the participants.

In contrast, our framework directly samples candidate position encodings under Boolean MPC and applies collision filtering to obtain hh distinct positions. These positions are then expanded into one-hot vectors to construct a fixed-weight ternary sparse secret. By performing collision detection before one-hot expansion, our approach avoids constructing and deduplicating prefix sums of encrypted one-hot vectors.

Importantly, directly transferring Min et al.'s sparse-secret generation algorithm to a GMW-based MPC framework would require redesigning and reanalyzing its underlying operations, particularly the Integer Indicator Function, OneHotVecGen, and the prefix-sum selection procedure. Their efficiency and security properties rely on CKKS homomorphic evaluation and SIMD processing and therefore do not directly carry over to Boolean MPC.

Overall, while both approaches aim to generate distributed CKKS secrets with prescribed distributions, they differ in their security assumptions, cryptographic primitives, sampling algorithms, and scalability characteristics. Moreover, our secret construction forms part of an integrated framework specifically designed for multi-user secure Transformer inference.

We will add a subsection to Appendix A.2 to briefly clarify the differences between our approach and that of Min et al.

AuthorsOpenReview ↗

Response to Reviewer VegE (Q2; Q3)

Q2: A practical cryptographic infrastructure indeed requires multiple Galois keys. Under our configuration with N=65,536N=65,536, the multi-user STI pipeline provisions 31 Galois keys, comprising rotation keys for steps ±2kk=014\\{\pm 2^k\\}_{k=0}^{14} and one complex-conjugation key. Other rotation steps can be realized by composing these supported rotations.

As specified in Algorithm 7, the communication cost reported in Table 4 consists of three components: the transmission of the common random polynomial aa from auxiliary server C0C_0 to C1C_1, and the exchange of their respective key-generation shares b(0)b^{(0)} and b(1)b^{(1)}. Since these values are represented in RNS form, with N=65,536N=65,536, 40 RNS limbs, and 64-bit storage per coefficient, each RNS polynomial occupies

65,536×40×8=20,971,520 bytes=20 MiB.65,536\times40\times8=20,971,520\ \text{bytes}=20\ \text{MiB}.

Our configuration uses 39 limbs in QQ and one auxiliary limb in PP, resulting in 39 gadget-decomposition components. Each component requires a separate set of aa, b(0)b^{(0)}, and b(1)b^{(1)}, yielding a total communication cost of 20×3×39=2,34020\times3\times39=2,340 MiB (2,453.67 MB) per key.

This overhead can be reduced in two ways. First, as supported by Algorithm 7, generating the common random polynomial aa from a public seed eliminates its full transmission, reducing the communication cost per key to approximately 1,635.78 MB. Second, when the required multiplicative depth is lower, hybrid key-switching decomposition [1] can reduce the number of decomposition components by using a larger auxiliary modulus PP. This provides a tradeoff between key-generation communication and the modulus requirements of key switching, subject to the overall security and depth constraints.

[1] Han, Kyoohyung, and Dohyeong Ki. "Better bootstrapping for approximate homomorphic encryption." In Cryptographers’ Track at the RSA Conference, pp. 364-390. Cham: Springer International Publishing, 2020.

We will include these discussions on Galois-key generation in the appendix.

Q3: We acknowledge the reviewer's concern. As stated in Section 6.5, “The gain comes from sharing otherwise underutilized packing capacity across independent users, rather than accelerating the HE operators or enlarging the packing capacity.” This limitation stems from the finite number of SIMD slots available in CKKS ciphertexts. For example, with N=65,536N=65,536, each ciphertext provides 32,768 slots. When a user's input reaches 32,767 tokens, the request occupies the entire packing capacity, including the reserved output slot, leaving no additional slots for other users. Consequently, the multi-user efficiency gains reported in Table 8 disappear in this case. To further illustrate this limitation, the following table presents theoretical efficiency estimates for varying input sequence lengths and numbers of users under N=65,536N=65,536.

Table: Theoretical multi-user speedup under varying input sequence lengths (N=216N=2^{16}).

Input tokens LLU=1U=1U=4U=4U=16U=16
1271.000×1.000\times4.000×4.000\times16.000×\mathbf{16.000\times}
10231.000×1.000\times4.000×4.000\times16.000×\mathbf{16.000\times}
20471.000×1.000\times4.000×4.000\times16.000×\mathbf{16.000\times}
40951.000×1.000\times4.000×4.000\times8.000×8.000\times
81911.000×1.000\times4.000×4.000\times4.000×4.000\times
163831.000×1.000\times2.000×2.000\times2.000×2.000\times
327671.000×1.000\times1.000×1.000\times1.000×1.000\times

The table above presents theoretical upper bounds on multi-user speedup under varying input sequence lengths, assuming ideal slot utilization and negligible batching overhead. With N=65,536N=65,536, each CKKS ciphertext provides 32,768 SIMD slots. As the input sequence length increases, fewer independent requests can share the same packing capacity, gradually reducing the achievable multi-user speedup. For 16 users, the ideal speedup remains 16×16\times for input lengths up to 2,047 tokens, decreases to 8×8\times at 4,095 tokens, and eventually reaches 1×1\times at 32,767 tokens. These estimates illustrate the inherent packing-capacity limitation of our framework rather than an improvement in the underlying HE operators.

Reviewer_isqJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a secure inference system based on SSE and threshold CKKS. The authors design a distributed key generation protocol facilitated by two auxiliary servers, reducing initialization computation and communication overhead through techniques such as Boolean-first secret generation and deferred RNS conversion. By employing an extended interleaved batching mechanism, the system enables multiple users to share a single Transformer inference pipeline while ensuring the secure recovery of each user's specific output. Experimental results demonstrate that deferred RNS conversion yields significant performance gains. In evaluations using the LLaMA3-8B model across three text classification tasks, joint inference achieves a speedup proportional to the number of users compared to serial inference, while maintaining output consistency with single-user encrypted inference.

Critical strengths

Efficient Cryptographic Protocol Design: The authors utilize two non-colluding auxiliary servers to generate dense and sparse keys which follow specified distribution within a Boolean domain. Experiments demonstrate that deferred RNS conversion effectively reduces intermediate computations, leading to significant improvements in both latency and communication overhead.

Effectively addresses key incompatibility issues. Through a unified threshold CKKS key system, multiple users can independently encrypt data and perform shared computations without needing to provide the full decryption private key to the server.

The experimental validation is quite comprehensive, covering aspects such as inference speed, key correctness, distribution, and scalability.

Critical weaknesses

The actual deployment conditions are quite stringent. Decryption requires the collaboration of multiple users; if a user leaves or is unable to participate, it may prevent other users from obtaining the result.This poses a challenge to the practical utility of the framework.

Questions for authors

The end-to-end evaluation in the paper is based solely on natural language classification tasks. Given that real-world large language model services typically involve autoregressive text generation, would the authors discuss the applicability of this framework to multi-step generation tasks?

To enable the merging of ciphertexts from multiple users into a single round of Transformer inference, the system requires a predetermined set of users for each batch and a static allocation of slots, and all users in the same batch must be online to participate in the M-of-M decryption process. In what scenarios is this system applicable, and does this requirement limit the framework's usability?

3 comments
AuthorsOpenReview ↗

Response to Reviewer isqJ

We thank the reviewer for these constructive comments and insightful questions. We address each concern below.

Q1: For multi-step autoregressive generation, KV caching is commonly employed to reduce redundant computation. We believe our current framework is well suited to KV-cache construction during prefill and single-step generation. However, during subsequent decoding steps, each request processes only one new token, leaving many ciphertext slots unused. Although these slots are initially underutilized, subsequent attention operations, including QK computation, softmax, and attention-value (AV) multiplication, require accessing the cached keys and values, which can substantially increase slot utilization and limit the additional acceleration achievable with the current packing strategy. Nevertheless, since our framework builds on MOAI's interleaved batching (IB) technique, extending IB with a KV-cache-aware ciphertext layout could enable substantial acceleration for multi-step generation.

We will include these discussions in the appendix.

W1&Q2: A representative application scenario is scheduled multi-user inference, which is common in LLM-based applications. When multiple users periodically invoke Transformer inference at synchronized times, such workloads naturally align with our M-of-M deployment model. However, for irregular workloads, where only a subset of users submits requests in a given round, requiring all M participants to remain available for threshold decryption limits practical applicability.

Our framework can be extended to support t-of-M threshold decryption by replacing additive resharing with Shamir-based resharing during the secret-distribution phase, performed by the two non-colluding auxiliary servers, and allowing any t authorized participants to contribute decryption shares. The primary additional setup cost arises from random polynomial generation and multipoint evaluation in Shamir resharing, whereas threshold decryption becomes more efficient because fewer participants need to generate and exchange decryption shares. The table below compares the 9-of-16 Shamir and 16-of-16 additive configurations, demonstrating this tradeoff between setup (Dense share handoff) overhead and threshold-decryption efficiency. Compared with the 16-of-16 configuration, the 9-of-16 configuration increases setup (dense share handoff) time from 0.542 s to 2.276 s, but reduces threshold-decryption time from 1.406 s to 0.730 s and communication from 125.946 MB to 70.829 MB.

Table: Setup (dense-share handoff) and threshold-decryption costs.

MethodSetup (Dense share handoff) Time (s)Setup (Dense share handoff) Comm. (MB)Threshold Decryption Time (s)Threshold Decryption Comm. (MB)
Additive 16-of-160.542671.1021.406125.946
Shamir 9-of-162.276671.1020.73070.829

We will include these discussions in the appendix.

Thank you for the responses. I would like to keep the positive rating.

AuthorsOpenReview ↗

Response to Reviewer isqJ

Thank you for your response and support.

Reviewer_tCKNOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies non-interactive secure transformer inference for multiple data owners. It proposes a reusable, two-server-assisted setup for threshold CKKS with sparse-secret encapsulation (SSE). The two non-colluding setup servers use party-contributed randomness to generate a collective dense ternary secret and a balanced fixed-Hamming-weight sparse secret in the Boolean-sharing domain. The construction delays conversion to RNS arithmetic shares until the final indicator vectors have been formed, after which the servers jointly generate the public, evaluation, and cross-secret switching keys and transfer shares of the long-term dense secret to the participating parties.

Critical strengths

S1. The paper addresses an important gap between single-owner homomorphic transformer inference and a setting with multiple data owners. The decomposition into reusable setup, collective key ownership, interleaved online evaluation, and threshold-controlled output release is coherent and well motivated. S2. The Boolean-first construction is technically thoughtful. In particular, the dense rejection sampler, collision-filtered sparse-support sampler, private expansion, and delayed Boolean-to-RNS conversion directly address the need to preserve the dense and fixed-weight sparse secret distributions required by SSE. S3. The paper does more than present an isolated cryptographic primitive. It integrates the setup with a large-model encrypted-inference pipeline and evaluates distributional correctness, setup latency and communication, LAN and WAN scaling, cryptographic-parameter scaling, and task-level inference correctness.

Critical weaknesses

W1. The end-to-end security claim is not yet concretely instantiated. Theorem 2 depends on the pre-smudging error range, the decryption-failure term beta_dec, the statistical distances delta_i, and the anticipated number of decryption queries tau. However, the evaluation states only sigma_smdg = 2^28 and does not give numerical values or bounds for these other quantities. Consequently, it is difficult to verify that the chosen flooding noise simultaneously provides the claimed statistical protection, supports the expected number of decryptions, and preserves the required CKKS precision. Similarly, epsilon_Key is treated largely as an assumed joint bound for the adopted and adapted key-generation procedures. A more explicit theorem or reduction covering the exact cross-secret construction and its error aggregation would make the main security claim more convincing. W2. The reported setup cost does not appear to represent the complete key material needed for the evaluated transformer. Table 4 includes only one Galois key, K = 1, whereas a full transformer implementation may require a larger rotation-key set. The paper does not state the actual number of rotation keys used by the LLaMA3-8B pipeline or report the corresponding full setup latency, communication, and key-storage footprint. This omission makes it difficult to assess the cost of deploying the complete system. W3. The interpretation of the 15.998x result needs a stronger baseline and clearer accounting. The result is close to the number of packed users because it compares one shared evaluation with separate full evaluations, and the paper attributes the gain to filling otherwise unused packing capacity rather than accelerating the HE operators. It is unclear whether the separate baseline is serial, whether an equally provisioned concurrent or conventionally batched baseline was considered, and exactly which packing, communication, extraction, and threshold-decryption costs are included in the multi-user interface measurement. Given that one encrypted inference takes approximately 426,000 seconds, reporting only the relative speedup and a very small overhead percentage can obscure the system's absolute practicality. W4. Independent users are independent data owners but not independent cryptographic domains after setup. They belong to a pre-established collective-key cohort, and the concrete system uses an M-of-M access structure in which all parties participate in output decryption. This creates availability, churn, and straggler concerns and limits the applicability of the multi-user claim. Dynamic admission and departure are deferred to future work, and the paper provides no experimental evaluation of a t-of-M alternative.

Questions for authors

Q1. Could the authors provide a complete numerical instantiation of the online security bound? In particular, what are the assumed number of decryption queries tau, the pre-smudging error bounds, beta_dec, and the resulting delta_i or total statistical-security loss for sigma_smdg = 2^28? What output precision remains after this noise is added? Q2. How many Galois or rotation keys are actually required by the evaluated LLaMA3-8B and MOAI pipeline? Please report the complete setup latency, communication volume, public-key-material size, and storage cost for that full key set rather than K = 1. Q3. How exactly was the Separate latency in Table 8 obtained? Was it U serial executions, U concurrent executions under the same four-GPU budget, or another configuration? Which components are included in the multi-user interface time, particularly network transfer and synchronized threshold decryption? A resource-normalized throughput and latency comparison would help establish the practical benefit. Q4. In the M-of-M deployment, what happens when one registered party is offline or refuses to provide a decryption share? Can the authors quantify the security and performance cost of the mentioned t-of-M resharing option and clarify which results would change under that access structure? Q5. For the affine cross-secret switching-key construction, how are the per-server error distributions selected so that their aggregate has the reference CKKS key-generation distribution assumed by the proof? Please state the resulting correctness and security parameters explicitly.

5 comments
AuthorsOpenReview ↗

Response to Reviewer tCKN (W1 & Q1; W2 & Q2)

We thank the reviewer for these constructive comments and insightful questions. We address each concern below.

W1&Q1: The smudging-noise parameters underlying Theorem 2 are discussed in Appendix E.5 (Noise-Flooding Parameter Selection). Smudging and noise flooding share the same principle of masking the underlying ciphertext error with additional noise. As specified in Eq. (40), the calibrated flooding-noise standard deviation is

σiOF=12τ 2sF/2Bical,\sigma_i^{\mathrm{OF}}=\sqrt{12\tau}\,2^{s_F/2}B_i^{\mathrm{cal}},

where BicalB_i^{\mathrm{cal}} denotes the pre-flooding ciphertext-noise bound, sFs_F is the statistical-security parameter in bits, and τ\tau is the assumed upper bound on adversarial decryption queries. Decryption correctness additionally requires avoiding modular wraparound under the remaining ciphertext modulus qq, with the centered representation bounded by q/2q/2.

In our evaluation, we set σsmdg=228\sigma_{\mathrm{smdg}}=2^{28} to limit the impact of smudging noise on inference accuracy. As shown in the table below, increasing this parameter substantially degrades the decrypted results. We further examine the tradeoff among numerical precision, statistical security, and the decryption-query budget.

For a representative pre-flooding error of 10−210^{-2}, a CKKS scale of 2402^{40}, and N=65,536N=65,536, the assumed noise-estimation model yields log⁡2Bical=25.356\log_2 B_i^{\mathrm{cal}}=25.356. With σsmdg=228\sigma_{\mathrm{smdg}}=2^{28} and τ=1\tau=1, Eq. (40) gives sF≈1.70s_F\approx1.70 bits. Conversely, adopting OpenFHE's reference setting of sF=30s_F=30 and τ=1\tau=1 requires σiOF≈242.149\sigma_i^{\mathrm{OF}}\approx2^{42.149} under the same noise estimate, which would substantially impair numerical precision. If the flooding standard deviation is constrained to 2282^{28}, the corresponding pre-flooding error budget is approximately 5.5×10−75.5\times10^{-7} under this estimation model. Achieving such a restrictive error budget may require additional noise-management mechanisms, increasing inference overhead.

Table: Threshold decryption errors with 16 parties, N=65536N=65536, and scale Δ=240\Delta=2^{40}.

Per-party σ\sigmaMaximum absolute real-part errorReal-part RMS error
2242^{24}0.0540.011
2262^{26}0.2150.044
2282^{28}0.8790.177
2302^{30}3.4120.707
2322^{32}14.2882.828
2362^{36}246.89245.253
2422^{42}14357.8402895.953

We will provide a detailed discussion of the smudging-noise parameter selection in the appendix.

W2&Q2: We report the number of Galois keys required by the LLaMA3-8B inference pipeline, together with the measured setup latency, communication cost, and key-storage requirements. As shown in the table below, the complete setup takes 134.35 s for 16 users. The inference pipeline requires 31 Galois keys, comprising 30 rotation keys for steps {±2k}k=014\{\pm 2^k\}_{k=0}^{14} and one complex-conjugation key. Other rotation steps can be realized by composing these supported rotations. The Galois-key generation dominates the setup cost, requiring 121.26 s, 64.95 GB of communication, and 43.31 GB of key storage. The storage reported for dense-share handoff represents the aggregate storage across all 16 users.

Table: Measured setup components at N=216N=2^{16}, h=32h=32, ∣Q∣=36|Q|=36, and ∣P∣=1|P|=1, with 16 users. Galois keys cover rotations {±2k}k=014\{\pm 2^k\}_{k=0}^{14} and complex conjugation.

ComponentTime (s)Comm. (GB)Storage (GB)
Dense secret0.1130.0410.039
Sparse secret0.2100.0420.039
Threshold PK0.1880.0580.039
Dense$\to$Sparse KSK0.1460.0580.039
Sparse$\to$Dense KSK4.9022.0951.397
Galois keys (31)121.25964.94743.314
Relin. material4.0262.1341.397
Dense-share handoff0.6040.6210.310
filtered setup time134.352----

We will include a table summarizing the overall setup time in the appendix to supplement the RQ4 evaluation.

AuthorsOpenReview ↗

Response to Reviewer tCKN (W3 & Q3; W4 & Q4)

W3&Q3: The independent-inference baseline executes user requests sequentially while fully exploiting the batching technique employed by MOAI. Under our evaluated hardware configuration, this execution mode already saturates the available computational resources, leaving no additional capacity for concurrent execution. Regarding the multi-user interface overhead, the MU Interface measurements in Table 8 already include user-input packing, output extraction, threshold decryption, and other associated overheads. We will provide an additional table detailing the breakdown of these costs.

Table: Inference scalability.

UUSeparate (s)MU interface (s)Multi-user (s)Speedup
1426378.1420.415426172.2841.000×1.000\times
41704173.5400.804426603.5023.995×3.995\times
166818527.7402.746426210.62915.998×\mathbf{15.998\times}

Table: MU interface overhead at N=216N=2^{16} and M=16M=16.

UUPlacement/Packing (s)Extraction (s)Threshold Decryption (s)Other (s)MU Interface (s)
10.2640.0100.1210.0200.415
40.3480.0380.3930.0250.804
160.9960.1311.5660.0532.746

We will add a supplementary table to provide further details on the RQ4 evaluation.

W4&Q4: Under the current M-of-M deployment, threshold decryption cannot be completed if any registered participant is offline or withholds its decryption share, since all M participants must contribute to satisfy the threshold requirement. Adopting a t-of-M scheme would improve availability by allowing any t authorized participants to complete decryption.

The primary protocol changes involve the auxiliary servers' resharing of the dense and sparse secrets during setup and the subsequent threshold-decryption procedure. Specifically, converting additive shares into Shamir shares requires random polynomial generation and multipoint evaluation, increasing setup computation. In contrast, threshold decryption requires only t participants to generate and exchange partial-decryption shares, reducing communication and execution time.

The table below compares additive 16-of-16 and Shamir 9-of-16 sharing. Shamir resharing increases setup time from 0.542 s to 2.276 s, while maintaining the same communication volume of 671.102 MB. However, it reduces threshold decryption time from 1.406 s to 0.730 s (48.08%) and communication from 125.946 MB to 70.829 MB (43.76%).

Table: Setup (dense-share handoff) and threshold-decryption costs.

MethodSetup (dense-share handoff) Time (s)Setup (dense-share handoff) Comm. (MB)Threshold Dec Time (s)Threshold Dec Comm. (MB)
Additive 16-of-160.542671.1021.406125.946
Shamir 9-of-162.276671.1020.73070.829

This improvement in availability comes at the cost of reduced collusion resistance. Under the semi-honest model, a t-of-M threshold protects the shared secret against coalitions of at most t−1 corrupted participants. Accordingly, moving from 16-of-16 to 9-of-16 reduces the maximum tolerated number of corrupted participants from 15 to 8.

We will include this discussion in the appendix.

AuthorsOpenReview ↗

Response to Reviewer tCKN (W1 & Q5)

W1&Q5:

Cross-Secret Switching-Key Generation

Both auxiliary servers independently sample their local errors from the same RLWE error distribution χloc\chi_{\mathrm{loc}} used in the original SSE switching-key construction.

As specified in Algorithm 7, Each resulting switching-key component satisfies

\qquad E_\ell=e_{\ell,0}+e_{\ell,1},$$ where $s_x$ and $s_y$ denote the source and destination secrets, respectively, and $e_{\ell,i}\leftarrow\chi_{\mathrm{loc}}^N$. Consequently, $$B_\ell+a_\ell s_y=w_\ell s_x+E_\ell,$$ preserving the standard CKKS key-switching relation. ### Correctness Analysis Let $B_{\mathrm{loc}}$ denote the support bound of $\chi_{\mathrm{loc}}$, such that $$\operatorname{supp}(\chi_{\mathrm{loc}}) \subseteq[-B_{\mathrm{loc}},B_{\mathrm{loc}}]\cap\mathbb Z.$$ Since the local errors are independently sampled, the aggregate error satisfies $$\lVert E_{\ell}\rVert_{\infty}\le 2B_{\mathrm{loc}},$$ with coefficient distribution $\chi_{\mathrm{loc}}*\chi_{\mathrm{loc}}$ and standard deviation $\sqrt{2}\sigma_{\mathrm{loc}}$. For a sampler truncated to $[-\lfloor6\sigma_{\mathrm{loc}}\rfloor,\lfloor6\sigma_{\mathrm{loc}}\rfloor]$, we have $B_{\mathrm{loc}}=\lfloor6\sigma_{\mathrm{loc}}\rfloor$. These bounds characterize the key-generation error introduced by our two-server construction. ### Security Analysis As stated in Lemma 6, “Corrupted local sampling records are preserved consistently with the backend’s private-input convention.” Thus, the simulator can reconstruct the corrupted server's switching-key share $B_{\ell,b}$ using its local sampling records and derive the complementary share as $$\widehat B_{\ell,1-b}=B_\ell-B_{\ell,b}.$$ For the corresponding real public output $B_\ell=B_{\ell,0}+B_{\ell,1}$, this yields $\widehat B_{\ell,1-b}=B_{\ell,1-b}$, preserving the share-aggregation relation exactly. Moreover, if auxiliary server $C_0$ is corrupted, the adversary can remove its local error contribution, obtaining $$B_\ell-e_{\ell,0} =-a_\ell s_y+w_\ell s_x+e_{\ell,1}.$$ Since the honest server's error $e_{\ell,1}$ retains the original RLWE error distribution $\chi_{\mathrm{loc}}$, the residual component preserves the noise distribution and algebraic structure of the original SSE switching key. Accordingly, the security analysis relies on the underlying multiparty key-generation simulation and the same RLWE and circular-security assumptions as the original SSE construction. Our configuration uses $N=2^{16}$, sparse-secret Hamming weight $h=32$, and switching-key moduli satisfying $\log_2(QP)<1747$ and $\log_2(qp)<121$, targeting 128-bit security under the corresponding SSE parameter settings.
Reviewer_sa6ZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates secure transformer inference in multi-user scenarios, a crucial real-world problem. To allow different users to share a single homomorphic evaluation, a threshold-CKKS framework supporting sparse-secret encapsulation (SSE) is constructed. This framework utilizes two non-colluding auxiliary servers to generate a collective dense secret satisfying the target distribution and a collective sparse secret with a fixed Hamming weight in the Boolean-sharing domain. Subsequently, the system generates a collective public key and evaluation keys, distributes long-term secret-key shares to users, and finally completes the inference task through interleaved packing and synchronized threshold decryption. Each user also recovers their own output. I believe this paper's multi-user encrypted inference scenario and corresponding scheme are valuable.

Critical strengths
  1. I like the problem identified and the solution proposed in this paper. Multiple users sharing encrypted inference requires not only batching but also a common key domain and distributed decryption authority. The paper further addresses the requirements of SSE for secret distributions and fixed Hamming weight, which are significant for secure inference based on homomorphic encryption.
  2. The design is ingenious. For example, the paper retains secret generation in the Boolean-sharing domain and only performs RNS conversion on the final positive and negative indicator vectors. This avoids prematurely expanding a large number of intermediate one-hot vectors across RNS limbs, and the paper also explains the benefits of this optimization well through experiments.
  3. The experiments are conducted and analyzed from different perspectives. Although I believe some experiments are incomplete, this is understandable considering the experimental cycle of HE-based secure inference.
Critical weaknesses
  1. I think the biggest problem is that the benefit of SSE for full-model inference has not been analyzed. The approximately 16× speedup in Table 8, I believe, mainly comes from merging requests from 16 users, but this does not seem to measure the contribution of SSE itself to full transformer inference. I hope the authors can compare the inference time and bootstrapping overhead with and without SSE enabled.
  2. End-to-end experiments of the complete system are also insufficient. Since the main contribution of this paper is a multi-user secure inference framework, the component-level results are not sufficient to fully verify the overall correctness and actual overhead. I hope the authors will report at least one representative set of full end-to-end multi-user executions, while verifying the output accuracy and actual end-to-end latency.
  3. The threat model appears to have some issues. Appendix B allows an adversary to simultaneously corrupt the inference server, a subset of users, and one auxiliary server. However, the end-to-end system definition in Appendix E.1 assumes that both auxiliary servers are honest during setup, and Theorem 3 is based on this system definition. This affects the system's trust assumptions, and I think the authors should carefully analyze this point.
Questions for authors
  1. As mentioned in the Weaknesses, I would like to know the differences in inference time and overhead between SSE-enabled threshold CKKS and threshold CKKS with dense-secret bootstrapping.
  2. Could you report a set of end-to-end experiments, i.e., complete experiments of shared transformer inference, per-user output extraction, and threshold decryption with the specified smudging noise?
  3. Could you explain the problems with the threat model in Weakness 3? Does the end-to-end guarantee cover the case where the inference server, a subset of users, and one auxiliary server collude simultaneously?
1 comment
AuthorsOpenReview ↗

Response to Reviewer sa6Z

We thank the reviewer for these constructive comments and insightful questions. We address each concern below.

W1&Q1: We evaluated inference with and without SSE, and the results in the table below confirm the observation in the Introduction: “For instance, in MOAI Zhang et al. (2026)’s LLaMA3-8B evaluation, the Feed-Forward Network (FFN) expands the encrypted state from 4,096 to 14,336 ciphertexts under column packing while consuming 13 levels. With only 11 post-bootstrap levels, dense-secret bootstrapping requires a refresh within this widened state, …”. The additional overhead without SSE is dominated by the FFN, which requires 14,336 additional bootstrap operations per Transformer layer. The two RMSNorm operations and attention softmax contribute another 6 and 64 bootstraps per layer, respectively. Across 32 layers, these account for 460,992 additional bootstraps. As shown in the table below, SSE reduces inference time by 27.69% and the total bootstrap count by 35.11%.

Table: Inference time and bootstrap count with and without SSE.

MetricWith SSEWithout SSEReduction with SSE
Inference time (s)426,378.142589,679.539163,301.397 (27.69%)
Bootstrap count851,9681,312,960460,992 (35.11%)

We will include these results in the appendix to further substantiate the performance benefits of SSE.

W2&Q2: Table "Inference scalability" already reports these measurements in our paper. The “Multi-user” column presents the complete end-to-end inference time, while “MU Interface” isolates the non-inference overhead, including user-input aggregation, output extraction, threshold decryption, and other overhead. Specifically, user-input aggregation includes client-side encryption, data transmission, and server-side aggregation. Threshold decryption covers local partial decryption, share exchange, and final decryption. Other overhead includes server-side distribution of output ciphertexts, memory management across execution stages, connection establishment and acknowledgment, message serialization and deserialization, and runtime verification.

Table: Inference scalability.

UUSeparate (s)MU interface (s)Multi-user (s)Speedup
1426378.1420.415426172.2841.000×1.000\times
41704173.5400.804426603.5023.995×3.995\times
166818527.7402.746426210.62915.998×\mathbf{15.998\times}

Table: MU interface overhead at N=216N=2^{16} and M=16M=16.

UUPlacement/Packing (s)Extraction (s)Threshold Decryption (s)Other (s)MU Interface (s)
10.2640.0100.1210.0200.415
40.3480.0380.3930.0250.804
160.9960.1311.5660.0532.746

We will include an additional table providing a detailed breakdown of the RQ4 experimental results.

W3&Q3: The threat model is consistent with the system model in Section 3.1, which explicitly states that “The auxiliary servers retain no secret-key state after setup.” This reflects the practical separation of key generation from cryptographic operations: the auxiliary servers participate only in setup and do not retain secret-key state during online inference.

Although setup and online inference occur sequentially, security proofs do not inherently revoke knowledge acquired by an adversary. Therefore, our end-to-end security definition in Appendix E.1 explicitly adopts a phase-separated adversarial model, representing the joint view as (VI,⊥set,VI,Son)(V^{\mathrm{set}}_{\mathcal I,\bot}, V^{\mathrm{on}}_{\mathcal I,S}). In contrast, (VI,bset,VI,S,bon)(V^{\mathrm{set}}_{\mathcal{I},b}, V^{\mathrm{on}}_{\mathcal{I},S,b}) would impose a stronger security requirement by allowing adversarial knowledge to accumulate across phases, which is beyond our stated model.

Importantly, auxiliary-server corruption is not omitted from the setup security proof. Throughout the setup analysis, we consider AI,bset\mathcal A^{\mathrm{set}}_{\mathcal I,b}, which allows the corruption of up to one auxiliary server. Thus, our security analysis accounts for a corrupted inference server, a subset of corrupted users, and one corrupted auxiliary server under the specified phase-separated model, without permitting the auxiliary server's setup observations to be reused in the online adversarial view.

Comments on the paper

4 comments
AuthorsOpenReview ↗

The revised manuscript has been submitted.

We have completed the manuscript revisions in response to the reviewers' comments. Specifically, we added Appendix A to explain how the two-auxiliary-server design addresses Challenge C3, including its motivation and the tradeoff between efficiency and security assumptions. We expanded the comparison with Min et al. in Appendix B.2 (formerly A.2) to cover secret-generation procedures and key-generation noise, and updated Table 10 to explicitly acknowledge that Min et al.'s approach supports dense-secret generation. We also clarified that the small error terms in Algorithm 6 (relinearization-key generation) and Algorithm 7 (key-switching-key generation) are sampled from the standard RLWE error distribution.

Furthermore, we added experimental details addressing the reviewers' concerns. Appendix I evaluates the performance benefits of SSE; Appendix J provides a detailed breakdown of multi-user STI overhead, including user-input aggregation, output extraction, and threshold decryption; Appendix K reports the setup-time measurements for multi-user STI; and Appendix L discusses smudging-noise parameter selection.

Finally, we added Appendix M to discuss the extension to t-of-M threshold settings, the communication costs and potential optimizations of Galois-key generation, and the applicability of our framework to multi-step autoregressive generation.

All revisions have been highlighted in blue in the revised manuscript for ease of reference.

AuthorsOpenReview ↗

Further Revisions

Following the reviewer's suggestions, we have further revised the manuscript. Specifically, we have added a sentence to the Introduction to clarify the trade-off between setup efficiency and security assumptions introduced by the two-auxiliary-server setting, while retaining the detailed discussion in Appendix A. In addition, we have expanded the proof of Lemma 6 (CKKS/SSE Key Generation) in Appendix F.2 to provide a more detailed explanation of the proof procedure.

Dear reviewers,

Please be aware that little time remains for the author-reviewer discussion, and this discussion phase will close in a few hours. As the deadline is approaching, we kindly request that you refrain from asking for additional experiments. Please summarize your final position concisely.

Sincerely, AC

AuthorsOpenReview ↗

Appreciation

We sincerely thank the AC and all reviewers for their valuable feedback and support throughout the review process!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.