acceptodds
Under review as a conference paper at ICLR 2027

Error Propagation-Aware Scale Design for Efficient Homomorphic Encryption-Based LLM Inference

Abstract

Homomorphic encryption (HE) enables privacy-preserving inference by allowing neural networks to operate directly on encrypted data, but its computational cost remains a major obstacle to deploying large language models in practice. In particular, CKKS-based inference consumes ciphertext modulus through homomorphic multiplications and requires costly bootstrapping when the available modulus is exhausted. In this work, we propose an error-propagation-aware scale design for efficient CKKS-based privacy-preserving LLM inference. We characterize the numerical errors introduced by individual homomorphic operations and analyze how they propagate through subsequent Transformer computations. Based on this analysis, we quantify the contribution of each local error to the final inference error and determine the precision required for individual operations. We then allocate operation-wise scales and modulus levels accordingly, avoiding unnecessarily conservative precision while maintaining the target inference accuracy. As a result, more computation can be performed within a given modulus chain, reducing the frequency of costly bootstrapping operations and improving overall inference efficiency. Compared with THOR, our method reduces modulus consumption by 30.0% and the number of bootstrapping operations by 83.6% for the standard Transformer. For the HE-friendly Transformer, our method reduces modulus consumption by 33.6% and the number of bootstrapping operations by 80% compared with PowerFormer.

Primary area:
optimization
Keywords:
Large Language Models, Homomorphic Encryption, Privacy-Preserving Machine Learning, Efficient Inference, Error Analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DqwaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes sensitivity-weighted scale allocation for CKKS Transformer inference. It models local squared error as a scale-dependent term plus a nonlinear approximation-error floor, estimates propagation to the final encoder representation using Jacobians on training samples, and minimizes a modulus-consumption proxy under per-sample predicted-error constraints. Bootstrapping configurations are also retuned. In 12-layer BERT experiments, the authors report lower modulus use and fewer bootstraps, with plaintext/ciphertext prediction mismatches of 0/408 on MRPC and 3/277 on RTE.

Critical strengths
  1. The paper addresses an important practical bottleneck in privacy-preserving Transformer inference and offers a clear intuition for allocating precision according to downstream sensitivity instead of using a uniform scale.

  2. The implementation study reports operation-level scales, bootstrapping configurations, and ciphertext-versus-plaintext task accuracy on two Transformer settings. These details make the system's direction concrete and provide useful starting points for follow-up work.

Critical weaknesses
  1. The propagated-error estimate lacks sufficient validation. Section 3 uses an expectation identity and high dimensionality to justify a sample-wise approximation, without establishing the required concentration conditions or validating the assumed error statistics. Figure 1 shows local-error fits for only one sample and encoder, and it reports no held-out comparison of predicted and measured final-representation errors. Although Table 4 shows encouraging task accuracy, it does not directly validate this estimator. The basis for choosing E_max = 5 to preserve classification decisions is also unexplained. These gaps make it difficult to assess the reliability of the error budget used for scale selection.

  2. The optimization specification and comparisons need clarification. Section 5 minimizes modulus consumption under per-sample error constraints, whereas Appendix B describes error minimization with box clipping. Algorithm 1 appears consistent with Section 5, but it does not specify the projection onto its coupled sample constraints. Experimentally, Section 6.1 acknowledges that speedups combine backend acceleration with scale and bootstrapping optimization, yet no same-backend uniform-scale control isolates the proposed allocation's contribution. Finally, the abstract's 83.6% bootstrap reduction relative to THOR uses an ATLAS-derived count, while Appendix F.2 explicitly excludes that count from direct quantitative comparison. This inconsistency weakens the headline comparison.

Questions for authors
  1. Can the authors compare predicted and measured squared L2 errors at the final encoder output on held-out inputs, report the discrepancies and the fraction of inputs with measured error above E_max = 5, and explain how this threshold was selected? This would assess generalization of the training-based error model. Please also clarify applicability to out-of-range inputs, given that the 20 RTE training examples were excluded from the scale analysis.

  2. Algorithm 1 appears consistent with Section 5 under its stated scale conversion, whereas the prose in Appendix B describes error minimization and box clipping. Please reconcile these descriptions and specify the objective, sample-specific feasible sets, and projection procedure actually used in the implementation.

  3. Can optimized and uniform scales be compared on the same backend, model, and nonlinear implementations, with both configurations tuned to comparable accuracy requirements? Where feasible, please separate the contribution of bootstrapping retuning, or explain why it cannot be isolated. Please also reconcile the abstract's 83.6% reduction relative to THOR with Appendix F. 2's statement that the ATLAS-derived count is not used for direct quantitative comparison.

1 comment
AuthorsOpenReview ↗

Author Response to Reviewer

We thank the reviewer for the constructive feedback and for recognizing the importance of sensitivity-aware precision allocation and our implementation-level evaluation. Our error propagation model uses a first-order approximation, where local errors propagate through their Jacobians. Under zero-mean, independent, and isotropic errors, the expected squared error amplification is characterized by the squared Frobenius norm divided by the input dimension. While this does not provide a deterministic bound, supplementary experiments comparing the Frobenius-based estimate, spectral-norm bound, and observed amplification (N = 32 and N = 1024) support our modeling choice. We provide point-by-point responses to the reviewer's questions and comments below.

Question 1:

Can the authors compare predicted and measured squared L2 errors at the final encoder output on held-out inputs, report the discrepancies and the fraction of inputs with measured error above E_max = 5, and explain how this threshold was selected? This would assess generalization of the training-based error model. Please also clarify applicability to out-of-range inputs, given that the 20 RTE training examples were excluded from the scale analysis.

Answer:

The final stage of BERT performs binary classification using a separating hyperplane. From this geometric perspective, the minimum inter-class distance provides a natural reference for selecting an error tolerance.

We therefore chose Emax = 5 as a conservative squared L2 error budget for the full 128 × 768 Transformer encoder output. Specifically, we computed the minimum squared Euclidean distance between the 768-dimensional CLS representations of correctly classified samples from different classes. The minimum squared distances were 15.42 for RTE and 103.20 for MRPC.

Since Emax = 5 constrains the error over the entire 128 × 768 output, it also bounds the error in the CLS representation alone. We selected this threshold to be smaller than the observed minimum squared inter-class distances, aiming to limit the impact of approximation errors on classification accuracy.

Regarding out-of-range inputs, 20 RTE training samples were excluded because their activations exceeded the supported polynomial approximation ranges. While the original activation functions are bounded within the intended operating ranges, their polynomial approximations are not necessarily globally bounded and may diverge rapidly outside these ranges. This limitation is inherent to PowerFormer's polynomial approximation-based architecture, rather than our proposed scale optimization. Consequently, reliable error estimation cannot be guaranteed for these out-of-range inputs.

Question 2:

Algorithm 1 appears consistent with Section 5 under its stated scale conversion, whereas the prose in Appendix B describes error minimization and box clipping. Please reconcile these descriptions and specify the objective, sample-specific feasible sets, and projection procedure actually used in the implementation.

Answer:

Algorithm 1 follows the optimization problem in Section 5: the objective minimizes modulus consumption, and the feasible set is the intersection of the sample-specific feasible sets C_j. In Line 9, projection onto each C_j enforces the corresponding sample's error constraint. Thus, the error constraints of all samples are incorporated into the optimization.

Question 3: Can optimized and uniform scales be compared on the same backend, model, and nonlinear implementations, with both configurations tuned to comparable accuracy requirements? Where feasible, please separate the contribution of bootstrapping retuning, or explain why it cannot be isolated. Please also reconcile the abstract's 83.6% reduction relative to THOR with Appendix F.2's statement that the ATLAS-derived count is not used for direct quantitative comparison.

Answer:

We compare our method with Zephyr, a GPU-optimized 30-bit CKKS system, using the same HE backend, Transformer model, and nonlinear approximations. Our contribution is reducing modulus consumption and bootstrapping frequency rather than accelerating the underlying HE implementation.

Under the same backend, bootstrapping time decreases from 36.54 s to 6.87 s (81.2% reduction). Scale allocation itself does not directly improve execution time, but its reduced modulus consumption enables bootstrapping parameter retuning and fewer bootstrapping operations. Therefore, the individual speedup contributions of these two optimizations cannot be meaningfully isolated, as they are interdependent.

The original THOR paper does not report the number of bootstrapping operations. Therefore, we used the counts from the ATLAS reproduction, which is already cited in our manuscript. We will revise Appendix F.2 to remove the statement that these counts are not used for quantitative comparison.

Reviewer_WKuyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an error-propagation-aware scale allocation framework for efficient CKKS-based Transformer inference. The method models local numerical errors, estimates their propagation through Jacobian-based sensitivity analysis, and optimizes operation-wise scales to reduce modulus consumption and bootstrapping frequency. Experiments on BERT-based classification tasks report substantial inference-time improvements over THOR and PowerFormer.

Critical strengths

The paper addresses an important computational bottleneck in HE-based inference. Allocating numerical precision according to error propagation is a reasonable approach, and the reported reductions in modulus consumption and bootstrapping operations suggest potential practical benefits.

Critical weaknesses
  • The novelty and practical significance are insufficiently established in the context of recent HE-based LLM research. Adaptive HE parameter selection based on neural-network error tolerance has already been investigated in AutoPrivacy [1]. More recent studies, including Encryption-Friendly LLM Architecture [2], EncryptedLLM [3], and MOAI [4], have substantially advanced efficient encrypted Transformer inference. However, the manuscript does not adequately position its contribution against these developments. Despite repeatedly motivating the work through LLM inference, the experiments are limited to BERT-based classification on MRPC and RTE. The proposed method is not evaluated on modern decoder-only LLMs, autoregressive generation, or longer-context inference. Consequently, its relevance and additional technical value for contemporary HE-based LLM inference remain unclear.
  • The experimental results do not adequately substantiate the claimed efficiency improvements of the proposed method. The reported 4.45× and 3.97× speedups combine error-aware scale allocation with changes in the underlying HE implementation and bootstrapping configurations. The independent contribution of the proposed scale optimization is not quantified. Furthermore, the headline THOR bootstrapping comparison relies on a count derived from a different implementation rather than a directly comparable measurement. Controlled ablations under an identical backend and comparisons with more recent optimized HE inference frameworks are essential to establish the actual benefits of the proposed method.

[1] Q. Lou, S. Bian, and L. Jiang, "AutoPrivacy: Automated Layer-wise Parameter Selection for Secure Neural Network Inference," NeurIPS, 2020. [2] D. Rho et al., "Encryption-Friendly LLM Architecture," ICLR, 2025. [3] L. De Castro et al., "EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption," ICML, 2025. [4] L. Zhang et al., "MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer Inference," ICLR, 2026.

Questions for authors
  • Can the authors provide controlled experiments using an identical HE backend to separately quantify the speedups attributable to error-aware scale allocation and bootstrapping optimization?
  • How does the proposed method perform when integrated with recent HE inference frameworks such as MOAI? Can the authors demonstrate meaningful additional improvements on larger decoder-only language models or realistic autoregressive inference workloads?
3 comments
AuthorsOpenReview ↗

Author Response to Reviewer

We thank the reviewer for the constructive feedback and for highlighting the importance of clearly establishing the novelty and practical benefits of our method. Regarding the novelty of our approach, AutoPrivacy uses deep reinforcement learning to select HE parameters at the layer level in a hybrid HE–garbled-circuit framework. In contrast, our method analytically models numerical error propagation using Jacobian-based amplification factors and optimizes operation-wise scaling factors for both linear and nonlinear operations through ADMM under error constraints. Thus, our contribution differs from AutoPrivacy in both the error modeling approach and the granularity of parameter optimization. We will clarify this distinction and include AutoPrivacy in the related work. We provide point-by-point responses to the reviewer's questions and comments below.

Question 1:

Can the authors provide controlled experiments using an identical HE backend to separately quantify the speedups attributable to error-aware scale allocation and bootstrapping optimization?

Answer:

The reported 4.45× and 3.97× speedups include implementation-level benefits, particularly from Zephyr's GPU-optimized 30-bit CKKS system. Our contribution is reducing modulus consumption and bootstrapping frequency, rather than accelerating the HE implementation itself.

Under the same Zephyr backend, end-to-end inference time decreases from 107.94 s to 86.71 s (1.24× speedup), while bootstrapping time decreases from 36.54 s to 6.87 s (81.2% reduction). Scale allocation alone does not directly reduce execution time; instead, it reduces modulus consumption, enabling bootstrapping parameter optimization and fewer bootstrapping operations. Thus, the two optimizations contribute jointly to the speedup rather than providing independent acceleration, and their effects cannot be treated as separate, additive speedups. We will clarify this distinction in the revised manuscript.

Question 2:

How does the proposed method perform when integrated with recent HE inference frameworks such as MOAI? Can the authors demonstrate meaningful additional improvements on larger decoder-only language models or realistic autoregressive inference workloads?

Answer:

Our method is designed for general Transformer architectures and is not limited to BERT. Therefore, it is potentially applicable to recent privacy-preserving LLM inference frameworks, including MOAI, EncryptedLLM, and Encryption-Friendly LLM Architecture. However, we have not yet experimentally evaluated these integrations or larger decoder-only LLMs. We will explore these extensions in future work.

I appreciate the authors' detailed responses. However, most of my concerns remain unresolved.

  • The result that the speedup attributable to the proposed optimization is 1.24×, not the originally reported 4× or more, substantially reduces the significance of the experimental results. Also, while bootstrapping time decreases by approximately 30 seconds, the total inference time decreases by only about 20 seconds. Is there an increase in execution time elsewhere? A more detailed analysis is needed.
  • The paper emphasizes LLM inference, and I believe that demonstrating effectiveness at a realistic LLM scale is important for establishing the significance at this time. However, the answer does not adequately support this claim. I understand that conducting such experiments within the limited rebuttal period may be difficult, but this does not resolve the underlying limitation in the paper's contribution.

For these reasons, I maintain my original score.

  • I understand that AutoPrivacy is a different technical approach and operates within a GAZELLE-based hybrid framework. Nevertheless, its core idea is DRL-based layerwise HE parameter search, which makes it possible alternative for the proposed optimization method. I am also aware of several subsequent studies in this direction. Comparisons with those methods would help establish the advantages of the proposed method. However, since I did not request such comparisons in my original review, I do not consider this point a basis for my current score.
AuthorsOpenReview ↗

Clarification on Execution Time and Evaluation Scope

We thank the reviewer for the additional feedback.

The difference between the reduction in bootstrapping time and the overall inference time can be explained by changes in ciphertext levels. Our scale optimization reduces modulus consumption and changes bootstrapping locations, thereby altering the levels at which subsequent operations are performed. Since CKKS computational costs depend on the number of active RNS moduli, these changes can increase or decrease the execution time of other operations.

We acknowledge that our current evaluation is limited to encoder-based Transformer inference and does not establish performance gains for larger decoder-only LLMs. We will clarify this limitation, along with the level-dependent execution costs, in the revised manuscript.

Reviewer_jehQOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Unlike conventional CKKS-based privacy-preserving machine learning (PPML) methods that use a common scaling factor across operations, this paper adopts Grafting (Cheon 2025), a recently proposed scale management technique, to assign different scaling factors to individual operations. Thus, the proposed method reduces unnecessary modulus consumption and bootstrapping operations, thereby improving computational efficiency. In particular, the authors analyze how the local error introduced by each operation affects the final model accuracy and optimize the scaling factors accordingly.

Critical strengths

The paper addresses the inefficiency of conventional approaches that use a common precision level and scaling factor across all operations, potentially leading to unnecessary modulus consumption. The proposed operation-wise scaling factor optimization is particularly promising for Transformer-based models, where bootstrapping constitutes a major computational bottleneck. By reducing unnecessary modulus consumption, the method has the potential to improve the overall efficiency of homomorphic inference.

Critical weaknesses

The paper formulates an optimization problem over the scaling factors (\Delta), which are transformed into variables (z) to obtain a convex formulation. However, in practical CKKS implementations, scaling factors are commonly chosen as powers of two. Table 2 also shows that, although different scaling factors are assigned to different operations, they are still expressed as powers of two. This raises a concern about the practical applicability of the continuous optimization formulation. However, the optimization problem is convex only when the transformed variables (z) are treated as continuous real-valued variables, while the scaling factors (\Delta) used in practice are typically restricted to discrete values, such as powers of two. Even if the continuous solutions are rounded to feasible scaling factors, the relatively large gaps between consecutive powers of two may introduce substantial deviations from the theoretical optimum. Therefore, it would be useful to clarify how the continuous optimization results are mapped to practical scaling factors and to evaluate the performance degradation, if any, caused by this discretization.

Questions for authors
  • The paper uses the Frobenius norm to quantify error propagation across operations. Have the authors considered using the spectral norm instead to characterize worst-case error amplification? Such an approach could provide a complementary perspective on how the maximum error propagates through the network and affects the final inference accuracy.

  • The paper reports a reduction of approximately 30–33% in modulus consumption, while the number of bootstrapping operations is reduced by 80–83%. Does a 30–33% reduction in modulus consumption lead to such a substantial reduction in the number of bootstrapping operations? Furthermore, Table 3 shows that the bootstrapping execution time is reduced by approximately 90–94%. Could the authors explain why the reduction in bootstrapping execution time is even greater than the reduction in the number of bootstrapping operations?

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer

We thank the reviewer for the constructive feedback and for recognizing the potential of our operation-wise scale optimization. Regarding the concern about continuous optimization and practical scaling factors, our optimization uses continuous variables to obtain a convex formulation. After optimization, we recover the scaling factors and round their logarithmic bit precision upward to the nearest integer, obtaining powers of two. Since the estimated error in our model is non-increasing with respect to each scaling factor, upward rounding preserves the feasibility of the modeled error constraints. Although discretization may increase modulus consumption, the rounding overhead is less than one bit per optimized scale. We will clarify this conversion procedure and its implications in the revised manuscript. We provide point-by-point responses to the reviewer's questions and comments below.

Question 1:

The paper uses the Frobenius norm to quantify error propagation across operations. Have the authors considered using the spectral norm instead to characterize worst-case error amplification? Such an approach could provide a complementary perspective on how the maximum error propagates through the network and affects the final inference accuracy.

Answer:

The spectral norm provides a rigorous worst-case error amplification bound, but it can be overly conservative for random perturbations. In contrast, under the zero-mean isotropic error assumption used in our analysis, the expected squared error amplification is characterized by the squared Frobenius norm divided by the input dimension.

To illustrate this distinction, we compare the amplification of 50 random perturbations with the spectral-norm bound and the Frobenius-based estimate for N = 32 and N = 1024 (see supplementary material). The results show that the spectral-norm bound is considerably more conservative, whereas the Frobenius-based estimate better reflects the observed amplification. Moreover, as error propagation estimates are repeatedly applied across successive layers, the gap between the spectral-norm bound and the Frobenius-based estimate can grow rapidly, making the spectral-norm approach increasingly conservative. These observations support our use of the Frobenius norm to model expected error propagation rather than worst-case amplification.

Question 2:

The paper reports a reduction of approximately 30–33% in modulus consumption, while the number of bootstrapping operations is reduced by 80–83%. Does a 30–33% reduction in modulus consumption lead to such a substantial reduction in the number of bootstrapping operations? Furthermore, Table 3 shows that the bootstrapping execution time is reduced by approximately 90–94%. Could the authors explain why the reduction in bootstrapping execution time is even greater than the reduction in the number of bootstrapping operations?

Answer:

The reduction in bootstrapping operations is not proportional to the reduction in modulus consumption, as it depends on the bootstrapping locations and the number of ciphertexts involved.

In the baseline CKKS implementation (Appendix F), bootstrapping occurs at the attention, residual, and feed-forward stages. Our 30–33% reduction in modulus consumption eliminates bootstrapping at the attention and feed-forward stages, leaving only the residual stage. Since the feed-forward hidden dimension is four times larger, it requires more ciphertexts for bootstrapping. Consequently, the total bootstrapping count decreases from 10 to 2 operations per encoder (80% reduction). The larger reduction in bootstrapping time (90–94%) likely reflects differences in the underlying HE implementations, particularly Zephyr's GPU-optimized 30-bit CKKS system. Our contribution is reducing modulus consumption and bootstrapping frequency, not accelerating bootstrapping itself. As we have not separately quantified these implementation effects, we do not attribute the additional speedup to our optimization.

The authors' responses to the W1 and Q1 are clear and have adequately addressed my concerns. However, I believe that further clarification is needed regarding Q2.

According to Table 3(b), Powerformer requires 10 bootstrapping operations per encoder (120 in total), whereas the proposed method requires only 2 per encoder (24 in total). However, the total bootstrapping execution time decreases from 103.72 s to 6.87 s. Dividing these times by the corresponding numbers of bootstrapping operations yields an average execution time of approximately 0.864 s and 0.286 s per operation, respectively, a difference of approximately a factor of three. This shows that the reduction in bootstrapping execution time may result not only from the reduced number of bootstrapping operations but also from differences in the underlying HE implementations, particularly the GPU-optimized CKKS implementation provided by Zephyr.

Therefore, the current execution-time comparison appears to reflect both the benefits of the proposed scale optimization and the performance differences between the underlying HE implementations. To more clearly demonstrate the performance gains attributable to the proposed method, I believe a comparison using the same HE implementation and experimental conditions, or an ablation study that separates the contributions of the different optimization components, would be necessary.

I also have a concern regarding the reported 83.6% reduction in bootstrapping operations compared with THOR. According to Table 1, the bootstrapping count for THOR is taken from ATLAS rather than the original THOR paper. However, it is unclear whether this count directly reflects the original THOR configuration or is based on a different experimental setup or implementation used in ATLAS. Therefore, the authors need to clarify whether the bootstrapping count reported in ATLAS is the same as that in the original THOR implementation.

AuthorsOpenReview ↗

Clarification on Bootstrapping Performance and THOR Comparison

We thank the reviewer for the follow-up comments and for acknowledging our responses to W1 and Q1.

Regarding bootstrapping performance, we provide an additional comparison table in the supplementary material. Under the same Zephyr backend, the bootstrapping count decreases from 120 to 24 (80.0%), while the total bootstrapping time decreases from 36.54 s to 6.87 s (81.2%). The average execution times per operation are similar (0.305 s and 0.286 s), indicating that the reduction in bootstrapping time is primarily attributable to fewer bootstrapping operations. The additional speedup relative to PowerFormer largely reflects differences in the underlying HE implementations, particularly Liberate.FHE and Zephyr. We will also incorporate this comparison table into the revised manuscript to clarify the contribution of our optimization independently of implementation-level acceleration.

Regarding THOR, the original THOR paper does not report bootstrapping counts. Our reported 83.6% reduction is based on the THOR configuration reproduced in ATLAS, which is already cited in our manuscript. We cannot confirm that this configuration is identical to the original THOR implementation. We will clarify this distinction in the revised manuscript and explicitly identify the comparison as based on the ATLAS reproduction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.