- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes sensitivity-weighted scale allocation for CKKS Transformer inference. It models local squared error as a scale-dependent term plus a nonlinear approximation-error floor, estimates propagation to the final encoder representation using Jacobians on training samples, and minimizes a modulus-consumption proxy under per-sample predicted-error constraints. Bootstrapping configurations are also retuned. In 12-layer BERT experiments, the authors report lower modulus use and fewer bootstraps, with plaintext/ciphertext prediction mismatches of 0/408 on MRPC and 3/277 on RTE.
Critical strengths
-
The paper addresses an important practical bottleneck in privacy-preserving Transformer inference and offers a clear intuition for allocating precision according to downstream sensitivity instead of using a uniform scale.
-
The implementation study reports operation-level scales, bootstrapping configurations, and ciphertext-versus-plaintext task accuracy on two Transformer settings. These details make the system's direction concrete and provide useful starting points for follow-up work.
Critical weaknesses
-
The propagated-error estimate lacks sufficient validation. Section 3 uses an expectation identity and high dimensionality to justify a sample-wise approximation, without establishing the required concentration conditions or validating the assumed error statistics. Figure 1 shows local-error fits for only one sample and encoder, and it reports no held-out comparison of predicted and measured final-representation errors. Although Table 4 shows encouraging task accuracy, it does not directly validate this estimator. The basis for choosing E_max = 5 to preserve classification decisions is also unexplained. These gaps make it difficult to assess the reliability of the error budget used for scale selection.
-
The optimization specification and comparisons need clarification. Section 5 minimizes modulus consumption under per-sample error constraints, whereas Appendix B describes error minimization with box clipping. Algorithm 1 appears consistent with Section 5, but it does not specify the projection onto its coupled sample constraints. Experimentally, Section 6.1 acknowledges that speedups combine backend acceleration with scale and bootstrapping optimization, yet no same-backend uniform-scale control isolates the proposed allocation's contribution. Finally, the abstract's 83.6% bootstrap reduction relative to THOR uses an ATLAS-derived count, while Appendix F.2 explicitly excludes that count from direct quantitative comparison. This inconsistency weakens the headline comparison.
Questions for authors
-
Can the authors compare predicted and measured squared L2 errors at the final encoder output on held-out inputs, report the discrepancies and the fraction of inputs with measured error above E_max = 5, and explain how this threshold was selected? This would assess generalization of the training-based error model. Please also clarify applicability to out-of-range inputs, given that the 20 RTE training examples were excluded from the scale analysis.
-
Algorithm 1 appears consistent with Section 5 under its stated scale conversion, whereas the prose in Appendix B describes error minimization and box clipping. Please reconcile these descriptions and specify the objective, sample-specific feasible sets, and projection procedure actually used in the implementation.
-
Can optimized and uniform scales be compared on the same backend, model, and nonlinear implementations, with both configurations tuned to comparable accuracy requirements? Where feasible, please separate the contribution of bootstrapping retuning, or explain why it cannot be isolated. Please also reconcile the abstract's 83.6% reduction relative to THOR with Appendix F. 2's statement that the ATLAS-derived count is not used for direct quantitative comparison.
Author Response to Reviewer
We thank the reviewer for the constructive feedback and for recognizing the importance of sensitivity-aware precision allocation and our implementation-level evaluation. Our error propagation model uses a first-order approximation, where local errors propagate through their Jacobians. Under zero-mean, independent, and isotropic errors, the expected squared error amplification is characterized by the squared Frobenius norm divided by the input dimension. While this does not provide a deterministic bound, supplementary experiments comparing the Frobenius-based estimate, spectral-norm bound, and observed amplification (N = 32 and N = 1024) support our modeling choice. We provide point-by-point responses to the reviewer's questions and comments below.
Question 1:
Can the authors compare predicted and measured squared L2 errors at the final encoder output on held-out inputs, report the discrepancies and the fraction of inputs with measured error above E_max = 5, and explain how this threshold was selected? This would assess generalization of the training-based error model. Please also clarify applicability to out-of-range inputs, given that the 20 RTE training examples were excluded from the scale analysis.
Answer:
The final stage of BERT performs binary classification using a separating hyperplane. From this geometric perspective, the minimum inter-class distance provides a natural reference for selecting an error tolerance.
We therefore chose Emax = 5 as a conservative squared L2 error budget for the full 128 × 768 Transformer encoder output. Specifically, we computed the minimum squared Euclidean distance between the 768-dimensional CLS representations of correctly classified samples from different classes. The minimum squared distances were 15.42 for RTE and 103.20 for MRPC.
Since Emax = 5 constrains the error over the entire 128 × 768 output, it also bounds the error in the CLS representation alone. We selected this threshold to be smaller than the observed minimum squared inter-class distances, aiming to limit the impact of approximation errors on classification accuracy.
Regarding out-of-range inputs, 20 RTE training samples were excluded because their activations exceeded the supported polynomial approximation ranges. While the original activation functions are bounded within the intended operating ranges, their polynomial approximations are not necessarily globally bounded and may diverge rapidly outside these ranges. This limitation is inherent to PowerFormer's polynomial approximation-based architecture, rather than our proposed scale optimization. Consequently, reliable error estimation cannot be guaranteed for these out-of-range inputs.
Question 2:
Algorithm 1 appears consistent with Section 5 under its stated scale conversion, whereas the prose in Appendix B describes error minimization and box clipping. Please reconcile these descriptions and specify the objective, sample-specific feasible sets, and projection procedure actually used in the implementation.
Answer:
Algorithm 1 follows the optimization problem in Section 5: the objective minimizes modulus consumption, and the feasible set is the intersection of the sample-specific feasible sets C_j. In Line 9, projection onto each C_j enforces the corresponding sample's error constraint. Thus, the error constraints of all samples are incorporated into the optimization.
Question 3: Can optimized and uniform scales be compared on the same backend, model, and nonlinear implementations, with both configurations tuned to comparable accuracy requirements? Where feasible, please separate the contribution of bootstrapping retuning, or explain why it cannot be isolated. Please also reconcile the abstract's 83.6% reduction relative to THOR with Appendix F.2's statement that the ATLAS-derived count is not used for direct quantitative comparison.
Answer:
We compare our method with Zephyr, a GPU-optimized 30-bit CKKS system, using the same HE backend, Transformer model, and nonlinear approximations. Our contribution is reducing modulus consumption and bootstrapping frequency rather than accelerating the underlying HE implementation.
Under the same backend, bootstrapping time decreases from 36.54 s to 6.87 s (81.2% reduction). Scale allocation itself does not directly improve execution time, but its reduced modulus consumption enables bootstrapping parameter retuning and fewer bootstrapping operations. Therefore, the individual speedup contributions of these two optimizations cannot be meaningfully isolated, as they are interdependent.
The original THOR paper does not report the number of bootstrapping operations. Therefore, we used the counts from the ATLAS reproduction, which is already cited in our manuscript. We will revise Appendix F.2 to remove the statement that these counts are not used for quantitative comparison.