- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents OBELISK, a system for efficient non-interactive Transformer inference under CKKS FHE. It jointly optimizes three execution decisions: the modulus-level budget and nonlinear operators, intermediate-ciphertext lifetimes through Process-in-Pieces (PiP), and real vs. complex representations through Hybrid Real-Complex Representation (HRC). The authors' evaluation covers full 12-layer BERT-base inference on four GLUE validation sets, multiple GPUs, and component-level profiling and ablations. The authors report a 6.6× reduction in amortized server evaluation time and a 6.0× reduction in peak GPU memory relative to MOAI on an H200. The deployed system also runs on a 24 GiB RTX 4090, with accuracy losses of up to 2.52 percentage points relative to its plaintext checkpoints.
Critical strengths
- The memory reduction makes encrypted inference feasible on more accessible hardware. The system reduces the working set enough to run encrypted BERT inference on a 24 GiB GPU, whereas the evaluated baselines require GPUs with substantially more memory. Memory breakdowns, attention and FFN granularity sweeps, and successful runs across multiple GPUs support this result. This is a meaningful improvement in deployment feasibility.
- The paper provides convincing evidence that the execution optimizations complement one another. The level schedules explain how the budget constrains tiling and representation choices. The PiP/HRC factorial ablations show that tiling substantially reduces memory with little runtime overhead, while selective complex pairing reduces evaluation time. Validation of the bootstrap cost model across two libraries further supports the design. Together, these results demonstrate useful opportunities to optimize execution after the packing layout has been chosen
Critical weaknesses
-
The main performance comparison does not establish the speedup at comparable accuracy. Appendix F acknowledges that Table 1 compares computational cost without matching accuracy. OBELISK loses up to 2.52 percentage points relative to its plaintext model. Appendix H reports larger accuracy losses than THOR on RTE and MRPC, but these comparisons use different fine-tuned models and do not establish their relative accuracy under controlled conditions. A comparison on the same evaluation data, allowing each system its required model adaptation, would clarify how much of the speed advantage holds at comparable accuracy. This matters because the design uses a shorter modulus chain, and the authors attribute most of the remaining accuracy loss to homomorphic noise.
-
The contribution of budget–operator co-design to the overall speedup is not fully isolated. The base system already includes fused plaintext–ciphertext multiplication, a cumulative rotation-key strategy, and cached bootstrap encoding. Table G.3 shows that these additional optimizations reduce execution time to 32% of the budget-only configuration in an instrumented two-layer experiment. Although the paper measures these improvements and separately evaluates PiP and HRC, it does not provide an end-to-end ablation of budget–operator co-design. Several alternative budgets in Table B.1 also have timing results but do not produce decryptable outputs. This leaves uncertainty about how much the budget and operator choices contribute to the total speedup and how alternative numerically valid configurations compare in accuracy and performance.
Questions for authors
-
What accuracy does MOAI achieve on the evaluated tasks, and how does it compare with OBELISK? Without this information, how should we interpret the reported speedup in terms of the accuracy–performance tradeoff?
-
Appendix B.1 reports timings for several alternative budgets that do not produce decryptable outputs. How did you account for numerical correctness when selecting the budget? Did you compare the selected configuration with other budgets that produce correct outputs, including those using fewer special-modulus primes?
-
How does the 93.7-bit core-SVP estimate in Appendix A support the claim of 128-bit classical security for the deployed sparse secret? Please specify the estimator settings and the standard configuration used for comparison.
-
How do OBELISK’s GELU, softmax, and LayerNorm approximations differ from MOAI’s in multiplicative depth and accuracy? How much of the reported speedup comes from these differences and their effect on bootstrapping, rather than from the other execution optimizations?
Official Comment by Authors
We sincerely thank you for the careful reading and the insightful comments.
Response to Q1/W1 - Speedup at comparable accuracy
The speedup holds at comparable accuracy. General Response 1 traces most of the earlier drop to implementation errors (miscalibration) rather than to noise on the shorter chain. After the fixes, OBELISK loses at most 0.36 pp, close to the losses MOAI and THOR report. MOAI and THOR use their own fine-tuned models, so we compare the accuracy each system loses under encryption. The table gives encrypted accuracy (%), with that loss in parentheses.
In a controlled comparison on THOR's released MRPC model and the same 408 validation inputs, OBELISK is within one input of THOR at 17.9× its throughput on an RTX PRO 6000 (19.1× on an A100). We rebuilt only OBELISK's per-task constants from the MRPC training data. MOAI has not released its task models, and THOR has released only its MRPC model. THOR's accuracy and A100 time are as reported, and we measured the other times.
Response to Q2/W2 - The co-design ablation and alternative budgets
In General Response 2, the budget–operator co-design makes the baseline with all three base techniques 2.94× faster (row 3 to row 5). Of this, the operator choices give 1.68× and the budget 1.75×. The base techniques themselves give 1.42× over our MOAI rerun. Table G.3's 32% came from adding the base techniques to our cheaper budget in an instrumented two-layer run, where weight encoding takes a larger share.
Every alternative budget we re-ran gives the same predictions as the deployed (2, 16, 5), which remains the cheapest. Accuracy thus does not separate the budgets, so cost alone decides. We ran L = 17 to 21, and L = 16 with fewer special primes, through all 12 layers on 256 MRPC inputs.
The submitted note that budgets with L ≥ 17 do not decrypt was wrong. Each budget gets 224 right, against 223 for float64 plaintext. Budgets with L ≤ 15 need a third bootstrap per layer, so we excluded them. The revised 128-bit bound gives L = 19 and 20 one fewer special prime than the submission. Times are for one layer on an RTX PRO 6000. Every column is relative to the deployed budget. Cosine compares each input's last-layer [CLS] vector under the two budgets.
Response to Q3 - 128-bit security
The deployed configuration now reaches 128.2 bits of security through adopting SSE, with less than 1% of runtime overhead. The 93.7 bits in Appendix A came from the estimator's rough core-SVP mode and do not support 128 bits, and the full estimator puts the submitted secret at about 121 bits. We appreciate this question, which led us to the fix. General Response 1 gives the estimator settings and the HE Standard's largest 128-bit point.
Response to Q4 - Operator choices against MOAI
Most of the operator gain comes from LayerNorm, which takes 2 levels instead of MOAI's 20. A layer then needs 30 levels instead of MOAI's 63, which halves the full-width bootstraps (row 4 of General Response 2, 1.68×) and allows the shorter chain (rows 4 to 5, 1.75×).
Each of our operators stays within 0.05 pp of the exact model's 90.68%. We use the public TextAttack checkpoints of our main results, as MOAI's are not public. MOAI's LayerNorm and softmax diverge as released on these checkpoints, so we refit them on our training inputs. Its GELU is accurate only on [−17, 10], while inputs here reach 150, and a same-degree refit is worse. Accuracy covers 7,020 validation inputs, one operator replaced at a time in float64.
Thank you again for the thoughtful review.