acceptodds
Under review as a conference paper at ICLR 2027

Mygo: Cross-Module Co-Design for Compact Non-Interactive Transformer Inference with Fully Homomorphic Encryption

Abstract

Fully homomorphic encryption (FHE) enables non-interactive secure Transformer inference, but fitting the complete computation into a compact configuration while hiding request shape remains challenging. We present MYGO, which addresses this challenge by co-designing three components under a fixed deployment contract: (i) a compact FHE configuration, using RNS-CKKS with a common level-4 post-bootstrap interface; (ii) fixed execution for request-shape privacy, designing Standardized Interleaved Batching (SIB) with encrypted validity metadata to hide within-container occupancy and sequence lengths for 1–128 variable-length inputs; and (iii) cross-module numerical compilation, jointly distilling FFN affine layers and cubic activations, selecting site-specific minimax circuits, and folding representation factors into affine maps to avoid separate online scaling operations. MYGO's BERT-base achieves 92.09% plaintext accuracy on SST-2. Its full encrypted graph processes 128 inputs in 2,788 s with 22.37 GiB peak GPU memory on one RTX 5090, taking only 2.1% longer than with one active input. Against a matched control, evaluation-key payload, post-bootstrap resident GPU memory, and synchronized bootstrapping latency decrease by 50.3%, 35.1%, and 22.9%, respectively. The same configuration executes LLaMA-3-8B's complete first decoder layer with median output MAE of against FP32. These results demonstrate the feasibility of resource-efficient, non-interactive FHE Transformer inference on a single consumer GPU.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Fully homomorphic encryption, Privacy-preserving Transformer inference, Request-shape privacy, Ciphertext packing, Cryptographic–Numerical Co-Design, Low-Depth Polynomial Approximation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_5NBrOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Mygo, a system for non-interactive Transformer inference under CKKS fully homomorphic encryption. It combines a compact ring configuration, a common post-bootstrap interface allowing four rescale steps, fixed-capacity batching with encrypted metadata to hide occupancy and sequence lengths, and cross-module numerical compilation. The numerical design includes jointly distilled FFN layers with cubic activations and scaling factors folded into affine operations. The evaluation reports plaintext student accuracy on three GLUE tasks, independent encrypted BERT layer tests, complete-graph runtime measurements, resource and numerical ablations, and continuous encrypted execution of LLaMA-3-8B’s first decoder layer. The complete BERT graph processes 128 inputs in 2,788 seconds with 22.37 GiB peak GPU memory on an RTX 5090. However, end-to-end encrypted classification accuracy is not established.

Critical strengths
  1. The paper clearly specifies its intended privacy guarantee. The leakage definition and conditional argument distinguish protected occupancy and sequence lengths from observable metadata. This provides useful clarity, although fixed-capacity batching largely follows existing work, and the practical benefit and incremental cost of encrypted metadata remain unestablished.

  2. The implementation provides useful feasibility and resource measurements. The paper integrates low-degree model adaptation, site-specific approximations, and affine fusion into a compact FHE configuration. The ring-only control quantifies resource savings, while numerical-alignment ablations and continuous encrypted execution of one LLaMA decoder layer provide supporting evidence. These are useful implementation results, but do not by themselves establish substantial methodological novelty or end-to-end encrypted model accuracy.

Critical weaknesses
  1. End-to-end encrypted classification accuracy is not established. Appendix F.1 states that the reported task accuracies are plaintext student results using standard attention, normalization, and pooler operators. The encrypted BERT probes initialize each layer from its reference input, so they do not measure accumulated error through the complete model. Appendix H.2 also acknowledges classification drift in several multi-input runs and limits those results to timing and memory evidence. The complete-graph measurements therefore demonstrate execution, but not the accuracy of the resulting predictions. This is a central validation gap for the proposed encrypted inference system.

  2. The co-design offers limited methodological novelty, and its comparative benefit is not established. Fixed-capacity interleaved batching largely follows MOAI. The three elements of the numerical-compilation contribution also provide limited novelty: jointly training low-degree polynomial activations and surrounding affine layers is an established FHE model-adaptation strategy; selecting approximation intervals and circuits for individual sites is an incremental refinement of range-based approximation; and absorbing representation factors into weights and biases is standard affine fusion. The paper does not clearly identify a new algorithm or demonstrate a nontrivial advantage from coordinating these ingredients. Table 2 further combines published timings across different hardware, libraries, capacities, model adaptations, and evaluation scopes. The ring-only control quantifies specific resource savings, but does not establish an end-to-end advantage at comparable accuracy and security.

Questions for authors
  1. What accuracy does the full encrypted BERT model achieve on held-out examples with one active input and with all 128 lanes occupied? What causes the classification drift reported in Appendix H.2, and how often do encrypted predictions differ from those of the plaintext student?

  2. What evidence shows that Mygo improves runtime and memory use over a baseline at comparable accuracy and security? Can you provide an end-to-end comparison on the same hardware, or ablations that separate the effects of the smaller ring, numerical redesign, and underlying FHE library?

  3. Section 3.1 reports a 125.90-bit MATZOV/default estimate and acknowledges that the 128-bit target is not a verified lower bound. What security level is supported for the deployed configuration? What parameter changes would be needed to reach the target under the chosen attack-cost model, and how would they affect runtime and memory?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 5NBr (Part 1/3, W1–W2)

W1: End-to-end encrypted classification accuracy is not established. Appendix F.1 states that the reported task accuracies are plaintext student results using standard attention, normalization, and pooler operators. The encrypted BERT probes initialize each layer from its reference input, so they do not measure accumulated error through the complete model. Appendix H.2 also acknowledges classification drift in several multi-input runs and limits those results to timing and memory evidence. The complete-graph measurements therefore demonstrate execution, but not the accuracy of the resulting predictions. This is a central validation gap for the proposed encrypted inference system.

The calibrated N=216N=2^{16}, Level-13 teacher carries ciphertexts through all 12 layers, pooler, and classifier on the same fixed 256-input SST-2 validation cohort. Both runs achieve 93.36% label accuracy (239/256) and 256/256 agreement with both exact and circuit-matched FP64 references.

Both runs compute all 768 columns; light probes observe four layer-specific columns, full probes all 768. Fitting uses 2,048 training examples; plaintext diagnostics on the fixed validation cohort set positivity margins before FHE.

Thank you for emphasizing the need for final classifier outputs. Table 1's three-task accuracies describe the plaintext compact student with standard attention, LayerNorm, and pooler and the same cubic FFNs. Its independent layer probes start from matching reference inputs. Q1 gives the compact student's continuous results.

Changes: Section 6.1/Table 1 labels plaintext/local results (p. 7, lines 342–361); Table 2/Figure 7 reports teacher classification and coverage (p. 8, lines 378–398). G.2–G.3 specify cohorts and reference models (pp. 37–38, lines 1982–2020; S1).


W2: The co-design offers limited methodological novelty, and its comparative benefit is not established. Fixed-capacity interleaved batching largely follows MOAI. The three elements of the numerical-compilation contribution also provide limited novelty: jointly training low-degree polynomial activations and surrounding affine layers is an established FHE model-adaptation strategy; selecting approximation intervals and circuits for individual sites is an incremental refinement of range-based approximation; and absorbing representation factors into weights and biases is standard affine fusion. The paper does not clearly identify a new algorithm or demonstrate a nontrivial advantage from coordinating these ingredients. Table 2 further combines published timings across different hardware, libraries, capacities, model adaptations, and evaluation scopes. The ring-only control quantifies specific resource savings, but does not establish an end-to-end advantage at comparable accuracy and security.

Across the LN1–FFN–LN2 interfaces, MYGO chooses output representations to meet the next module's approximation interval and Boot domain within the shared level budget. The representation factors are distinct from CKKS scale and fold into adjacent affine maps. The compact student's cubic FFN fits four Rescales and places Boot after FC2 contracts 3,072 ciphertexts to 768. SIB combines fixed-capacity interleaving with encrypted validity to keep the public execution graph fixed while protecting occupancy and individual lengths.

Table 19 reports representation-factor controls. The N=215N=2^{15}, Level-4 Layer-0 study fixes 100 SST-2 inputs (1,675 valid tokens), parameters, packing, levels, ciphertext counts, and 41 Boot timing events. Each arm uses three fresh processes; boundary entries are medians over valid coordinates.

Table 19C: outside-domain fraction (%)LN1 corePre-LN2 residual
Matched BASE00
FFN-bridge factors removed28.76239.497

A separate attention fixture repeats one five-token example across 128 lanes, with three fresh processes per arm (mean ± sample SD). Both selected Table 19A arms use the same degree-31 Mod-8 inverse on [0.1,3.0][0.1,3.0].

Table 19A armMean relative ℓ1\ell_1 error (%)
Compact Exp with restoration1.234±0.1171.234\pm0.117
Compact scale-free Exp0.978±0.0570.978\pm0.057

The folded scale change adds no online multiplication, level, or Boot.

Changes: Section 5.1 defines the interface (p. 5, lines 249–259); Section 6.2 explains the bridge/attention controls (p. 9, lines 462–485). H.3 clarifies existing Table 19A/C workloads, matched BASEs, and statistics (pp. 42–44, lines 2256–2326; S2).

AuthorsOpenReview ↗

Response to Reviewer 5NBr (Part 2/3, Q1–Q2)

Q1: What accuracy does the full encrypted BERT model achieve on held-out examples with one active input and with all 128 lanes occupied? What causes the classification drift reported in Appendix H.2, and how often do encrypted predictions differ from those of the plaintext student?

For the N=215N=2^{15}, Level-4 compact student, all three configurations execute 12 layers, pooler, and classifier.

Table 13 cohortFHE label correctFP32 label correctPrediction agreement
One unlabeled author example——1/1
127 distinct labeled inputs + author example66/127112/12768/128
One labeled example replicated 128 times62/128128/12862/128

The mixed cohort uses SST-2 validation inputs. On its 127 labeled inputs, 67/127 predictions agree and 60/127 differ; the full-container 68/128 includes the shared unlabeled example. The 2,788 s run is the replicated-lane diagnostic on one RTX 5090. Single/mixed input runs use an RTX PRO 6000.

At the first encoder observation, the shared example's RMSE is 0.04146 with one input and 11,469.4 in the mixed container; aggregate mixed RMSE is 11,348.2. These measurements cover valid tokens in 16 fixed hidden columns. The example's final-logit maximum error is 0.07031/110.615 (single/mixed), while its predicted class is unchanged.

The FP32 student uses standard attention, LayerNorm, and pooler with the same cubic FFNs. Thus the differences include circuit approximation and encrypted execution. Occupancy, neighboring inputs, keys, and encryption randomness vary together: the traces locate divergence by the first layer end without isolating a cause.

Changes: G.2 separates labeled accuracy, agreement, and replicated-lane diagnostics in Table 13 (pp. 37–38, lines 1982–2006). G.3 reports the paired traces, references, and observation scope (p. 38, lines 2008–2020). S1 and the CSV retain per-lane outcomes.


Q2: What evidence shows that Mygo improves runtime and memory use over a baseline at comparable accuracy and security? Can you provide an end-to-end comparison on the same hardware, or ablations that separate the effects of the smaller ring, numerical redesign, and underlying FHE library?

At fixed QPQP, Level-4 Boot output, and topology, the compact ring saves 50.3% key payload, 35.1% post-Boot resident memory, and 22.9% synchronized Boot latency. W2's numerical controls hold the ring and online schedule fixed.

The encoder measurements use one RTX PRO 6000, the same Cheddar build, N=216N=2^{16} chain, scale 2402^{40}, Level-13 Boot output, 256-lane capacity, and 12-layer endpoint. Each row is one run.

Tables 3/14: light probesEncoder prefix (s)Full run (s)CUDA-pool used peak (GiB)
MYGO teacher4,338.284,376.6837.22
Reference port6,945.996,945.9925.58

Prefixes include setup and probes; MYGO's full run also includes its head. MYGO uses its calibrated teacher and 256 active inputs (6,508 tokens); the port uses author weights, fixed shifts, and one five-token example. The 37.5% shorter prefix describes these recorded workloads with their respective checkpoints and active cohorts. See W1 for the accuracy results and Q3 for the security estimates.

For the reference port, Table 14 also records 8,757.00 s for the earlier version. The 20.7% decrease to 6,945.99 s compares versions that differ in both the implementation and the Cheddar build.

Encrypted validity hides occupancy and individual lengths within the public profile and permitted leakage. The 134 ciphertexts comprise 128 full-C pair masks, 1 row mask, 4 Attention length fields, and 1 compatibility shift field (F.3):

Table 16 configurationMetadata ciphertextsRaw RNS payload (GiB)
Compact student1340.262
Level-13 teacher1341.243

The teacher prepares metadata once per batch: Encode alone takes 6.73 s (0.154% of the light-probe run) within 8.32 s complete preparation, including encryption; online gauge takes 20.24 s.

Changes: Section 6.1/Table 3 identifies the common-backend cohorts (p. 8, lines 412–426); G.4/Table 14 records both port versions and their timing scopes (pp. 38–39, lines 2023–2053). Section 6.2 separates ring and numerical controls (p. 9, lines 434–485); H.1/Table 18A gives paired ring costs (p. 42, lines 2222–2225). G.7/Table 16 lists the metadata inventory and component costs (p. 40, lines 2106–2126; S2–S4).

AuthorsOpenReview ↗

Response to Reviewer 5NBr (Part 3/3, Q3)

Q3: Section 3.1 reports a 125.90-bit MATZOV/default estimate and acknowledges that the 128-bit target is not a verified lower bound. What security level is supported for the deployed configuration? What parameter changes would be needed to reach the target under the chosen attack-cost model, and how would they affect runtime and memory?

The compact configuration's main-domain minimum is 125.90 bits under the stated MATZOV/GSA model, below the 128-bit target. This estimate concerns the dense main secret (weight 16,384) on the exposed Q16PQ_{16}P basis, with log⁡2(Q16P)=880.998757\log_2(Q_{16}P)=880.998757.

The deployed weight-32 auxiliary Boot secret has a different exposure basis: log⁡2(qp)=96.982967\log_2(qp)=96.982967. On that actual short basis, the evaluated primal BDD hybrid, dual MITM hybrid, and analytical meet-in-the-middle costs are 176.6608, 147.5764, and 210.8826 bits; their minimum is 147.5764 bits. The main-domain four-attack set comprises uSVP, BDD, dual, and dual hybrid. Estimates use m=Nm=N, Gaussian error standard deviation 3.2, and the stated balanced-ternary model for random-sign fixed-weight secrets.

A larger-ring candidate keeps the compact full-QPQP product, short-basis primes, Level-4 Boot output, and operator topology. At N=216N=2^{16}, main-secret weight 32,768 and auxiliary weight 32, its main-domain four-attack minimum is 272.1371 bits. The short-domain dual MITM hybrid estimate is 170.6900 bits.

Table 18A: fixed-QPQP resource controlN=215N=2^{15}N=216N=2^{16}
Synchronized Boot (ms)19.27825.018
Evaluation-key payload (GiB)8.59717.304
Post-Boot resident GPU memory (GiB)15.9424.56

These are paired resource microbenchmarks at the compact profile's modulus budget.

A second option is an estimator-only modulus-budget screen. Keeping N=32768N=32768, main-secret weight 16,384, error standard deviation 3.2, and m=Nm=N, a synthetic effective modulus 28682^{868} gives a four-attack minimum of 128.0290 bits. This analytical point still requires a compatible RNS chain, Boot precision, and refresh schedule; it has no measured execution cost.

The N=216N=2^{16} teacher is specified separately by its Level-13 chain, with log⁡2(QP)=1745.415737\log_2(QP)=1745.415737 and log⁡2(qp)=112.013493\log_2(qp)=112.013493. The reported attack costs apply to the compact chain and its fixed-QPQP control.

Changes: Section 3.1 identifies the deployed dense/sparse domains and their estimates (p. 3, lines 146–152). D.2 gives the short-basis attacks, larger-ring candidate, analytical budget screen, and separate teacher-chain binding (pp. 24–25, lines 1290–1329). H.1/Table 18A provides measured resource costs (p. 42, lines 2222–2225; S3).

Reviewer_GCqJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MYGO implements non-interactive CKKS BERT-base inference at ring dimension N = 2^15, using four levels between bootstraps. A fixed execution graph and encrypted masks protect request-shape privacy. On an RTX 5090, the full 128-input graph takes 2,788 s, or 21.78 s/input, at 22.37 GB peak memory.

Critical strengths
  1. The paper formally addresses leakage of batch occupancy and input lengths. Resource measurements remain stable across 1 to 128 active inputs.
  2. A matched N = 2^16 control supports the reported resource savings, and the complete graph fits one consumer GPU.
Critical weaknesses
  1. End-to-end encrypted correctness is not established. Task accuracies are plaintext, and layer-local probes cannot reveal accumulated errors. Appendix H.2 reports classification drift, leaving it unclear whether the headline 2,788 s run produces correct predictions.
  2. The full-basis security estimate is 125.90 bits, below the 128-bit target, and the deployed sparse short-basis configuration lacks an estimate. Differences in hardware and timing scope further weaken the efficiency comparison.
Questions for authors
  1. Clarification, decisive for my score: What are the full 12-layer encrypted accuracy and agreement with plaintext? Did the 2,788 s run classify correctly, and what causes the observed drift?
  2. Clarification: Can you provide parameters meeting 128-bit security, quantify their performance cost, and estimate security for the deployed sparse short-basis configuration?
  3. Clarification: How are inputs longer than 128 tokens handled? Does request-shape privacy still hold when additional containers are required?
3 comments
AuthorsOpenReview ↗

Response to Reviewer GCqJ (Part 1/3, W1–W2)

W1: End-to-end encrypted correctness is not established. Task accuracies are plaintext, and layer-local probes cannot reveal accumulated errors. Appendix H.2 reports classification drift, leaving it unclear whether the headline 2,788 s run produces correct predictions.

The calibrated N=216N=2^{16}, Level-13 teacher completes all 12 layers, the pooler, and the classifier, with 239/256 label accuracy (93.36%) and 256/256 agreement with both exact and circuit-matched FP64 references. The light-probe execution takes 4,376.68 s through the classifier on one RTX PRO 6000. We appreciate your request to connect execution time to final predictions.

Both runs compute the full-width model on the same fixed 256-input validation cohort (6,508 valid tokens). Light probes observe four layer-specific columns per layer; full probes observe all 768. Fitting uses 2,048 training examples; plaintext diagnostics on this validation cohort set positivity margins before FHE. Final-layer maximum errors against circuit-matched FP64 are 0.00674 on the light profile's sampled valid coordinates and 0.0509 on the full profile's valid coordinates.

Table 1 reports plaintext adaptation and independent-layer probes initialized from each layer's reference input. Q1 below gives the compact student's continuous classification results, including the original 2,788 s run.

Changes: Section 6.1 distinguishes adaptation and local probes in Table 1 (p. 7, lines 342–361) from continuous classification in Table 2 and Figure 7 (p. 8, lines 378–398). Appendices G.2–G.3 record the cohorts, reference models, and observation widths (pp. 37–38, lines 1982–2020).


W2: The full-basis security estimate is 125.90 bits, below the 128-bit target, and the deployed sparse short-basis configuration lacks an estimate. Differences in hardware and timing scope further weaken the efficiency comparison.

MYGO combines dense-secret inference with sparse-secret Boot encapsulation. We assess each secret on the basis exposed by its evaluation keys:

Compact key domainSecret weightlog⁡2\log_2 exposed modulusEstimated cost (bits)
Main Q16PQ_{16}P16,384880.998757125.90
Auxiliary short qpqp3296.982967147.5764

The return switching key targets the main secret. The main-domain estimate is below the 128-bit target. The main four-attack set is uSVP, BDD, dual, and dual hybrid, with MATZOV costs, GSA shapes, m=Nm=N, and discrete Gaussian error σ=3.2\sigma=3.2. Balanced ternary secrets approximate the implementation's fixed-weight, random-sign secrets. The short-basis value is the minimum of the evaluated finite sparse-secret costs: primal BDD hybrid 176.6608, dual MITM hybrid 147.5764, and analytical MITM 210.8826 bits.

Table 3 compares the encoders on one RTX PRO 6000, with the same patched Cheddar build, N=216N=2^{16} Level-13 chain, scale 2402^{40}, 256-lane capacity, and full-width 12-layer endpoint. Its light-probe rows give:

MeasurementMYGO teacherReference port
Active inputs / valid tokens256 / 6,5081 / 5
Encoder prefix, including setup and probes (s)4,338.286,945.99
Peak CUDA-pool use (GiB)37.2225.58

MYGO uses its calibrated teacher; the reference port retains the authors' weights and fixed shifts on one five-token example. The 37.5% shorter prefix describes these executed workloads, whose weights and inputs differ. The Level-13 timing chain is separate from the compact Level-4 chain and its fixed-QPQP ring control; it does not inherit their security estimates.

Changes: Section 3.1 identifies the actual key exposures (p. 3, lines 146–152); D.2 gives attack models and the short-basis assessment (pp. 24–25, lines 1290–1307). Table 3 and G.4 state the common backend, endpoints, and active cohorts (p. 8, lines 412–426; pp. 38–39, lines 2023–2053).

AuthorsOpenReview ↗

Response to Reviewer GCqJ (Part 2/3, Q1–Q2)

Q1: Clarification, decisive for my score: What are the full 12-layer encrypted accuracy and agreement with plaintext? Did the 2,788 s run classify correctly, and what causes the observed drift?

The original RTX 5090 run returned 62 correct predictions on 128 copies of one labeled SST-2 example, with 62/128 FP32 agreement. The FP32 student predicts that example's label correctly.

In the mixed run, label correctness uses 127 labeled inputs, while prediction agreement covers all 128 lanes (Table 13):

Continuous runFHE label correctPrediction agreement
Compact student: one example replicated 128 times62/12862/128
Compact student: one active unlabeled exampleUnavailable1/1
Compact student: 127 labeled inputs + that example66/12768/128
Level-13 teacher: 256 inputs, both probe runs239/256256/256

Compact rows use the FP32 student; the teacher agrees with both exact and circuit-matched FP64 references. In the mixed run, FP32 classifies 112/127 labels correctly; labeled prediction agreement is 67/127, with 60/127 differences.

The student reference has the same cubic FFNs and standard attention, LayerNorm, and pooler. Its 16-column probes show divergence by the first encoder output: RMSE on the shared example is 0.04146 with one input and 11,469.4 in mixed lane 0. Maximum final-logit errors are 0.07031 and 110.615, while that example's class stays unchanged. Keys, encryption randomness, occupancy, and neighbors change together. These traces locate the first observed divergence without isolating its cause; errors include circuit approximation and encrypted execution.

Changes: G.2 reports the four cohorts and explicit denominators (pp. 37–38, lines 1982–2006); G.3 gives drift and reference scope (p. 38, lines 2008–2020). Table 2 gives teacher predictions (p. 8, lines 378–388); S1/CSV provide lane records.


Q2: Clarification: Can you provide parameters meeting 128-bit security, quantify their performance cost, and estimate security for the deployed sparse short-basis configuration?

The implemented larger-ring control increases NN from 2152^{15} to 2162^{16} while retaining compact QPQP, short primes, post-Boot Level 4, and operator topology. Its main-domain four-attack minimum is 272.1371 bits, and its short-domain dual MITM hybrid estimate is 170.6900 bits. The main-secret weight increases from 16,384 to 32,768; auxiliary weight remains 32.

Table 18A reports the paired microbenchmarks:

QuantityCompact N=215N=2^{15}Fixed-QPQP N=216N=2^{16} control
Synchronized Boot (ms)19.27825.018
Evaluation-key RNS payload (GiB)8.59717.304
Post-Boot resident GPU memory (GiB)15.9424.56

For the deployed compact chain, W2 gives 125.90 bits on the main domain, below 128, and a 147.5764-bit minimum evaluated short-basis cost. At N=215N=2^{15}, substituting a synthetic effective modulus 28682^{868} in the main-domain instance gives 128.0290 bits. This is an estimator-only budget screen requiring a compatible RNS and refresh schedule; it has no measured execution costs. The measured resources above apply to the fixed-QPQP control. The separate Level-13 teacher chain does not inherit its estimates.

Changes: D.2 specifies the parameters used for each estimate and separates the synthetic screen and teacher chain (p. 25, lines 1307–1329). H.1/Table 18A gives paired Boot, key-payload, and memory costs (p. 42, lines 2222–2225).

AuthorsOpenReview ↗

Response to Reviewer GCqJ (Part 3/3, Q3)

Q3: Clarification: How are inputs longer than 128 tokens handled? Does request-shape privacy still hold when additional containers are required?

Both evaluated BERT profiles accept up to 128 tokens including special tokens, excluding longer examples without truncation. For full-slot SIB, Bmax⁡Mmax⁡=N/2B_{\max}M_{\max}=N/2. Appendix E gives the following layout derivation at N=215N=2^{15}:

Public token capacity Mmax⁡M_{\max}128256512
Sequence lanes Bmax⁡B_{\max}1286432

The 256/512-token entries are layout derivations, not measured longer-sequence executions. Each larger capacity requires its own compiled profile with full-sequence attention. Independent windows omit cross-window attention.

Encrypted validity protects occupancy and individual lengths within each selected public capacity. Within a fixed profile, each additional container uses the same encrypted template. Privacy compares supported requests under the same public contract and equal permitted leakage, assuming fixed transcripts, fresh independent encryption randomness, and CPA-EK security for the complete public/evaluation-key bundle against an honest-but-curious CSP. For n>0n > 0 input sequences, minimum allocation without cover containers uses K=⌈n/Bmax⁡⌉K=\lceil n/B_{\max}\rceil and reveals (K−1)Bmax⁡<n≤KBmax⁡(K-1)B_{\max} < n\leq KB_{\max}. Container count, timing, key reuse, and profile choice remain visible. With public 128/256-token choices and no cover routing, selecting the smallest fitting profile for one input reveals 129–256 tokens when the 256-token profile is chosen. Batch-maximum routing instead constrains the longest input.

Changes: Section 2 separates public capacity and private occupancy (p. 3, lines 118–130). C.1–C.2 state transcript and privacy conditions (pp. 18–19, lines 918–976); E gives derived capacities and routing leakage (pp. 27–28, lines 1453–1467). Section 6 records the tested 128-token limit (p. 7, lines 333–338).

Reviewer_kK8xOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MYGO, a framework for efficient and privacy-preserving Transformer inference using fully homomorphic encryption (FHE). The framework jointly optimizes three aspects of encrypted inference: cryptographic configuration, data packing, and numerical computation. The authors evaluate MYGO using BERT-base on three classification benchmarks and report full encrypted BERT inference on a single RTX 5090 GPU. Additional experiments examine memory consumption, runtime, numerical approximation accuracy, and execution consistency across request shapes. The paper also demonstrates encrypted execution of the first decoder layer of LLaMA-3-8B to investigate the applicability of the proposed techniques beyond BERT.

Critical strengths
  • Its motivation of reducing high computational and memory requirements of FHE significantly that limit the practical deployment of Transformer models is good.
  • The quantitative measurements of runtime, memory usage, and numerical approximation errors, along with ablation studies are good.
  • The fixed execution structure for variable-length requests is another practically relevant feature.
Critical weaknesses
  • The comparison with existing methods is not sufficiently controlled to support strong performance claims. Table 2 compares MYGO against systems running on different hardware, including A100, H200, and RTX 6000 Ada GPUs, with different batch capacities and evaluation scopes. Some baselines evaluate only the Transformer encoder, whereas others include additional model components.
  • The applicability of the proposed framework to larger and more general Transformer models remains limited.
Questions for authors
  • Could the authors provide a more detailed analysis of the classification drift observed in the multi-input encrypted experiments? In particular, does this issue arise from encrypted batching, numerical approximation errors, or interactions between different components of the computation graph?
  • How sensitive is the proposed numerical compilation approach to changes in the input data distribution?
  • How the system handles inputs that fall outside the registered numerical ranges?
  • How does the framework balance the trade-off between hiding request shapes and avoiding unnecessary computation?

Generally, I think the motivation and method is good and would raise my scores if the method can be extended to larger-scale models (e.g., 27B and beyond).

4 comments
AuthorsOpenReview ↗

Response to Reviewer kK8x (Part 1/3, W1–W2)

Thank you for highlighting the need to align hardware, capacity, and evaluation endpoints.

W1: The comparison with existing methods is not sufficiently controlled to support strong performance claims. Table 2 compares MYGO against systems running on different hardware, including A100, H200, and RTX 6000 Ada GPUs, with different batch capacities and evaluation scopes. Some baselines evaluate only the Transformer encoder, whereas others include additional model components.

Table 3 uses a common full-width 12-layer encoder endpoint on one RTX PRO 6000 Blackwell Server Edition, with the same Cheddar build, N=216N=2^{16} chain, scale 2402^{40}, post-Boot Level 13, and 256 configured lanes.

Table 3: light-probe measurementsMYGOReference port
Active inputs2561
Valid tokens6,5085
Encoder prefix (s)4,338.286,945.99
Peak CUDA-pool use (GiB)37.2225.58
Peak pool reservation (GiB)47.2247.22

The prefix includes setup and the arm's light probes; each column is one run. MYGO uses its calibrated teacher, while the reference port retains the authors' weights, fixed shifts, and original five-token example. The 37.5% shorter prefix describes these workloads with different weights and active cohorts.

The optimized MYGO path uses Cheddar's native Chebyshev evaluator. The optimized reference port reuses encoded public masks and adds the scalar LayerNorm bias before row masking. For the reference port's encoder workload, full light-probe time went from 8,757.00 s in the earlier version to 6,945.99 s in the optimized version (Table 14). The 20.7% decrease compares versions that differ in both the implementation and the Cheddar build.

Published system reports keep their hardware, capacity, and endpoint labels. The fixed-QPQP ring control is reported separately.

Changes: Section 6.1/Table 3 makes the common encoder endpoint and different cohorts explicit (p. 8, L412–426). Appendix G.4/Table 14 documents both versions of the reference port and their timing scopes (pp. 38–39, L2023–2053; S2–S3).


W2: The applicability of the proposed framework to larger and more general Transformer models remains limited.

For LLaMA-3-8B, we execute the complete first decoder layer with original weights at N=215N=2^{15}. Numerical settings are frozen before testing 2,553 eight-token WikiText-2 test prefixes excluded from calibration, with three independent FHE runs per batch. The path includes RMSNorm, RoPE, causal grouped-query attention, and the full 14,336-wide SwiGLU FFN. All 11 audited boundaries have zero observed domain violations. Median pooled output MAE/P99 against CPU FP32 are 4.652×10−44.652\times10^{-4}/1.679×10−31.679\times10^{-3}.

For Qwen3.6-27B, a common 5,120-column Level-3 ciphertext handoff connects 48 Gated DeltaNet and 16 Gated Attention layers within the post-Boot Level-4 schedule; each includes a full 17,408-wide SwiGLU FFN. Q5 reports the completed 64-layer text prefill and vocabulary endpoint.

Numerical fidelity of the hybrid mixer is measured separately on a Gated DeltaNet head-group fixture: synthetic projected activations, official Layer-0 weight slices, eight tokens in two active lanes, one Q/K head, three V heads, and an eight-column output tile. The encrypted path covers convolution/SiLU, Q/K normalization, triangular solve, and gated RMSNorm before residual refresh. All 12 observed stage groups meet predefined tolerances. Final Level-3 handoff RMSE is 6.867×10−46.867\times10^{-4} against the CPU polynomial reference.

Changes: Section 6.3 distinguishes LLaMA layer fidelity from Qwen full-model execution (p. 10, L488–529). Figure 11 reports LLaMA layer fidelity; I.1 specifies its protocol (p. 44, L2333–2355). Appendix J documents the separate Gated DeltaNet head-group fixture and its validation (p. 48, L2563–2586).

AuthorsOpenReview ↗

Response to Reviewer kK8x (Part 2/3, Q1–Q2)

Q1: Could the authors provide a more detailed analysis of the classification drift observed in the multi-input encrypted experiments? In particular, does this issue arise from encrypted batching, numerical approximation errors, or interactions between different components of the computation graph?

All rows evaluate the 12-layer encoder, pooler, and classifier. Divergence is visible by the first observed encoder output.

Table 13: continuous classification(N,LBoot)(N,L_{\rm Boot})ActiveLabel correctAgreement
Student, replicated input(215,4)(2^{15},4)128 copies62/12862/128
Student, single input(215,4)(2^{15},4)1Unlabeled1/1
Student, mixed inputs(215,4)(2^{15},4)12866/12768/128
Teacher, both probe runs(216,13)(2^{16},13)256239/256256/256

Student agreement uses FP32; teacher agreement uses both exact and circuit-matched FP64 references. Each student row reports one run; the two teacher runs use the same 256-input cohort.

The original 2,788 s RTX 5090 run repeats labeled sample 379 in all lanes. FP32 is correct in 128/128 lanes; FHE is correct and agrees in 62/128. This is a replicated-lane diagnostic. The other student runs use an RTX PRO 6000. The single-active author example has no label. The mixed container includes that example plus 127 distinct, frozen SST-2 validation inputs: FHE gives 66/127 (51.97%) accuracy versus 112/127 (88.19%) for FP32. Labeled predictions agree on 67/127 and differ on 60/127; total agreement is 68/128.

Appendix G.3 gives the corresponding diagnostics:

Shared author exampleSingle-activeMixed, lane 0
First-layer RMSE, 16 observed columns0.0414611,469.4
Maximum final-logit error0.07031110.615

The mixed cohort's aggregate first-layer RMSE is 11,348.2. RMSE uses decoded real values on valid tokens; the shared example's prediction agrees with FP32 in both runs. FP32 uses standard attention, LayerNorm, and pooler with the same cubic FFNs, so deviations combine circuit approximation and encrypted execution. Because keys, encryption randomness, occupancy, and neighbors change together, these runs locate the first observed divergence but do not isolate its cause.

Both teacher runs compute all 768 columns per layer, observing four layer-specific/all 768 columns. Final-layer maxima versus circuit-matched FP64 are 0.00674/0.0509 over the respective observed columns; both runs preserve all final predictions.

Changes: G.2/Table 13 identifies the configuration and cohort for each classification result; G.3 records layer-end errors, reference definitions, and changing variables (pp. 37–38, L1982–2020). Section 6.1/Table 2 reports the separate teacher results (p. 8, L378–398). S1/CSV supplies lane-level records.


Q2: How sensitive is the proposed numerical compilation approach to changes in the input data distribution?

Calibration fixes approximation intervals, guards, representation factors, and Boot domains before FHE. The N=216N=2^{16}, Level-13 teacher fits numerical tables on 2,048 training examples; plaintext diagnostics on its same fixed 256-input validation cohort set positivity margins. Input-distribution changes can alter coverage of these frozen domains.

The separate N=215N=2^{15}, Level-4 Layer-0 controls use 100 fixed SST-2 inputs, 1,675 valid tokens, and three fresh processes per arm. Selected factors are set to one and affine compensation is recompiled, keeping parameters, CKKS scales, packing, topology, levels, ciphertext counts, and 41 Boot timing events fixed.

Table 19C, selected boundaries: outside-domain (%)LN1 corePre-LN2 residual
Matched BASE0.0000.000
FFN-bridge-ID28.76239.497

These are median fractions with ∣x∣>1|x| > 1 over valid pre-Boot coordinates. The bridge arm uses its dedicated three-run BASE, separately from the main batch. Individual factor removal gives 8.55%, 28.77%, 39.50%, and 0.36% at the four corresponding boundaries, versus zero observed with BASE.

A separate attention control uses one five-token example replicated across 128 lanes and three processes per arm. At fixed reciprocal circuit, scale cancellation lowers mean relative ℓ1\ell_1 error from 1.234% to 0.978%.

Changes: Section 5.1 explains joint interface domains (p. 5, L249–259); Section 6.1 identifies teacher fitting/margins (p. 8, L387–388). Section 6.2/H.3 clarifies fixed-schedule controls and distinct attention/boundary workloads (p. 9, L462–485; pp. 42–44, L2256–2326; S2).

AuthorsOpenReview ↗

Response to Reviewer kK8x (Part 3/3, Q3–Q5)

Q3: How the system handles inputs that fall outside the registered numerical ranges?

Valid activations follow the compiled polynomial and Boot graph. A polynomial extrapolates beyond its registered interval, where its in-interval error bound no longer applies. Crossing a Boot domain can invalidate the refresh approximation even when execution finishes with finite outputs.

Denominator guards and invalid-slot safe substitutions handle their designated cases. Other valid activations continue through the same fixed polynomial and Boot circuits.

Changes: Appendix F.2 states the fixed-graph behavior outside registered intervals and the designated guard/substitution scope (p. 32, L1678–1682; S2).


Q4: How does the framework balance the trade-off between hiding request shapes and avoiding unnecessary computation?

The public profile fixes the workload, including padded tokens and inactive lanes. Batching amortizes that graph across active inputs; encrypted validity protects occupancy and individual lengths for supported requests with the same public contract and permitted leakage. The guarantee assumes fixed transcripts, fresh independent encryption randomness, and CPA-EK security for the complete public/evaluation-key bundle against an honest-but-curious CSP. Profile choice, container count, and submission timing remain visible.

Full-slot packing satisfies Bmax⁡Mmax⁡=N/2B_{\max}M_{\max}=N/2. Increasing public token capacity therefore reduces lane capacity at fixed NN. Both evaluated BERT profiles use Mmax⁡=128M_{\max}=128 and exclude longer examples without truncation; larger capacities require a compatible compiled profile. Across 11 compact client profiles, every request has 902 ciphertexts and 1.762 GiB raw RNS payload.

Table 16: metadata payloadCiphertextsRaw RNS payload (GiB)
Compact student, N=215N=2^{15}1340.262
Level-13 teacher, N=216N=2^{16}1341.243

Metadata is prepared once per teacher batch. In the light-probe run, metadata Encode alone takes 6.73 s (0.154% of 4,376.68 s), within 8.32 s of preparation including encryption; the online gauge takes 20.24 s.

Changes: Section 2 states the public contract/privacy conditions (p. 3, L118–130). Appendix E explains capacity and routing leakage (pp. 27–28, L1453–1467); G.7/Table 16 itemizes payload, preparation, Encode, and gauge scopes (p. 40, L2106–2126; S4).


Q5: Generally, I think the motivation and method is good and would raise my scores if the method can be extended to larger-scale models (e.g., 27B and beyond).

We appreciate your suggestion to test the method at this scale. MYGO completes Qwen3.6-27B's text prefill with original text weights at N=215N=2^{15} and post-Boot Level 4. One fixed public eight-token input occupies one lane of the 2,048-lane geometry. Its prior CPU trace fixes coefficients and representation factors before encrypted execution.

A common 5,120-column Level-3 ciphertext bank connects all 48 Gated DeltaNet and 16 Gated Attention layers, each with a full 17,408-wide SwiGLU FFN. Ciphertexts remain on one RTX PRO 6000 through all 64 layers, final RMSNorm, and full last-position vocabulary projection, without intermediate decryption or encryption.

Table 28: completed text prefillMeasurement
Decoder layers / vocabulary tiles64 / 1,940
Encrypted last-position logits248,320
Homomorphic evaluation145,173.10 s (40.33 h)
FHE program wall time145,292.72 s
Final RMSNorm and vocabulary head235.64 s
Sampled peak device memory77.33 GiB

Independent structural validation checks the output records, native states, and 53,324 joined timing scopes.

Changes: Section 6.3 and Appendix J document the fixed input and prior CPU trace, the complete prefill and vocabulary endpoint, and the measured resource use (p. 10, L517–523; pp. 47–48, L2530–2554). S5 summarizes the complete prefill measurements.

Thank you for the clarifications and additional results. I think this paper meets the thresold for poster and I have raised my score to positive.

Comments on the paper

2 comments
AuthorsOpenReview ↗

MYGO: response to the reviews (Part 2/2, points 3–5)

3. Resource effects and controlled comparison

The paired ring measurements fix QPQP, short-basis primes, post-Boot Level 4, and operator topology. Table 18A reports:

MetricN=215N=2^{15}N=216N=2^{16} controlCompact reduction
Evaluation-key payload8.597 GiB17.304 GiB50.3%
Post-Boot resident memory15.94 GiB24.56 GiB35.1%
Synchronized Boot19.278 ms25.018 ms22.9%

The common-backend encoder comparison in Table 3 uses one RTX PRO 6000, the same Cheddar build, N=216N=2^{16} Level-13 chain, 256 configured lanes, and a full-width 12-layer encoder endpoint. Prefix times including setup and light probes are 4,338.28 s for MYGO and 6,945.99 s for the reference port. MYGO uses its calibrated teacher on 256 inputs/6,508 valid tokens; the reference retains author weights and fixed shifts on one five-token example.

Changes: Table 3 states the common execution conditions, actual active cohorts, endpoints, and memory measure (p. 8, L412–426). Appendix G.4 documents the optimized configurations and their timing scopes (p. 38, L2023–2042). Section 6.2 and Table 18A connect ring dimension to the paired resource measurements (p. 9, L434–438; p. 42, L2222–2225).


4. Complete 27B encrypted prefill

With original Qwen3.6-27B text weights, a common 5,120-column Level-3 ciphertext handoff connects 48 Gated DeltaNet and 16 Gated Attention layers within the post-Boot Level-4 schedule. Every layer includes the full 17,408-wide SwiGLU FFN. At N=215N=2^{15}, one RTX PRO 6000 executes all 64 layers, final RMSNorm, and the full 248,320-logit last-position projection:

Complete-prefill measurement (Table 28)Result
Homomorphic evaluation40.33 h
Sampled peak device memory77.33 GiB
Intermediate decryption/re-encryptionNone

The run processes one fixed eight-token input. Its prior CPU trace fixes the numerical coefficients and representation factors.

The LLaMA-3-8B first-layer evaluation uses original weights and frozen settings on 2,553 test prefixes excluded from calibration.

Changes: Section 6.3 presents LLaMA first-layer fidelity and the complete Qwen prefill (p. 10, L488–523). Appendix I.1 specifies the LLaMA test protocol (p. 44, L2333–2355). Appendix J and Table 28 detail the Qwen execution and resource measurements (pp. 47–48, L2530–2554).


5. Key exposures and request-shape privacy

Under the stated MATZOV/GSA models, the compact main-domain estimate is 125.90 bits, below 128, and the deployed short-basis minimum evaluated sparse-attack cost is 147.5764 bits. Appendix D.2 evaluates a larger-ring alternative at fixed QPQP and Level 4; Table 18A gives its measured costs. The Level-13 teacher uses a separate chain.

Encrypted validity protects occupancy and individual lengths within a public capacity profile, under the stated CPA-EK assumptions and fixed transcript structure. Profile selection, container counts, and submission timing remain visible. Both evaluated BERT profiles accept up to 128 tokens including special tokens.

Changes: Section 3.1 and Appendix D.2 distinguish exposed key domains, attack estimates, and parameter alternatives (p. 3, L146–152; pp. 24–25, L1290–1329). Section 2 and Appendix E define public capacity and routing leakage (p. 3, L118–130; pp. 27–28, L1453–1467). Appendix G.7 itemizes metadata costs (p. 40, L2106–2126).

AuthorsOpenReview ↗

MYGO: response to the reviews (Part 1/2, points 1–2)

Dear Area Chair and Reviewers,

Thank you for the careful reading and specific suggestions. Substantive revisions are marked in red; paper locations follow each response.

MYGO coordinates representation, approximation domains, and refresh placement across a fixed packed graph. The compact FFN places refresh after FC2 contracts the width from 3,072 to 768, while affine-folded factors align adjacent modules without extra online conversions.

1. Continuous encrypted classification

The calibrated BERT teacher (N=216N=2^{16}, post-Boot Level 13) completes all 12 encoder layers, the pooler, and the classifier on a fixed cohort of 256 SST-2 validation inputs, achieving 93.36% label accuracy (239/256) and 256/256 prediction agreement with both exact and circuit-matched FP64 references. The light- and full-probe runs both compute the full 768-column model and produce identical final predictions.

Fitting uses 2,048 training examples; plaintext diagnostics on the fixed validation cohort set positivity margins before FHE. The light-probe run takes 4,376.68 s through the classifier on one RTX PRO 6000.

Table 13 reports the compact N=215N=2^{15}, Level-4 student's replicated, single-active, and mixed-input runs separately, using its own checkpoint and numerical configuration.

Changes: Section 6.1 distinguishes plaintext adaptation, independent-layer probes, and continuous classification, and specifies teacher fitting, references, and observation widths (pp. 7–8, L342–398). Appendices G.2–G.3 report the cohort-specific prediction counts and layer-end traces (pp. 37–38, L1982–2020).


2. What the numerical co-design contributes

At N=215N=2^{15} and Level 4, the Layer-0 controls hold parameters, packing, levels, ciphertext counts, and 41 Boot timing events fixed. On 100 fixed SST-2 inputs, removing the LN1–FFN–LN2 bridge factors changes domain coverage as follows (selected Table 19C entries):

RepresentationLN1-core violationsPre-LN2-residual violations
Matched BASE0 observed0 observed
Bridge factors set to one28.762%39.497%

Each arm uses three fresh FHE processes; entries are medians over valid coordinates. The calibrated path has zero observed violations at both interfaces under the same online graph and refresh schedule.

With the reciprocal circuit fixed, cancelling the exponential output scale at the next module lowers mean relative ℓ1\ell_1 error from 1.234% to 0.978%. It uses one five-token example replicated across 128 lanes, with three fresh processes per arm. The folded cancellation adds no online multiplication, Boot, or level.

Changes: Section 5.1 makes the producer–consumer representation interface explicit (p. 5, L249–259). Section 6.2 and Appendix H.3 explain which quantities the existing controls isolate, state the unchanged schedule, and distinguish the attention fixture from the 100-input boundary audit (p. 9, L462–485; pp. 42–44, L2256–2326).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.