acceptodds
Withdrawn from ICLR 2027

Selective Homomorphic Inference via Encryption Boundary Selection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Homomorphic encryption (HE) can protect data during inference, but encrypted computation remains costly for transformer models. We study how the choice of encryption boundary affects representation exposure, task performance, and latency. We compare three strategies across DistilBERT, Mistral-7B, and Qwen-2.5-3B on multiple NLP tasks: encrypting only the classifier head, encrypting a representation for transfer to an assumed trusted component, and performing projection and prediction over ciphertext. Head-only encryption leaves informative plaintext representations exposed, while deeper encrypted computation incurs substantial latency and, in our experiments, often reduces accuracy. Encrypting a pooled representation for transfer, then decrypting it for downstream plaintext computation inside the trusted component, offers a favorable tradeoff among the evaluated strategies. On Mistral-7B with GoEmotions, this approach achieves 55.81% Micro-F1 versus 58.30% for plaintext inference, with 26.60 ms measured latency. These results suggest that pooled and moderately projected representations are promising boundary choices under the stated trust model.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Homomorphic Encryption, Privacy-Preserving Inference, Encryption Boundary Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Y7PPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates encryption boundary selection for privacy-preserving transformer inference. It compares head-only homomorphic computation, encrypted representation transfer with trusted downstream execution, and deeper encrypted projection and prediction. Experiments on DistilBERT, Mistral-7B, and Qwen2.5-3B examine task performance, representation leakage, and computational latency. The authors identify representation-level encryption as a favorable operating point under their hybrid trust model.

Critical strengths

None.

Critical weaknesses
  • The central methodological contribution is insufficient. The proposed Repr-HE configuration performs no homomorphic computation and relies on trusted plaintext execution. The authors' own AES-GCM baseline achieves comparable utility with lower latency. Consequently, the primary result provides limited justification for the proposed HE-based framework. Moreover, the trusted component is only simulated, leaving the practical deployment assumptions unvalidated.
  • The experimental comparisons do not establish a meaningful privacy–utility–efficiency advantage. The evaluated strategies rely on different trust assumptions and protect different portions of the inference pipeline. Furthermore, the extremely poor performance of Proj-Poly-HE appears to arise from unresolved encrypted-execution or implementation issues. These results provide insufficient evidence for the proposed boundary-selection methodology.
Questions for authors
  • Given that AES-GCM provides equivalent protection with better efficiency under the Repr-HE trust model, what technical advantage does the proposed framework offer over conventional encrypted representation transfer?
  • Can the authors demonstrate the proposed boundary-selection methodology in a realistic deployment with explicitly defined trusted and untrusted components and comparable security guarantees?
Reviewer_5AEpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Similar to some prior works in this space, instead of computing the entire inference under homomorphic encryption (HE), the authors suggest to use HE selectively, i.e., use HE for only specific stages of inference evaluation. However, existing works do not analyze how different encryption boundaries affect privacy, utility and cost of the inference. In this paper the authors study how the choice of encryption boundary impacts privacy, utility and latency incurred in computation of transformer inference using HE. In particular, the paper models boundary selection as an optimization problem balancing leakage, task performance, and cost of homomorphic evaluation. The authors then identify and evaluate three main configurations: 1. HE-Linear, in which the backbone and projection are evaluated in plaintext, after which the projected representation is encrypted and the classifier is computed under HE. 2. Repr-HE, in which the backbone is evaluated in a trusted environment, the output is encrypted, and the ciphertext is then released to an untrusted environment. It is subsequently passed to another trusted environment, which decrypts the ciphertext and performs the projection and classifier computation in plaintext. 3. Proj-Poly-HE, which is essentially the same as Repr-HE, except that the projection, polynomial activation, and classification are performed using HE. The authors than empirically evaluate the three configurations on DistilBERT, Mistral-7B, and Qwen-2.5-3B across SST-2, IMDB, MNLI, and GoEmotions.

Critical strengths

None

Critical weaknesses
  1. The privacy component (based on ROC-AUC) in the objective function is attack specific of a particular probing classifier g. However, an adversary has full-knowledge of the model weights and can follow any arbitrary attack strategy. In fact such general adversary is considered in their threat model. Thus, the privacy term L_priv is chosen attack specific and is not a well-formalized privacy notion, making the optimization objective under-specified.
  2. The leakage metric does not seem to capture input privacy. ROC-AUC metric captures how well task labels can be inferred from intermediate representations. However, the motivating concern is protection of sensitive inputs. A representation may reveal substantial information about the input even if label inference is difficult. The paper should justify why this metric is appropriate for its privacy objective.
  3. HE-Linear configuration lacks motivation. In HE-Linear, the backbone and projection are evaluated in plaintext, so the server already observes an input-derived representation before encryption. If the server also has the classifier parameters, it can simply compute the classifier output directly. It is unclear what privacy guarantee is provided by encrypting only the classifier computation.
  4. Repr-HE is basically trusted plaintext execution with communication of a single encrypted intermediate representation. The representation is encrypted only while crossing an untrusted boundary, then decrypted and processed in plaintext inside a trusted component. No computation is performed on ciphertext, and the paper itself notes that standard authenticated encryption would suffice. Are the authors essentially proposing that the entire inference be executed within a trusted environment? I am struggling to identify the novelty of this approach.
  5. Repr-HE configuration is not well-motivated. It is unclear why execution must leave a trusted environment after computing the representation and then enter another trusted environment for the remaining computation. Since the untrusted component appears to only relay ciphertext, the paper should provide a scenario that justifies this boundary.
  6. In main body, the paper states that Proj-Poly-HE computes the projection, polynomial activation and classifier in HE. However, the appendix states that trusted environment is used for intermediate ciphertexts decryption and re-encryption. This additional trusted decryption weaken both the security interpretation and the comparison with genuine HE inference.
  7. The latency evaluation omits relevant costs especially for Repr-HE configuration, as the trusted boundary is only simulated as a single process, and the reported numbers exclude network communication, TEE transitions, etc. This is problematic because the entire motivation of Repr-HE involves moving an encrypted representation across trust boundaries.
Questions for authors

See weaknesses

Reviewer_QMEdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies where to place an encryption boundary in transformer-based classification pipelines. It formulates boundary selection as a weighted combination of representation leakage, task-performance degradation, and cryptographic cost. It evaluates three principal configurations: HE-Linear, which encrypts classifier computation after plaintext representation extraction and projection; Repr-HE, which encrypts representations for transfer and decrypts them for trusted plaintext computation; and Proj-Poly-HE, which performs projection, a quadratic activation, and classification through staged encrypted computation. Experiments cover DistilBERT, Mistral-7B, and Qwen2.5-3B across sentiment, emotion, and natural-language-inference tasks. Additional experiments examine representation boundaries, sensitive-attribute inference, input reconstruction, matched plaintext execution, and authenticated encryption. The main empirical conclusion is that pooled or moderately projected representations offer useful operating points within the evaluated prototype. For example, Repr-HE achieves 55.81% Micro-F1 on Mistral-7B/GoEmotions, compared with 58.30% for plaintext inference, at a reported latency of 26.60 ms.

Critical strengths
  1. The paper addresses a practical question: where should encryption begin? The paper examines which parts of a model’s processing pipeline should be protected to balance privacy, prediction quality, and computation time. It tests several models and tasks and reports results across three runs, providing useful evidence about different design choices For example, Table 5 shows that summarizing the full token representations into one smaller vector reduces reported latency from 744.80 ms to 27.10 ms, while Micro-F1 decreases only from 57.64% to 56.42%. This suggests that choosing a compact representation can substantially reduce encryption-related costs while preserving most prediction quality. Table 9 also examines how much of the original text can be reconstructed from different representations, adding useful information beyond classification performance.

  2. The paper highlights a practical privacy pitfall: encryption cannot protect information that has already been exposed. In the evaluated HE-Linear configuration, the classifier operates on encrypted data, but the server can still access informative representations produced earlier. Table 3 shows that these exposed representations allow strong task-label inference. This provides a useful design lesson: adding encrypted computation is insufficient unless sensitive information is protected before it reaches an untrusted component

Critical weaknesses
  1. The evidence does not yet convincingly establish which encryption boundary should be preferred. The paper’s central contribution is a procedure for choosing a boundary by balancing privacy, accuracy, and cost. However, the choices reported in Table 7 do not follow from Table 5 under the standard min–max interpretation of the stated normalization procedure. For example, with weights of 0.60 for privacy, 0.20 for accuracy, and 0.20 for cost, recalculating the objective across the nine representation-boundary candidates gives approximately 0.3994 for Layer 8 and 0.5372 for Layer 16, where lower is better. Table 7 nevertheless selects Layer 16. The paper does not fully specify the normalization procedure or candidate set, so this discrepancy requires clarification rather than an assumption that the results are incorrect. It directly affects confidence in the claimed preferred boundaries. The main comparisons also change who is trusted to see plaintext. HE-Linear exposes an earlier representation to the server, while Repr-HE assumes trusted handling of that representation and the later calculations. Therefore, the observed privacy differences cannot be attributed to boundary placement alone. A comparison using the same trust assumptions would better isolate the benefit of each boundary. Finally, the paper’s own AES-GCM baseline achieves almost identical prediction quality to Repr-HE with lower latency: 22.10 ms versus 26.60 ms. Since Repr-HE performs no calculations on encrypted data, these results support efficient encrypted transfer between trusted components, but do not demonstrate a practical advantage from homomorphic encryption itself. The paper therefore needs stronger evidence for the additional value of its proposed selection framework.

  2. The privacy experiments lack enough detail to judge the strength of the attacks. The paper tests whether an attacker can predict sensitive information or reconstruct the original text from intermediate representations. However, it provides limited information about how these attackers are trained and evaluated, including the separation of training and test data, the reconstruction model, and how prediction scores are combined across labels. These details matter because a low attack success rate could reflect either less information in the representation or an attacker that fails to recover the available information. Clearer experimental descriptions and checks of attack effectiveness are needed to support the comparisons between representation boundaries.

Questions for authors
  1. Please report the packing layout, number of ciphertexts and encryptions, rotations, matrix-evaluation method, and inclusion or exclusion of key/context initialization. How do these explain approximately 8,000 ms versus 11.25 ms for the two Mistral encryption measurements, and the roughly elevenfold Mistral–Qwen HE-Linear latency difference?

  2. How does a configuration with trusted backbone and projection, followed by encryption of (z) and an untrusted HE classifier, compare with Repr-HE and AES-GCM transfer? For the intended deployment, what additional protection or functionality justifies the staged encrypted computation, given its intermediate trusted decrypt–re-encrypt operations?

  3. For HE-Linear and Repr-HE, do the plaintext and encrypted evaluations use the same model weights, preprocessing, and prediction thresholds? Please report a comparison that changes only the encryption procedure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.