acceptodds
Under review as a conference paper at ICLR 2027

VeilMoE: Hierarchical Routing Envelopes for Privacy-Aware Mixture-of-Experts Inference

Abstract

Sparse Mixture-of-Experts (MoE) models activate only a few experts per token, but input-dependent routing exposes expert activity that can reveal information about the input. We present \method, which keeps the route used for computation inside a trusted controller while exposing a coarser execution structure to the accelerator. The controller maps the native Top- route to a hierarchical envelope; experts in each touched group receive the same quantized capacity and run through transformed SwiGLU kernels. Under the stated execution assumptions, we prove exact request-local semantics in real arithmetic, characterize the exposed routing structure, and bound slot expansion. On OLMoE-1B-7B, reduces routing-only Top-1 token reconstruction from 37.29% to 4.34% and Banking77 intent inference from 84.38% to 27.47%, with a utility change of percentage points. A trainable attacker using raw rows and the envelope has low accuracy without target-epoch training examples, although it still uses 256 target-epoch requests for checkpoint selection; target-epoch training substantially increases accuracy. Full-decoder Qwen utility changes by to percentage points at the tested settings. With matched grouped-expert backends, OLMoE serving costs 7.10 Plain latency in prefill and 1.83 at 64-row cached decode. The privacy guarantees cover routing structure; transformed values and physical timing remain separate sources of leakage.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Privacy, MOE

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_n448OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the issue of MoE expert routing leaking semantic information and proposes hiding the expert trace through expert grouping, capacity quantization, and uniform padding to improve security. The authors systematically design expert mapping (encoding and decoding) methods based on the invertible transformation properties of SwiGLU and provide theoretical proofs. The paper provides extensive experiments and analysis.

Critical strengths
  1. This paper studies a very important and necessary problem.
  2. The authors provide extensive experiments and theoretical analysis to demonstrate the effectiveness of VeilMoE.
Critical weaknesses
  1. Expert routing selection does indeed involve information leakage, but it is not necessarily semantic information. At least when I used the MMLU dataset for classification with Mixtral 8x7B (I did related experiments before), I did not observe a similar phenomenon. Therefore, although the paper provides extensive experiments and theoretical analysis, merely presenting attack effects before and after defense on some datasets (OpenWebText, AG News, etc.) is not convincing. It is impossible to determine whether this reflects the threat model and the actual defense effect in real-world scenarios. It is suggested to directly use existing attack methods to evaluate the defense effect.

  2. The threat model is not clear. Hiding the invocation relationships of experts does not directly hide the objects actually computed by the GPU. This defense method seems to lose effectiveness under side-channel attacks.

Questions for authors

See Weakness

1 comment

As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.

Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.

Thank you all for your time and constructive engagement

Reviewer_LeMDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VeilMoE reduces routing disclosure during sparse MoE inference by replacing exact expert activity with hierarchical group envelopes. A trusted controller preserves the original Top-k assignments, pads every expert in a touched group to a shared quantized capacity, and applies covariant SwiGLU transformations. The paper proves real-arithmetic correctness, characterizes structural leakage, and bounds padding overhead. On OLMoE, routing-only token recovery falls from 37.29% to 4.34% at a=16a=16, while downstream utility changes by -0.22 percentage points. Qwen experiments provide additional utility and routing evidence. Matched-backend latency overhead is 7.10x for prefill and 1.83x for 64-row decode. Values and physical timing are explicitly outside the structural guarantee.

Critical strengths
  1. Sections 4.1-4.3 clearly connect observable routing structure, semantic correctness, and padding cost, providing a useful analytical framework.

  2. Evaluation spans two backbones, utility, fidelity, attacks, and matched serving backends, with candid reporting of residual leakage.

Critical weaknesses
  1. [W1] Zero covers undermine practical load hiding. Section 5.5 uses zero covers, which remain zero under Eq. (13)'s linear transforms. The GPU can therefore identify covers and recover expert loads; Table 15 reports approximately 0.998 classification AUROC and 0.999 load-recovery R2R^2. COV-X is tested separately offline and remains distinguishable. The restricted structural theorem remains valid, but practical protection and runtime are not demonstrated together under the same configuration.

  2. [W2] Trusted-controller placement and cost are unclear. Routing, packing, transforms, cover bookkeeping, and merging require a trusted controller. Table 4 uses one GPU, while Appendix A.4 attributes controller-related operations to CUDA without specifying their trust boundary. Veil-O has only an abstract logical-call prototype (Table 19), leaving the implementation and cost of its whole-tick access contract unverified.

Questions for authors
  1. W1: Can the GPU count nonzero rows in the canonical zero-cover configuration? If prevented, what mechanism does this and where is its cost included?
  2. W1: Do any existing leakage and latency results share the same nonzero-cover configuration, layer set, and key-refresh schedule? Please identify the configuration and results.
  3. W2: Which controller operations run on trusted hardware versus the untrusted GPU? Does Table 4 measure Veil-S, Veil-O, or an emulation? Are isolation, transfer, and oblivious-access costs included?
1 comment

As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.

Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.

Thank you all for your time and constructive engagement

Reviewer_8LCnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In MoE inference, which experts a token is routed to can leak information about the input. VeilMoE keeps the exact routing inside a trusted controller and exposes only coarse information to the GPU: which expert groups are used, and one padded capacity per group. Expert weights are hidden with invertible linear transforms, so standard SwiGLU kernels can still run. The paper proves exactness and characterizes what the GPU observes. On OLMoE and Qwen-MoE, token-reconstruction and intent-inference attacks drop substantially with small utility loss, at a cost of roughly 1.8x to 7x latency.

Critical strengths
  1. Routing leakage in MoE is a real and timely problem, and the paper separates the leakage channels (structure, trusted calls, values, timing) clearly. This makes it easy to see what is and is not protected.

  2. The paper is honest about its limitations. It openly reports that value attacks recover a lot of signal, which I appreciate.

Critical weaknesses
  1. The core idea is not new. Routing leakage is a form of access-pattern or volume leakage, and the defense (group the accesses, pad to quantized sizes) is the standard padding idea from ORAM and volume-hiding encrypted search. The MoE-specific part is the grouping hierarchy, which reads more as engineering than as a conceptual contribution. The theorems are largely true by construction: the GPU sees the envelope because the system is designed to show it the envelope. What the envelope itself leaks is only measured empirically.

  2. The overall protection is weak. With target-epoch training data, value attacks reach 55% on Banking77 and 86% on AG News, so the transformed values leak more than the routing ever did. Structural protection on AG News is also small (88% to 84%). On cost, at coarse settings the slot expansion (about 7x) approaches simply running all experts (8x for OLMoE), which hides the route completely; that baseline is missing. Finally, the trusted controller is assumed but not implemented in a real TEE.

Questions for authors
  1. How does VeilMoE compare, in both cost and leakage, to the trivial baseline of running all experts for every token?

  2. Given the value-channel results, what deployment would actually be protected end to end? Is frequent rekeying assumed, and what does it cost in practice?

  3. What is the overhead once the controller actually runs inside a TEE?

1 comment

As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.

Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.

Thank you all for your time and constructive engagement

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.