acceptodds
Under review as a conference paper at ICLR 2027

VeilMoE: Efficient Secure MoE Inference with Faithful Sparse Execution

Abstract

Secure mixture-of-experts (MoE) inference extends privacy-preserving Transformer inference beyond dense computation to sparse expert activation. However, existing secure MoE frameworks impose routing or expert-capacity restrictions that force single-expert selection, dropped assignments, or altered selection semantics that deviate from native plaintext MoE execution. They also overlook the differences in weight visibility across open-source and closed-source deployments, incurring unnecessary cryptographic overhead. In this work, we present VeilMoE, an efficient two-party privacy-preserving inference framework based on function secret sharing (FSS) that protects input-dependent routing while preserving exact, drop-free Top- semantics and native expert sparsity in both settings. In the closed-source setting, VeilMoE-C secretly permutes and masks the expert bank in the offline phase and reveals only selected anonymized slots, with the resulting residual linkability formally characterized and empirically quantified. In the open-source setting, VeilMoE-O employs a route-independent access pattern to conceal the selected experts while avoiding cryptographic evaluation of the full expert bank. Extensive experiments show that VeilMoE reduces end-to-end online communication and latency by up to and , respectively, compared with the state-of-the-art FSS-based dense secure-inference baseline.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
secure inference, mixture-of-experts, privacy-preserving machine learning, function secret sharing, secure multiparty computation, sparse expert activation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2dn9OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

VeilMoE is a two-party secure inference framework for mixture-of-experts (MoE) models, built on function secret sharing (FSS). It keeps the model’s native Top-k routing and drops no assignments, which existing systems give up: SecMoE forces a single expert and CryptoMoE caps expert capacity. The servers rank masked router scores once, then use that ranking in one of three variants. VeilMoE-C shuffles the private expert bank and reveals only anonymized expert slots. VeilMoE-O selects public experts with secret-shared one-hot selectors. VeilMoE-CG does the same oblivious selection over private, shared weights. In every variant, only k FFNs are evaluated cryptographically.

Critical strengths

VeilMoE keeps native Top-k selection and drops no assignments, instead of changing routing to make the cryptography easier. Capacity limits cost up to 6.75 points, and the paired bootstrap intervals exclude zero.

The framework ranks masked router scores once, then branches into three variants depending on whether weights are public or private and whether membership may be revealed. Online communication falls by up to 7.2×.

Critical weaknesses
  • Limited technical novelty. VeilMoE mostly combines existing primitives. it builds on CmpAgg ranking and FSS inference primitives and that its contribution is “their integration with protected expert access”

-lacks baselines. The only cryptographic baseline is Dense-SIGMA. it runs on the same stack and backend, evaluates all N experts, and then masks out the unselected ones. Several comparisons that would place the gains in context are missing, for example, SecMoE and CryptoMoE. None of the HE- or hybrid-HE/MPC systems the paper cites (e.g., Hao et al. 2022, Pang et al. 2024, Lu et al. 2025) are evaluated, even with their dense-MoE cost estimated.

  • lack of security proof.Theorem 1 rests on a single-paragraph argument, Proposition 1 in Appendix A.3. It says the simulator “composes the primitive simulators” in a hybrid model, but it never constructs a simulator or gives a hybrid sequence for any variant.

  • the end to end deployment is not evaluated. The abstract describes the gains as “end-to-end online communication and latency,” but none of the measurements cover a working deployment.

Questions for authors

Beyond integrating CmpAgg, threshold differencing, and Beaver-style gathers, which part of the protocol do you regard as a new technical contribution?

1 comment

We thank the reviewer for recognizing drop-free Top-k and our communication savings.

Q1 / Weakness 1. What is the technical contribution beyond existing primitives?

SecMoE selects one expert; CryptoMoE caps expert capacity. Replacing native Top-k with Top-1 or dropping selected assignments changes checkpoint computation; Our policy study (Table 2) shows losses up to 6.75 points at capacity c=1. We preserve drop-free Top-k while controlling access leakage and avoiding all-expert evaluation.

FSS material is generated before the route is known. Secure Top-k alone cannot align selected weights, gates and activation corrections. For v^W~−rvW~=vW~\hat v\widetilde W-r_v\widetilde W=v\widetilde W, both terms must use the same hidden expert.

Our protocol makes one protected ranking control these dependencies. C jointly permutes bank and routing correspondence; O/CG derive rank-wise one-hot selectors to gather parameters and matching corrections. O uses public-bank scans; CG selects private shares through offline correlations and a masked selector opening. This design evaluates only k FFNs cryptographically, without capacity drops, under public/private-weight access policies.

Weakness 2. Missing secure MoE baselines

We added encrypted CryptoMoE runs to compare with VeilMoE-CG (ours) on OLMoE and DeepSeekMoE, batch=1, 4/16 tokens. The table reports mean online backbone costs for our membership-hiding variant.

ModelTokensCG (ours) GBCrypto GBCG (ours) sCrypto sCrypto diff.†
OLMoE40.20015.363815.62629.91+2.36%
OLMoE160.804719.960869.001775.52-4.77%
DeepSeekMoE40.369610.641137.471461.06+8.14%
DeepSeekMoE161.486036.4456141.953940.94-7.40%

In these settings, CG uses 24.53-28.79x fewer online bytes. The two hosts each have dual AMD EPYC 7532 CPUs, 503 GB RAM and an A100 80 GB. CG uses GPU/FSS; CryptoMoE uses CPU HE/OT with 48 workers/party. Historical CG runs use 1-Gbps LAN; added runs use the same link without new shaping.

CG and CryptoMoE use the same configuration. Dense-SIGMA remains the same-backend control.

† Differences use Table 3 (t=2.0)'s 1.31/2.46 GB for OLMoE/DeepSeek, assuming per-token normalization and scaling by 4/16. They are within 8.14%. The original Xeon 8468/3-Gbps setup differs from ours; latency gaps alone do not imply implementation mismatch. Source

CryptoMoE's release covers balanced routing, not full encrypted inference. Our reproduction follows the paper and cited libraries and uses the authors' expert MatMul from SPU PR #1271. Code.

SecMoE's full public code was unavailable to us; Top-1 also changes native Top-8/Top-6.

Weakness 3. Explicit simulator and hybrid sequence

We have expanded Appendix A.3 with per-variant simulators and execution-order hybrids, specifying the functionality, corrupted view, replacements and retained correlations.

For example, the single-opening gather uses Δ=Q−A\Delta=Q-A, C=ABC=AB and Zb=ΔBb+CbZ_b=\Delta B_b+C_b. Given the corrupted party's shares and output, its simulator samples Δ,Ab\Delta,A_b uniformly, sets Cb=Zb−ΔBbC_b=Z_b-\Delta B_b, and determines the other opening share as Δ−(Qb−Ab)\Delta-(Q_b-A_b). Fresh masks and independently split correlations make this a joint-view simulation.

The complete argument preserves dependencies among masks, keys, selected weights and activation corrections. C's simulator receives the declared ordered membership leakage; O/CG use route-independent access schedules. The proof is stated under the paper's semi-honest, single-online-party corruption and trusted-preprocessing assumptions.

Weakness 4. Meaning of end-to-end

Our timing covers one connected secure Transformer-backbone execution: input hidden states pass through every attention, normalization, routing, expert and residual block to the final hidden states. It is neither an isolated MoE microbenchmark nor a sum of extrapolated layer timings. The boundary excludes embedding lookup, the vocabulary head and an autoregressive serving loop.

This is an end-to-end online computation measurement, not a claim of production deployment. SIGMA likewise uses end-to-end secure-inference terminology and separates offline and online costs. Zero-valued benchmark payloads do not remove the cryptographic operators or restrict the protocol to zero weights; nonzero secure correctness and checkpoint-based fidelity are checked separately.

Reviewer_BF5dOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces VeilMoE, a two-party privacy-preserving inference framework built on function secret sharing. It aims to protect dynamic expert routing while preserving exact Top-k execution without dropping tokens. VeilMoE avoids performing encrypted computations across the entire pool of experts, focusing instead only on the k selected experts. It offers three protocol variants, tailored to whether the model weights are public or private. Specifically, VeilMoE-C employs anonymous slot distribution and includes an analysis of residual linkability, VeilMoE-CG hides expert membership at the cost of higher overhead and VeilMoE-O utilizes oblivious expert gathering. Experiments conducted on OLMoE and DeepSeekMoE demonstrate that, compared to dense FSS baselines, VeilMoE reduces online communication volume by up to 7.2x and online latency by up to 2.9x.

Critical strengths

First, unlike prior works that either alter the routing mechanism, drop tokens, or cryptographically evaluate all experts, VeilMoE preserves exact Top-k semantics while computing only the k selected experts. Because it does not modify the underlying model architecture, it works directly with existing MoE checkpoints without sacrificing accuracy, while still delivering substantial communication and latency gains.

The second strength is the paper's transparent and practical treatment of security across different deployment models. Instead of relying on a single scheme, the authors distinguish between private and public weight settings and tailor distinct protocols for each. Especially, regarding the high efficiency variant VeilMoE-C, the paper avoids overpromising total confidentiality. It explicitly formalizes the disclosure of routing information and quantifies the residual linkability that arises from repeated queries, thereby providing a realistic assessment of the trade-off between efficiency and privacy.

Critical weaknesses

My primary concern is that the claimed efficiency gains largely disappear when strong privacy is required. The headline speedups come from VeilMoE-C, yet the paper's own measurements show that its anonymized slot access still leaks substantial document structure. Closing this leakage with VeilMoE-CG makes DeepSeekMoE LAN latency (37.47 s) no faster than the dense baseline (36.98 s), while requiring more preprocessing storage and exposing plaintext weights to the dealer. In addition, the empirical evaluation is limited. Timing experiments only test batches of 4 to 16 tokens using zero weights and a synthetic router, and compare solely against a dense baseline rather than prior sparse systems like CryptoMoE. Since the work primarily composes existing cryptographic building blocks, a more realistic evaluation is needed to establish its practical value.

Questions for authors

1.Regarding the privacy and latency tradeoff noted in the first weakness, in what concrete scenarios are VeilMoE-C and VeilMoE-CG practically preferable, and can VeilMoE-CG avoid exposing plaintext weights to the dealer?

2.Following the second weakness, could you provide a direct runtime and communication comparison against prior sparse secure MoE work such as CryptoMoE under a matched setup?

3.How do the VeilMoE variants perform on longer prefill sequences and autoregressive decoding with KV caching?

2 comments

We thank the reviewer for recognizing native Top-k execution and our explicit privacy analysis.

Q1. When are C and CG preferable, and must the dealer see private weights?

Concrete scenarios. Consider a vendor's proprietary technical-support assistant using two non-colluding cloud providers. Under its leakage contract, C protects queries, weights and intermediate values without exposing router scores, gate coefficients or the native expert-to-slot mapping.

When to choose CG. A legal-document assistant may also need to conceal associations between queries about the same confidential case. CG hides the membership/linkage channel. In our attacks, near-perfect domain classification requires labeled target-model traces; native-expert recovery with an unrelated public proxy remains near random. Unsupervised tests show anonymous structural linkage without recovering plaintext or native-expert relationship. CG addresses this linkage and stronger auxiliary information under the same online adversary model.

Communication benefit. For DeepSeekMoE/4, CG reduces online traffic from 2520.482 MiB to 352.467 MiB, although LAN latency is 37.47 s versus 36.98 s for Dense-SIGMA. Local bank scans limit LAN gains; fewer bytes reduce link occupancy and bandwidth-dependent transfer time on constrained WANs.

Dealer visibility. CG's online protocol does not require dealer access to weights. In alternative preprocessing, the dealer gives random R to the owner, who sends M=W+R only to the online parties. Random correlations let them derive [AW]=[A]M-[AR] and [rW]=[r]M-[rR]; selector-dependent corrections follow the same expansion. The owner only masks weights. Online messages and rounds are unchanged; generation/conversion are offline. The dealer must not receive M or collude with an online party.

Q2. Direct comparison with CryptoMoE

Membership-hiding VeilMoE-CG has lower measured communication and latency than CryptoMoE in all four settings. Values are means (batch=1, 4/16 tokens).

Each host has dual AMD EPYC 7532 CPUs, 503 GB RAM and an A100 80 GB. CG uses GPU/FSS; CryptoMoE uses CPU HE/OT with 48 workers/party. Historical CG runs use 1-Gbps LAN; added runs use the same link without new shaping.

ModelTokensCG comm. (GB)CryptoMoE comm. (GB)CG latency (s)CryptoMoE latency (s)Crypto comm. diff.†
OLMoE40.20015.363815.62629.91+2.36%
OLMoE160.804719.960869.001775.52-4.77%
DeepSeekMoE40.369610.641137.471461.06+8.14%
DeepSeekMoE161.486036.4456141.953940.94-7.40%

† Table 3 (t=2.0) reference: interpret OLMoE/DeepSeek's 1.31/2.46 GB per token and scale by 4/16. Communication agrees within 8.14%. Our hardware/network differ from its Xeon 8468/3-Gbps LAN, preventing direct latency replication; the time gap alone does not imply an implementation mismatch. Source

CG and CryptoMoE use the same configuration. Dense-SIGMA remains the same-backend control.

CryptoMoE releases balanced-routing code, not its full encrypted pipeline. Our reproduction follows the paper and cited libraries, using the authors' expert MatMul from SPU PR #1271. Code.

We prioritize multi-expert CryptoMoE: no complete public SecMoE implementation was available to us, and its Top-1 restriction changes these models' native Top-8/Top-6 routing.

Q3. Longer prefill and KV-cached decoding

The 4/16-token settings do not cap the protocol. Increasing tokens fourfold raises CG communication by 4.02x for both models; latency rises 4.42x for OLMoE and 3.79x for DeepSeekMoE. Near-linear bytes are consistent with repeating fixed-k expert work per token; latency also includes attention and local execution.

SecMoE benchmarks eight tokens in one batch. CryptoMoE's 0.5K-4K study varies batch size (8-64), rather than timing one equally long encrypted input. Long single-sequence prefill and KV-cached decoding are directions for future secure-MoE serving research, building on our expert-selection protocol.

Additional concern: zero-weight timing

Zero payloads simplify benchmarking; the protocol equations do not require zero weights. The timed graph executes the cryptographic operators rather than skipping them because their inputs are zero. We separate architecture-sized timing, nonzero secure correctness tests and checkpoint-based routing/numerical checks. SIGMA's public benchmark likewise zero-initializes inputs and weights before secure execution. This methodology measures the secure computation path; our nonzero tests establish functionality separately.

Thank you for the responses. I prefer to keep my original rating.

Reviewer_CJ1ROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Prior secure mixture-of-experts (MoE) inference designs can use only single-expert selection, dropped assignments, and/or altered selection semantics. They overlook the weight visibility differences across open-source and closed-source cases, introducing significant cryptographic overhead. This paper proposes VeilMoE, a two-party privacy-preserving inference framework based on function secret sharing (FSS) that protects input-dependent routing while preserving exact, drop-free Top-k semantics and native expert sparsity in both settings. In the closed-source case, VeilMoE-C permutes and masks the expert bank in the offline phase and shows only selected anonymized slots. On the contrary, in the open-source case, VeilMoE-O uses a route-independent access pattern to conceal the selected experts without cryptographic evaluation of the full expert bank.

Critical strengths
  1. The secure MoE inference problem is relevant. VeilMoE tries to preserve top-kk routing and evaluate every selected expert without capacity-based drops. Different schemes for for private and public weights make the tradeoff between weight visibility and routing leakage explicit.

  2. Reusing protected ranks and sparse execution are implemented in a coherent way. VeilMoE turns shared ranks into shuffled expert membership or secret gather selectors.

  3. The leakage and measurement scope are handled well. VeilMoE quantifies what the membership stream can leak, presents the protocol’s trust assumptions, and distinguishes synthetic timing results from task-accuracy and numerical-fidelity checks.

Critical weaknesses
  1. I did see the advantage of VeilMoE over all prior secure MoE system. In table 3, the authors compared the runtime of VeilMoE with that of Dense-SIGMA, its dense FSS baseline. The SecMoE and CryptoMoE comparisons instead emulate their routing restrictions in plaintext to measure accuracy effects in Table 2. However, they do not run those systems’ cryptographic protocols. The authors explicitly admit that they did not make a matched runtime comparison with SecMoE or CryptoMoE, because the systems differ in protocol, client participation, preprocessing, and routing. So the paper supports a speedup over Dense-SIGMA, but not a runtime advantage over those other secure MoE systems. Those emulations help show the cost of altered routing, but they do not show how VeilMoE compares with those systems’ cryptographic runtimes.

  2. Is a secure MoE inference practical? The major results use architecture-sized stacks with zero attention and expert payloads, a synthetic router, and a four-token batch. However, correctness checks use separate, much smaller settings. The paper does NOT establish performance on full, nonzero-weight inference, long sequences, autoregressive decoding, or multi-user workloads at all.

  3. VeilMoE is just conditionally secure on different threat models. For open-source and close-source cases, the threat models are different. VeilMoE-C intentionally shows which anonymized expert slots were selected for each token and layer. The paper’s attacks show those patterns can reveal useful information, such as document similarity or domain. That is a privacy leak, but the threat model permits. VeilMoE-O and CG hide that selection pattern from the online servers under the modeled protocol. CG’s preprocessing dealer, however, sees the private weights, so the model is not hidden from that dealer.

Questions for authors

Please address the first and second points I raised in the weakness section.

  1. What is the performance comparison between VeilMoE, SecMoE or CryptoMoE?

  2. Is the VeilMoE inference practical for full, nonzero-weight inference, long sequences, autoregressive decoding, or multi-user workloads?

2 comments

We thank the reviewer for recognizing drop-free Top-k execution and our explicit leakage and measurement scope.

Q1. Performance compared with SecMoE and CryptoMoE

We now complement Table 2's routing-policy study with actual encrypted CryptoMoE runs, compared with VeilMoE-CG (ours) on OLMoE and DeepSeekMoE, batch=1, 4/16 tokens. The table gives mean online backbone costs. CG hides membership and uses 24.53-28.79x fewer bytes in these measurements.

ModelTokensCG (ours) GBCrypto GBCG (ours) sCrypto sCrypto diff.
OLMoE40.20015.363815.62629.91+2.36%
OLMoE160.804719.960869.001775.52-4.77%
DeepSeekMoE40.369610.641137.471461.06+8.14%
DeepSeekMoE161.486036.4456141.953940.94-7.40%

The two hosts each have dual AMD EPYC 7532 CPUs, 503 GB RAM and an A100 80 GB. CG uses GPU/FSS; CryptoMoE uses CPU HE/OT (48 workers/party). Historical CG runs use 1-Gbps LAN; added runs use the same link without new shaping. CG and CryptoMoE use the same configuration.

The last column assumes Table 3 (t=2.0)'s 1.31/2.46 GB are per-token OLMoE/DeepSeek costs, scaled by 4/16: agreement within 8.14%, not matched measurements. Hardware and the original 3-Gbps LAN differ from ours. Source

CryptoMoE's release covers balanced routing, not full encrypted inference. We follow its paper and cited libraries, including the authors' expert MatMul from SPU PR #1271. Code.

We prioritize CryptoMoE: SecMoE's full public code was unavailable to us and Top-1 changes native Top-8/Top-6.

Q2. Nonzero inference, long sequences, decoding and multi-user workloads

Nonzero weights. Nonzero weights are supported and have been tested in two-party execution. Section 5's two-party CG test covers all 32 outputs, with maximum absolute error 6.47×10−46.47\times10^{-4}. Appendix B, Table 6 reports a real-weight, 16-token OLMoE check: 100% selected-expert agreement and mean/max MoE-block errors of 0.0007485/0.84625. Blocks receive captured inputs and feed a plaintext-backbone splice, rather than accumulated encrypted states.

Full-stack timing measures cryptographic cost; zero payloads do not remove attention, routing or FFN operators. SIGMA's benchmark also uses zero initialization.

Scaling and serving. The measured token counts are workload choices, not protocol caps. From 4 to 16 tokens, CG communication increases approximately 4x on both models, consistent with repeating fixed-k expert work per token. SecMoE benchmarks eight tokens in one batch. CryptoMoE's 0.5K-4K study varies batch size (8-64), rather than timing one equally long encrypted input. Long single-sequence prefill, KV-cached decoding and multi-user serving are directions for future secure-MoE serving research, building on our expert-selection protocol.

Weakness 3. What C protects, when CG is needed, and dealer visibility

Protection. VeilMoE-C, VeilMoE-O and VeilMoE-CG share the semi-honest, non-colluding online-server setting; they differ in weight visibility and permitted access leakage. C protects inputs, private weights and intermediate values under its stated leakage contract. It exposes neither plaintext routing scores and gate coefficients nor the native expert-to-slot map.

Attacks and CG. The attacks depend on auxiliary information. Near-perfect domain classification uses labeled traces from the target model. Native-expert recovery with the tested unrelated public proxy remains near random guessing. Unsupervised tests reveal anonymous structural linkage, but can not recover plaintext or native expert identities. CG is appropriate when membership/linkage itself must be hidden, or stronger auxiliary information is in scope.

WAN benefit. CG's communication advantage is particularly relevant to bandwidth-constrained WANs: fewer bytes reduce link occupancy and bandwidth-dependent transfer time even when local scans limit LAN speedups.

Dealer visibility. Dealer access to plaintext weights is also avoidable through alternative preprocessing. The dealer gives a random R to the owner, who sends M=W+R only to the online parties. The dealer supplies random correlations; the parties derive [AW]=[A]M-[AR] and [rW]=[r]M-[rR], with selector-dependent corrections obtained analogously. The owner only masks weights; the dealer generates random material without receiving M. The online protocol, messages and rounds are unchanged; the extra generation/conversion occurs offline. The dealer remains non-colluding and trusted for correct preprocessing.

VeilMoE-C provides a relaxed routing-privacy guarantee relative to CryptoMoE and SecMoE. It intentionally reveals which anonymized expert slots are selected for each token and layer. Although the native expert identities remain hidden, the disclosed trace preserves recurrence and co-activation structure within a permutation epoch, enabling document linkage or clustering and, with suitable auxiliary information, domain inference. Its efficiency should therefore be assessed in light of this additional permitted leakage.

VeilMoE-CG removes VeilMoE-C’s membership leakage and provides a comparable routing-confidentiality objective to CryptoMoE and SecMoE. However, it achieves this under different trust assumptions: two non-colluding outsourced servers and a trusted preprocessing dealer, which accesses plaintext private weights in the current implementation. Consequently, its security guarantee is not established under the same threat model as these prior works.

Yes, this work shows a better performance than SecMoE and CryptoMoE. But I do not know how much benefit is from the modified threat model.

Yes, dealer access to plaintext weights is also avoidable through alternative preprocessing. However, the authors should count this latency into the performance comparison.

I feel like the papers sacrifices security and improves performance, so I maintain the same score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.