- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MoE-PACT, a system for efficient MoE inference under Confidential Computing. It combines optimized encrypted CPU–GPU transfers, searched base-delta bit partitioning, and fused GPU kernels for weight reconstruction. Experiments on three MoE models show reduced memory usage and improved decoding throughput with limited accuracy degradation.
Critical strengths
1.The paper addresses a practical bottleneck in confidential MoE inference. The measurements on actual Confidential Computing hardware make the results more convincing. 2.The system integrates several useful optimizations, with promising throughput improvements across different MoE models and batch sizes.
Critical weaknesses
1.The contribution of the bit-partition search is not clearly isolated. Section 4 shows that nearly all tensors select the same partition, which is then fixed in the implementation. A comparison with a fixed bit split or a simpler 6-bit quantization baseline would help establish whether the search and clustering are actually necessary. 2.The baseline comparisons need stronger support. EARTH and MoE-APEX are reimplemented, and EARTH's accelerator design is not included. Moreover, the batch-size-one CC measurements use single, short runs for Mixtral. This makes it difficult to assess the reliability of the reported speedups. 3.The confidentiality implications of routing-dependent transfers are not sufficiently evaluated. At batch size one, delta fetching exposes transfer sizes and timing to the untrusted host, as the authors acknowledge. The paper does not quantify this leakage or evaluate the performance cost of preventing it.
Questions for authors
1.How much of the performance gain remains compared with a fixed 6-bit quantization scheme using the same caching policy and GPU kernels? 2.Can the authors provide repeated CC throughput measurements and evaluate the security-performance trade-off of fixed-size delta fetching?
3 comments
Thank you for the response and the additional details. The quantization error and perplexity comparison against the conventional high/low split clearly demonstrates the value of the searched bit partitioning, and your clarifications regarding the CC measurement variance and paper scope make sense. I will maintain my positive rating.
Thank you for your assistance in improving our paper.
Thank you very much for reviewing our paper and for your comments on how to improve it. We have tried to address them as best we can.
EARTH and MoE-APEX are our reimplementations because their authors have not released their code.
The batch-size-one measurements in CC mode were run once because they take significantly more time. They also show less variability in decode latency, because H2D transfers are an even larger bottleneck under CC. We therefore limited them to a shorter decode length and a single run. In normal mode, where we ran three prompts per configuration, throughput varied by a median of about 3% on the Mixtral models.
The security implications will be studied in future work, as this paper focuses on the performance bottlenecks of NVIDIA CC mode and on designing a system that achieves higher performance under it.
Regarding the bit-partition search, every other partition has a larger quantization error, which is the objective the search minimizes. Even the second-best partition has a median of 15% (Mixtral-8×7B), 12% (Qwen3), and 6% (Mixtral-8×22B) higher error per tensor than the selected one. Larger error results in higher perplexity: on Mixtral-8×7B the conventional high/low split used by EARTH gives a perplexity of 4.2450, against 3.8530 for our searched split.
We have uploaded the updated PDF, with all changes shown in blue. If you have further comments, we would be happy to hear them and address them as best we can.