- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses the issue of MoE expert routing leaking semantic information and proposes hiding the expert trace through expert grouping, capacity quantization, and uniform padding to improve security. The authors systematically design expert mapping (encoding and decoding) methods based on the invertible transformation properties of SwiGLU and provide theoretical proofs. The paper provides extensive experiments and analysis.
Critical strengths
- This paper studies a very important and necessary problem.
- The authors provide extensive experiments and theoretical analysis to demonstrate the effectiveness of VeilMoE.
Critical weaknesses
-
Expert routing selection does indeed involve information leakage, but it is not necessarily semantic information. At least when I used the MMLU dataset for classification with Mixtral 8x7B (I did related experiments before), I did not observe a similar phenomenon. Therefore, although the paper provides extensive experiments and theoretical analysis, merely presenting attack effects before and after defense on some datasets (OpenWebText, AG News, etc.) is not convincing. It is impossible to determine whether this reflects the threat model and the actual defense effect in real-world scenarios. It is suggested to directly use existing attack methods to evaluate the defense effect.
-
The threat model is not clear. Hiding the invocation relationships of experts does not directly hide the objects actually computed by the GPU. This defense method seems to lose effectiveness under side-channel attacks.
Questions for authors
See Weakness
As the rebuttal period draws to a close, I encourage reviewers to engage with the authors’ responses and revised manuscript, clearly identifying which concerns have been addressed and which remain unresolved.
Given the limited time remaining, please prioritize evaluating the evidence and clarifications already available in the current revision. If you nevertheless consider additional experiments essential to support the paper’s claims, please specify what is needed and why. Even if the authors cannot complete these experiments within the remaining time, a discussion of the underlying concerns and the available evidence would help inform our final assessment.
Thank you all for your time and constructive engagement