- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents WaterMoE, a watermarking method for the Mixture-of-Experts (MoE) LLMs. It embeds watermark through the expert routing process. The experiments demonstrate that WaterMoE is efficient, can achieve high fidelity over datasets with different complexity levels.
Critical strengths
The proposed watermarking mechanism for MoE LLMs is novel and interesting.
Critical weaknesses
As noted by the authors, WaterMoE requires access to the model and its routing configuration, limiting its applicability to provider-side verification.
WaterMoE modifies the expert routing paths, causing deviations in the generated texts compared to those of the clean model. The proposed approach relies on the inherent redundancy of MoE models, which allows experts with similar routing scores can be dynamically selected without significantly affecting output quality. However, it remains unclear whether such rich redundancy is a general property of MoE models, with each expert having sufficiently many alternatives exhibiting similar routing scores to enable reliable selection. The evaluation is limited to only two MoE models, which appears to be insufficient to establish whether such redundancy is a general property or is specific to the models evaluated. A broader evaluation across diverse MoE architectures and model scales would help assess the generalizability of the proposed approach.
Questions for authors
How are the routing scores defined and computed? What is the rationale for selecting 0.2 as the threshold for the difference in routing scores between experts?