- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
MoE-GRPO extends LiveEdit with reinforcement learning to improve multi-hop reasoning after lifelong multimodal knowledge editing. It updates the expert generator and router while freezing the backbone, using singular-value perturbation, a max-based objective, asymmetric gradient clipping, and a composite reward.
Critical strengths
MoE-GRPO targets the gap between retaining edited facts and using them in multi-hop reasoning, with improvements on BLIP2 and LLaVA. The study evaluates multiple backbones and ablates several rewards, clipping, gradient, and perturbation choices. Applying the method to LEMoE provides limited evidence that it can extend beyond LiveEdit.
Critical weaknesses
-
Eq. (8) requires a likelihood for the joint action of expert matrices and router outputs, but the paper specifies stochasticity only over expert singular values. It defines neither the induced action density nor the resulting importance ratio and reference KL. The main text decomposes the current matrices, whereas Appendix K fixes the singular vectors to the pretrained basis. If old and new policies use different bases, their distributions may have different support, leaving the objective undefined.
-
In Eq. (15), the negative sign means that maximizing the reward discourages positive residual similarity and can favor concentrating capacity in fewer experts, contrary to the claimed compatibility and co-activation. Eq. (16) compares vectors whose lengths grow with the expert pool without defining alignment or padding. Both rewards use parameter norms rather than routing probabilities, so they do not directly measure routing consistency. The main objective also contains no multi-hop reward, making the claim of direct multi-hop supervision inaccurate.
-
The evaluation definitions are inconsistent. Eq. (31) compares a rephrased input with the unedited model's original output, which conflicts with generalization to the new fact. Eq. (32) defines an indicator metric, while Figures 4 and 7 use unexplained partial credit. The metric and decoding procedure are therefore unclear.
-
The claimed small trade-off is not supported at 1,000 BLIP2 edits, where reliability and text generalization each drop by about 20 points relative to LiveEdit. MiniGPT4 gains are small, and the 100-edit LEMoE result is nearly unchanged. Some reported gains also use different baselines or edit counts from those stated: 30.48% is relative to LEMoE rather than LiveEdit, while 2.17% comes from another edit-count comparison.
-
The max/min objectives are not tuned separately, and the paper does not report matching the Gaussian perturbation energy. Freezing the backbone does not establish gradient underestimation, and Eq. (11) appears to apply the noise scale twice in the later stage.
-
The AI-use statement limits generative AI to language editing, whereas Appendix D.1 uses GLM-5.1 to synthesize auxiliary QA data. This disclosure should be corrected.
Questions for authors
-
How is the joint action density defined, and how are the importance ratio and reference KL computed? Do old and new policies use the same singular basis and support?
-
Why does the negative similarity in Eq. (15) encourage compatible residuals? How are the different-length vectors in Eq. (16) aligned, and why do parameter norms measure routing consistency?
-
Is Eq. (31) incorrect? What metric and decoding protocol produce the tables and the partial scores in Figures 4 and 7? Which baselines and edit counts define the reported gains?
-
After tuning learning rate and KL separately, does max still outperform min? Does SVD perturbation remain better when Gaussian perturbation energy is matched? At comparable editing quality and cost, does MoE-GRPO outperform further training of LiveEdit?