- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the performance discrepancy between Teacher Forcing (TF) training and Autoregressive Generation (AG) in parameter-routing-based Multimodal Knowledge Editing. The authors identify "routing mismatch"—where optimal parameters selected during training are not reliably activated during generation—as a primary cause of generation failures. To mitigate this, they propose a "Generation-Aligned Optimization" framework featuring a Generation-Aligned Prefix (GAP) loss and Semantic-Anchor Gating. The GAP loss truncates gradient propagation after the first incorrectly generated token to prevent knowledge corruption, while Semantic-Anchor Gating uses frozen multimodal features to stabilize routing decisions. The method is evaluated on VLKEB and ComprehendEdit benchmarks using multiple MLLM backbones.
Critical strengths
-
The paper provides a solid empirical diagnosis of why dynamic inference methods (like ELDER or MELO) fail during autoregressive generation. Isolating the "routing mismatch" from general exposure bias offers a useful perspective on the fragility of dynamic parameter selection in editing.
-
The experiments are extensive, evaluating across multiple recent MLLM backbones (Qwen2.5-VL, LLaVA-OneVision) and comparing against a wide array of both static and dynamic inference editing methods.
Critical weaknesses
-
The paper heavily frames the Generation-Aligned Optimization and GAP loss as a "sequential decision-making process" inspired by Reinforcement Learning (RL) (Section 3.3). However, a closer inspection of Equation (4) reveals that the GAP loss is essentially a simple Truncated Cross-Entropy Loss—it merely masks out the loss computation after the first incorrectly predicted token. Dressing up a basic sequence-masking heuristic as a novel RL framework is a severe overstatement and detracts from the paper's academic rigor. Furthermore, training on model-generated prefixes is a highly standard technique (e.g., Scheduled Sampling) to mitigate exposure bias, limiting the theoretical novelty of this approach.
-
The proposed Semantic-Anchor Gating resolves the issue of hidden-state drift by using a frozen CLIP model to compute cosine similarities for activation decisions. This is a highly standard practice utilized by numerous prior memory-based or retrieval-based editing methods (e.g., SERAC). Presenting this as a core methodological contribution lacks conceptual innovation.
-
The design philosophy of the GAP loss—penalizing the model the moment it deviates from the exact literal string of the ground truth—raises significant concerns regarding exact-match overfitting. While this strategy artificially inflates exact-match metrics like Rel(G) and T-G(G), it forces the model into rote memorization of specific answer templates. The paper completely fails to investigate whether this strict token-matching enforcement degrades the model's open-ended generation capabilities, semantic flexibility, or general multimodal QA performance.
-
The authors attempt to strictly distinguish "routing mismatch" from "exposure bias." However, in a dynamic routing network where routing depends on the generated prefix, routing mismatch is fundamentally a direct symptom and downstream consequence of exposure bias. The conceptual decoupling feels somewhat forced and lacks theoretical depth.
Questions for authors
see weakness