- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines cross-capability interference between understanding and generation in unified multimodal models, focusing on the limitations of passive gradient statistics in predicting update outcomes. The proposed Active Mechanism-Separating Probe (AMSP) uses low-rank probes on a temporary optimizer branch to distinguish action-relevant hypotheses and select updates under a source-feasibility constraint. Experiments on Show-o2-7B and InternVL-U-4B using the Chop & Learn benchmark report higher mean joint understanding–generation scores than the compared baselines, alongside an analysis of destructive-update repair and constructive-transfer retention.
Critical strengths
-
The paper identifies a potential limitation of passive gradient statistics, illustrating that updates with similar gradient characteristics may lead to different cross-capability outcomes. The distinction between transfer sign and intervention value provides a reasonable motivation for investigating alternative update decisions.
-
AMSP presents a structured high-level procedure involving probe selection, finite-response measurement, hypothesis filtering, and intervention selection. Using a temporary optimizer branch allows probing without modifying the original training state, which is a reasonable design choice for stateful optimizers.
Critical weaknesses
1. Insufficient specification and justification of the core methodology.
The core decision mechanism relies on a hypothesis set , predictive response distributions , and robust separation bounds, yet their construction and calibration are insufficiently specified. The constrained/reduced update actions and fallback policy also lack implementation details. Given the limited set of predefined actions, the methodological distinction from existing active experimental design and robust action-selection approaches remains insufficiently established. While Appendix B provides a conditional first-order analysis, it does not establish the reliability of the proposed decisions under response-model misspecification, a concern further supported by the limited misspecification detection reported in Appendix F. These gaps hinder reproducibility and make the technical contribution difficult to assess.
2. Limited empirical support for the motivation and generality.
The reported 3.70-percentage-point degradation is introduced without sufficiently clarifying its controlled experimental setting. While Figures 1 and 3 illustrate limitations of passive statistics, the conditions and prevalence of this phenomenon beyond the controlled setting remain unclear. Both backbones are evaluated solely on the small-scale Chop & Learn benchmark, without larger-scale or more diverse understanding–generation evaluations. The joint-score improvements on InternVL-U are statistically inconclusive, and the wall-clock and memory overhead of active probing is not empirically evaluated. These limitations weaken the evidence for the method's generalizability and practical benefits.
Questions for authors
-
How are the hypothesis set , predictive response distributions , and robust separation bounds constructed and calibrated in practice? Could the authors clarify the implementation of the constrained/reduced update actions and fallback policy?
-
What distinguishes AMSP from existing active experimental design and robust action-selection methods, given its limited set of predefined actions? What supports the reliability of its decisions under response-model misspecification, particularly considering the limited detection rates in Appendix F?
-
How were the R0/R2 examples in Figure 1 selected, and what evidence supports the observability gap beyond the controlled setting? Are there additional results on larger or more diverse understanding–generation benchmarks, as well as measured wall-clock and memory overhead relative to the baselines?