- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the semantic transferability between understanding and generation branches in unified multimodal models (UMMs). The authors propose cross-branch semantic steering, which extracts layer-wise steering vectors from the hidden states of one branch using contrastive samples and injects them into the other branch during inference. Moreover, they construct UMMSteer, a contrastive dataset covering 51 concept pairs across seven semantic categories, to extract steering vectors and evaluate their cross-branch transferability. Experiments show that steering vectors from the understanding branch can effectively control image generation, while those from the generation branch fail to meaningfully influence understanding. The authors analyze this asymmetry through subspace alignment and self-steering experiments, suggesting differences in the semantic representations learned by the two branches. Additional experiments across different UMM architectures examine the generalizability of these findings and explore the potential of steering to improve semantic faithfulness in image generation.
Critical strengths
-
Asymmetric cross-branch transferability. Steering vectors extracted from the understanding branch can effectively control image generation, while those extracted from the generation branch fail to influence understanding. This finding provides insights into the relationship between understanding and generation in UMMs.
-
Comprehensive experiments. The authors evaluate 51 concept pairs across seven semantic categories, multiple steering methods, and different UMM architectures. Additional ablations and control experiments further support the effectiveness of understanding-to-generation steering.
Critical weaknesses
-
Lack of Novelty. The paper's main findings regarding the representational differences and limited alignment between understanding and generation largely overlap with prior work. UniFork [1] has analyzed the different layer-wise representation alignment patterns of understanding and generation, while GapEval [2] has revealed inconsistencies between the two capabilities and limited cross-task knowledge transfer. Although both works are cited, the paper does not adequately discuss how its findings differ from or extend these studies. While cross-branch steering provides a different experimental approach, its additional insights beyond prior work remain unclear.
-
Insufficient evidence for the claimed representation mismatch. As shown in Figure 5(a), generation-derived steering vectors fail to reliably modify the target semantic attribute even when applied back to the generation branch. This raises the possibility that the observed Gen-to-Und failure is caused by ineffective semantic direction extraction rather than an inherent representational mismatch between the two branches.
[1] Li et al., UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generation, 2025. https://arxiv.org/abs/2506.17202
[2] Wang et al., Quantifying the Gap between Understanding and Generation within Unified Multimodal Models, 2026. https://arxiv.org/abs/2602.02140
Questions for authors
-
Please clarify the main differences and additional insights of this work compared with prior studies such as UniFork and GapEval.
-
Could the failure of Gen-to-Und steering be due to ineffective steering vector extraction rather than representation mismatch? Please provide further clarification or additional experiments to distinguish these possibilities.
final justification
Thank you for the responses and additional experiments. They address some of my concerns, but also show that the original conclusions do not hold across all the tested settings. UniPic-1 shows the opposite asymmetry, K-Steering enables reverse transfer for counting, and Nexus-Gen supports transfer in both directions at the shared visual interface.
I appreciate that the authors have revised their claims. However, it is still unclear why transfer succeeds in some settings and fails in others. It is difficult to obtain reasonable, concrete conclusions according to the current experiment results. I therefore maintain my weak reject rating. I encourage the authors to study these differences further and clarify what determines whether transfer works.