- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates the perception–reasoning asymmetry arising from merging reasoning-specialized LLMs into multimodal large language models (MLLMs). The authors observe that model merging can improve reasoning performance while degrading fine-grained visual perception. To address this issue, they propose PASA, a two-stage post-merge alignment framework. In the first stage, the merged model is supervised to generate structured visual evidence through SFT. In the second stage, the authors introduce asymmetric GRPO, where reasoning-prompted responses provide pseudo-answer supervision for perception-prompted trajectories. Experimental results across different model scales and merging strategies demonstrate consistent improvements on multiple multimodal benchmarks.
Critical strengths
- The framework is intuitive. PASA leverages different prompted behaviors of the same model to construct a self-training signal. The asymmetric supervision strategy is intuitive and does not require a separate teacher model during the RL stage.
- The empirical improvements are considerable. The experiments cover multiple model scales (2B, 8B, and 14B) and different merging strategies. PASA achieves consistent improvements over direct merging across several multimodal benchmarks, including MMMU, MathVista, MMStar, and OCRBench.
Critical weaknesses
- The proposed method is motivated by the observation that the reasoning-prompted role achieves substantially higher answer accuracy than the perception-prompted role. However, this comparison is conducted after the structured SFT stage, which has already modified the model's perception-oriented behavior. It is therefore unclear whether the observed gap is a consequence of model merging or is primarily induced by the different prompting strategies and the structured SFT stage.
- The paper categorizes several multimodal benchmarks into reasoning-oriented and perception-oriented groups, although these benchmarks generally involve both capabilities. Their performance differences do not necessarily establish a clean separation between reasoning improvement and perception degradation.
- The effectiveness of asymmetric supervision may be strongly coupled with the specifically designed structured SFT stage. One particularly interesting result is that PASA outperforms gold-answer GRPO after the structured SFT stage, achieving 69.5 versus 66.7 on MathVista. However, the reported results suggest that this advantage may be closely related to the specific SFT data construction rather than asymmetric supervision alone. In particular, Table 3a reports that reasoning-anchor GRPO without Stage I achieves 65.1 on MathVista, slightly below the 65.5 achieved by gold-answer GRPO without Stage I in Table 2. After introducing structured SFT, however, the corresponding scores become 69.5 and 66.7, respectively. This suggests a potentially important interaction between the structured SFT stage and the subsequent asymmetric RL procedure, although the current experiments do not fully isolate this interaction. The structured SFT stage trains the model on synthetic visual-evidence trajectories following a specific schema, thereby modifying the distribution of perception-prompted responses. It is possible that this initialization particularly benefits the subsequent reasoning-anchor supervision, for example, by making the pseudo-answers more compatible with the model's current output distribution or providing more informative reward variations during GRPO. However, the paper does not sufficiently investigate these possibilities.
Questions for authors
See weakness
5 comments
Isolating the Stage-I × Stage-II interaction and validating both mechanisms (W3)
Yes. Stage I increases Reasoning-Anchor GRPO's advantage over Gold-Answer GRPO across all five benchmarks (positive relative-to-Gold interaction throughout) because Stage I stabilizes parsable structured trajectories while Stage-II supplies a distributionally reachable target with denser within-group agreement variance.
-
Robustness across Stage-I SFT variants (Table 3a–c, MMMU / MathVista / MMStar / Hallusion). Zeroing SFT loss on final-answer tokens (
Answer-Masked Stage I + PASA) reaches59.0 / 69.5 / 56.5 / 48.2vs. default59.2 / 69.5 / 56.6 / 48.1, preserving the full69.5MathVista score without terminal-answer SFT supervision. Replacing externalQwen3-VL-Maxtraces with parser-filtered self-generated traces from the merged model itself still reaches58.7 / 69.0 / 56.4 / 47.8(69.0vs.66.8±0.17for Gold GRPO), while structured traces outperform unstructured traces (65.3 / 46.7vs.63.9 / 45.2on MathVista / Hallusion, Table 3a). -
Complete 2×4 Stage-I × Stage-II comparison and rollout diagnostics. Note also that
Gold-answer GRPOuses privileged reference answers () during RL, whereas PASA uses zero ground-truth RL labels (Majority[1] is the matched label-free RL baseline). With Stage I, PASA surpasses even privilegedGold GRPOacross all five benchmarks:
| Stage I | Stage-II Target | MMMU | MathVista | MMStar | OCRBench | Hallusion |
|---|---|---|---|---|---|---|
| No | None (Direct Merge) | 56.7 | 63.4 | 51.2 | 79.4 | 45.5 |
| No | Majority () [1] | 57.3 | 66.0 | 53.1 | 80.2 | 46.1 |
| No | Gold () | 58.2 | 65.5 | 53.4 | 80.2 | 46.4 |
| No | Reasoning () | 57.7 | 65.1 | 53.4 | 80.9 | 47.0 |
| Yes | None (Stage I) | 57.4 | 65.3 | 53.7 | 80.5 | 46.7 |
| Yes | Majority () [1] | 57.8 | 66.7 | 53.9 | 82.1 | 46.5 |
| Yes | Gold () | 58.8 | 66.7 | 54.3 | 81.9 | 46.9 |
| Yes | Reasoning (, PASA) | 59.2 | 69.5 | 56.6 | 83.2 | 48.1 |
| Lift | Stage I on | +1.5 | +4.4 | +3.2 | +2.3 | +1.1 |
| Interaction | (vs. Gold) | +0.9 | +3.2 | +2.3 | +0.6 | +0.6 |
Matched 3-seed means (mean ± sample SD, ) for the three With-Stage-I RL rows are 57.8±0.20 / 66.4±0.19 / 54.0±0.10 / 82.3±0.21 / 46.4±0.16 (Majority), 58.7±0.13 / 66.8±0.17 / 54.3±0.14 / 82.0±0.05 / 47.1±0.11 (Gold), and 59.2±0.06 / 69.4±0.21 / 56.4±0.25 / 83.2±0.09 / 48.1±0.15 (PASA), likewise giving positive of +1.0 / +3.0 / +2.1 / +0.5 / +0.4 across all five benchmarks.
| Rollout Metric () | No Stage I + | Stage I + Majority | Stage I + Gold | Full PASA |
|---|---|---|---|---|
| Format valid | 23.8% | 98.4% | 98.0% | 98.6% |
| Target hit () | 66.7% | 74.0% | 69.0% | 77.0% |
| Agreement variance | 52.0% | 67.0% | 62.0% | 71.0% |
These diagnostics directly confirm both mechanisms you proposed:
- Without Stage I (
65.1),Direct Mergesatisfies the structured<caption>...</caption>+\boxed{}format on only23.8%of rollouts (vs.98.6%after Stage I), leaving only52.0%nonzero agreement-variance groups as updates are spent fixing output syntax. - With Stage I (
69.5vs.66.7,+3.2), once structured trajectories are parsable (98.0%–98.6%), external gold answers still miss all rollouts more frequently on hard visual-math items (69.0%hit;62.0%nonzero agreement-variance groups), whereas same-model comes from a much stronger role thanMajority(75.3%reasoning vs.47.2%greedy perception) and is distributionally compatible with the shared policy (77.0%hit;71.0%nonzero agreement-variance groups) while co-evolving upward (75.3%83.4%). We will add this 2×4 table and rollout analysis to Section 4.4 and streamline Section 3's role-update presentation for clarity.
[1] Wei et al., Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO, 2025.
Dear Reviewer,
Thank you again for your thoughtful review and for the time you have spent evaluating our paper.
We have carefully addressed the concerns raised in your review and hope that our rebuttal has helped clarify the issues you pointed out.
As the discussion period is coming to an end, we would like to kindly ask if you have any remaining questions or if there is anything further we can clarify. We would be more than happy to provide additional explanations.
Thank you very much for your time and consideration.
We thank Reviewer
72Gbfor the constructive review and for appreciating our paper's soundness, intuitive parameter-shared cross-role design, and consistent gains across model scales and merge strategies. We address all three concerns below.Does the role gap arise from model merging rather than prompting and Stage-I SFT? (W1)
Yes, the positive reasoning-over-perception role gap is directly caused by cross-modal merging prior to any SFT. Evaluating both roles (Preason and Pperc) on the exact same
6,000queries across{Unmerged Source, Merged} × {Before, After Stage-I SFT}isolates each factor cleanly:Source MLLM, perception exceeds reasoning by14.9points (69.6%vs.54.7%). Applying the identical 20,000-trace Stage-I SFT (Source + Stage I) improves both roles slightly while leaving the gap virtually unchanged at−14.2(71.2%vs.57.0%), matching Table 2b (Source + Stage I:55.2 / 64.2vs. PASA:59.2 / 69.5on MMMU / MathVista).Direct Merge) immediately raises reasoning accuracy (+19.2) while lowering perception accuracy (−27.1), reversing the gap by+46.3points (−14.9→+31.4before SFT;+28.1after SFT).Evidence of perception degradation and recovery beyond benchmark grouping (W2)
Yes, perception degradation after merging and recovery under PASA hold directly on fine-grained, task-decoupled measurements without relying on benchmark grouping (which Section 4.1 and Appendix A.2 use only as an operational summary):
Exact text, document, and formula reading on
OCRBench(Tables 1 & 2a).OCRBenchuses deterministic string/exact matching with zero LLM judge, and deductive reasoning cannot guess unseen characters or digits in an image.OCRBenchdrops after merging (82.5→79.4at 2B,82.9→76.2at 8B,82.9→78.9at 14B) and recovers under PASA (83.2,83.3,83.7), surpassing Gold GRPO (82.0±0.05) and Majority (82.3±0.21) [1].Same-image, same-question perception-role recovery ([W1]). Holding the image and question identical across
6,000queries, merging drops Pperc accuracy (69.6%→42.5%pre-SFT;71.2%→47.2%post-SFT), and Stage-II PASA restores it to77.5%(+30.3).Blank-image visual-dependence test. Replacing input images with a blank white canvas drops PASA's accuracy from
69.5%→9.6%on MathVista (−59.9) and56.6%→5.7%on MMStar (−50.9), confirming direct reliance on visual evidence.Answer-blind visual-evidence audit (Figure 3a & Table 6). Scoring only image-observable facts in answer-stripped MathVista traces raises the Visual Grounding Score from
64.7(Direct Merge) →68.2(Stage I) →75.9(Full PASA).