- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates a staged GRPO algorithm for real-photograph scenarios where intermediate steps are difficult to verify precisely. The first stage rewards output structure and final answer quality, while the second replaces the answer reward with rubric-based scores for selected reasoning steps. The authors introduce MOSAIC, a visual missing-word inference task covering 12 training languages and three unseen languages, with a four-step decomposition derived from human strategies. Experiments on Qwen and InternVL demonstrate the method's effectiveness (e.g., increasing Qwen2.5-VL-7B's average MOSAIC accuracy from 81.1% to 85.3%). The paper also validates the model-proposed steps on SpatiaLQA.
Critical strengths
- The authors support the method's effectiveness and robustness with sufficient comparative experiments. MOSAIC covers multiple languages and two model families, while SpatiaLQA evaluates a different task with a model-proposed reasoning structure.
Critical weaknesses
- There is a fundamental gap between the paper's core claim and its evaluation. The paper claims to study multimodal reasoning in real-photograph scenarios, focusing on the difficulty of rewarding intermediate steps. However, i) the evaluation primarily uses a multilingual task whose characteristics are difficult to connect directly to this motivation; ii) the focus on word filling is too narrow, and small models already achieve very high scores on MOSAIC. As a result, the task cannot comprehensively assess multimodal reasoning ability.
- The method lacks sufficient core novelty. Its central ideas, including GRPO, staged reinforcement learning, and dense rewards, have already been validated and published.
- The authors manually specify four reasoning steps—Sentence Analysis, Image Analysis, Object Exclusion, and Grammar and Context—and require the model to follow them. This may limit, to some extent, the trained model's generalization to other tasks.
Questions for authors
- How were the weights for the individual components of the final reward determined in each of the two GRPO training stages?
- Could the authors evaluate the method on more representative benchmarks, including OOD benchmarks such as MM-star, MathVista, and MMMU, to support the stated goal of studying multimodal reasoning in real photographs?
- Could the authors report the performance of other representative models, such as Qwen3.5-9B, on MOSAIC? This would help establish the task's level of difficulty.
2 comments
Thank you to the authors for their detailed response. However, I still find that the objective of "testing whether semantic step rewards support reasoning across languages" is only weakly connected to image-grounded reasoning, and I remain concerned about the method's generalizability to other domains. Furthermore, Qwen3.5-9B already achieves a relatively high accuracy of 85.86% on MOSAIC, raising further concerns about the benchmark's difficulty. Therefore, I have decided to maintain my current score.
We thank the reviewer for the detailed feedback. Below, we clarify our design choices and present additional experimental results addressing the main concerns.
W1: A gap between the motivation and evaluation; MOSAIC is narrow and already easy for small models
A: MOSAIC is a multimodal question-answering task grounded in photorealistic images, formulated as missing-word inference from an image and an incomplete caption. Multilinguality concerns the language of the input caption; the underlying task remains reasoning jointly over visual and linguistic evidence. We deliberately vary the input language while keeping intermediate reasoning in English to test whether semantic step rewards support reasoning across languages, rather than improvements restricted to English-specific input patterns. Evaluation on languages unseen during RL training further tests transfer beyond the training languages. The multilingual design therefore directly serves our investigation of image-grounded reasoning.
The characterization that small models already achieve very high scores overlooks the reported results. Average accuracy across the six evaluated models ranges from 67.4% to 82.3%. Qwen2.5-VL-3B achieves 71.7% overall and only 34.5% on Hindi, leaving substantial room for improvement. Moreover, the evaluation extends beyond missing-word inference to SpatiaLQA, a challenging public benchmark requiring spatial reasoning and action-plan generation with a different, model-proposed decomposition.
W2: The method lacks novelty because GRPO, staged training, and dense rewards already exist
A: Our contribution is the design and empirical study of task-specific semantic rewards for photorealistic reasoning whose intermediate steps lack uniquely verifiable targets. We define reasoning steps with explicit roles and corresponding evaluation criteria, allowing a frozen, off-the-shelf judge to supervise them without reasoning-demonstration SFT or dedicated reward-model training.
Our experiments establish how this supervision can be used: introducing global and step-wise rewards jointly from the initial model performs poorly, whereas step-wise training after global-reward initialization improves final-answer performance and outperforms continued outcome-based training. SpatiaLQA further demonstrates the framework with a different, model-proposed decomposition. The prior existence of GRPO, staged training, and process rewards does not by itself account for these task-specific reward designs and empirical findings.
W3: The manually prescribed four steps may limit generalization to other tasks
A: The four steps are specific to MOSAIC; the framework does not require other tasks to reuse them. SpatiaLQA directly demonstrates adaptation to a different task: we use five model-proposed reasoning steps and corresponding evaluation criteria for spatial action-plan generation. The framework improves F1 from 40.20 to 44.70, with Stage 2 contributing a further 2.06 points. This provides evidence that the method extends beyond MOSAIC’s manually specified decomposition through task-adapted steps and rubrics.
Q1: How were the reward-component weights determined?
A: We selected the reward weights through grid search on 100 validation examples using Qwen2.5-VL-3B and used the same weights across model sizes, without size-specific tuning.
Q2: Could the method be evaluated on additional OOD benchmarks such as MMStar, MathVista, and MMMU?
A: SpatiaLQA already evaluates our framework on a challenging public spatial-reasoning benchmark, using model-proposed steps for action-plan generation. This directly tests adaptation to a different task beyond MOSAIC. Extending the framework to further tasks requires task-specific steps and rubrics followed by two-stage training, which carries substantial computational cost.
Q3: Could more recent models, such as Qwen3.5-9B, be evaluated on MOSAIC?
A: We additionally evaluated Qwen3.5-9B on MOSAIC using the zero-shot task instruction and GPT-based semantic scoring.
The newer and larger Qwen3.5-9B achieves accuracy comparable to our staged Qwen2.5-VL-7B model. This shows that our training brings the smaller, earlier model to a performance level comparable to the newer model on MOSAIC. Moreover, Qwen3.5-9B still makes errors on approximately 14% of examples, indicating that the task remains challenging even for a more recent model.