- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper targets multimodal reasoning with reinforcement learning and introduces VRGym, which generates problems with verifiable answers and structured guidance, together with OPGG, which progressively masks and removes that guidance based on rollout accuracy. Across two Qwen-VL backbones, OPGG achieves the highest average accuracy among the evaluated methods on VRGym and six external benchmarks.
Critical strengths
- VRGym provides a useful way to generate multimodal reasoning problems with correct, controllable guidance.
- The method also improves performance on six external benchmarks without using guidance at test time.
Critical weaknesses
-
The comparison does not isolate the benefit of the OPGG schedule. OPGG uses procedural derivations, while the hint-based baselines use oracle-generated traces. The ablations in §4.4 test components of OPGG, but the comparison with other schedules still mixes differences in guidance and scheduling. I would like to see the schedules compared using the same procedural derivations, including a simple predetermined withdrawal schedule.
-
Permanent guidance withdrawal is not directly tested. Algorithm 1 freezes a task’s budget once it reaches 2, so later failures cannot restore guidance. The ablations do not test this decision. Please report each task’s unguided accuracy before and after graduation, and test whether allowing guidance to return improves performance.
Questions for authors
- How does OPGG compare with a predetermined withdrawal schedule when both use the same procedural guidance?
- How does each task’s unguided accuracy change before and after graduation?
- What happens if guidance can be restored when a graduated task’s performance declines? 4.How does providing Scaf-GRPO with the procedural solution rather than only the final answer affect its performance?
3 comments
Questions
-
We are not entirely sure what the reviewer means by a "predetermined withdrawal schedule." If this refers to withdrawing guidance based only on training steps, we believe such a design is less suitable. It would require testing many scheduling strategies to determine how much guidance should be withdrawn at each step. More importantly, it would not indicate whether the model has actually learned to solve problems without relying on guidance. In contrast, OPGG adjusts guidance based on group accuracy. Similar accuracy-based strategies are also used in previous hint-based methods. For example, Scaf-GRPO [1] increases the number and strength of hints when the model fails to produce a correct answer. Therefore, we believe that using group accuracy to schedule guidance is a simple and reasonable choice.
Nevertheless, following the reviewer's suggestion, we also conducted an experiment with a linear schedule. Specifically, we linearly increase the budget from 0 at the beginning of training to 2 at the end of training. The results are shown in the table below.
Method VRGym MathVista MathVision MathVerse DynaMath MMMU MMMU-Pro Avg. OPGG (default) 23.66 71.14 27.96 49.28 52.05 54.35 38.05 45.21 linear 22.31 70.64 26.52 48.56 50.11 54.38 38.03 44.36 -
We have provided additional results addressing this concern in our response to Weakness 2.
-
As shown in the additional experiments in our response to Weakness 2, allowing guidance to be restored does not substantially affect the final performance. We also do not observe a large difference in training speed. When guidance restoration is allowed, ρ fluctuates between 1.95 and 2 for graduated tasks. We report the actual training time below. Allowing guidance restoration increases the average training time per step by approximately 10%.
OPGG (default) With guidance restoration Relative training time per step 1.000x 1.105x -
First, we would like to clarify that Scaf-GRPO [1] uses oracle-generated reasoning and adds hints when the model fails to produce a correct answer. Therefore, the difference is not simply whether a solution is provided. We acknowledge that our original description may have been unclear, and we have revised the related work section accordingly. Furthermore, as discussed in our response to Weakness 1, we have added experiments in Section 4.4 where three existing hint-based RL baselines are trained using our procedural guidance. These results provide additional evidence for the effectiveness of procedural guidance independently of OPGG.
[1] Zhang, Xichen, et al. "Scaf-grpo: Scaffolded group relative policy optimization for enhancing llm reasoning." International Conference on Learning Representations. 2026.
About Presentation
In addition, we learn the reviewer's concern about the clarity of our writing, and we apologize if some parts of the paper were difficult to follow. Since the reviewer raised issues with the presentation, we would appreciate it if you could point out specific sections or sentences that were unclear or ambiguous. This would help us better understand the issues and make targeted improvements to the paper. We sincerely appreciate your feedback and would like to improve the clarity of our work as much as possible.
Reminder: Response to Our Rebuttal
Dear Reviewer miiw,
We apologize for our late response, since we needed additional time to conduct supplementary experiments and provide additional experimental results and details.
We have also added an "Additional Summary of Paper" in the general comment section to further clarify the key contributions of our paper.
As the discussion period is approaching its deadline, we would greatly appreciate it if you could take a moment to review our rebuttal and share your feedback.
Thank you very much for your time and consideration.
Best regards, The Authors
We thank the reviewer for taking the time to review our paper. Based on the feedback from all three reviewers, we realized that some parts of our paper were not clearly written, which may have led to misunderstandings. To address this, we have added a comment titled "Reiterating the Contributions" to clarify and restate the key contributions of our work.
Weaknesses
We agree with the reviewer. As clarified in our "Reiterating the Contributions" comment, our paper focuses on procedural guidance rather than OPGG. Therefore, we have added new experiments in Section 4.4, where we replace the guidance used by existing hint-based RL baselines with procedural guidance while keeping their original scheduling mechanisms unchanged. This allows us to evaluate the effect of procedural guidance independently of OPGG's scheduling mechanism. The results show that procedural guidance further improves the performance of existing on-policy hint-based methods. In particular, Scaf-GRPO improves across all seven benchmarks, with the largest gain on MathVision (+4.93 percentage points) and an average improvement of +1.78 percentage points with procedural guidance.
As explained in our "Reiterating the Contributions" comment, OPGG is a simple and intuitive approach designed to demonstrate the feasibility of procedural guidance. We do not expect small changes to its design to substantially affect performance, as also shown in our other ablation studies. Nevertheless, we have conducted additional experiments to address the reviewer's concerns.
(1) Performance on graduated and ungraduated tasks
Below, we report the avg@8 results for graduated and ungraduated tasks, both overall and across the task categories included in training. We observe that graduated tasks are generally easier by design and benefit more from guidance. In contrast, ungraduated tasks tend to be more difficult, making it harder for the model to fully benefit from guidance. This is consistent with Limitation 2, where we acknowledge that our task design may not be perfectly balanced and that some tasks may still be too difficult for the model.
We also report the proportion of graduated and ungraduated tasks in each category below.
(2) Removing the permanent guidance withdrawal constraint
Following the reviewer's suggestion, we also evaluate OPGG without the permanent guidance withdrawal constraint. The results are shown below.
The results show that removing this constraint has little effect on the overall performance. Therefore, this alternative design is also a reasonable choice. As emphasized earlier, OPGG is only one possible way to use procedural guidance, and we chose a simple and intuitive design to demonstrate its feasibility. Our main focus remains on evaluating the benefits of procedural guidance rather than optimizing OPGG's scheduling mechanism.