- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes a two-stage framework that first trains direct answering and tool-use capabilities separately using Direct/Tool GRPO, and then learns when to use tools through OPD. The overall idea is clear and the results are promising, but the current experiments don't fully justify the need for decoupling or show whether the model actually learns better tool routing.
Critical strengths
-
Separating Capability Learning from Routing Learning through Direct/Tool GRPO and OPD is intuitive and well-motivated.
-
The method improves accuracy across multiple visual benchmarks while reducing the tool-call rate, showing benefits in both effectiveness and efficiency.
Critical weaknesses
-
Higher accuracy and fewer tool calls could simply mean the model has become stronger at solving tasks without tools. The paper doesn't clearly separate capability gains from routing improvements, so it's hard to conclude that the model has actually learned better tool routing.
-
Missing key baselines. I'd like to see two additional comparisons:
-
End-to-end Joint RL: Compare the full two-stage approach against joint RL under the same data, rollout budget, and training compute. Otherwise, it's hard to justify why decoupling is necessary.
-
Separate Experts + OPD: Train independent Direct-only and Tool-only experts, then merge them through OPD. This would help explain why Stage I needs a shared dual-mode expert instead of two separate experts.
-
-
Stage II introduces several changes at once, making it hard to tell where the gains come from. I think some important baselines are missing, such as off-policy distillation with SFT, random mode assignment, and always-direct/always-tool teachers.
-
Qwen2.5-VL-7B doesn't have crop-and-zoom tool-use prior, while many recent open-source VLMs (after Qwen3-VL) already do. It's unclear whether the proposed method would still be effective for models with more mature tool-use capabilities.
-
In Appendix I. The text mentions 150 training steps and reports results from steps 101–150, but Figure 5 only shows 50 steps. This is strange.
Questions for authors
See weakness
4 comments
Response to Reviewer AsHk (2/2)
Q3: Stage II introduces several changes at once, making it hard to tell where the gains come from. I think some important baselines are missing, such as off-policy distillation with SFT, random mode assignment, and always-direct/always-tool teachers.
A3: Thanks for your suggestions.
Following your suggestions, we conduct three additional comparisons: (1) off-policy SFT using expert-generated trajectories, (2) on-policy distillation with random mode assignments, and (3) on-policy distillation with always-direct or always-tool teachers. The results are summarized below.
| Method | V* Bench | HRBench-4K | HRBench-8K | HRBench Avg. |
|---|---|---|---|---|
| SFT | 83.25 | 74.25 | 67.25 | 70.75 |
| OPD + Random Assignment | 89.01 | 73.00 | 68.88 | 70.94 |
| OPD + Always-Direct | 89.53 | 74.88 | 67.25 | 71.07 |
| OPD + Always-Tool | 89.53 | 74.38 | 68.75 | 71.57 |
| Ours | 90.05 | 76.38 | 71.25 | 73.82 |
As shown in the table, our method consistently outperforms all ablation variants across the three benchmarks. Compared with off-policy SFT, our on-policy approach achieves substantial accuracy improvements, supporting the effectiveness of distillation on student-generated trajectories. Furthermore, Our method outperforms both random assignment and fixed-mode teachers, indicating the importance of selecting supervision based on the relative strengths of the two modes for each question. Together, these results support the effectiveness of combining question-specific mode assignment with on-policy self-distillation in Stage II.
Q4: Qwen2.5-VL-7B doesn't have crop-and-zoom tool-use prior, while many recent open-source VLMs (after Qwen3-VL) already do. It's unclear whether the proposed method would still be effective for models with more mature tool-use capabilities.
A4: Thanks for your suggestion.
We would like to emphasize that our method is motivated by the instability arising from jointly optimizing mode capabilities and routing, rather than limitations in the backbone's tool-use capabilities. While stronger tool-use priors may improve the model's ability to solve questions with tools, they do not inherently eliminate inter-mode competition during RL, which can still lead to unstable training behavior.
Following your suggestion, we conduct additional experiment to validate the effectiveness of our framework on Qwen3-VL-8B-Instruct,which have stronger tool-use priors. The results are summarized below.
| Model | V* Bench | HRBench 4K | HRBench 8K | HRBench Avg. |
|---|---|---|---|---|
| Qwen3-VL-8B Instruct | 87.43% | 76.38% | 69.12% | 72.75% |
| Stage I model | 88.48% | 79.50% | 70.12% | 74.81% |
| Stage II model | 90.58% | 81.00% | 73.38% | 77.19% |
As shown in the table, stage II consistently improves upon Stage I across all benchmarks, with gains of 2.10 percentage points on VStar and 2.38 percentage points on HRBench overall. These results demonstrate that our two-stage framework remains effective on models with stronger tool-use priors, supporting its applicability beyond models with limited initial tool-use capabilities.
Q5: In Appendix I. The text mentions 150 training steps and reports results from steps 101–150, but Figure 5 only shows 50 steps. This is strange.
A5: Thank you for this feedback.
We apologize for the errors in Appendix I. The experiment was conducted for 50 training steps, as shown in Figure 5. The references to "150 training steps" and "steps 101–150" were typographical errors, which we have corrected in the revised manuscript.
Cool, I've carefully read the authors' response, and I'm overall satisfied with it. Most of my previous concerns have been addressed, so I've decided to increase my score.
Thank you for your encouraging feedback and for recognizing our efforts to address your concerns. We sincerely appreciate your thoughtful comments and constructive suggestions, which have helped us further strengthen our work.
Response to Reviewer AsHk (1/2)
We sincerely thank the reviewer for their constructive feedback.
Q1: Higher accuracy and fewer tool calls could simply mean the model has become stronger at solving tasks without tools. The paper doesn't clearly separate capability gains from routing improvements, so it's hard to conclude that the model has actually learned better tool routing.
A1: We thank the reviewer for the suggestion.
To distinguish improved question-dependent routing from a simple preference for direct answers, we conduct an additional evaluation of the question-level optimal routing rate on HRBench. For each question, we identify the preferred mode by comparing the empirical accuracies of Direct and Tool, assigning ties to Direct to avoid unnecessary tool calls. We then measure how often the model's autonomous routing decision matches this preference. Stage II improves the optimal routing rate from 55.00% to 76.50% on HRBench-4K and from 52.63% to 73.25% on HRBench-8K, yielding gains of 21.50 and 20.63 percentage points, respectively. These substantial improvements directly demonstrate that Stage II learns to make more appropriate tool-use decisions for individual questions, rather than simply reducing tool usage.
This finding is further supported by Table 2 in our original paper, where our method consistently outperforms heuristic and learned routing baselines, including an LLM-based router, under comparable tool-use budgets. Together, these results provide consistent evidence that the improvements stem from more effective question-dependent routing, rather than solely from stronger direct-answering capabilities.
Q2: Missing key baselines. I'd like to see two additional comparisons:
A2: Thanks for your suggestions.
We first clarify the motivations behind our two key design choices and then provide additional comparisons to substantiate their effectiveness.
Our primary motivation for decoupling is to avoid the training instability observed in joint RL, where mode collapse or tool overuse can cause optimization to concentrate on a single mode, limiting learning opportunities for the other. Regarding the motivation for using a shared expert, we do not claim it is the only viable approach to decoupling. Rather, it provides a more parameter-efficient solution by learning both mode capabilities within a single model, avoiding the additional training and model-management overhead of maintaining two independent experts during the distillation process.
Following your suggestions, we conduct additional experiments comparing our framework against (1) end-to-end joint RL under the same training budget with Mini-o3-like and DeepEyes rewards, and (2) independently trained Direct-only and Tool-only experts followed by on-policy distillation. The results are summarized below.
As shown in the table, our method consistently outperforms both compute-matched joint RL baselines, indicating that the improvements are not simply due to additional training computation. Furthermore, while separate experts provide an alternative for decoupling, our shared-expert approach achieves higher accuracy across all three benchmarks with a more parameter-efficient training design. These results further support both the effectiveness of decoupling capability learning from routing and the practical advantages of using a shared expert.