- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper addresses a tangible shortcoming of existing agentic visual reasoning models: they invoke tools indiscriminately, such that the gains tools provide on difficult questions are largely offset by the errors they introduce on easy ones. Therefore, the authors propose Beacon, trained on Qwen3-VL-8B-Instruct via a two-stage SFT-then-RL pipeline. The RL stage contributes two mechanisms: (1) Necessity-Aware Adaptive Reward (NAAR, Equation 1), which gives full reward to correct tool-free answers when tool-free reasoning succeeds within the rollout group, discounted 0.25 reward to correct code-assisted answers in the same case, and full reward to code-assisted solutions when tool-free reasoning fails; (2) Hint-guided Capability Expansion (HCE, Section 3.4.2), which uses Gemini 3.1 Pro to generate answer-free hints for all-incorrect rollout groups, re-samples with hints, then removes the hints during optimization to transfer the discovered behavior to the hint-free policy. Across 13 benchmarks, Beacon achieves the highest average score among open-source 8B models, ranking first on 11 benchmarks.
Critical strengths
- By formalizing Mode Adaptiveness and Tool Effect and introducing multi-rollout stability (Section 2.1), the paper provides the community with a principled evaluation protocol applicable to any agentic visual reasoning system. The decomposition of Tool-Gain and Tool-Harm is particularly illuminating, revealing that tool use in prior models is essentially zero-sum (Figure 3c). Even if the specific training method is surpassed, this analytical contribution will remain useful.
- The group-conditioned online labeling scheme (Equation 1) avoids the pitfalls of prior adaptive rewards documented in related work. Table 4 shows that NAAR alone improves MA_mean from 56.30 to 59.68 and delta-TE from +1.39 to +2.54.
Critical weaknesses
- Gemini 3.1 Pro is used at every stage: generating 15K SFT trajectories, refining trajectories, producing hints for all-incorrect groups during RL, and judging answers at evaluation time. No experiment isolates the contribution of NAAR and HCE from the gains of distilling a frontier model. For instance, would NAAR still help if the SFT data came from Qwen3-VL-72B instead of Gemini? The SFT-only model already achieves 56.91 Overall Acc (Table 4), surpassing all baselines except Metis, suggesting that Gemini-quality SFT data accounts for a substantial proportion of the improvement.
- The full NAAR+HCE framework improves Overall Acc from 56.91 (SFT) to 58.98 (+2.07 points, Table 4), a relatively modest margin. Furthermore, Beacon's MA_text is 22.91% (Table 3), meaning it invokes tools on over three-quarters of text-easy questions. The narrative of "adaptive tool invocation" overstates the actual behavior—a more accurate characterization would be "a tool-using model with high tool efficacy."
Questions for authors
- How sensitive are the results to the 0.25 discount factor in NAAR (Eq. 1)? Have you tried 0.0, 0.5, or 0.75? A sweep over this parameter is needed to understand whether NAAR is robust or requires careful tuning.
- Can you provide results where Gemini 3.1 Pro is replaced by an open-source model (e.g., Qwen3-VL-72B or InternVL-78B)? This would clarify whether the proposed framework's benefits are contingent on access to a frontier proprietary model.