- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GRATS, a two-stage reinforcement learning framework for multimodal reasoning that first constructs an explicit relational graph from image-question inputs and then performs graph-centric chain-of-thought reasoning over the generated structure. Its main contribution is a stage-specific credit assignment scheme that separately computes graph-level and reasoning-level rewards and applies the corresponding advantages to graph-construction and reasoning tokens, with the goal of improving structured visual perception and reducing reasoning drift toward language priors. The method is evaluated with Qwen3-VL-4B/8B on seven multimodal reasoning benchmarks covering spatial perception, counting, chart understanding, mathematical reasoning, and comprehensive multimodal evaluation, where it consistently outperforms GRPO/DAPO-based baselines; the paper further provides stage-wise ablations, attention analyses, sensitivity studies, qualitative cases, and a computational cost breakdown to support the effectiveness of the proposed graph-centric training framework.
Critical strengths
-
Clear and potentially useful decomposition of multimodal reasoning. GRATS explicitly separates relational graph construction from graph-centric reasoning and introduces stage-specific credit assignment for the two phases. This is a meaningful design choice because it targets a real limitation of outcome-only RL in multimodal settings: errors in visual structure extraction and errors in subsequent reasoning otherwise receive the same coarse supervision. The proposed formulation is conceptually simple and could be broadly useful beyond the specific benchmarks considered here.
-
Strong and relatively broad empirical gains. The method is evaluated on seven benchmarks spanning spatial perception, counting, chart reasoning, mathematical reasoning, and general multimodal evaluation, and consistently outperforms GRPO/DAPO baselines under both 4B and 8B settings. The improvements are particularly large on several tasks, suggesting that the graph-centric representation and two-stage optimization provide substantial practical benefit rather than only marginal gains.
Critical weaknesses
-
The paper does not directly validate whether the proposed joint training actually improves or preserves graph-construction quality. A central claim of GRATS is that stage-specific credit assignment improves both perception and reasoning. However, the current evidence for the perception stage is indirect: Figure 3 reports a combined training reward, Appendix E evaluates downstream QA accuracy, and Figure 4 analyzes attention patterns, but none directly measures the correctness of the generated relational graphs. This is particularly important because both stages share the same policy, so reasoning-stage optimization may also alter graph-generation behavior. Moreover, the graph-level reward itself depends substantially on downstream reasoning correctness and perplexity rather than direct graph supervision, with the leak-controlled reasoning score receiving 80% weight in the Phase-1 reward. Therefore, the current results establish that the generated graphs are useful for downstream answering, but do not establish that graph perception itself is improved or even preserved under joint two-stage optimization.
-
The practical training efficiency of the two-stage framework is insufficiently characterized. GRATS samples multiple graphs per input and multiple reasoning trajectories per graph, while the reasoning reward uses a GPT-4o-based correctness score. The computational analysis emphasizes that the additional phase-wise advantage computation is negligible, but the reported runtime shows that Phase-1 and Phase-2 rollouts themselves dominate training time (596.69s and 543.81s per iteration, respectively). The paper does not report the latency or monetary cost of GPT-4o judging, nor whether this cost is included in Figure 5. As a result, the current analysis does not clearly quantify the end-to-end training overhead of GRATS relative to standard GRPO.
Questions for authors
-
Clarification: Can the authors provide a direct evaluation of graph-construction quality, particularly comparing Stage-1-only training with the full joint Stage-1/Stage-2 training? Since the two stages share the same policy, this would clarify whether reasoning-stage optimization preserves, improves, or degrades the quality of the generated relational graphs.
-
Clarification: Since the graph-level reward relies heavily on downstream reasoning correctness and perplexity rather than direct graph supervision, how do the authors ensure that a graph with high reward is actually visually and relationally correct, rather than merely sufficient for producing the correct final answer?
-
Clarification: How is GPT-4o-based reward computation implemented during training? In particular, how many judge evaluations are required, are requests batched/cached/asynchronous, and is GPT-4o latency included in the runtime reported in Figure 5? A total end-to-end training cost comparison with standard GRPO would materially clarify the practical overhead of the method.
-
Clarification: Training data scale and train/evaluation separation. The training set contains only approximately 8K unique examples, sampled from the same seven benchmark families used for evaluation, while GRATS achieves substantial gains on several datasets. Could the authors clarify the exact training/evaluation split for each benchmark, especially VSR, which is described as being used for both training and evaluation, and confirm whether the training and evaluation instances are strictly disjoint at both the sample and image levels? In addition, have the authors examined performance as a function of training-set size (e.g., 25%, 50%, 75%, and 100% of the training data)? Such a scaling analysis would help determine whether the reported gains consistently arise from the proposed training framework or are sensitive to the particular small, in-domain training set.
-
Clarification: Could the authors clarify the apparent inconsistency between Table 1 and Appendix Table 2? Appendix Table 2 is labeled as Qwen3-VL-4B, but its full-GRATS results on TallyQA/ChartQA/MathVista (75.0/77.9/58.8) match the 8B results in Table 1 rather than the reported 4B results (68.8/76.7/58.0).