- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims at the performance of large reasoning models (LRMs) on chart editing tasks. The authors propose REChart, a two-stage training framework that improves both editing fidelity and reasoning efficiency. In the first stage, 200k high-quality structured reasoning trajectories are synthesized via a role-specialized multi-agent Reason-Score-Refine workflow. In the second stage, the model is further optimized via GRPO with a hybrid reward function. Experiments on the ChartEdit and ChartMIMIC benchmarks show that REChart achieves state-of-the-art performance among open-source models of comparable scale.
Critical strengths
-
REChart introduces two complementary technical advances that address the overthinking problem at both the data and optimization levels. The multi-agent trajectory synthesis pipeline is a clever approach to producing high-quality reasoning supervision. The episode-level efficiency reward in the RL stage is also well motivated.
-
The method delivers strong improvements on both dimensions. REChart outperforms all evaluated open-source baselines across three benchmark settings. The consistency of gains across multiple benchmark suites and task types further strengthens confidence in the results.
-
The constructed corpus is a substantial and valuable resource for the research community. The multi-stage cascaded generation pipeline with rigorous quality filtering produces high-quality training data covering a diverse taxonomy of chart editing operations. Public release of this dataset will enable substantial further work on chart reasoning and efficient multimodal code generation.
Critical weaknesses
-
Nearly all quality evaluation relies on GPT-4o as an automated judge, following the ChartEdit benchmark protocol. The paper does not complement this with deterministic, objective metrics such as exact code syntax accuracy, quantitative structural similarity of chart elements, or pixel-level visual distance metrics. VLM-based scoring is known to exhibit biases, variability, and imperfect alignment with human judgment for fine-grained visual details. Without supporting objective metrics, the magnitude of reported improvements is harder to independently verify and interpret.
-
All experiments are conducted exclusively on Qwen3-VL-8B. It remains unclear whether the findings scale to larger LRMs (e.g., 32B or 72B parameters), where overthinking patterns may differ and the relative benefits of structured SFT and efficiency RL may be smaller or larger.
-
The work focuses exclusively on static Python chart editing with Matplotlib/Seaborn. It does not explore generalization to other visualization formats (e.g., SVG, HTML, interactive dashboards), diagram editing, or broader multimodal code generation tasks. While the core ideas around efficient reasoning should conceptually transfer, the specific benefits and optimal design choices may differ substantially in other domains.
-
While the efficiency reward clearly reduces token usage, the paper provides limited insight into exactly how it alters reasoning behavior. It is not fully clear whether the model truly learns to reason more efficiently or whether it instead learns to skip certain types of analysis or avoids harder examples. The brief error analysis in the appendix documents some shifts in error types but does not deeply characterize which reasoning steps are dropped, retained, or compressed. A more fine-grained qualitative analysis of reasoning content would strengthen the interpretation of the results.
-
The thinking budget implementation uses a hard token cutoff with an injected "stop reasoning" prompt, which is a relatively brute-force approach. More natural adaptive stopping mechanisms would be more elegant and potentially more robust. The RL stage also uses only a small discrete set of budget values rather than supporting continuous adaptive reasoning depth.
Questions for authors
Refer to weakness part.
3 comments
Response to Reviewer Vybt (Part 2/2)
W3: Task scope.
A3: Thank you for the question. REChart targets instruction-conditioned chart editing through executable Python visualization code. Our experiments on ChartEdit and ChartMimic evaluate this setting, alongside direct chart reproduction. SVG/HTML generation, interactive dashboards, and diagram editing involve additional output formats and task requirements beyond the scope of this work. Our claims concern efficient reasoning for the evaluated chart-generation and editing tasks. Cross-domain transfer is already acknowledged as a direction for future research.
W4: Qualitative analysis of reasoning behavior.
A4: Thank you for this suggestion. We have added qualitative results to Appendix G (marked in blue) across the reasoning error categories in Table 6: visual hallucination, edit intent drift, and details missing. These qualitative results illustrate different types of improvement after RL, including:
- More accurate data interpretation, such as correcting heatmap data alignment.
- Reduced edit-intent drift, such as avoiding unrequested titles.
- Better preservation of essential details, such as the exact chart type, point ordering after interpolation, and value annotations.
W5. Thinking-budget implementation.
A5: Thank you for the question.
-
Design rationale. Figure 6 in Appendix A.4 shows that Qwen3-VL-8B performs best at approximately 1,024–2,048 reasoning tokens, depending on task difficulty, and that longer reasoning does not consistently improve performance. These findings motivate explicit control of reasoning cost. As a practical implementation, we use hard truncation at 4,096 tokens, above the observed optimal range, with a stop-reasoning prompt to enforce the budget and a discrete set of budgets during RL to provide controlled training conditions at different cost levels.
-
Empirical effectiveness. Under this budget-control implementation, REChart improves chart-editing performance while reducing reasoning-token usage (Tables 4 and 5). These results demonstrate the effectiveness of our approach in achieving a favorable quality–efficiency trade-off.
-
Future extensions. Input-dependent stopping policies and soft penalties for budget overruns are promising directions for future work. These extensions could help models learn when to stop and adjust reasoning length more flexibly according to task difficulty. We have added this discussion to the Limitations section as future work (marked in blue).
Dear Reviewer Vybt,
Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period will end in less than 24 hours, we gently request that you read the rebuttal and check whether your concerns are addressed.
Thanks again for your time and help in improving the quality of this paper.
Regards from the authors.
Response to Reviewer Vybt (Part 1/2)
We sincerely thank the reviewer for the careful and constructive review.
W1: Reliability of evaluation and additional objective metrics.
A1: Thank you for highlighting the importance of objective evaluation. Our evaluation already includes execution success rates and ChartMimic’s low-level metrics (Text, Layout, Type, Color), which programmatically compare chart attributes extracted during code execution without GPT-4o judgment. We further report objective results on ChartEdit, covering image similarity (SSIM and MSE), code similarity (CodeBLEU), and chart-element matching adapted from ChartMimic (Text, Layout, Type, Color, and their average).
REChart ranks second in SSIM, MSE, and CodeBLEU, achieves the highest Text and Color scores, and obtains the highest average low-level score. It also improves over REChart-SFT on all reported metrics except Layout, which remains nearly unchanged (98.49 vs. 98.50). These results provide independent evidence of improved image and code similarity and chart-element fidelity, supporting the performance gains observed in our GPT-4o-based evaluation.
W2: Generalization to larger reasoning models.
A2: Thank you for the question. As stated in our Limitations section, we evaluate the full SFT and RL training pipeline on Qwen3-VL-8B due to computational constraints, leaving training at 32B and larger scales for future work. Our preliminary analysis already examines reasoning-budget effects across multiple model scales, including Qwen3-VL-32B. The paper therefore distinguishes the broader analysis of reasoning behavior from the evaluation of training gains on the 8B backbone.