acceptodds
Under review as a conference paper at ICLR 2027

REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models

Abstract

Chart editing requires inferring and modifying visualization code from a reference chart image based on an editing instruction, challenging fine-grained visual reasoning, instruction following, and executable code synthesis capabilities of MLLMs. Large reasoning models (LRMs) with extended Chain-of-Thought (CoT) reasoning are suitable for tackling such complex multimodal tasks. However, our preliminary study reveals an "inverted-U" relationship between reasoning length and chart-editing performance: Excessive reasoning often leads to "overthinking," where models drift toward hallucinated visual details or get stuck in redundant reasoning loops. To address the gap, we introduce REChart, a two-stage training framework that provides process-level supervision over intermediate reasoning steps, improving both editing fidelity and reasoning efficiency. First, we synthesize 200k high-quality reasoning trajectories for supervised fine-tuning from a large image-instruction-code pool, using a role-specialized agentic Reason-Score-Refine workflow that iteratively refine the chart code toward higher quality. Second, we optimize the model via reinforcement learning with two complementary rewards: a fidelity reward evaluating code correctness, visual fidelity, and structural consistency, and an efficiency reward that assigns each rollout a random thinking budget, truncates the reasoning process, and credits the final reasoning segment according to its contribution to the output. On the ChartEdit and ChartMIMIC benchmarks, our model achieves state-of-the-art chart-editing performance among open-source models of comparable scale, while mitigating overthinking and reducing average reasoning token usage by 79.0% under a maximum thinking budget of 16,384 tokens compared with the base model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Chart Editing, Large Reasoning Models, Reasoning Efficiency, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VybtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims at the performance of large reasoning models (LRMs) on chart editing tasks. The authors propose REChart, a two-stage training framework that improves both editing fidelity and reasoning efficiency. In the first stage, 200k high-quality structured reasoning trajectories are synthesized via a role-specialized multi-agent Reason-Score-Refine workflow. In the second stage, the model is further optimized via GRPO with a hybrid reward function. Experiments on the ChartEdit and ChartMIMIC benchmarks show that REChart achieves state-of-the-art performance among open-source models of comparable scale.

Critical strengths
  1. REChart introduces two complementary technical advances that address the overthinking problem at both the data and optimization levels. The multi-agent trajectory synthesis pipeline is a clever approach to producing high-quality reasoning supervision. The episode-level efficiency reward in the RL stage is also well motivated.

  2. The method delivers strong improvements on both dimensions. REChart outperforms all evaluated open-source baselines across three benchmark settings. The consistency of gains across multiple benchmark suites and task types further strengthens confidence in the results.

  3. The constructed corpus is a substantial and valuable resource for the research community. The multi-stage cascaded generation pipeline with rigorous quality filtering produces high-quality training data covering a diverse taxonomy of chart editing operations. Public release of this dataset will enable substantial further work on chart reasoning and efficient multimodal code generation.

Critical weaknesses
  1. Nearly all quality evaluation relies on GPT-4o as an automated judge, following the ChartEdit benchmark protocol. The paper does not complement this with deterministic, objective metrics such as exact code syntax accuracy, quantitative structural similarity of chart elements, or pixel-level visual distance metrics. VLM-based scoring is known to exhibit biases, variability, and imperfect alignment with human judgment for fine-grained visual details. Without supporting objective metrics, the magnitude of reported improvements is harder to independently verify and interpret.

  2. All experiments are conducted exclusively on Qwen3-VL-8B. It remains unclear whether the findings scale to larger LRMs (e.g., 32B or 72B parameters), where overthinking patterns may differ and the relative benefits of structured SFT and efficiency RL may be smaller or larger.

  3. The work focuses exclusively on static Python chart editing with Matplotlib/Seaborn. It does not explore generalization to other visualization formats (e.g., SVG, HTML, interactive dashboards), diagram editing, or broader multimodal code generation tasks. While the core ideas around efficient reasoning should conceptually transfer, the specific benefits and optimal design choices may differ substantially in other domains.

  4. While the efficiency reward clearly reduces token usage, the paper provides limited insight into exactly how it alters reasoning behavior. It is not fully clear whether the model truly learns to reason more efficiently or whether it instead learns to skip certain types of analysis or avoids harder examples. The brief error analysis in the appendix documents some shifts in error types but does not deeply characterize which reasoning steps are dropped, retained, or compressed. A more fine-grained qualitative analysis of reasoning content would strengthen the interpretation of the results.

  5. The thinking budget implementation uses a hard token cutoff with an injected "stop reasoning" prompt, which is a relatively brute-force approach. More natural adaptive stopping mechanisms would be more elegant and potentially more robust. The RL stage also uses only a small discrete set of budget values rather than supporting continuous adaptive reasoning depth.

Questions for authors

Refer to weakness part.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Vybt (Part 1/2)

We sincerely thank the reviewer for the careful and constructive review.

W1: Reliability of evaluation and additional objective metrics.

A1: Thank you for highlighting the importance of objective evaluation. Our evaluation already includes execution success rates and ChartMimic’s low-level metrics (Text, Layout, Type, Color), which programmatically compare chart attributes extracted during code execution without GPT-4o judgment. We further report objective results on ChartEdit, covering image similarity (SSIM and MSE), code similarity (CodeBLEU), and chart-element matching adapted from ChartMimic (Text, Layout, Type, Color, and their average).

ModelSSIM ↑MSE ↓CodeBLEU ↑Text ↑Layout ↑Type ↑Color ↑Low-level ↑
ChartCoder-8B0.66394890.220.294962.5594.5073.4171.7775.557
MSRL-8B0.66494407.140.331071.0194.8767.7767.0875.181
RRVF-ChartMimic-7B0.67985479.490.211364.6794.6172.7867.3574.854
GLM4.1V-9B0.66545204.270.271475.2398.4777.2674.1381.271
GPT-4o0.66665049.000.275276.7398.67‾\underline{98.67}79.4474.87‾\underline{74.87}82.427‾\underline{82.427}
Qwen3-VL-32B0.66434977.710.292376.96‾\underline{76.96}98.6879.38‾\underline{79.38}74.4482.365
Qwen3-VL-8B0.64895290.240.283366.9298.3976.0371.1378.116
REChart-SFT0.64834927.550.307672.3798.5075.6872.1879.681
REChart0.6685‾\underline{0.6685}4629.36‾\underline{4629.36}0.3271‾\underline{0.3271}78.0098.4976.9876.2782.434

REChart ranks second in SSIM, MSE, and CodeBLEU, achieves the highest Text and Color scores, and obtains the highest average low-level score. It also improves over REChart-SFT on all reported metrics except Layout, which remains nearly unchanged (98.49 vs. 98.50). These results provide independent evidence of improved image and code similarity and chart-element fidelity, supporting the performance gains observed in our GPT-4o-based evaluation.

W2: Generalization to larger reasoning models.

A2: Thank you for the question. As stated in our Limitations section, we evaluate the full SFT and RL training pipeline on Qwen3-VL-8B due to computational constraints, leaving training at 32B and larger scales for future work. Our preliminary analysis already examines reasoning-budget effects across multiple model scales, including Qwen3-VL-32B. The paper therefore distinguishes the broader analysis of reasoning behavior from the evaluation of training gains on the 8B backbone.

AuthorsOpenReview ↗

Response to Reviewer Vybt (Part 2/2)

W3: Task scope.

A3: Thank you for the question. REChart targets instruction-conditioned chart editing through executable Python visualization code. Our experiments on ChartEdit and ChartMimic evaluate this setting, alongside direct chart reproduction. SVG/HTML generation, interactive dashboards, and diagram editing involve additional output formats and task requirements beyond the scope of this work. Our claims concern efficient reasoning for the evaluated chart-generation and editing tasks. Cross-domain transfer is already acknowledged as a direction for future research.

W4: Qualitative analysis of reasoning behavior.

A4: Thank you for this suggestion. We have added qualitative results to Appendix G (marked in blue) across the reasoning error categories in Table 6: visual hallucination, edit intent drift, and details missing. These qualitative results illustrate different types of improvement after RL, including:

  1. More accurate data interpretation, such as correcting heatmap data alignment.
  2. Reduced edit-intent drift, such as avoiding unrequested titles.
  3. Better preservation of essential details, such as the exact chart type, point ordering after interpolation, and value annotations.

W5. Thinking-budget implementation.

A5: Thank you for the question.

  1. Design rationale. Figure 6 in Appendix A.4 shows that Qwen3-VL-8B performs best at approximately 1,024–2,048 reasoning tokens, depending on task difficulty, and that longer reasoning does not consistently improve performance. These findings motivate explicit control of reasoning cost. As a practical implementation, we use hard truncation at 4,096 tokens, above the observed optimal range, with a stop-reasoning prompt to enforce the budget and a discrete set of budgets during RL to provide controlled training conditions at different cost levels.

  2. Empirical effectiveness. Under this budget-control implementation, REChart improves chart-editing performance while reducing reasoning-token usage (Tables 4 and 5). These results demonstrate the effectiveness of our approach in achieving a favorable quality–efficiency trade-off.

  3. Future extensions. Input-dependent stopping policies and soft penalties for budget overruns are promising directions for future work. These extensions could help models learn when to stop and adjust reasoning length more flexibly according to task difficulty. We have added this discussion to the Limitations section as future work (marked in blue).

Dear Reviewer Vybt,

Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period will end in less than 24 hours, we gently request that you read the rebuttal and check whether your concerns are addressed.

Thanks again for your time and help in improving the quality of this paper.

Regards from the authors.

Reviewer_gYsHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

REChart studies chart editing: from a reference chart image and an edit instruction, without access to the source code, the model outputs executable code for the edited chart. A preliminary study on ChartEdit reports an inverted-U relation between thinking budget and performance. The method builds a 709,784-triplet corpus from arXiv charts and synthesizes 207,936 reasoning trajectories with a multi-agent Reason-Score-Refine workflow that keeps only score-improving reasoning episodes. It then applies GRPO with a fidelity reward (equal-weight CodeBLEU, SVG-structure and embedding similarity) and an efficiency reward for the fidelity gain of the last 512-token reasoning segment under a random thinking budget. Built on Qwen3-VL-8B-Thinking, it reaches 79.7 on ChartEdit and 77.2 on Direct Mimic with 79.0% fewer reasoning tokens.

Critical strengths

1, A clear problem-to-solution narrative, partly validated by the post-training error analysis (visual hallucinations 125 → 105 → 84).

2,Substantial data engineering and strong 8B results, outperforming Qwen3-VL-32B-Thinking on ChartEdit (79.7 vs 74.6).

Critical weaknesses

1, The fidelity reward is heuristic. The three dimensions and their equal weights are motivated only conceptually and are not ablated.The visual term compares the output with the instruction plus the pre-edit reference image rather than with the rendered target, which can favor under-editing, and CodeBLEU is computed against model-generated reference code. Since the same dimensions also select the SFT trajectories, the pipeline risks overfitting to its own metric.

2, The preliminary study is not rigorous enough to be a main contribution. The "optimal" budget is chosen post hoc, the "longest" budget is unspecified, and no variance is reported. Because non-executable code scores zero on every dimension, the drop at long budgets may come from execution failures after truncation rather than from reasoning drift.

3, The efficiency reward trades accuracy for cost. Budgeted GRPO without it scores higher on ChartEdit (80.24 vs 79.71) and Direct Mimic (78.96 vs 77.15); most of the token reduction already comes from SFT (−70.8%); and the −79.0% figure is measured at a 16,384-token budget while accuracy is evaluated at 4,096.

4, Possible ground-truth leakage. The Planner receives the ground-truth code, the Critic is marked GT-visible in Figure 3, and the Critic's corrections are inserted as the model's own first-person self-corrections. The student thus imitates reasoning informed by information that is unavailable at test time.

Questions for authors

A, Can you ablate the three fidelity terms and their weights, and justify using the pre-edit reference rather than the rendered target in the visual term?

B, How do you prevent ground-truth details from leaking into the SFT reasoning text, and how does the model perform with GT-hidden planning?

6 comments
AuthorsOpenReview ↗

Response to Reviewer gYsH (Part 1/3)

We sincerely thank the reviewer for the careful and constructive review.

W1 & Q_A: Fidelity reward components, weights, and visual comparison target

A1: Thank you for suggesting these ablations. We conducted additional RL runs initialized from the SFT model. Each run used 50 updates with a batch size of 256, corresponding to 12,800 training sample instances, and otherwise followed the hyperparameters used for our model in the paper. We examined the reward components, their relative weights, and the visual comparison target.

For compactness, we report Overall scores for ChartEdit without source code (CE), ChartMIMIC Customized (CM-C), and ChartMIMIC Direct (CM-D). Overall is the mean of the code-level and chart-level scores for CE, and the mean of the low-level and high-level scores for CM-C and CM-D. Values below are calculated from the reported component scores and rounded to two decimal places.

  • Reward component ablation. Removing the visual term lowers Overall on all three benchmarks. Removing the structural term slightly improves CE but lowers both CM-C and CM-D, whereas removing the code term improves CM-C but lowers CE and CM-D. These results support complementary contributions across tasks. The full reward achieves the highest CM-D Overall and remains competitive on CE and CM-C.

    ConfigurationCE OverallCM-C OverallCM-D Overall
    Full reward74.1462.7570.94
    w/o sviss_{\mathrm{vis}}72.1062.0670.45
    w/o sstructs_{\mathrm{struct}}74.2261.5170.29
    w/o scodes_{\mathrm{code}}73.5863.2870.26
  • Reward weight sensitivity. No tested unequal weighting improves performance across all three tasks. A higher structural weight gives the best CE Overall, a higher visual weight gives the best CM-C Overall, and equal weighting gives the best CM-D Overall. These results support equal weighting as a reasonable default across the evaluated tasks.

    Weights (code / structure / visual)CE OverallCM-C OverallCM-D Overall
    0.33 / 0.33 / 0.3374.1462.7570.94
    0.50 / 0.25 / 0.2573.6562.7070.72
    0.25 / 0.50 / 0.2574.7062.1170.24
    0.25 / 0.25 / 0.5074.3562.9370.21
  • Visual comparison target. Our visual reward uses the pre-edit reference image to assess preservation of chart content and visual properties, together with the instruction to assess the requested modification. To evaluate this heuristic choice empirically, we compared it with a variant using the rendered target image under the same 50-update RL setup.

    Visual reward inputCE OverallCM-C OverallCM-D Overall
    Reference image + instruction74.1462.7570.94
    Target image73.3663.0370.83

    Using the reference image and instruction yields higher Overall on CE and CM-D, while using the rendered target image performs better on CM-C. The target-image variant therefore provides no consistent improvement across the evaluated tasks, supporting our current choice.

AuthorsOpenReview ↗

Response to Reviewer gYsH (Part 2/3)

W3: Separating RL Gains from the Efficiency Reward’s Trade-off

A3: Thank you for the question. We have updated the manuscript with matched-budget evaluations at 4,096 and 16,384 tokens, reporting Overall scores and mean thinking-token usage from the same runs. The corresponding revisions are marked in blue.

  • Matched-budget re-evaluation. We first compare the full RL model with REChart-SFT to measure the gains from the complete RL stage. ChartMIMIC rows use all 600 examples per task, including generated-code failures. The following comparison measures the complete RL stage beyond SFT, not the efficiency reward in isolation.

    TaskBudgetSFT Overall / tokensFull RL Overall / tokensScore changeToken reduction
    ChartEdit409673.4 / 4,067.679.7 / 2,997.2+6.326.3%
    Direct409665.75 / 3,446.076.39 / 2,889.0+10.6516.2%
    Customized409660.39 / 3,541.269.16 / 2,451.3+8.7830.8%
    ChartEdit1638472.9 / 5,909.078.4 / 3,465.8+5.541.3%
    Direct1638466.30 / 8,575.876.03 / 5,556.1+9.7335.2%
    Customized1638461.36 / 6,738.666.69 / 4,036.8+5.3240.1%

    For example, at 16,384 on ChartEdit, the complete RL stage improves Overall from 72.9 to 78.4 while reducing mean thinking length from 5,909.0 to 3,465.8 tokens, a 41.3% reduction. Relative to Qwen3-VL-8B at that same budget, the gain is 11.1 points with 48.6% fewer thinking tokens. These matched-run results show an RL-stage improvement beyond SFT, but do not attribute all of that improvement to the efficiency term.

  • Marginal effect of efficiency reward. We then compare Budgeted GRPO without the efficiency reward against the full model at the same budget.

    TaskBudgetBudgeted GRPO Overall / tokensFull RL Overall / tokensScore changeToken reduction
    ChartEdit409680.24 / 3,854.179.71 / 2,997.2-0.5322.2%
    Direct409677.42 / 2,984.476.39 / 2,889.0-1.033.2%
    Customized409665.81 / 2,915.269.16 / 2,451.3+3.3615.9%
    ChartEdit1638479.75 / 5,081.478.41 / 3,465.8-1.3431.8%
    Direct1638477.94 / 5,902.676.03 / 5,556.1-1.915.9%
    Customized1638463.27 / 4,571.966.69 / 4,036.8+3.4211.7%

    The efficiency reward reduces thinking-token usage in all six comparisons. At a budget of 16,384, it reduces thinking tokens by 31.8% on ChartEdit and 5.9% on Direct Mimic, with Overall decreases of 1.34 and 1.91 points, respectively. On Customized Mimic, it reduces thinking tokens by 11.7% while improving Overall by 3.42 points. These results show a task-dependent trade-off between performance and reasoning cost.

AuthorsOpenReview ↗

Response to Reviewer gYsH (Part 3/3)

W4 & Q_B: Potential ground-truth leakage in SFT trajectories

A4: Thank you for raising this concern. We audited the existing trajectories for potential leakage and compared the full pipeline with a GT-hidden alternative to assess its effects on downstream performance and trajectory construction.

  • Leakage audit. Ground-truth access during trajectory construction creates a risk that the reasoning text contains details unavailable from test-time inputs. We audited 500 Planner outputs and 1,000 Critic outputs from the previously collected dataset, using GPT-6 luna to flag potential leakage and human review to assess the reported cases.

    Audited outputSample sizeAutomatically flaggedHuman-verified leakage
    Planner5001.60%0.20%
    Critic1,0004.30%1.00%

    The audit identified a low rate of confirmed leakage. Since human review focuses on automatically flagged outputs, these results do not establish that all remaining outputs are leakage-free.

  • GT-hidden comparison. We additionally constructed trajectories with a GT-hidden pipeline in which neither the Planner nor the Critic receives the target code. Ground-truth code remains available to the external scoring stage for evaluating and selecting candidates.

    • Downstream SFT performance. To examine whether the construction advantage translates into downstream gains, we trained SFT models on the two trajectory sources using the same training budget of 100 updates and an effective batch size of 256, corresponding to 25,600 training sample instances per run.

      Trajectory sourceCE OverallCM-C OverallCM-D Overall
      Full pipeline67.8758.7860.99
      GT-hidden pipeline69.0955.2958.09

      GT-hidden training obtains higher CE Overall, while the full pipeline obtains higher CM-C and CM-D Overall. Target-code access during construction thus does not yield a uniform downstream advantage, and useful SFT supervision can also be obtained with GT-hidden planning and critique.

    • Trajectory construction success. We also evaluated trajectory construction success on the sampled cases. Pass@k is the proportion of cases with at least one candidate reaching the selection threshold of 65 within the first k rounds.

      PipelinePass@3Pass@6Pass@9Avg. tokens
      Full24.75%35.17%42.58%2086.33
      GT-hidden9.67%13.50%15.50%1779.10

      The full pipeline achieves higher construction success at every round budget. At nine rounds, it yields qualifying trajectories for 42.58% of cases versus 15.50% for GT-hidden, a difference of 27.08 percentage points. Although GT-hidden uses fewer tokens on average, the full pipeline produces qualifying training trajectories for a larger proportion of source tasks within the same round limit.

Reviewer_cq9HOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper observes an inverted-U relationship between reasoning length and chart-editing performance for large reasoning models (LRMs), and proposes REChart, which has two parts:

  1. Data.
    • About 709K chart–code–edit triplets built from roughly 100K CC BY 4.0 arXiv papers (2024), organized by a six-category edit taxonomy.
    • About 200K SFT trajectories produced by a multi-agent Reason-Score-Refine (RSR) pipeline: a Planner (which sees the GT code), a Reasoner, a Scorer, a Critic whose refinements are written into the trajectory as self-correction, and a Finalizer.
  2. RL.
    • GRPO on 82K samples with budgeted rollouts: thinking is truncated at a random budget of 512–5120 tokens and the model then answers after a "Time is up" prompt.
    • The reward combines fidelity (code, structure, and visual embedding similarity) with an MRT-style episode-level efficiency reward.

The 8B model outperforms open models of similar size on ChartEdit and ChartMimic (direct and customized). It improves on the base model by 13.3 points on average while cutting mean reasoning tokens by 79.0%. Ablations add SFT, Base GRPO, Budgeted GRPO, and the efficiency reward cumulatively.

Critical strengths

S1. A large, realistic data resource and a complete, well-documented pipeline. The corpus starts from real arXiv charts and is filtered in stages: 854K extracted figures → 622K charts → 521K chart–code pairs → 709K triplets, with counts reported at each stage (Table 10). The authors commit to releasing it. It is closer to real-world chart distributions than purely synthetic chart corpora. For the chart-to-code community, this resource may be the paper's most lasting contribution.

S2. Transparent RL configuration and a cumulative ablation with token accounting. The RL hyperparameters are fully listed (Table 17: chunk size 512, budget sampled uniformly from {1..10}×512, KL 0.01, group size 8). Every ablation stage reports both score and token count (Tables 4–5), and the headline numbers (13.3 points, 79.0%, the 25.5–37.0% reduction relative to Base GRPO) can be reproduced from the tables. This makes the efficiency–fidelity trade-off measurable, although as discussed below the paper overstates it.

S3. A timely problem with motivation across model families. Performance degrading with longer reasoning is shown for Qwen3-VL 4B / 8B / 32B, GLM-4.1V, and Gemini 3.1 Pro. Figure 1b analyzes how error types change with budget. Overthinking in vision-to-code editing is understudied, and the paper makes a reasonable case that it matters.

Critical weaknesses

W1. The evaluation protocol is inconsistent and probably unfavorable to the baselines.

  • Two budgets. App. E.2 sets the thinking budget for all open-source reasoning models to 4096 tokens. Table 5 and the abstract's 79.0% token figure use a 16,384 budget. Table 4 does not say which budget it uses.
  • The 4096 setting suits REChart. Left alone, the base Qwen3-VL-Thinking models use 6.5K–9.6K tokens (Table 5), so at 4096 they are routinely cut off by "Time is up". REChart was trained specifically on 512–5120 truncation with the same prompt, so it has learned to answer well after truncation.
  • The paper's own evidence argues against a single budget. Figure 6a shows the optimal budget differs across models, yet the main tables use one fixed budget.
  • Consequence. The claim that the 8B model beats 32B-Thinking, and the efficiency–accuracy trade-off, may partly come from this protocol, and accuracy and tokens may come from different settings.

W2. Most of the efficiency gain comes from SFT imitation, not from the RL mechanism. Of the total reduction (8,328 → 1,749 tokens), the SFT step alone (8,328 → 2,436) accounts for about 90% (5,892 of 6,579 tokens). The synthetic trajectories themselves average about 2,086 tokens (Table 12), so the shorter outputs after SFT largely just imitate short teacher trajectories. The abstract presents the 79.0% alongside the RL efficiency reward, which invites readers to credit RL for it.

Questions for authors
  1. At what thinking budget were the scores in Tables 2–4 obtained? Do Table 4 scores and Table 5 tokens come from the same evaluation run?
  2. Please report accuracy and tokens together, at both 4096 and 16,384 (or no truncation), for REChart, the base model, and all open reasoning baselines. Please also report each baseline at its own optimal budget, and an accuracy-vs-budget curve from 512 to 16,384.
  3. Did you run near-duplicate detection (perceptual hash plus embedding) or paper-level (arXiv ID) exclusion between the 622K training charts and the ChartEdit/ChartMimic test images? Were the ChartEdit samples used to build the taxonomy drawn from the test split?
  4. For Figure 1b and Table 6, were errors annotated by humans or by an LLM, how many traces per budget, and with what κ? What were the pass rate and judge agreement in the 1,000-pair human audit? What is the inter-expert agreement for the taxonomy clustering?
7 comments
AuthorsOpenReview ↗

Response to Reviewer cq9H (Part 1/6)

We sincerely thank the reviewer for the careful and constructive review.

Q1: Clarifying the Budgets Used for Scores and Token Counts

Thank you for the question.

  • Thinking budget for Tables 2–4. The scores for open-source reasoning models in Tables 2–4 were obtained with a maximum thinking budget of 4,096 tokens, following Appendix E.2.

  • Evaluation settings for Tables 4 and 5. Table 4 scores and Table 5 token statistics were obtained from different evaluation runs, with maximum thinking budgets of 4,096 and 16,384 tokens, respectively. Table 5 reports actual mean thinking-token usage under the 16,384-token cap. The original 79.0% reduction therefore describes reasoning-token savings under this setting.

  • Additional matched-budget results. To clarify the relationship between performance and reasoning cost, we provide additional results pairing task scores with actual mean thinking-token usage at both 4,096 and 16,384 tokens. These results, together with the complete six-budget comparison, are reported in our response to Q2.

AuthorsOpenReview ↗

Response to Reviewer cq9H (Part 2/6)

Q2: Performance and Reasoning Cost Across Matched Budgets

Thank you for requesting a budget-matched comparison. We report task scores and average thinking-token usage at matched budgets of 4,096 and 16,384 on ChartEdit and ChartMIMIC, compare each model at its best observed budget, and evaluate performance across six budgets from 512 to 16,384.

  • Matched-budget comparison. At a thinking budget of 4,096, REChart-RL improves over the base model, Qwen3-VL-8B, by 6.2–18.6 points across the three tasks and reduces thinking-token usage by 27.8–38.0% on the two ChartMIMIC tasks. At 16,384, it improves scores by 5.1–16.6 points while reducing thinking-token usage by 43.8–58.9% across all three tasks.

    • Table 1. Scores and thinking-token usage at the 4096 and 16384 thinking budgets.

      ModelBudgetChartEdit score / tokensDirect score / tokensCustomized score / tokens
      Qwen3-VL-8B4,09667.30 / 2,271.457.75 / 3,998.162.94 / 3,952.9
      Qwen3-VL-32B4,09674.60 / 3,997.566.02 / 4,061.269.49 / 4,066.6
      GLM-4.1V4,09668.60 / 3,165.767.36 / 3,675.862.28 / 3,698.5
      REChart-SFT4,09673.43 / 4,035.465.75 / 3,445.560.39 / 3,540.7
      REChart-RL4,09679.71 / 2,997.276.39 / 2,888.269.16 / 2,450.4
      Qwen3-VL-8B16,38467.26 / 6,493.059.40 / 10,470.061.60 / 9,817.4
      Qwen3-VL-32B16,38473.80 / 11,362.865.90 / 15,006.369.56 / 15,169.1
      GLM-4.1V16,38468.40 / 3,740.765.59 / 5,256.361.96 / 5,286.2
      REChart-SFT16,38472.91 / 5,982.366.30 / 8,575.261.36 / 6,737.9
      REChart-RL16,38478.41 / 3,465.876.03 / 5,555.266.69 / 4,036.0
    • We additionally compare the three RL variants at the same budgets. No variant consistently achieves the highest score across tasks. The efficiency reward reduces thinking tokens by 16.3–47.3% relative to Base GRPO and by 3.2–28.2% relative to Budgeted GRPO, providing a consistent reduction in reasoning cost.

      Table 2. Scores and average thinking-token usage of RL ablations under 4096 and 16384 thinking budget.

      ModelBudgetChartEdit score / tokensDirect score / tokensCustomized score / tokens
      Base GRPO4,09679.60 / 3,979.176.50 / 3,449.367.25 / 3,522.8
      Budgeted GRPO4,09680.24 / 3,854.177.42 / 2,983.665.81 / 2,914.5
      REChart-RL4,09679.71 / 2,997.276.39 / 2,888.269.16 / 2,450.4
      Base GRPO16,38478.80 / 5,635.575.65 / 8,254.467.59 / 7,662.6
      Budgeted GRPO16,38479.75 / 5,081.477.94 / 5,901.663.27 / 4,571.0
      REChart-RL16,38478.41 / 3,465.876.03 / 5,555.266.69 / 4,036.0
  • Comparison at each model’s best observed budget. We select the highest-scoring budget for each model on each task from the six-budget sweep. Under this comparison, REChart-RL exceeds the base model by 12.06 points on ChartEdit, 17.25 points on ChartMIMIC Direct, and 5.72 points on ChartMIMIC Customized.

    • Table 3. Each model at its best observed budget.

      ModelChartEdit best budget / score / tokensDirect best budget / score / tokensCustomized best budget / score / tokens
      Qwen3-VL-8B2,048 / 67.65 / 1,458.916,384 / 59.40 / 10,470.08,192 / 63.44 / 6,512.4
      Qwen3-VL-32B1,024 / 76.00 / 1,022.38,192 / 68.41 / 7,985.216,384 / 69.56 / 15,169.1
      GLM-4.1V4,096 / 68.60 / 3,165.72,048 / 68.83 / 2,038.31,024 / 62.59 / 1,024.0
      REChart-SFT4,096 / 73.43 / 4,035.48,192 / 68.08 / 5,295.72,048 / 62.97 / 2,032.4
      REChart-RL4,096 / 79.71 / 2,997.2512 / 76.65 / 512.04,096 / 69.16 / 2,450.4
  • Performance across reasoning budgets. We evaluate thinking budgets of 512, 1,024, 2,048, 4,096, 8,192, and 16,384. REChart-RL outperforms the base model and REChart-SFT on all three tasks at every evaluated budget. It also outperforms all other evaluated open reasoning baselines on ChartEdit and ChartMIMIC Direct at every budget.

    • Table 4a. ChartEdit.

      Model5121,0242,0484,0968,19216,384
      Qwen3-VL-8B65.8567.2667.6567.3067.4767.26
      Qwen3-VL-32B75.7076.0074.8074.6073.8073.80
      GLM-4.1V61.7065.8067.0068.6068.6068.40
      REChart-SFT71.8873.0372.8373.4372.6872.91
      REChart-RL78.2778.5278.8779.7178.5078.41
AuthorsOpenReview ↗

Response to Reviewer cq9H (Part 3/6)

Q2: Performance and Reasoning Cost Across Matched Budgets

Table 4b. ChartMIMIC Direct.

Model5121,0242,0484,0968,19216,384
Qwen3-VL-8B56.9158.3857.5057.7557.8159.40
Qwen3-VL-32B64.1166.3366.4666.0268.4165.90
GLM-4.1V65.1068.2468.8367.3666.4365.59
REChart-SFT65.8464.1767.4365.7568.0866.30
REChart-RL76.6576.3075.7276.3976.3376.03

Table 4c. ChartMIMIC Customized.

Model5121,0242,0484,0968,19216,384
Qwen3-VL-8B60.1061.1960.7362.9463.4461.60
Qwen3-VL-32B65.9166.3367.6169.4968.3769.56
GLM-4.1V59.5662.5961.8162.2861.4361.96
REChart-SFT59.0861.9062.9760.3962.5461.36
REChart-RL65.6765.6864.9169.1667.7866.69

Comments on the paper

2 comments

Dear reviewers,

Please be aware that little time remains for the author-reviewer discussion. As the deadline is approaching, we kindly request that you refrain from asking for additional experiments. Please summarize your final position concisely.

Sincerely, AC

AuthorsOpenReview ↗

Summary of Revisions

Dear Area Chair and Reviewers,

We sincerely thank you for your time and valuable feedback during the review and rebuttal period.

We are grateful that reviewers recognized the strengths of our work, including: (1) the substantial and valuable chart-editing corpus (Reviewers cq9H, gYsH, and Vybt), (2) the agentic reasoning construction pipeline and well-motivated efficiency reward (Reviewer Vybt), (3) the strong performance of the model (Reviewers gYsH and Vybt), and (4) the timely problem and clear motivation (Reviewers cq9H and gYsH).

During the rebuttal period, we have addressed the reviewers' concerns through additional experiments, analyses, and clarifications. Key revisions have been incorporated into the manuscript, with additional results provided in our responses.

Reviewer Feedback

  • Reviewer gYsH appreciated our detailed responses and extensive additional experiments. The reviewer confirmed that all the questions were resolved and maintained a positive score.

  • Reviewer cq9H raised concerns about budget consistency, the attribution of efficiency gains, potential training–test overlap, and annotation reliability. We provided budget-controlled comparisons, training-stage ablations, overlap-exclusion analyses, and annotation-agreement statistics. The reviewer has not yet provided follow-up feedback.

  • Reviewer Vybt requested objective evaluation metrics, deeper analysis of reasoning behavior, and clarification of generalization and budget control. We provided additional objective metrics and qualitative examples, and clarified the scope and limitations of our experiments. The reviewer has not yet provided follow-up feedback.

Additional Experiments and Analyses

  • Matched-Budget Evaluation: We reported scores and thinking-token usage at matched budgets, compared performance across six budgets, and evaluated each model at its best observed budget. (Reviewer cq9H W1 and Q1–Q2, Table 4)

  • Preliminary-Study Validation: We evaluated the corrected longest-budget condition, reported paired bootstrap confidence intervals, and repeated the budget comparison on the same 660 cases with valid renderings throughout. (Reviewer gYsH W2, Figure 1a and Table 5)

  • Human Validation of Data Quality: We reported the 1,000-example human audit, including annotator agreement and comparison with the automatic judge. (Reviewer cq9H Q4)

  • Trajectory-Leakage Analysis: We audited Planner and Critic outputs for potential leakage and evaluated both trajectory-construction success and downstream SFT performance with ground-truth-hidden planning and critique. (Reviewer gYsH W4 and Q_B)

  • Fidelity-Reward Ablation: We tested removal of individual reward components, alternative component weights, and rendered-target supervision for the visual reward. (Reviewer gYsH W1 and Q_A)

  • Training–Test Overlap Analysis: We screened training charts against benchmark images and recomputed model scores after excluding flagged test entries. (Reviewer cq9H Q3)

  • Additional Objective Evaluation: We supplemented LLM-based evaluation with SSIM, MSE, CodeBLEU, and ChartMIMIC low-level metrics. (Reviewer Vybt W1)

  • Qualitative Reasoning Analysis: We added examples illustrating improvements in visual interpretation, edit-intent adherence, and preservation of essential chart details after RL. (Reviewer Vybt W4, Appendix G)

Clarifications and Scope

  • Preliminary-Study Settings: We clarified the thinking and code-generation budgets and specified how the best observed budgets were selected. (Reviewer gYsH W2, Figure 1a, Appendix A.2–A.3, Table 5)

  • Headline Performance and Efficiency Results: We revised the headline claims to report score improvements and token reductions under the same thinking budget. (Reviewer cq9H W1–W2, Reviewer gYsH W3, Abstract and Conclusion)

  • Use of ChartEdit in Data Construction: We clarified that ChartEdit examples informed the general editing taxonomy rather than serving as content or instruction templates for data generation. (Reviewer cq9H Q3)

  • Generalization Scope: We clarified that the complete training pipeline is evaluated on the 8B backbone for Python chart generation and editing, with larger-scale training and broader visualization formats left for future work. (Reviewer Vybt W2–W3, Limitations)

  • Thinking-Budget Control: We explained the rationale for explicit budget control and added adaptive stopping policies and soft budget penalties as future directions. (Reviewer Vybt W5, Limitations)

Thank you once again to all the reviewers for your constructive feedback, which has helped us strengthen the experimental evidence and improve the clarity of our manuscript. We believe these additional experiments and revisions have strengthened our work and adequately addressed the concerns raised.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.