acceptodds
Under review as a conference paper at ICLR 2027

ChartRevise: A Dataset and Evaluation Protocol for Exact Chart Editing via Code

Abstract

Chart editing requires cross-modal edit grounding, realizing a requested visual change in the code that draws it, with necessary related updates and without altering unrelated content. Existing benchmarks emphasize either code executability or chart quality, but their metrics do not clearly distinguish request completion from missed coupled updates and gratuitous changes. We introduce ChartRevise, a structured dataset and evaluation protocol for exact program-grounded chart editing. For dataset construction, we build on the grammar of graphics to systematically cover chart-editing operations, using source-program checks to verify their applicability across chart types and libraries. To improve edit exactness, our pipeline checks individual requirements and guides repair or exclusion when they are unmet. The resulting dataset contains 92,438 records covering 344 edit types across 20 chart types and three plotting libraries. For evaluation, our reference-free protocol separately measures atomic requirement completion, identifies gratuitous changes, and detects missed coupled updates. These checks are combined with successful execution and rendering to determine exact-edit success. Across five models and four external benchmarks, fine-tuning yields relative gains of 16% in mean requirement recall and 22% in mean exact-edit rate.

Primary area:
datasets and benchmarks
Keywords:
Dataset, Benchmark, Chart Editing, Multi-Modal

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LNBiOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ChartRevise, a dataset and evaluation protocol for program-grounded chart editing. It evaluates whether models complete explicit instructions, make necessary coupled updates, and avoid gratuitous changes, covering both under-editing and over-editing. The dataset contains 92,438 examples across 20 chart types, three plotting libraries, and 344 edit types. Its reference-free evaluation combines LLM-based code judging with execution checks. Experiments across five models and four external benchmarks show stronger requirement completion and exact-edit performance, while improvements in preventing unrelated edits are less consistent.

Critical strengths

1.The dataset is constructed based on chart grammar, which is an interesting idea. It is diverse in chart types, edit types, and plotting libraries, with automatic audits to improve data quality.

2.The evaluation considers both under-editing and over-editing, including missed coupled updates and unnecessary changes. This is very interesting and diagnostically useful.

Critical weaknesses

1.The LLM judge only checks code without seeing the rendered charts. This can miss visual problems such as overlapping elements, even when the code looks correct.

2.Fine-tuning reduces unnecessary edits in only half of the comparisons. The results are more convincing for reducing under-editing than over-editing.

3.Human validation finds only 69.5% of compound examples fully correct, which raises concerns about using them to teach exact editing.

Questions for authors

1.How much would the exact-edit results change if the judge also checked the rendered charts?

2.Why does fine-tuning increase unnecessary edits in some settings? Are there common types of over-editing?

3.What are the main errors in the compound training examples, and could these errors explain some of the remaining failures after fine-tuning?

4 comments
AuthorsOpenReview ↗

Response (Part 1)

We sincerely thank the reviewer for the thoughtful and constructive feedback and for recognizing the contributions of our work. We appreciate the insightful suggestions and have conducted additional analyses to address the concerns, as detailed below.

W1 & Q1. Limitations of Code-Only Evaluation

R: We thank the reviewer for highlighting this limitation. Our code-only protocol supports fine-grained verification of explicit requirements and coupled updates, including in non-executable programs and for changes that are difficult to assess visually. We agree that rendered charts can reveal visual defects that code inspection may miss.

Robustness of exact-edit results. To directly assess the impact of visual evidence, we re-evaluated outputs from all five model families using both code and rendered charts. On ChartEdit, the fine-tuned exact-edit rates change by only 1–3 percentage points for four of five models. Importantly, fine-tuning continues to improve exact-edit rates for all five models on both ChartEdit and Chart2Code-L2 under code-and-image evaluation. Thus, the direction of the main fine-tuning gains is preserved when visual evidence is included.

Comparison of judge strategies. We evaluated all three strategies on reference programs from four benchmarks:

Reference programs judged complete (%)ChartEditChart2Code-L2ChartSyncChartM3\mathrm{ChartM}^{3}
Visual-only79.675.795.634.5
Visual + code95.292.798.577.0
Code-only (ours)93.681.198.381.3

Visual-only judging identifies fewer reference edits, particularly on ChartM3\mathrm{ChartM}^{3}, where subtle or non-visible changes can be difficult to verify. Incorporating code evidence substantially increases reference AppliedRate, supporting the importance of program-level inspection.

Scope of visual issues. Our protocol evaluates visual constraints when they are part of the editing requirements. For example, 57 of 1,405 ChartEdit tasks explicitly require avoiding overlap. Nevertheless, implicit layout defects may remain undetected by code-only judging. We will clarify this limitation and distinguish exact-edit verification from general visual-quality assessment, which can be evaluated using complementary metrics such as ChartScore from ChartEdit.

W2 & Q2. Inconsistent Improvements in Over-Editing

R: We agree that fine-tuning improves requirement completion more consistently than it reduces gratuitous changes. However, these regressions do not necessarily indicate unreliable supervision. Our analysis suggests a completion–preservation trade-off: fine-tuning encourages models to implement more requested operations, which can increase the opportunities for unintended changes, particularly when models have limited capacity to manage complex requests.

This trade-off is particularly evident in two settings:

  1. Weaker base models on long requests. On Chart2Code-L2, weaker base models leave most requirements unfulfilled and introduce fewer gratuitous changes simply because they perform fewer edits. Fine-tuning enables them to perform more operations, improving requirement completion and coupled-update recall (Table 3), but also increasing gratuitous changes. For Phi-3.5-Vision, requirement recall improves at every instruction depth, including beyond the training distribution, while gratuitous changes increase only for requests with at least 15 requirements (Figure 6). This suggests that preservation becomes more challenging as request complexity exceeds the training distribution.
  2. Near-ceiling benchmarks. On ChartSync, base completion is already high, leaving limited room for further improvement. However, gratuitous changes can still increase slightly, even when completion gains are small.

Common over-editing errors. Our additional analysis identifies modifications to text or labels (31%) and changes to colors or styles (28%) as the most frequent categories of gratuitous edits.

These findings help explain why improving requirement completion does not necessarily improve preservation. They also identify complex multi-operation requests as an important remaining challenge for exact chart editing.

AuthorsOpenReview ↗

Response (Part 2)

W3 & Q3. Compound-Data Noise and Its Impact on Fine-Tuning

R: The compound records still provide high-quality editing supervision, with 95.6% of their individual requirements correctly satisfied. The lower task-level correctness (69.5%) reflects our strict all-or-nothing criterion over 2–12 requirements. At 96% per-requirement correctness, 79.3% of records are expected to satisfy every requirement under an independent-error assumption (measured: 81.5%), even before preservation is considered.

The main remaining errors are gratuitous changes, typically code left over from earlier attempts or reasoning comments (79% of defective compound records). The human audit also applies stricter preservation judgments, counting unrequested code changes even when they do not affect the rendered chart.

Impact on fine-tuning. These errors may contribute to some remaining failures, but are unlikely to be their primary cause, for three reasons:

  1. Limited training noise. Clear editing defects in compound records account for only 0.75% of the training data.
  2. Persistent completion failures. Among tasks still incomplete after fine-tuning, 61–84% on ChartEdit and 87–99.8% on Chart2Code-L2 were also incomplete for the corresponding base models. Thus, most remaining completion failures reflect limitations already present in the base models rather than failures newly introduced by fine-tuning.
  3. Model capability limitations. Preservation and executability remain challenging for weaker models on longer requests. As discussed in W2 & Q2, this is consistent with a completion–preservation trade-off.

Despite the remaining noise, fine-tuning improves exact-edit rates for all five models on ChartEdit (Table 3), demonstrating that the editing supervision provides useful learning signals.

Dear reviewer,

We sincerely thank you again for the thoughtful and constructive feedback. We have carefully addressed the concerns through additional experiments, analyses, and clarifications. We believe your suggestions have helped us strengthen the paper and better communicate its contributions.

We truly appreciate the time and effort you have dedicated to reviewing our work. And, we would be happy to address any further concerns.

Best Regards

Thank you for the clarifications and the additional experiments. I’m keeping my original rating. Wish you good luck!

Reviewer_jxsEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ChartRevise introduces a dataset and evaluation protocol for exact chart editing through plotting code. Grammar-derived object-action templates are instantiated against compatible source programs and checked for completion, coupled updates, and preservation. The release contains 92,438 records across 344 edit types, 20 chart types, and three plotting libraries. A reference-free evaluator and transfer experiments examine editing behavior on external benchmarks.

Critical strengths
  1. Distinguishing instruction completion, coupled updates, and preservation gives exact chart editing a clear motivation beyond plausibility or executability.

  2. The grammar-grounded resource and decomposed evaluation protocol provide a useful contribution. Transfer experiments across five models and four external benchmarks, together with human audits, offer substantive evidence of value.

Critical weaknesses
  1. Compound-task quality. Human-audited correctness is 84.2% overall and 69.5% for compound requests. Characterizing the compound failures would clarify where the exact-edit supervision remains noisy.

  2. Evaluator reliability. The 93% agreement on audited executable outputs is encouraging. Separate evidence for coupled updates and preservation in difficult cases would strengthen the proposed exactness measures.

  3. Reference-program scoring. The low ChartM3 reference AppliedRate needs interpretation. Distinguishing reference errors, adaptation errors, and judge errors matters when using that result to assess editing quality.

  4. Transfer regressions. Some completion and exact-edit gains coexist with preservation or executability losses. A targeted error breakdown would clarify the useful operating range of the supervision. These limitations qualify a valuable dataset contribution.

Questions for authors
  1. Which errors account for the lower compound-task correctness?
  2. How reliable are coupled-update and preservation judgments in difficult cases?
  3. What explains the low ChartM3 reference AppliedRate?
  4. Where are preservation and executability regressions concentrated after fine-tuning?
4 comments
AuthorsOpenReview ↗

Response (Part 1)

We sincerely thank the reviewer for the thoughtful and constructive feedback and for recognizing the contributions of our work. We appreciate the insightful suggestions and have conducted additional analyses to address the concerns, as detailed below.

W1 & Q1. Dataset Noise and Breakdown on Compound Samples

R: Gratuitous changes (79%) account for most of the lower compound correctness:

Error typeFailed Records (n=61)
Gratuitous change48 (79%)
Failed completion29 (48%)
Missed coupled update7 (11%)

Gratuitous changes are the most frequent failure type (79%). Notably, 20 records (33% of failures) satisfy all requested edits and fail solely due to unrequested changes. These frequently involve residual code, intermediate editing steps, or comments left by the generation model. Our human audit applies strict code-level preservation checks, including changes that do not affect the rendered chart. We will include this breakdown and clarify the audit criteria in the revision.

W2 & Q2. Coupled Updates and Preservation Evaluation Reliability

R: We thank the reviewer for highlighting the need for component-level reliability analysis. We conducted additional evaluations of requirement completion, coupled updates, and preservation.

  1. Requirement completion: The automatic judge achieves 94% agreement with human annotations.
  2. Coupled updates: Since the original human audit did not separately annotate coupled updates, we re-evaluated outputs from all five model families with a second judge (Qwen3.8-27B). The fine-tuning gains in coupled-update recall differ by only +0.3 points on ChartEdit and −1.0 points on Chart2Code-L2 on average, and by at most 2.2 points for any model.
  3. Preservation: The second judge independently confirms 87.5% and 94.5% of gratuitous changes flagged by the original judge on ChartEdit and Chart2Code-L2, respectively.

These results indicate that the measured fine-tuning gains are relatively stable across judges and that most flagged preservation errors are also identified by a second model.

W3 & Q3. Low ChartM3\mathrm{ChartM}^{3} Reference AppliedRate

R: We thank the reviewer for identifying this inconsistency. Upon investigation, we found that the reference AppliedRate in the appendix was inadvertently computed using a visual-only judge rather than the intended code-and-image judge described in Appendix B.5.

Re-evaluating the reference programs with the intended judge substantially increases AppliedRate across all four benchmarks:

Reference AppliedRate (%)ChartEditChart2Code-L2ChartSyncChartM3\mathrm{ChartM}^{3}
Visual-only (reported)79.675.795.634.5
Code-and-image (corrected)95.292.798.577.0

ChartM3\mathrm{ChartM}^{3} improves from 34.5% to 77.0%. Further inspection indicates that 33.4% of reference edits were not verified by the visual-only judge because their effects were invisible, subtle, or only partially visible. Among the remaining unsuccessful cases, we observe incomplete or incorrectly targeted reference edits and instructions referring to absent chart objects. These findings explain much of the original discrepancy while identifying remaining limitations in the benchmark references and judging process.

We will correct Tables 11 and 13, explicitly document the judge configuration, and clarify the distinction between scoring errors and genuine reference-program limitations.

AuthorsOpenReview ↗

Response (Part 2)

W4 & Q4. Preservation and Executability Regressions

R: We thank the reviewer for this suggestion. Our analysis identifies Chart2Code-L2 as the benchmark with the clearest preservation and executability regressions. After fine-tuning, four of five models introduce more gratuitous changes, and four exhibit lower execution rates (Tables 2 and 3). In contrast, ChartEdit shows reductions in gratuitous changes for four of five models.

We identify three patterns:

  1. Task complexity. Figure 6 shows that Phi-3.5-Vision improves requirement recall across instruction-depth groups, but its reduction in gratuitous changes reverses for requests with at least 15 requirements, exceeding ChartRevise’s maximum instruction depth of 12. This suggests that more complex requests create additional opportunities for unintended modifications and execution errors, particularly when the base model leaves more requirements unaddressed.
  2. Execution failure types. The regressions predominantly involve exception categories already present in base models, including data-shape mismatches caused by inconsistent updates.
  3. Gratuitous-change categories. The most common unintended modifications involve text or labels (31%) and colors or styles (28%).

These results suggest that a completion–preservation trade-off contributes to the observed regressions: fine-tuning encourages more complete edits, but preserving unrelated content becomes harder as request complexity increases. We will clarify this limitation and expand the failure analysis in the revision.

Thank you for the clarifications and the additional experiments. I’m keeping my original rating of 3.

Dear reviewer,

Thank you for taking the time to review our responses and additional analyses. We sincerely appreciate your feedback and consideration.

Best Regards

Reviewer_CirAOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ChartRevise is a dataset and evaluation protocol for changing a chart through its source program. It distinguishes satisfying an explicit request, making dependent updates such as adjusting a legend, and preserving unrelated chart content. A grammar-derived edit taxonomy, program applicability checks, execution, model judging, and atomic audits produce 92,438 records across 20 chart types and three plotting libraries. The evaluation decomposes instructions and coupled updates before judging outputs, without a reference edited program. Fine-tuning five vision-language models improves requirement recall in 19 of 20 external model–benchmark comparisons and exact-edit rate in 17, although preservation and executability do not improve consistently.

Critical strengths
  1. The decomposition addresses a real weakness in evaluating editing systems. A model can implement the requested change while leaving a stale dependent element or making an unrelated modification. The protocol makes those outcomes separately visible, and using a fixed requirement and coupled-update inventory across compared outputs is a sensible design choice. The distinction is borne out by the results: better requirement recall coexists with more gratuitous changes in half the comparisons, and execution rate decreases on average (Tables 2–3). Those negative results make the analysis more informative than a single aggregate quality score.

  2. The dataset has substantial coverage and useful evidence of external transfer. Its templates are grounded in source-program applicability rather than generated without checking for a valid target. Five model families are evaluated on four external benchmarks that do not supply training examples, and the construction includes a second model-family audit. The paired blinded human evaluation is also a worthwhile check on the automatic exact-edit verdicts, rather than assuming that the judge is correct.

Critical weaknesses
  1. The human agreement study validates output judgments conditional on the protocol's own requirement inventory, rather than independently validating that the inventory captures the whole task. Annotators are given the same atomic-requirement list as the automatic judge (Appendix E.4), and the reported human positive criterion refers to every listed requirement. A missed requirement can therefore be missed by both evaluations. The audit also contains only pairs where both outputs render, with just 17 Chart2Code-L2 pairs, so the 93% agreement should not be read as uniform reliability over the full evaluation population or over coupled-update inventory construction. This is consequential for a paper whose contribution includes an exactness metric.

  2. The release remains substantially noisy at the task level despite the audit pipeline. Table 7 gives 84.2% weighted task correctness, and only 69.5% for compound records. The 95.6% compound requirement-level figure is useful but does not make the compound tasks exact. This matters because these records are training targets for preservation and completion. The filtering ablation is directionally helpful, but the remaining task-level error rate and the inconsistent preservation gains make it hard to attribute the transfer improvement specifically to exactness, rather than to additional chart-editing supervision.

Questions for authors
  1. Can a human audit independently derive the explicit requirements and required coupled updates from the original instructions and source charts, without first seeing the automatic inventory? What omissions or ambiguities does that reveal, and does correcting the inventories change the exact-edit gains?
  2. Please report the main error categories in the 15% of failed single-action records and 30.5% of failed compound records from Table 7. How much is failed completion, missed coupled updates, or gratuitous change, and are these failures concentrated in particular libraries or edit categories?
  3. Is the pre-/post-filtering comparison in Table 15 controlled for training-data quantity and optimization budget? A matched-size comparison, together with independent inventory validation, would strengthen the claim that exactness filtering itself accounts for the gain.
4 comments
AuthorsOpenReview ↗

Response (Part 1)

We sincerely thank the reviewer for the thoughtful and constructive feedback and for recognizing the contributions of our work. We appreciate the insightful suggestions and have conducted additional analyses to address the concerns, as detailed below.

W1 & Q1. Requirement Inventory Completeness and Evaluation Reliability

R: Our human evaluation intentionally uses fixed requirement inventories to isolate judge reliability from inventory quality. Allowing annotators to derive their own inventories would mix errors in requirement identification with errors in judging model outputs. We acknowledge that this setup does not independently establish inventory completeness. We therefore conducted two additional analyses.

  1. Independent inventory validation. On 100 tasks (50 ChartEdit and 50 Chart2Code-L2), an independent model (Opus 5.5) derived requirements and coupled updates from the original instructions and source programs without seeing our inventories. Our inventories cover 97%/98% of independently derived requirements and 95%/80% of coupled updates, respectively.
  2. Evaluation reliability. Beyond the original human audit, we re-evaluated the outputs of all five model families with a second judge (Qwen3.8-27B). The fine-tuning gains change by less than 1.3 points on average: −0.3, +0.3 and −0.5 points for requirement recall, coupled-update recall and exact-edit rate on ChartEdit, and −1.2 and −1.0 points for requirement and coupled-update recall on Chart2Code-L2. Exact-edit gains on ChartEdit remain significant for all five models under both judges.

Regarding omissions and their impact, missed requirements mainly involve secondary constraints, such as keeping unrelated data unchanged, while missed coupled updates mostly concern labels or annotations in long requests. Although these omissions may affect individual exact-edit judgments, the limited changes in fine-tuning gains under the second judge suggest that they do not materially alter our main conclusions. We will clarify the scope of the original human agreement study.

AuthorsOpenReview ↗

Response (Part 2)

W2. Task-Level Data Noise and Uncertain Benefits of Filtering

R: We thank the reviewer for raising this concern. We acknowledge that some task-level noise remains, especially in compound records. Table 7 applies our strictest criterion: a record must execute, satisfy every requirement, and preserve all unrelated content. Under this criterion, weighted task-level correctness reaches 84.2%. We further explain the lower correctness of compound records, characterize the remaining errors and their scale, and show why the residual noise does not undermine the effectiveness of our supervision.

  1. Strict task-level correctness. Compound records contain 2–12 requirements. At 96% per-requirement correctness, only 79.3% of compound records are expected to satisfy every requirement under an independent-error assumption (measured: 81.5%), even before preservation is considered. Thus, high requirement-level correctness does not necessarily translate into equally high task-level correctness.
  2. Remaining noise and its scale. The human audit applies stricter preservation judgments than our pipeline, counting unrequested code changes, including leftover reasoning comments or unused code, as failures. Such gratuitous changes occur in 79% of defective compound records. However, compound records account for only 5% of the training data, and clear editing defects in these records, excluding borderline cases and instruction problems, account for only 0.75% of the full training set.
  3. Benefits of filtering. More importantly, our matched-size ablation shows that the filtered release achieves better exact-edit performance than unfiltered data with the same number of training records and optimization steps (Q3). This supports the benefit of our refinement pipeline beyond simply providing more training examples.

Remaining noise does not necessarily prevent effective learning. Synthetic instruction dataset are common to contain noise, but can still be very helpful to improve model performance [1]. More importantly, our matched-size ablation shows that the filtered release achieves higher exact-edit rates than unfiltered data under the same training budget (Q3).

Regarding inconsistent preservation gains, Section 4.3 and Figure 6 suggest that task complexity and model capability play a larger role. Fine-tuning encourages models to complete more requested edits, but longer requests create more opportunities for gratuitous changes. For Phi-3.5-V, fine-tuning reduces gratuitous changes on requests with fewer than 15 requirements, while the training records contain at most 12. Beyond this depth, preservation gains reverse despite continued improvements in requirement recall. This suggests that, beyond a certain level of task complexity, weak models that perform fewer edits may appear better at preservation simply because they leave more requirements unmet.

Overall, ChartRevise provides effective supervision for exact editing despite the remaining noise, while preservation on long, complex requests remains challenging.

[1] Wang, Yizhong, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. "Self-instruct: Aligning language models with self-generated instructions." In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pp. 13484-13508. 2023.

AuthorsOpenReview ↗

Response (Part 3)

Q2. Dataset Failure Types and Error Concentration

R: We conducted a detailed error analysis of the human-audited records, categorizing failures into four non-mutually exclusive types:

Error typeSingle-action (45 failures)Compound (61 failures)
Failed completion20 (44%)29 (48%)
Gratuitous change18 (40%)48 (79%)
Missed coupled update4 (9%)7 (11%)
Other10 (22%)3 (5%)

For single-action records, failed completion is the most frequent error (44% of failures, or 6.7% of all 300 audited records). All 20 affected programs execute successfully, and 16 preserve unrelated content. These errors primarily involve subtle misuses of plotting-library APIs.

For compound records, gratuitous changes are the most frequent defect (79% of failures), often involving residual code or unrequested modifications. Missed coupled updates are relatively uncommon in both groups (9% and 11% of failures, respectively). Other errors involve invalid instruction targets or broken source-program asset paths.

Our examination across plotting libraries, chart types, and edit categories did not reveal an obvious concentration of failures in a single group.

Overall, preservation is the principal remaining challenge in compound editing, while single-action failures more often involve incorrect implementation of requested operations.

Q3. Training Data Size and Budget Control in Filtering Ablation

R: We thank the reviewer for this helpful suggestion. We acknowledge that the original ablation in Table 15 did not control for training-data quantity and optimization budget. We therefore conducted a matched-size ablation using Phi-3.5-Vision, keeping the number of training records, and optimizer steps identical.

BenchmarkΔ Exact-edit (pp) ↑Δ Gratuitous / task ↓Δ Coupled recall (pp) ↑
ChartEdit+1.1−0.026+1.9
Chart2Code-L2+0.1+0.034+2.8

At matched size, the filtered release improves exact-edit rate by 1.1 percentage points on ChartEdit and 0.1 points on Chart2Code-L2, while increasing coupled-update recall by 1.9 and 2.8 points, respectively. Gratuitous changes decrease by 0.026 per task on ChartEdit, although they increase slightly on Chart2Code-L2. These results provide evidence that filtering improves editing performance beyond simply increasing the amount of training data, while preservation on complex requests remains challenging. We will replace the original ablation with this controlled comparison.

Dear reviewer,

We sincerely thank you again for the thoughtful and constructive feedback. We have carefully addressed the concerns through additional experiments, analyses, and clarifications. We believe your suggestions have helped us strengthen the paper and better communicate its contributions.

We truly appreciate the time and effort you have dedicated to reviewing our work. And, we would be happy to address any further concerns.

Best Regards

Comments on the paper

2 comments
Area_Chair_QS7XOpenReview ↗

Reviewers responses to authors

Dear reviewers,

Author-reviewer discussions end on November 18th. If possible, please respond to author comments by that time so that authors have opportunity for any additional responses.

Appreciate reviewers who already engaged in the discussion.

Thank you.

AuthorsOpenReview ↗

Summary of Rebuttal Discussion

Dear Reviewers and Area Chairs,

We sincerely thank the reviewers for their thoughtful feedback, and the Area Chair for facilitating the review and discussion process. ChartRevise introduces a grammar-grounded dataset and a reference-free evaluation protocol for exact chart editing that separately assesses instruction completion, necessary coupled updates, and preservation of unrelated content. We appreciate the reviewers’ recognition that our work “addresses a real weakness in evaluating editing systems” (CirA), that our dataset and evaluation protocol provide a “useful contribution” (jxsE), and that our evaluation is “very interesting and diagnostically useful” (LNBi), alongside their recognition of the dataset’s substantial coverage and evidence of external transfer.

Across all three reviews, the feedback converged on three main concerns: (1) the reliability of model judgments and derived requirement inventories, (2) the quality of the compound training data, and (3) inconsistent preservation after fine-tuning. Our rebuttal addresses these concerns through additional validation with cross-model verification on model judgement and inventories, a matched-size filtering ablation, and targeted error analyses. These additions support our main findings while clarifying the remaining limitations. Reviewer jxsE has acknowledged our additional results and clarifications.

We thank again the reviewers for their careful assessment and engagement in the rebuttal, and the Area Chair for coordinating the entire review process. We will incorporate the clarifications and additional analyses into the revised manuscript and would be happy to address any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.