acceptodds
Under review as a conference paper at ICLR 2027

EdiTikZ: Scientific Figure Editing from Revision Trajectories

10 views

Abstract

Vision-language models (VLMs) have shown strong performance in generating scientific figures from text or images. However, publication-ready figures often require iterative refinement, making scientific figure editing an important yet largely unexplored step toward interactive figure creation. Existing approaches rely on costly proprietary agentic systems, focus primarily on evaluation, or construct training supervision from synthetically generated edits. Instead, we leverage naturally occurring scientific revision and development trajectories as a scalable source of supervision. To this end, we introduce DaEdiTikZ, the first large-scale dataset of revision-derived scientific figure edits, constructed by mining 391K plausible TikZ edit pairs from arXiv, GitHub, and TeX SE and inferring 781K directed edit instructions with a VLM conditioned on rendered figures and TikZ code. We further introduce DaEdiTikZ-Bench, a human-refined benchmark with 690 instances, and train two compact Qwen3.5-based EdiTikZ models (4B and 9B) by jointly learning image-to-TikZ reconstruction and instruction-conditioned editing, followed by reinforcement learning (RL) with complementary rewards for rendered fidelity and edit application. Automatic evaluation places our 9B model above all tested baselines, while human evaluation with 9 annotators and 4,320 ratings places it above GPT-5.6-Sol and on par with Gemini-3.1-Pro. Under severe out-of-distribution shifts, it remains competitive with GPT-5.6-Sol near its 2K training sequence-length regime. Models and datasets will be released.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision language models, graphics program generation, scientific figure editing, reinforcement learning, small models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3DzwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • EdiTikZ mines plausible figure-edit pairs from scientific sources and infers editing instructions from images and code. Joint reconstruction and editing training, followed by multi-reward RL, produces compact models with strong results on a human-refined benchmark.
Critical strengths
  • Mining existing human-authored figures provides a substantial and useful source of editing supervision.
  • Group-level data separation, human-corrected benchmark instructions, and human evaluation strengthen the empirical evidence.
Critical weaknesses
  • Half of the audited training instructions contain some error. The consequences of these mismatches for training and the two reward signals remain insufficiently quantified.
  • The data-mixture ablations do not clearly establish whether joint training benefits from reconstruction supervision specifically or from additional training exposure.
Questions for authors
  1. Clarification: Were editing-only, sequential, and joint-training ablations matched for total training tokens and optimization steps?
  2. Evidence request: Is there evidence separating performance on clean and noisy instructions, or a controlled filtering ablation? This would clarify how reliably the method learns from the mined supervision.
2 comments

W1: Half of the audited training instructions contain some error. The consequences of these mismatches for training and the two reward signals remain insufficiently quantified.

Thank you for pointing this out. This is acknowledged as a limitation in our conclusion (p9, l478-479). Despite still containing errors, 82.9% of our instructions were rated either very good or good by human expert annotators with high agreement (weighted κ=0.79\kappa = 0.79) as seen in p4, l213-215. Moreover, these errors also contain hardly observable changes such as minimally increasing the size of an arrowhead which may even improve model robustness and generalization to non-meaning changing perturbations. Lastly, with growing capabilities of VLMs in the future, we expect the performance of our approach to increase as error rates go down.

W2: The data-mixture ablations do not clearly establish whether joint training benefits from reconstruction supervision specifically or from additional training exposure.

Q1: Were editing-only, sequential, and joint-training ablations matched for total training tokens and optimization steps?

Editing-only and reconstruction-only models were trained for 4 epochs on 754K samples, whereas joint and sequential training used two epochs over 1.51M combined examples (~3M combined samples for all four ablation settings). We also used the same effective batch size as indicated in Table 14, matching the optimization steps. Input images and instructions are masked from the loss and a maximum sequence length of 2K is used throughout training. Due to DaEdiTikZ being a subset of the extended DaTikZ-V4, all plausible figure editing pairs can be used for both editing and reconstruction. Standalone figures not included in DaEdiTikZ but in our extended DaTikZ-V4 can still be used for reconstruction supervision alone. Therefore, our joint approach does not rely on additional data.

Q2: Is there evidence separating performance on clean and noisy instructions, or a controlled filtering ablation? This would clarify how reliably the method learns from the mined supervision.

To assess sensitivity to noisy instructions, we introduced controlled perturbations of the DaEdiTikZ-Bench edit instructions. For each sample, we replaced one text span with either:

  1. A paraphrase that preserves meaning (positive)

  2. A visual property (e.g., annotation, data, relationship, …) that changes meaning (negative)

Afterwards, we judged if the outputs fulfill the changed requirement and report success under meaning-preserving/meaning-changing instructions conditioned on success under the original instruction. Our results are as follows:

Meaning-preserving perturbations

ModelSuccess (%) ↑Retention (%) ↑
GPT-5.6-Sol80.489.3
Qwen3.5-9B39.973.0
EdiTikZ-9B-SFT77.189.4
EdiTikZ-9B-RL87.894.7

Meaning-changing perturbations

ModelSuccess (%) ↑Adaptation (%) ↑
GPT-5.6-Sol78.184.1
Qwen3.5-9B39.062.6
EdiTikZ-9B-SFT63.870.3
EdiTikZ-9B-RL76.781.4

With the following takeaways:

  • EdiTikZ-9B-RL retains correctness under meaning-preserving changes in 94.7% of initially successful cases which is highest amongst all models.

  • RL does further improve the adaptation of visual requirements that change over SFT (81.4% vs. 70.3%) which is almost as good as GPT-5.6-Sol (84.1%). This demonstrates the scalability of our additional RL step post SFT, making it more robust to perturbations in the edit instructions that alter the figure.

  • Meaning-changing perturbations reduce overall success more for our models than for the non-meaning changing perturbations (e.g., 11.1% vs. 2.3% for EdiTikZ-9B-RL vs. GPT-5.6-Sol). However, this gap is mostly due to the very high performance on instructions that do not alter the figure and less so because it does not change properties when requested.

These findings indicate that our two-stage training approach learns very reliably from the mined supervision where it is robust to changes on the input edit instructions. We added our results to the revised paper (p8, Table 6) and thanks again for pointing this out.

Solved my concerns, so I will keep my rating.

Reviewer_Hy5VOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper is targeting scientific figure editing problem which is an important area but with not much exploration. Authors mined a large scale dataset for training VLM to edit scientific figures, using image input and instruction edit input as conditions to produce executable code for rendering the edited figure. Then authors trained a 4B and 9B Qwen3.5 models that joinly trained on image-to-TikZ reconstruction and instruction-conditioned editing, further improved with reinforcement learning. Finally authors created a benchmark and found that it performs better than close source models via human evaluation.

Critical strengths

S1) A crucial and highly practical problem selection. It will be very useful for researchers to iteratively edit the scientific figures.

S2) The contribution of this dataset tackles a major bottleneck in training models to edit scientific figures, which was caused by the severe scarcity of training data.

S3) The rationale for using GDPO instead of GRPO because of imbalance reward makes sense to me.

S4) The paper is overall well-written and easy to follow.

Critical weaknesses

W1) I feel this work did not fully explore the potential of multimodal inputs. The proposed EdiTikZ models rely entirely on the rendered image and text instructions inputs to generate the updated TikZ code, forcing the model to reconstruct the entire scene from scratch rather than making localized edits to the existing code. Since the source code is available in the training data, the omission of a joint training setup (image + edit -> code alongside code + edit -> code) is a missed opportunity. The authors need to provide empirical experiments demonstrating how their approach compares to a code-aware baseline, or clearly justify why source code inputs were intentionally discarded.

Questions for authors

See W1

5 comments

We thank the reviewer for highly regarding our work. Its focus was intended in case the underlying TikZ code of a scientific figure is not available, for example when researchers want to directly adapt a methodological figure or flowchart from a paper. But we agree that multimodal support with source would make EdiTikZ practically more useful.

We therefore extended to both settings by jointly fine-tuning Qwen3.5-9B where we split the editing supervision equally between image + edit -> code and image + code + edit -> code, while keeping the reconstruction supervision the same, yielding EdiTikZ-9B-Mix.

Additionally, we compare two more multimodal baseline settings:

(i) Code-only editing to see if providing the source figure helps

(ii) Image reconstruction followed by code + image + edit (2 stage) in case source code is unavailable

This leaves us with four multimodal settings. Results are below:

Source code unavailable: Direct editing

ModelTED ↓DSim ↑EA ↑SP ↑VQ ↑Avg ↑CR ↑AT ↓
GPT-5.6-Sol0.7640.7960.7350.7750.8230.67388.3%485
Qwen3.5-4B0.8060.4110.1460.1880.3040.24951.4%726
Qwen3.5-9B0.7810.5230.2410.3110.4300.34564.1%599
EdiTikZ-9B-Mix0.7270.7410.4970.5620.6530.54582.8%742

Source code unavailable: Reconstruction → Editing

ModelTED ↓DSim ↑EA ↑SP ↑VQ ↑Avg ↑CR ↑AT ↓
GPT-5.6-Sol0.8010.6230.5860.6220.6480.53568.7%360
Qwen3.5-4B0.8150.3590.1440.1670.2580.22345.5%831
Qwen3.5-9B0.8140.4510.2410.2690.3720.30454.9%850
EdiTikZ-9B-Mix0.7260.7170.4880.5370.6450.53280.3%761

Source code available: Code only

ModelTED ↓DSim ↑EA ↑SP ↑VQ ↑Avg ↑CR ↑AT ↓
GPT-5.6-Sol0.3620.8300.7820.8270.8350.78295.7%432
Qwen3.5-4B0.3650.8100.5260.7260.7660.69287.7%542
Qwen3.5-9B0.3450.8360.5930.7460.8010.72689.7%534
EdiTikZ-9B-Mix0.3140.8820.6470.8090.8450.77493.3%563

Source code available: Image + code

ModelTED ↓DSim ↑EA ↑SP ↑VQ ↑Avg ↑CR ↑AT ↓
GPT-5.6-Sol0.3660.8600.8140.8560.8640.80695.2%444
Qwen3.5-4B0.3500.8320.5450.7620.8010.71889.1%565
Qwen3.5-9B0.3280.8750.5880.8130.8500.76093.0%545
EdiTikZ-9B-Mix0.3070.8850.6950.8210.8540.79093.0%569

Three main findings are observed:

  • The available source code improves performance, in particular for smaller models. For example, Qwen3.5-4B improves from 0.249 Avg with image-only input to 0.718 with additional code as context. However, correctly applying the edits still remains difficult (e.g., 0.588 edit application score for Qwen3.5-9B) even when code is provided.

  • Removing the source image (code only) decreases performance for all models, making it valuable for improving grounding of edit instructions with the code. Additionally, code-only works better than not using it at all.

  • When source TikZ code is unavailable, directly editing the figure outperforms a two-step “first reconstruct then edit” pipeline for all models while requiring one less API call thus being more efficient. This directly supports our original focus on models that edit figures directly from images.

An important note:

Our EdiTikZ-9B-Mix model is evaluated on an intermediate checkpoint (middle point) as training takes 13 days. We also plan to use it on our multi-reward RL with GDPO post-training setup where we expect the results to further improve. Nevertheless, we already see that it improves editing when source code is available, where EdiTikZ-9B-Mix almost matches GPT-5.6-Sol’s performance (0.774 vs 0.782 Avg). We will update the results as training progresses.

Our experiments are added to the revised paper (please see Table 7 on page 10). We also plan to release both models and hope these additions can be useful to the scientific community.

Thanks again for your valuable feedback. If there is anything else, feel free to let us know.

Dear Reviewer Hy5V,

thanks again for your positive and valuable review. As the author response deadline is approaching soon, is there anything else we could clarify or provide? Else, if you think that our rebuttal addresses your concerns, we would of course be happy if you would increase your score.

Best regards, Authors

Thanks authors for the substantial effort on ablations. So if I interpreted the tables correctly, mixing input code into training would slightly improve the performance, but its still a bit difficult for editing task. Did you apply optional input dropout during training (i.e. randomly dropping code or image) to maintain the robustness? Also from the table I wonder which metric is more reliable according to authors' experience, as some of metrics are essentially measuring same aspect on visual similarity / quality. Overall I will keep my score as weak acceptance.

Reviewer_pH4gOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EdiTikZ, a framework for scientific figure editing based on revision-derived supervision. The authors construct DaEdiTikZ, containing approximately 391K plausible figure-edit pairs and 781K directional editing instances, together with a human-refined benchmark of 690 instances. They train compact VLMs through joint reconstruction and editing supervision, followed by reinforcement learning with complementary rewards for visual similarity and instruction following. Experiments demonstrate substantial improvements over pretrained models, with EdiTikZ-9B-RL achieving competitive performance against proprietary VLMs.

Critical strengths
  1. The proposed revision-mining pipeline provides a scalable approach to constructing scientific figure editing supervision, addressing an important limitation of existing synthetic editing datasets.

  2. DaEdiTikZ-Bench further provides human-refined editing instructions, making it a potentially valuable resource for evaluating scientific figure editing systems.

  3. EdiTikZ-9B-RL achieves the highest aggregate score among the evaluated models on DaEdiTikZ-Bench. Human evaluations also show competitive performance against substantially larger proprietary VLMs.

  4. The ablations in Tables 4 and 5 provide evidence for the benefits of reconstruction supervision and the complementary reward objectives. The improvements in compilation rate and edit application further strengthen the experimental findings.

Critical weaknesses
  1. Although the figure pairs originate from human-authored scientific artifacts, many are selected through semantic similarity rather than verified chronological revisions. Consequently, the resulting transformations may not always reflect realistic editing intentions.

  2. More importantly, the dataset quality analysis reports that approximately 50% of the inferred instructions contain at least one error, including omissions and misinterpretations. This raises concerns about the reliability of the training supervision.

  3. The paper lacks a sufficiently controlled analysis of how instruction noise and different sources of figure pairs affect downstream editing performance.

  4. The human evaluation covers 180 example-level annotations, considerably fewer than the 690 benchmark instances. The paper would benefit from stronger statistical evidence supporting comparisons with proprietary models.

  5. The out-of-distribution evaluation relies on synthetically generated editing instructions and reference-free VLM judging. Moreover, Table 16 shows substantial performance gaps relative to GPT-5.6-Sol, particularly in edit application and source preservation.

  6. The current framework reconstructs edited TikZ code from source images rather than directly modifying existing TikZ programs. This limits the evidence for its effectiveness in practical code-based scientific figure editing workflows.

Questions for authors
  1. How does the quality of automatically inferred editing instructions affect downstream performance? Could the authors provide controlled experiments comparing different instruction-quality subsets or verified chronological revisions against similarity-mined pairs?

  2. How robust are the reported improvements on genuinely independent editing tasks? Can the authors provide statistical significance tests for the human evaluation and clarify the extent to which the conclusions hold on the out-of-distribution datasets?

  3. How does EdiTikZ compare with approaches that directly edit existing TikZ source code? Since the proposed models regenerate the entire figure, it would be useful to understand whether the same benefits persist when source programs are available.

5 comments

W1: Although the figure pairs originate from human-authored scientific artifacts, many are selected through semantic similarity rather than verified chronological revisions. Consequently, the resulting transformations may not always reflect realistic editing intentions.

Thank you for pointing this out. As discussed on p3, l153-156, knowing exact verified chronological revisions is error prone because anchors that identify them such as references, captions, .tex file names often change between arXiv versions. Additionally, related figures from the same paper submissions cannot be identified that way since there are no shared anchors to begin with. We tried this approach initially but it provided much lower sample sizes with many unrelated figures being matched. Instead, we approximated it via within-context semantic similarity from a scientific image encoder where we evaluated the filtering threshold through manual inspection (Table 2) which we find that the vast majority are plausible editable transformations.

W2: More importantly, the dataset quality analysis reports that approximately 50% of the inferred instructions contain at least one error, including omissions and misinterpretations. This raises concerns about the reliability of the training supervision.

We agree and already acknowledged this as a limitation (p9, l478-479). To reduce it, we initially provided TikZ code alongside rendered figures as complementary grounding which has decreased the error rate substantially. 82.9% of instructions receive overall quality ratings of good or very good with high agreement of weighted κ=0.79\kappa = 0.79 (p4, l213-215) which does not indicate much noise. Issues also include barely observable cases such as moving a line slightly higher which may even improve model robustness and generalization which we test later. Moreover, with the growing capabilities of VLMs, we expect that the performance of our method will increase in the future.

W3: The paper lacks a sufficiently controlled analysis of how instruction noise and different sources of figure pairs affect downstream editing performance.

Q1: How does the quality of automatically inferred editing instructions affect downstream performance? Could the authors provide controlled experiments comparing different instruction-quality subsets or verified chronological revisions against similarity-mined pairs?

To assess instruction sensitivity, we added controlled perturbations of the DaEdiTikZ-Bench instructions. For each instance, we replaced one text span with either:

(i) a meaning-preserving paraphrase (positive)

(ii) a change to one requested visual property such as attribute, annotation, data, spatial relationship (negative)

We then judged whether each output reflects that preservation/change and report success under positive/negetive perturbation conditioned on success under the original instruction:

Meaning-preserving perturbations

ModelSuccess (%) ↑Retention (%) ↑
GPT-5.6-Sol80.489.3
Qwen3.5-9B39.973.0
EdiTikZ-9B-SFT77.189.4
EdiTikZ-9B-RL87.894.7

Meaning-changing perturbations

ModelSuccess (%) ↑Adaptation (%) ↑
GPT-5.6-Sol78.184.1
Qwen3.5-9B39.062.6
EdiTikZ-9B-SFT63.870.3
EdiTikZ-9B-RL76.781.4

We find that EdiTikZ-9B-RL retains correctness under meaning-preserving changes in 94.7% of initially successful cases. RL also improves adaptation to changed visual requirements over SFT (81.4% vs. 70.3%), approaching GPT-5.6-Sol (84.1%). These findings demonstrate the benefits of our subsequent RL step, yielding models that are very robust despite being trained on VLM-generated instructions. We will add our results to the revised paper (see p9, Table 6).

W4: The human evaluation covers 180 example-level annotations, considerably fewer than the 690 benchmark instances. The paper would benefit from stronger statistical evidence supporting comparisons with proprietary models.

Our human evaluation has 180 annotation groups, each containing outputs from eight models (4 baseline, 4 trained) evaluated on three criteria (edit application, source preservation, visual quality) with a likert-scale of 1-7, yielding 4,320 ratings across 9 annotators which we believe is substantial labelling effort. To complement this, we added paired annotator-level significance tests to quantify uncertainty in the model comparisons, as discussed under Q2.

W5: The out-of-distribution evaluation relies on synthetically generated editing instructions and reference-free VLM judging. Moreover, Table 16 shows substantial performance gaps relative to GPT-5.6-Sol, particularly in edit application and source preservation.

While we agree that the OOD evaluation uses synthetically generated edit instructions, the instruction-generation prompt is grounded in real-world edits by specifying editing intents and operations with nine diverse human-written examples (see appendix p26, l1396-1403).

Appendix Table 16 does show performance gaps, however as discussed on p8, l424-431, this evaluation is intended to be a difficult stress-test for testing the boundaries of EdiTikZ. It involves editing complex architectural diagrams, multi-panel plots, and rare charts from non-TikZ sources (e.g., Matplotlib, MATLAB) whose generated tokens is often 2-5 times higher than on our DaEdiTikZ-Bench and above our 2K-token training regime on average (Table 16, AT column). Despite this, our EdiTikZ-9B models show competitive performance with GPT-5.6-Sol near this regime. These post-training gains persist even much further outside the training horizon but with increasing gaps which is where the performance gaps in Table 16 come from. We see these findings (Figure 7 + Table 16) as a strength instead of its weakness.

W6: The current framework reconstructs edited TikZ code from source images rather than directly modifying existing TikZ programs. This limits the evidence for its effectiveness in practical code-based scientific figure editing workflows.

Q3: How does EdiTikZ compare with approaches that directly edit existing TikZ source code? Since the proposed models regenerate the entire figure, it would be useful to understand whether the same benefits persist when source programs are available.

We agree that available source code is important for practical editing workflow. We therefore added three settings alongside our original image-only evaluation:

(i): code-only editing

(ii): image-and-code editing

(iii): reconstruction followed by image-and-code editing (when source code is unavailable)

We also trained Qwen3.5-9B jointly on reconstruction, image-only editing, and image+code editing which we evaluated against three baseline models on four multimodal settings (please see p10, Table 7). We also plan to apply it on our RL setup once SFT is finished. Further details of our findings and takeaways are provided in our detailed response to reviewer Hy5v.

Q2: How robust are the reported improvements on genuinely independent editing tasks? Can the authors provide statistical significance tests for the human evaluation and clarify the extent to which the conclusions hold on the out-of-distribution datasets?

We carried out paired annotator-level tests with Holm correction. The combined-score difference between EdiTikZ-9B-RL and GPT-5.6-Sol is +0.669 (95% CI: [-0.244,1.681]). The difference from Gemini-3.1-Pro is -0.112 (95% CI: [-1.050,0.731]). 95% CIs for other models are available in Figure 6. As already discussed under W5, gains transfer beyond the training distribution, while competitiveness with GPT is concentrated near the 2K-token regime and larger gaps remain on longer outputs. We will add this to the updated version.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns. The new results provide useful evidence supporting the proposed method. However, two concerns remain:

  1. The instruction perturbation experiments demonstrate inference-time robustness but do not directly address the impact of noisy training supervision, given that approximately 50% of the generated instructions contain errors.

  2. The reported confidence interval for EdiTikZ-9B-RL versus GPT-5.6-Sol includes zero, so the claimed superiority is not statistically established. Furthermore, performance gaps on challenging OOD tasks remain substantial.

The rebuttal partially addresses my concerns, but these limitations remain. I will maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.