acceptodds
Under review as a conference paper at ICLR 2027

FigMirror: Ground It, Code It, Plot It

Abstract

Converting scientific figures into executable code has gained increasing attention, yet existing methods primarily focus on reproducing the reference figure itself. A more practical setting is to plot new data while preserving the visual style of a reference figure (e.g., color scheme and typography). Prior approaches mimic the reference through pixel-level optimization and struggle to carry its style to new data. We show that the key to this task lies in the coordinate grounding and coding capabilities present in modern computer-use models. We propose FigMirror, an agentic framework that unlocks these capabilities through *Grounded Measurement*, which locates visual elements by coordinates and measures their properties through executable code. We further introduce PlotTwin-Bench, an expert-curated benchmark with fine-grained code- and image-level style metrics. Experiments show that FigMirror consistently outperforms existing methods on reference-conditioned style transfer. All plots in this paper are generated by FigMirror, except those produced by other methods for comparison.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Scientific figure generation, Style transfer, LLM agents

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Sa1SOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a reference-conditioned style transfer task for scientific figures, rather than reproducing the reference figure itself. They propose a new method called FigMirror packaged as a skill that runs inside agentic coding harnesses. FigMirror is composed of two core components, Drawer and Reviewer, which run in a loop combined with grounded measurement and style checklist. The key component in their method is Grounded Measurement, which localizes the visual element carrying each style attribute by coordinates and reads the attribute’s value from the image with a short program. Style attributes are tracked in a Style Checklist, and generation is organized as a Drawer–Reviewer loop: the Drawer measures the attributes and renders a draft, and the Reviewer compares the draft with the reference and returns box-grounded Revision and Preserve Lists. The authors also construct a benchmark called PlotTwin-Bench that includes 50 hand-drawn figure-code pair and 350 LLM-based rewriten pairs across 12 chart types. The core experiments are run on a 150-reference subset, with GPT-5.5 used for both generation and evaluation. In style-transfer task, FigMirror achieves the best result among the compared methods ChartIR, ChartGalaxy-Prompt, METAL, and Plot2Code.

Critical strengths

The paper shifts chart-to-code generation from reproducing a reference figure to transferring its visual style onto the user’s own data. This better matches how researchers actually use well-crafted figures from top venues as templates, which differs it from existing researches. Also, the authors construct a new benchmark PlotTwin-Bench. If fully released, it might be beneficial to the future community.

Critical weaknesses
  1. The paper attributes its gains to Grounded Measurement, which repurposes GUI grounding for precise attribute measurement. Yet it never evaluates measurement accuracy, even though every reference has ground-truth code against which measured colors, line widths, and aspect ratios could be checked. The ablations point elsewhere. Removing Grounded Measurement raises S_code (from 64.4 to 67.4), even though the code channel scores exactly these attributes; the loss appears only in the vision channel. Moreover, in Claude Code the naked harness without the skill already reaches S = 75.5, far above all external baselines (≤ 58.1). Much of the reported margin therefore comes from the agentic harness rather than the proposed method. Without evidence for the mechanism, the contribution reduces largely to an engineered prompt bundle.
  2. The comparison of FigMirror with baselines seems unfair. FigMirror’s launch template encodes benchmark-specific rules that mirror the target-data generator, such as a panel contract tied to the data’s panel column and chart-type preservation. Its Reviewer also explicitly penalizes default matplotlib styling and dropped signature motifs, which are exactly what the two scoring channels reward. The paper does not state what instructions, execution access, or compute the baselines received. Even the naked-Codex ablation gets only a one-sentence prompt, so its 11.6-point drop conflates the method with the task contract. It is also unclear whether the skill was developed on references disjoint from the evaluation subset, which includes all 50 hand-curated figures.
Questions for authors
  1. Did the baselines and the naked-Codex variant receive the same task instructions as FigMirror’s launch template (panel contract, chart-type preservation, data_echo.md)? Please report a naked-harness baseline that receives the full launch template but not the skill. Was the skill developed on references disjoint from the evaluation subset?
  2. Which evaluator was used for the Claude Code, Grok Build, and GPT-5.6 experiments in Appendix A? Please re-score all Table 1 outputs with a VLM evaluator from a different model family (e.g., Claude or Gemini), and report whether the ranking and margins hold.
  3. Regarding the S_code increase without Grounded Measurement, please provide a per-attribute breakdown of the code channel (palette, line width, aspect, grid, spines, font, etc.) and directly report the error between Drawer-measured values and ground-truth reference code, with and without Grounded Measurement.
  4. How are point-based attributes such as line width and font size recovered from a screenshot with unknown DPI and figsize? Or are only relative ratios recovered?
  5. Why were only 150 of the 400 references evaluated? What are the results on the remaining 250? How were the qualitative examples in the appendix selected?
6 comments
AuthorsOpenReview ↗

Response to Reviewer Sa1S (Part 1/5)

We appreciate the reviewer for the detailed and constructive feedback and comments, which are definitely helpful for us to improve the quality of the paper. We have uploaded a revised version to accommodate all your suggestions. Moreover, following your suggestion, we will publicly release our complete PlotTwin-Bench dataset and the codebase. Below, we further make detailed clarifications for each of your questions:

W1: The paper attributes its gains to Grounded Measurement, which repurposes GUI grounding for precise attribute measurement. Yet it never evaluates measurement accuracy, even though every reference has ground-truth code against which measured colors, line widths, and aspect ratios could be checked. The ablations point elsewhere. Removing Grounded Measurement raises S_code (from 64.4 to 67.4), even though the code channel scores exactly these attributes; the loss appears only in the vision channel. Moreover, in Claude Code the naked harness without the skill already reaches S = 75.5, far above all external baselines (≤ 58.1). Much of the reported margin therefore comes from the agentic harness rather than the proposed method. Without evidence for the mechanism, the contribution reduces largely to an engineered prompt bundle.

Q3: Regarding the S_code increase without Grounded Measurement, please provide a per-attribute breakdown of the code channel (palette, line width, aspect, grid, spines, font, etc.) and directly report the error between Drawer-measured values and ground-truth reference code, with and without Grounded Measurement.

Thanks for raising these points. To clarify, our per-attribute decomposition shows that the ScodeS_{\text{code}} inversion in Table 2 of the paper came from a grid-parsing edge case in the original scorer (A1-a), and direct evaluation confirms that Grounded Measurement substantially reduces attribute-reading error toward the Oracle box (A1-b). For the harness comparison raised in W1, we evaluate FigMirror against a matched-instruction naked harness and execution-equipped baselines in A2 below.

AuthorsOpenReview ↗

Response to Reviewer Sa1S (Part 2/5)

a) Per-attribute error against the reference code and why ScodeS_{\text{code}} flipped in Table 2 of the paper.

Following your suggestion, we analyzed the ablation outputs from Table 2 of the paper attribute by attribute against the ground-truth reference code and found that FigMirror (Full) achieves lower error than w/o Grounded Measurement across almost all measured attributes, namely aspect ratio, line width, palette color, and legend frame (Table 1). The apparent ScodeS_{\text{code}} inversion in Table 2 of the paper occurred because FigMirror often configures major and minor grids in separate calls, such as ax.grid(True, which="major") followed by ax.grid(False, which="minor"). Our original code scorer ignored the which argument and parsed the second call simply as ax.grid(False), assigning 0 points on the grid attribute for 11.4% of FigMirror's tasks (compared with 2.3% for w/o Grounded Measurement and 0.3%–1.4% across external baselines). Updating our code scorer to handle the which argument resolves the inversion in Table 2 of the paper (Table 2). Under the submission's original code scorer, removing only executable pixel measurement lowers ScodeS_{\text{code}} on the full 400-reference benchmark (from 82.0 to 80.4), and Table 3 reports the breakdown across the active code-channel attributes. We have also included these per-attribute code-channel breakdowns in Appendix B.3 (Table 12) of the revised manuscript.

Table 1. Full attribute-error comparison on the ablation outputs from Table 2 of the paper (hand-curated set, GPT-5.5).

Attribute metricw/o Grounded MeasurementFigMirror (Full)Relative error reduction ↑
Rendered image aspect-ratio error (%) ↓4.051.7955.9%
Plotting code figsize aspect-ratio error (%) ↓5.003.1337.5%
Palette color distance (CIEDE2000) ↓8.107.369.2%
Code line-width error (%) ↓33.6925.0225.7%
Legend-frame error rate (%) ↓21.4314.2933.3%
Serif font / hidden spines / tick directionTiedTied—

Table 2. ScodeS_{\text{code}} on the ablation from Table 2 of the paper before and after scorer refinement (hand-curated set, GPT-5.5).

Scorer versionFigMirror (Full) ↑w/o Grounded Measurement ↑ΔScode\Delta S_{\text{code}} (w/o GM − Full) ↓
Original scorer64.467.4+3.0
Refined scorer67.166.7−0.4

Table 3. Additional ScodeS_{\text{code}} breakdown across active code-channel attributes when only executable pixel measurement is removed (400 references, GPT-6.1).

AttributeFigMirror (Full) ↑w/o Pixel Measurement ↑Δ\Delta (w/o PM − Full)
Serif font100.075.0−25.0
Figure aspect ratio85.180.9−4.2
Spines removed95.192.2−2.9
Palette81.079.1−1.9
Grid90.188.5−1.6
Legend frame92.592.1−0.4
Tick direction50.050.00.0
Line width65.165.7+0.6
Total ScodeS_{\text{code}}82.080.4−1.6

b) Intermediate measurement accuracy against the reference code.

Following your suggestion, beyond the final output errors in Table 1, we further evaluate the intermediate measurement step against the ground truth extracted from the reference code across 412 plot elements on the hand-curated set (now included in Section 5.5, Table 6 of the revised manuscript). Specifically, we freeze the attribute-measurement scripts and compare running them without grounding on the full figure (No grounding), on a Random box, on the bounding boxes localized by Grounded Measurement (Predicted box), and on the ground-truth bounding boxes from the reference code (Oracle box). As shown in Table 4, measuring without element grounding yields large errors, whereas Grounded Measurement cuts color error by more than 2×2\times, marker-size error by more than 2.4×2.4\times, and text-height error by nearly 9×9\times, approaching the Oracle box.

Table 4. Intermediate attribute measurement error against the reference ground truth (hand-curated set, 412 elements).

Measurement conditionColor (CIEDE2000) ↓Stroke width ↓Marker size ↓Text height ↓
No grounding (whole image)13.750.3880.7381.000
Random box (same panel)35.910.5000.6720.534
Grounded Measurement (Predicted box, GPT-6.1)5.940.3090.3040.099
Grounded Measurement (Predicted box, GPT-6 Astra)5.320.3240.2830.115
Oracle box (from reference code)4.540.3090.2330.015
AuthorsOpenReview ↗

Response to Reviewer Sa1S (Part 3/5)

W2: The comparison of FigMirror with baselines seems unfair. FigMirror’s launch template encodes benchmark-specific rules that mirror the target-data generator, such as a panel contract tied to the data’s panel column and chart-type preservation. Its Reviewer also explicitly penalizes default matplotlib styling and dropped signature motifs, which are exactly what the two scoring channels reward. The paper does not state what instructions, execution access, or compute the baselines received. Even the naked-Codex ablation gets only a one-sentence prompt, so its 11.6-point drop conflates the method with the task contract. It is also unclear whether the skill was developed on references disjoint from the evaluation subset, which includes all 50 hand-curated figures.

Q1: Did the baselines and the naked-Codex variant receive the same task instructions as FigMirror’s launch template (panel contract, chart-type preservation, data_echo.md)? Please report a naked-harness baseline that receives the full launch template but not the skill. Was the skill developed on references disjoint from the evaluation subset?

Thanks for the suggestion on experimental controls. We clarify the prompt setup and show in Table 5 that under identical launch instructions, FigMirror outperforms Naked Codex, while removing the two Reviewer clauses you noted slightly increases FigMirror's score (A2-a). We also confirm that the skill was developed on figures disjoint from PlotTwin-Bench and generalizes to the 250 untouched references (A2-b), and show in Table 6 that FigMirror maintains a clear margin when external baselines receive code-execution feedback under a comparable budget (A2-c).

a) Task instructions and naked harness with the full launch template.

We clarify that in the main paper, FigMirror used its launch template (data_echo.md, the panel contract, and chart-type preservation), Naked Codex in Table 2 of the paper used a direct one-sentence task prompt, and the external baselines received the reference image, target CSV, and style-transfer task prompt through their respective published pipelines. Following your suggestion, we add two controlled variants on the hand-curated set in Table 5: (1) Naked Codex receiving the complete FigMirror launch template without the skill, and (2) FigMirror (w/o default-style & motif rules), which removes the Reviewer clauses penalizing default Matplotlib styling and dropped signature motifs. As shown in Table 5, removing those two clauses actually yields a slightly higher score (80.2 vs. 78.9 on SS, and 62.3 vs. 59.0 on ScodeS_{\text{code}}). Under identical launch instructions, both FigMirror variants outperform Naked Codex on ScodeS_{\text{code}} and SS while cutting rendered canvas aspect-ratio error by roughly half (3.37% and 3.91% vs. 7.20%), which is consistent with our Claude Code results in Appendix A.2 (Table 6 of the paper), where adding the FigMirror skill to the naked harness raises SS from 75.5 to 80.5 and ScodeS_{\text{code}} by 9.1 points (55.7 to 64.8).

Table 5. Controlled comparison between Naked Codex receiving the full launch template, FigMirror without the default-style and motif rules, and full FigMirror (hand-curated set, GPT-6.1, GPT-6 Astra evaluator).

MethodScodeS_{\text{code}} ↑SvisionS_{\text{vision}} ↑SS ↑Rendered aspect-ratio error (%) ↓
Naked Codex (full launch template, no skill)55.389.677.67.20
FigMirror (w/o default-style & motif rules)62.389.880.23.91
FigMirror59.089.678.93.37

b) Was the skill developed on references disjoint from the evaluation subset?

Yes. We developed the FigMirror skill on a separate set of complex paper figures outside PlotTwin-Bench before constructing the evaluation benchmark and automated scorer, and we never optimized the skill against evaluator scores on the evaluation set. Moreover, as shown in A2-a (Table 5), removing the two Reviewer clauses on default styling and signature motifs slightly raises SS from 78.9 to 80.2. Evaluating the frozen skill and launch template on the remaining 250 untouched references of PlotTwin-Bench further confirms generalization to unseen figures, where FigMirror leads all baselines by a clear margin (Table 6 and A5).

Reviewer_bhyqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies reference-conditioned style transfer for scientific figures, where the goal is to visualize new data while preserving the visual style and composition of a reference figure, rather than reproducing its original data. The authors propose FigMirror, an agentic framework built around Grounded Measurement and a Drawer–Reviewer refinement loop. Grounded Measurement uses visual coordinate grounding to locate relevant figure elements and executable code to measure their style attributes, while the iterative loop identifies and repairs visual mismatches while preserving previously accepted choices.

The paper also introduces PlotTwin-Bench, a benchmark containing 400 reference figures across 12 chart types, with expert-curated figure–code pairs and reference-specific evaluation criteria. Experiments on 150 references show that FigMirror outperforms four chart-to-code baselines on the combined code- and image-level style-transfer metric. Additional experiments examine component ablations, iteration budgets, and portability across different models and coding-agent harnesses.

Critical strengths
  1. A practically meaningful problem formulation and a well-motivated method.

The paper clearly distinguishes reference-conditioned style transfer from conventional chart-to-code reproduction. Its central insight—that transferable visual attributes should be explicitly localized and measured rather than inferred solely by visual imitation—is intuitive and technically well motivated. The Drawer–Reviewer framework translates this insight into a concrete workflow, and the ablations provide evidence that both Grounded Measurement and iterative review contribute to the overall performance. This makes the work more than a simple prompt-engineering recipe.

  1. Consistent empirical gains supported by multiple complementary experiments.

On the hand-curated benchmark split, FigMirror achieves a combined score of 72.7, compared with 61.3 for the strongest baseline, ChartIR. On the augmented split, it achieves 76.4 versus 70.3. The component ablations further show that removing Grounded Measurement reduces the combined score from 72.7 to 69.1, while removing the Reviewer reduces it to 70.6. The paper also reports positive results across different model and agent-harness configurations. Together, these results provide meaningful evidence for the proposed approach.

Critical weaknesses
  1. The validation of the automated evaluation metric remains relatively limited.

The combined metric relies substantially on a VLM-based visual evaluator that identifies defects and assigns severity-dependent penalties. Although the paper includes a human preference study with over 300 pairwise judgments, the study covers only 50 references and five annotators. The reported Spearman correlation between the automated score and human win rate is 0.90, but its exact two-sided p-value is 0.083, so the evidence is suggestive rather than conclusive. Consequently, it remains uncertain how reliably the metric ranks methods across a broader range of figures and visual styles. This matters because the primary performance claims rely on the same automated evaluation protocol.

  1. The benchmark composition leaves the breadth of the demonstrated generalization insufficiently established.

PlotTwin-Bench contains 400 references, but 350 are produced by augmenting existing ChartMimic plotting code through an LLM-based rewriting pipeline, whereas only 50 are manually curated. The main evaluation uses 150 references, comprising all 50 hand-curated examples and 100 randomly sampled augmented examples. Although the authors report results separately for these two splits, the augmented subset is still generated through a specific, code-driven construction pipeline. The evidence therefore establishes performance on the proposed benchmark more directly than it establishes robustness across the broader variety of scientific figures encountered in practice.

Questions for authors
  1. Evaluation validity: Can the authors provide additional statistical evidence for the agreement between the automated metric and human preferences, such as confidence intervals or results broken down by chart type? This would help establish whether the reported ranking is robust beyond the five-method aggregate.

  2. Benchmark generalization: How sensitive are the conclusions to the benchmark's reference source? In particular, do the relative rankings of FigMirror and the baselines remain consistent when the augmented references are evaluated separately by chart type or structural complexity?

  3. Potential benchmark familiarity: Given that the augmented references originate from ChartMimic plotting code, what evidence supports the claim that the comparison measures reference-conditioned style transfer rather than partially benefiting from prior familiarity with the source figures or their underlying plotting patterns?

5 comments
AuthorsOpenReview ↗

Response to Reviewer bhyq (Part 1/3)

We appreciate the reviewer for the detailed and constructive feedback and suggestions, which are definitely helpful for us to improve the quality of the paper. We have uploaded a revision to incorporate all your suggestions. Below, we further provide detailed clarifications for each of your questions:

W1: The validation of the automated evaluation metric remains relatively limited. The combined metric relies substantially on a VLM-based visual evaluator that identifies defects and assigns severity-dependent penalties. Although the paper includes a human preference study with over 300 pairwise judgments, the study covers only 50 references and five annotators. The reported Spearman correlation between the automated score and human win rate is 0.90, but its exact two-sided p-value is 0.083, so the evidence is suggestive rather than conclusive. Consequently, it remains uncertain how reliably the metric ranks methods across a broader range of figures and visual styles. This matters because the primary performance claims rely on the same automated evaluation protocol.

Q1: Evaluation validity: Can the authors provide additional statistical evidence for the agreement between the automated metric and human preferences, such as confidence intervals or results broken down by chart type? This would help establish whether the reported ranking is robust beyond the five-method aggregate.

Thanks for raising this point. We clarify that in our original analysis, the Spearman test operated on N=5N = 5 method-level aggregates. Following your suggestion, we evaluate pair-level agreement with 95% bootstrap confidence intervals (A1-a) and break down the human preference study by chart type and panel structure (A1-b), with full details added to Appendix B.2 of the revised manuscript.

a) Pair-level agreement and confidence intervals.

Following your suggestion, to evaluate metric reliability beyond the five-method aggregate, we examine the individual pairwise comparisons from the human preference study and compute 95% bootstrap confidence intervals. Using vision evaluators from two model families (GPT-5.5, GPT-6 Astra, and Gemini 4 Argon), we compare the sign of ΔS\Delta S on each FigMirror-versus-baseline pair with the human majority vote. In Table 1, across all three evaluators, the automated metric aligns with the human majority at a rate comparable to human inter-rater agreement (73.0%–79.7% vs. 78.0%). In Table 2, we report the pairwise human win rates between FigMirror and each baseline. The 95% confidence interval lower bound is above 60% against every baseline, and FigMirror's top ranking holds in every leave-one-annotator-out check.

Table 1. Pair-level agreement between SS and the human majority vote on FigMirror-versus-baseline comparisons (hand-curated set, GPT-5.5 generation).

EvaluatorPair-level agreement with human majority ↑95% CISpearman ρ\rho (five methods) ↑
GPT-5.5 (Table 1 evaluator)73.0%[61.5, 83.2]0.90
GPT-6 Astra73.0%[64.2, 80.8]0.90
Gemini 4 Argon79.7%[70.7, 87.4]0.90
Human vs. human (inter-rater pairs)78.0%[68.7, 86.6]—

Table 2. Reference-bootstrapped human win rates on the hand-curated references (GPT-5.5 generation).

ComparisonFigMirror human win rate ↑95% CI
FigMirror vs. ChartIR77.2%[63.3, 88.7]
FigMirror vs. ChartGalaxy73.9%[62.3, 84.3]
FigMirror vs. Plot2Code76.4%[63.6, 87.7]
FigMirror vs. METAL79.1%[67.7, 89.3]
FigMirror overall (all baselines)76.6%[67.9, 84.2]

b) Human preference broken down by chart type and panel structure.

To verify that the aggregate human preference is not driven by a single dominant chart category, we partition the hand-curated references by chart family (spanning line and band plots, statistical distributions, dense 2D fields, and 3D or composite figures) as well as by single- versus multi-panel layout. As shown in Table 3, human annotators favor FigMirror across every chart family and layout group, with the narrowest confidence interval on multi-panel figures.

Table 3. Human win rate of FigMirror broken down by chart type and panel structure (hand-curated set, GPT-5.5 generation).

GroupFigMirror human win rate ↑95% CI
Line and band81.4%[63.3, 95.9]
Distribution and box75.6%[59.3, 89.4]
Scatter, heatmap, field74.1%[51.2, 92.9]
3D, network, polar, composite75.8%[60.0, 87.7]
Multi-panel78.3%[69.9, 86.0]
Single-panel66.7%[33.3, 89.7]
AuthorsOpenReview ↗

Response to Reviewer bhyq (Part 2/3)

W2: The benchmark composition leaves the breadth of the demonstrated generalization insufficiently established. PlotTwin-Bench contains 400 references, but 350 are produced by augmenting existing ChartMimic plotting code through an LLM-based rewriting pipeline, whereas only 50 are manually curated. The main evaluation uses 150 references, comprising all 50 hand-curated examples and 100 randomly sampled augmented examples. Although the authors report results separately for these two splits, the augmented subset is still generated through a specific, code-driven construction pipeline. The evidence therefore establishes performance on the proposed benchmark more directly than it establishes robustness across the broader variety of scientific figures encountered in practice.

Q2: Benchmark generalization: How sensitive are the conclusions to the benchmark's reference source? In particular, do the relative rankings of FigMirror and the baselines remain consistent when the augmented references are evaluated separately by chart type or structural complexity?

Thanks for this question. Regarding sensitivity to the reference source, FigMirror maintains a consistent lead over the strongest baseline on both the 50 hand-curated figures from published papers and the augmented ChartMimic figures across all five evaluators (see A3, Table 7); half of those hand-curated figures originate from non-Matplotlib tools such as R/ggplot2, GraphPad Prism, and Adobe Illustrator (Appendix C.1, Table 13 of the revised manuscript). Below we further examine whether the method rankings hold when the 100 augmented references in Table 1 of the paper are stratified by chart type, panel layout, and structural complexity (A2-a), and across the remaining 250 augmented references under two cross-family vision judges (A2-b).

a) Stratified rankings on the 100 augmented references in Table 1 of the paper.

To examine ranking consistency across chart types and complexity levels, we partition these 100 references along three dimensions: chart family, single- versus multi-panel layout, and structural complexity measured by tertiles of reference plotting-code length (≤128\le 128 lines, 129–161129\text{--}161 lines, and >161> 161 lines). As shown in Table 4, FigMirror ranks first among all five methods across every chart family, panel structure, and code-complexity tier, with its widest margin over the second-best method appearing on the most complex references (>161> 161 lines of reference code).

Table 4. Stratified performance (SS ↑) on the 100 augmented references in Table 1 of the paper by chart family, panel structure, and reference code complexity (GPT-5.5).

StratumReferencesFigMirror ↑ChartIR ↑ChartGalaxy ↑Plot2Code ↑METAL ↑FigMirror rank
All augmented10076.470.366.050.751.41 / 5
Scatter, heatmap, field3176.468.964.345.549.81 / 5
Distribution and box1976.676.268.354.257.01 / 5
Bar1878.169.470.254.855.11 / 5
Line and band1776.869.169.055.851.01 / 5
3D, network, polar, composite1573.767.958.246.443.81 / 5
Multi-panel9476.070.065.150.351.21 / 5
Single-panel682.274.680.556.356.21 / 5
Reference code ≤128\le 128 lines3577.972.467.750.850.41 / 5
Reference code 129–161129\text{--}161 lines3274.073.469.151.754.71 / 5
Reference code >161> 161 lines3377.164.961.249.649.41 / 5
AuthorsOpenReview ↗

Response to Reviewer bhyq (Part 3/3)

b) Full evaluation on the remaining 250 augmented references by chart family and panel count.

To test whether our conclusions extend beyond the 100-reference sample, we evaluate all methods on the remaining 250 augmented references of PlotTwin-Bench, including stronger baseline variants equipped with code-execution feedback and multiple refinement rounds. Table 5 reports the stratified results across chart families and main-panel counts under two vision judges (GPT-6 Astra and Gemini 4 Argon). Under both evaluators, FigMirror ranks first in every chart family and in every panel-count group from 3 to 6 main panels.

Table 5. Stratified performance (SS ↑) on the remaining 250 augmented references by chart family and main-panel count (GPT-6.1 generation).

JudgeGroupFigMirror ↑ChartIR (3 rounds + exec) ↑ChartIR (original) ↑ChartGalaxy ↑Plot2Code ↑METAL (+ exec) ↑FigMirror rank
GPT-6 AstraBar84.976.674.065.050.950.11 / 6
Line and band84.978.771.861.249.746.91 / 6
Distribution and box84.974.473.161.759.057.31 / 6
Scatter, heatmap, field83.278.273.271.649.952.01 / 6
3D, network, polar, composite86.072.669.563.640.640.91 / 6
3 main panels86.081.274.567.351.752.11 / 6
4 main panels84.277.874.166.050.750.21 / 6
5 main panels85.172.169.559.949.946.81 / 6
6 main panels86.264.772.256.931.239.21 / 6
8 main panels85.287.159.691.168.235.93 / 6
Gemini 4 ArgonBar83.170.868.852.140.137.81 / 6
Line and band82.772.963.549.940.134.81 / 6
Distribution and box83.575.972.452.944.241.61 / 6
Scatter, heatmap, field82.171.670.259.839.444.01 / 6
3D, network, polar, composite82.369.567.148.334.536.31 / 6
3 main panels82.576.169.158.737.440.11 / 6
4 main panels82.972.268.953.040.738.01 / 6
5 main panels82.968.065.046.440.137.21 / 6
6 main panels84.162.267.342.331.236.01 / 6
8 main panels75.574.153.178.171.426.22 / 6

Q3: Potential benchmark familiarity: Given that the augmented references originate from ChartMimic plotting code, what evidence supports the claim that the comparison measures reference-conditioned style transfer rather than partially benefiting from prior familiarity with the source figures or their underlying plotting patterns?

Thanks for raising this concern. We clarify that our benchmark design limits the influence of prior familiarity with ChartMimic in two ways. First, our augmentation pipeline uses ChartMimic only as a seed and iteratively modifies its color palette, typography, spines, grids, and panel layout, leaving almost none of the seed code unchanged (Table 6). Second, even if a model memorized an original ChartMimic script during pretraining, that script contains the old visual style and hardcoded source data, whereas our task requires plotting a newly synthesized dataset from a different scientific domain in the modified reference style. Consistent with this, on the 50 hand-curated references collected directly from published papers, which have no overlap with ChartMimic, FigMirror achieves a larger lead over the strongest baseline than on the augmented references under every evaluator (Table 7).

In addition, to rule out potential model- or harness-level bias from using GPT-family models in our augmentation pipeline, we evaluate non-GPT models and coding-agent harnesses in Appendix A, where FigMirror remains the leading method. FigMirror's top ranking also holds when we re-score the outputs from Table 1 of the paper with vision evaluators from other model families (Table 7).

Table 6. Code divergence between the final augmented reference scripts and their original ChartMimic seeds.

MetricValue
Median plotting-code length (ChartMimic seed →\to augmented reference)74 →\to 142 lines
Median share of final reference code lines absent from the ChartMimic seed98.0%

Table 7. FigMirror's lead over the strongest baseline (ChartIR) on the non-ChartMimic hand-curated references versus the augmented references across five evaluators (ΔS\Delta S ↑, GPT-5.5 generation).

Vision evaluatorHand-curated 50 ΔS\Delta S (FigMirror − ChartIR) ↑Augmented 100 ΔS\Delta S (FigMirror − ChartIR) ↑
GPT-5.5 (Table 1 evaluator)+11.4+6.1
GPT-6 Astra+9.9+7.6
Gemini 4 Argon+7.8+7.3
Gemini 3.8 Flash+6.0+5.3
Claude Opus 5.5+7.5+3.6
Reviewer_qhZbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulates reference-conditioned style transfer for scientific figures. Given a reference figure and new data, produce plotting code that renders the new data in the reference visual style. This paper focuses on a precise style measurement as the core difficulty and propose grounded measurement approach. A model localizes the element carrying each style attribute with bounding-box coordinates and writes a short program that reads the attribute value from those pixels. The approach is motivated by an analogy between figures and graphical interfaces, where frontier models are trained for coordinate grounding.

Critical strengths
  1. The problem is well motivated, and the task is useful. reusing the visual design of a good figure with new data is a real need.

  2. Reading style with code is a sound idea. asking a model to compute a color or a gutter width from pixels removes a class of errors in a principled way.

  3. The evaluation design is also well thought. scoring only attributes that depart from library defaults avoids rewarding generic plots.

  4. The proposed method is portable and runs in three harnesses with models from three families.

Critical weaknesses
  1. I think, the premise of the proposed method that GUI grounding is the enabling capability is an analogy, not a tested claim. Because the proposed method has two steps, first localize, then measure with code. The second step needs no GUI grounding at all, because a measurement script can segment flat fills or sharp edges over the whole image. It looks good that measurement code does the work but I don't find any discussion on this point, no measurement of grounding accuracy on plot elements, no comparison of models with and without computer-use training, and also no condition in which predicted boxes are replaced by random boxes or exact ones derived from the reference code.

  2. Continuing from the above point, I think pixel measurement may not be able to recover several of the attributes it targets. Many style attributes are specified in device-independent units, for instance font sizes and line widths in points, figure size in inches. Their pixel footprint depends on the DPI and figure size, which are not observable from a raster. Anti-aliasing, transparency and overlap also mean that the pixel mean over a region is not the specified color. The paper presents measurement as reading the exact value, but should discuss which attributes are identifiable from pixels and how the others are inferred.

  3. The proposed loop approach itself is not new. Drawer–Reviewer self-refinement seems to be inspired from the ChartIR and METAL. The novelty lies in code-based measurement and box-anchored feedback, so these two components carry the burden of proof, which lacks in the discussion.

  4. The ablations do not support measurement as the source of the gain. Removing grounded measurement raises S_code, and only the VLM-judged S_vision falls. S_code is the channel that checks exact attribute values against the reference code. If precise measurement are the key, it should help there most.

  5. The benchmark's construction introduces its own biases because the hand-curated references are re-implemented in Matplotlib, so every reference is Matplotlib-renderable by construction. The realistic case of a figure made in ggplot2, TikZ or a commercial tool is untested. The augmented references and target data are generated by an LLM. The claim that FigMirror leads in both channels also fails in the Grok setting, where ChartGalaxy has the higher code score.

  6. Overall, the paper is well written, but it seems like framing outruns the evidence.

Questions for authors
  1. Can you test the grounding hypothesis directly, for example with grounding accuracy on plot elements, random versus oracle boxes?
  2. What does the system score when the Drawer receives true attribute values read from the reference code?
  3. Why does removing grounded measurement increase S_code? What happens when only pixel measurement is removed, keeping the floor self-check and boxed reviewer feedback?
  4. How are point-based attributes (font size, line width) recovered from pixels without knowledge of DPI and figure size?
  5. How does performance change when the target data require a different structure from the reference?
  6. What are the tokens, wall-clock time and cost per figure for each method? How do baselines perform with a matched budget and the same tools?
  7. How do rankings change under an evaluator from a different model family? What is the pair-level human agreement and the run-to-run standard deviation of S?
  8. Which configuration files and model IDs produced Tables 1–3? How does S_code parse style set through rcParams or themes?
10 comments
AuthorsOpenReview ↗

Response to Reviewer qhZb (Part 1/6)

Thanks for the careful and constructive comments, which will definitely help us improve the quality of the paper. All your suggestions will be accommodated in our revised paper. Below, we provide detailed clarifications for each of your questions:

W1: I think, the premise of the proposed method that GUI grounding is the enabling capability is an analogy, not a tested claim. Because the proposed method has two steps, first localize, then measure with code. The second step needs no GUI grounding at all, because a measurement script can segment flat fills or sharp edges over the whole image. It looks good that measurement code does the work but I don't find any discussion on this point, no measurement of grounding accuracy on plot elements, no comparison of models with and without computer-use training, and also no condition in which predicted boxes are replaced by random boxes or exact ones derived from the reference code.

Q1: Can you test the grounding hypothesis directly, for example with grounding accuracy on plot elements, random versus oracle boxes?

Thanks for raising this question. Following your suggestion, to isolate the effect of computer-use training, we compare a base model (Qwen3.5-9B) with two counterparts fine-tuned for computer use on the same backbone (Fara1.5-9B and UI-Venus-2-9B), alongside GPT-6 Astra across 412 elements on the hand-curated set. In Table 1, computer-use training on the same 9B backbone increases bounding-box IoU by 2.5×2.5\times to 4×4\times. In Table 2, we freeze the measurement scripts and vary only the input box. Without element localization, whole-image measurement (No box) fails on local attributes such as text height (1.000) and marker size (0.738), and Qwen3.5-9B (base) performs no better than Random box. Adding computer-use training on the 9B backbone consistently lowers error across all four attributes relative to the base model, and stronger localization with GPT-6 Astra substantially reduces error across every attribute (e.g., color from 13.75 to 5.32 and text height from 1.000 to 0.115), approaching the Oracle box. The following experiments are included in Section 5.5 (Table 6) of the revised manuscript.

Table 1. Plot-element localization accuracy on the hand-curated set (412 elements).

ModelCorner-box IoU (95% CI) ↑Point hit rate (95% CI) ↑
Qwen3.5-9B (base)0.041 [0.027, 0.059]14.0% [9.7, 18.7]
Fara1.5-9B (computer-use)0.106 [0.080, 0.139]64.6% [57.2, 71.8]
UI-Venus-2-9B (computer-use)0.163 [0.126, 0.207]66.3% [59.6, 73.0]
GPT-6 Astra0.588 [0.545, 0.628]89.8% [84.3, 94.3]

Table 2. Attribute measurement error with frozen measurement scripts under different box sources (hand-curated set, 412 elements).

Box sourceModelColor (CIEDE2000) ↓Stroke width ↓Marker size ↓Text height ↓
No box (whole image)—13.750.3880.7381.000
Random box—35.910.5000.6720.534
Predicted boxQwen3.5-9B (base)35.880.5300.7320.781
Predicted boxFara1.5-9B (computer-use)24.090.4280.6530.667
Predicted boxUI-Venus-2-9B (computer-use)16.070.3680.5930.493
Predicted boxGPT-6 Astra5.320.3240.2830.115
Oracle box—4.540.3090.2330.015

Q2: What does the system score when the Drawer receives true attribute values read from the reference code?

Thanks for this question. To address your comment and to evaluate the upper bound on downstream generation performance under exact attribute measurement, we supply the ground-truth attribute values read from the reference code in the Drawer prompt. We use ScodeS_{\text{code}} as the evaluation metric here, as it directly verifies style parameters against the reference code and more accurately reflects gains in attribute fidelity. As shown in Table 3, this raises ScodeS_{\text{code}} from 59.0 to 80.8.

Table 3. Code-channel performance (ScodeS_{\text{code}}) when the Drawer receives ground-truth attribute values from the reference code (hand-curated set, GPT-6.1).

SettingScodeS_{\text{code}} ↑
FigMirror (Full)59.0
FigMirror (Full + ground-truth attributes)80.8
AuthorsOpenReview ↗

Response to Reviewer qhZb (Part 2/6)

W2: Continuing from the above point, I think pixel measurement may not be able to recover several of the attributes it targets. Many style attributes are specified in device-independent units, for instance font sizes and line widths in points, figure size in inches. Their pixel footprint depends on the DPI and figure size, which are not observable from a raster. Anti-aliasing, transparency and overlap also mean that the pixel mean over a region is not the specified color. The paper presents measurement as reading the exact value, but should discuss which attributes are identifiable from pixels and how the others are inferred.

Q4: How are point-based attributes (font size, line width) recovered from pixels without knowledge of DPI and figure size?

Thanks for pointing this out. We clarify that given a raster image with unknown DPI and figure size, Grounded Measurement measures canvas aspect ratios and relative pixel proportions (e.g., stroke width and text height relative to the panel). The Drawer uses these ratios to initialize point-based parameters such as line width and font size, and refines them through visual review across rounds. For color, the script takes the median over background-filtered pixels in an unoccluded crop to mitigate anti-aliasing, transparency, and overlap. As shown in Table 4, these measurements reduce error across aspect ratio, line width, and palette color compared with generating without measurement. We have updated Section 3 and Section 3.1 in the revised manuscript to distinguish directly measurable pixel properties from point-based parameters.

Table 4. Approximation error on style attributes with and without Grounded Measurement (hand-curated set, GPT-5.5).

Attribute metricw/o Grounded MeasurementFigMirror (Full)Relative error reduction ↑
Aspect-ratio error (%) ↓4.051.7955.9%
Line-width error (%) ↓33.6925.0225.7%
Palette color distance (CIEDE2000) ↓8.107.369.2%

W3: The proposed loop approach itself is not new. Drawer–Reviewer self-refinement seems to be inspired from the ChartIR and METAL. The novelty lies in code-based measurement and box-anchored feedback, so these two components carry the burden of proof, which lacks in the discussion.

Thanks for raising this point. While ChartIR and METAL apply inference-time Drawer–Reviewer refinement to same-data chart reproduction (Section 2), FigMirror uses this refinement paradigm to optimize a different target: reference-conditioned style transfer to new data. Supporting this transfer target when the underlying data and layout change requires auxiliary mechanisms beyond iterative refinement alone. We therefore pair the loop with localized code-based measurement (evaluated in A1 and A5-b) and box-anchored Reviewer feedback; in Table 2 of the paper, removing the Reviewer and its boxed feedback drops SS from 72.7 to 70.6 (ScodeS_{\text{code}} from 64.4 to 62.7 and SvisionS_{\text{vision}} from 77.2 to 74.9).

AuthorsOpenReview ↗

Response to Reviewer qhZb (Part 3/6)

W4: The ablations do not support measurement as the source of the gain. Removing grounded measurement raises S_code, and only the VLM-judged S_vision falls. S_code is the channel that checks exact attribute values against the reference code. If precise measurement are the key, it should help there most.

Q3: Why does removing grounded measurement increase S_code? What happens when only pixel measurement is removed, keeping the floor self-check and boxed reviewer feedback?

Thanks for raising this question. Our per-attribute decomposition shows that the ScodeS_{\text{code}} inversion in Table 2 of the paper came from a grid-parsing edge case in the original scorer (A5-a), and that removing only pixel measurement while keeping the rest of the loop intact reduces ScodeS_{\text{code}} across the benchmark (A5-b).

a) Why removing Grounded Measurement raised ScodeS_{\text{code}} in Table 2 of the paper?

Following your suggestion, we analyzed the ablation outputs attribute by attribute and found that FigMirror achieves lower error across almost all measured attributes (Table 5). The score inversion came from an edge case in our static code scorer. In Matplotlib, ax.grid accepts a which parameter (which="major" or which="minor"), whereas our scorer ignored the which parameter and parsed ax.grid(False, which="minor") simply as ax.grid(False). Most methods write a single ax.grid(True) call and are unaffected, while FigMirror frequently configures major and minor grids in separate calls, causing the scorer to misread its active major grid as off and assign 0 points on the grid attribute for 11.4% of FigMirror's tasks (compared with 2.3% for w/o Grounded Measurement and 0.3%–1.4% across external baselines). Updating the parser to handle the which parameter resolves the inversion in Table 2 of the paper (Table 6).

Table 5. Full attribute-error comparison on the ablation outputs from Table 2 of the paper (hand-curated set, GPT-5.5).

Attribute metricw/o Grounded MeasurementFigMirror (Full)Relative error reduction ↑
Rendered image aspect-ratio error (%) ↓4.051.7955.9%
Plotting code figsize aspect-ratio error (%) ↓5.003.1337.5%
Palette color distance (CIEDE2000) ↓8.107.369.2%
Code line-width error (%) ↓33.6925.0225.7%
Legend-frame error rate (%) ↓21.4314.2933.3%
Serif font / hidden spines / tick directionTiedTied—

Table 6. ScodeS_{\text{code}} on the ablation from Table 2 of the paper before and after scorer refinement (hand-curated set, GPT-5.5).

Scorer versionFigMirror (Full) ↑w/o Grounded Measurement ↑ΔScode\Delta S_{\text{code}} (w/o GM − Full) ↓
Original scorer64.467.4+3.0
Refined scorer67.166.7−0.4

b) What happens when only pixel measurement is removed, keeping floor self-check and boxed Reviewer feedback?

Following your suggestion, to test this condition, we ran a controlled ablation across the full 400-reference benchmark that removes only executable pixel measurement (w/o Pixel Measurement) while keeping the floor self-check and boxed Reviewer feedback unchanged. Under the submission's original code scorer, removing pixel measurement reduces ScodeS_{\text{code}} by 1.6 points (from 82.0 to 80.4, 95% CI [−2.9, −0.4]) and increases rendered canvas aspect-ratio error from 2.41% to 3.98%. Table 7 shows the score change across the active code-channel attributes (also included in Appendix B.3, Table 12 of the revised manuscript).

Table 7. Per-attribute ScodeS_{\text{code}} ablation when only pixel measurement is removed (400 references, GPT-6.1).

AttributeFigMirror (Full) ↑w/o Pixel Measurement ↑Δ\Delta (w/o PM − Full)
Serif font100.075.0−25.0
Figure aspect ratio85.180.9−4.2
Spines removed95.192.2−2.9
Palette81.079.1−1.9
Grid90.188.5−1.6
Legend frame92.592.1−0.4
Tick direction50.050.00.0
Line width65.165.7+0.6
Total ScodeS_{\text{code}}82.080.4−1.6

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.