- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a reference-conditioned style transfer task for scientific figures, rather than reproducing the reference figure itself. They propose a new method called FigMirror packaged as a skill that runs inside agentic coding harnesses. FigMirror is composed of two core components, Drawer and Reviewer, which run in a loop combined with grounded measurement and style checklist. The key component in their method is Grounded Measurement, which localizes the visual element carrying each style attribute by coordinates and reads the attribute’s value from the image with a short program. Style attributes are tracked in a Style Checklist, and generation is organized as a Drawer–Reviewer loop: the Drawer measures the attributes and renders a draft, and the Reviewer compares the draft with the reference and returns box-grounded Revision and Preserve Lists. The authors also construct a benchmark called PlotTwin-Bench that includes 50 hand-drawn figure-code pair and 350 LLM-based rewriten pairs across 12 chart types. The core experiments are run on a 150-reference subset, with GPT-5.5 used for both generation and evaluation. In style-transfer task, FigMirror achieves the best result among the compared methods ChartIR, ChartGalaxy-Prompt, METAL, and Plot2Code.
Critical strengths
The paper shifts chart-to-code generation from reproducing a reference figure to transferring its visual style onto the user’s own data. This better matches how researchers actually use well-crafted figures from top venues as templates, which differs it from existing researches. Also, the authors construct a new benchmark PlotTwin-Bench. If fully released, it might be beneficial to the future community.
Critical weaknesses
- The paper attributes its gains to Grounded Measurement, which repurposes GUI grounding for precise attribute measurement. Yet it never evaluates measurement accuracy, even though every reference has ground-truth code against which measured colors, line widths, and aspect ratios could be checked. The ablations point elsewhere. Removing Grounded Measurement raises S_code (from 64.4 to 67.4), even though the code channel scores exactly these attributes; the loss appears only in the vision channel. Moreover, in Claude Code the naked harness without the skill already reaches S = 75.5, far above all external baselines (≤ 58.1). Much of the reported margin therefore comes from the agentic harness rather than the proposed method. Without evidence for the mechanism, the contribution reduces largely to an engineered prompt bundle.
- The comparison of FigMirror with baselines seems unfair. FigMirror’s launch template encodes benchmark-specific rules that mirror the target-data generator, such as a panel contract tied to the data’s panel column and chart-type preservation. Its Reviewer also explicitly penalizes default matplotlib styling and dropped signature motifs, which are exactly what the two scoring channels reward. The paper does not state what instructions, execution access, or compute the baselines received. Even the naked-Codex ablation gets only a one-sentence prompt, so its 11.6-point drop conflates the method with the task contract. It is also unclear whether the skill was developed on references disjoint from the evaluation subset, which includes all 50 hand-curated figures.
Questions for authors
- Did the baselines and the naked-Codex variant receive the same task instructions as FigMirror’s launch template (panel contract, chart-type preservation, data_echo.md)? Please report a naked-harness baseline that receives the full launch template but not the skill. Was the skill developed on references disjoint from the evaluation subset?
- Which evaluator was used for the Claude Code, Grok Build, and GPT-5.6 experiments in Appendix A? Please re-score all Table 1 outputs with a VLM evaluator from a different model family (e.g., Claude or Gemini), and report whether the ranking and margins hold.
- Regarding the S_code increase without Grounded Measurement, please provide a per-attribute breakdown of the code channel (palette, line width, aspect, grid, spines, font, etc.) and directly report the error between Drawer-measured values and ground-truth reference code, with and without Grounded Measurement.
- How are point-based attributes such as line width and font size recovered from a screenshot with unknown DPI and figsize? Or are only relative ratios recovered?
- Why were only 150 of the 400 references evaluated? What are the results on the remaining 250? How were the qualitative examples in the appendix selected?
6 comments
Response to Reviewer Sa1S (Part 2/5)
a) Per-attribute error against the reference code and why flipped in Table 2 of the paper.
Following your suggestion, we analyzed the ablation outputs from Table 2 of the paper attribute by attribute against the ground-truth reference code and found that FigMirror (Full) achieves lower error than w/o Grounded Measurement across almost all measured attributes, namely aspect ratio, line width, palette color, and legend frame (Table 1). The apparent inversion in Table 2 of the paper occurred because FigMirror often configures major and minor grids in separate calls, such as ax.grid(True, which="major") followed by ax.grid(False, which="minor"). Our original code scorer ignored the which argument and parsed the second call simply as ax.grid(False), assigning 0 points on the grid attribute for 11.4% of FigMirror's tasks (compared with 2.3% for w/o Grounded Measurement and 0.3%–1.4% across external baselines). Updating our code scorer to handle the which argument resolves the inversion in Table 2 of the paper (Table 2). Under the submission's original code scorer, removing only executable pixel measurement lowers on the full 400-reference benchmark (from 82.0 to 80.4), and Table 3 reports the breakdown across the active code-channel attributes. We have also included these per-attribute code-channel breakdowns in Appendix B.3 (Table 12) of the revised manuscript.
Table 1. Full attribute-error comparison on the ablation outputs from Table 2 of the paper (hand-curated set, GPT-5.5).
| Attribute metric | w/o Grounded Measurement | FigMirror (Full) | Relative error reduction ↑ |
|---|---|---|---|
| Rendered image aspect-ratio error (%) ↓ | 4.05 | 1.79 | 55.9% |
| Plotting code figsize aspect-ratio error (%) ↓ | 5.00 | 3.13 | 37.5% |
| Palette color distance (CIEDE2000) ↓ | 8.10 | 7.36 | 9.2% |
| Code line-width error (%) ↓ | 33.69 | 25.02 | 25.7% |
| Legend-frame error rate (%) ↓ | 21.43 | 14.29 | 33.3% |
| Serif font / hidden spines / tick direction | Tied | Tied | — |
Table 2. on the ablation from Table 2 of the paper before and after scorer refinement (hand-curated set, GPT-5.5).
| Scorer version | FigMirror (Full) ↑ | w/o Grounded Measurement ↑ | (w/o GM − Full) ↓ |
|---|---|---|---|
| Original scorer | 64.4 | 67.4 | +3.0 |
| Refined scorer | 67.1 | 66.7 | −0.4 |
Table 3. Additional breakdown across active code-channel attributes when only executable pixel measurement is removed (400 references, GPT-6.1).
| Attribute | FigMirror (Full) ↑ | w/o Pixel Measurement ↑ | (w/o PM − Full) |
|---|---|---|---|
| Serif font | 100.0 | 75.0 | −25.0 |
| Figure aspect ratio | 85.1 | 80.9 | −4.2 |
| Spines removed | 95.1 | 92.2 | −2.9 |
| Palette | 81.0 | 79.1 | −1.9 |
| Grid | 90.1 | 88.5 | −1.6 |
| Legend frame | 92.5 | 92.1 | −0.4 |
| Tick direction | 50.0 | 50.0 | 0.0 |
| Line width | 65.1 | 65.7 | +0.6 |
| Total | 82.0 | 80.4 | −1.6 |
b) Intermediate measurement accuracy against the reference code.
Following your suggestion, beyond the final output errors in Table 1, we further evaluate the intermediate measurement step against the ground truth extracted from the reference code across 412 plot elements on the hand-curated set (now included in Section 5.5, Table 6 of the revised manuscript). Specifically, we freeze the attribute-measurement scripts and compare running them without grounding on the full figure (No grounding), on a Random box, on the bounding boxes localized by Grounded Measurement (Predicted box), and on the ground-truth bounding boxes from the reference code (Oracle box). As shown in Table 4, measuring without element grounding yields large errors, whereas Grounded Measurement cuts color error by more than , marker-size error by more than , and text-height error by nearly , approaching the Oracle box.
Table 4. Intermediate attribute measurement error against the reference ground truth (hand-curated set, 412 elements).
| Measurement condition | Color (CIEDE2000) ↓ | Stroke width ↓ | Marker size ↓ | Text height ↓ |
|---|---|---|---|---|
| No grounding (whole image) | 13.75 | 0.388 | 0.738 | 1.000 |
| Random box (same panel) | 35.91 | 0.500 | 0.672 | 0.534 |
| Grounded Measurement (Predicted box, GPT-6.1) | 5.94 | 0.309 | 0.304 | 0.099 |
| Grounded Measurement (Predicted box, GPT-6 Astra) | 5.32 | 0.324 | 0.283 | 0.115 |
| Oracle box (from reference code) | 4.54 | 0.309 | 0.233 | 0.015 |
Response to Reviewer Sa1S (Part 3/5)
W2: The comparison of FigMirror with baselines seems unfair. FigMirror’s launch template encodes benchmark-specific rules that mirror the target-data generator, such as a panel contract tied to the data’s panel column and chart-type preservation. Its Reviewer also explicitly penalizes default matplotlib styling and dropped signature motifs, which are exactly what the two scoring channels reward. The paper does not state what instructions, execution access, or compute the baselines received. Even the naked-Codex ablation gets only a one-sentence prompt, so its 11.6-point drop conflates the method with the task contract. It is also unclear whether the skill was developed on references disjoint from the evaluation subset, which includes all 50 hand-curated figures.
Q1: Did the baselines and the naked-Codex variant receive the same task instructions as FigMirror’s launch template (panel contract, chart-type preservation, data_echo.md)? Please report a naked-harness baseline that receives the full launch template but not the skill. Was the skill developed on references disjoint from the evaluation subset?
Thanks for the suggestion on experimental controls. We clarify the prompt setup and show in Table 5 that under identical launch instructions, FigMirror outperforms Naked Codex, while removing the two Reviewer clauses you noted slightly increases FigMirror's score (A2-a). We also confirm that the skill was developed on figures disjoint from PlotTwin-Bench and generalizes to the 250 untouched references (A2-b), and show in Table 6 that FigMirror maintains a clear margin when external baselines receive code-execution feedback under a comparable budget (A2-c).
a) Task instructions and naked harness with the full launch template.
We clarify that in the main paper, FigMirror used its launch template (data_echo.md, the panel contract, and chart-type preservation), Naked Codex in Table 2 of the paper used a direct one-sentence task prompt, and the external baselines received the reference image, target CSV, and style-transfer task prompt through their respective published pipelines. Following your suggestion, we add two controlled variants on the hand-curated set in Table 5: (1) Naked Codex receiving the complete FigMirror launch template without the skill, and (2) FigMirror (w/o default-style & motif rules), which removes the Reviewer clauses penalizing default Matplotlib styling and dropped signature motifs. As shown in Table 5, removing those two clauses actually yields a slightly higher score (80.2 vs. 78.9 on , and 62.3 vs. 59.0 on ). Under identical launch instructions, both FigMirror variants outperform Naked Codex on and while cutting rendered canvas aspect-ratio error by roughly half (3.37% and 3.91% vs. 7.20%), which is consistent with our Claude Code results in Appendix A.2 (Table 6 of the paper), where adding the FigMirror skill to the naked harness raises from 75.5 to 80.5 and by 9.1 points (55.7 to 64.8).
Table 5. Controlled comparison between Naked Codex receiving the full launch template, FigMirror without the default-style and motif rules, and full FigMirror (hand-curated set, GPT-6.1, GPT-6 Astra evaluator).
| Method | ↑ | ↑ | ↑ | Rendered aspect-ratio error (%) ↓ |
|---|---|---|---|---|
| Naked Codex (full launch template, no skill) | 55.3 | 89.6 | 77.6 | 7.20 |
| FigMirror (w/o default-style & motif rules) | 62.3 | 89.8 | 80.2 | 3.91 |
| FigMirror | 59.0 | 89.6 | 78.9 | 3.37 |
b) Was the skill developed on references disjoint from the evaluation subset?
Yes. We developed the FigMirror skill on a separate set of complex paper figures outside PlotTwin-Bench before constructing the evaluation benchmark and automated scorer, and we never optimized the skill against evaluator scores on the evaluation set. Moreover, as shown in A2-a (Table 5), removing the two Reviewer clauses on default styling and signature motifs slightly raises from 78.9 to 80.2. Evaluating the frozen skill and launch template on the remaining 250 untouched references of PlotTwin-Bench further confirms generalization to unseen figures, where FigMirror leads all baselines by a clear margin (Table 6 and A5).
Response to Reviewer Sa1S (Part 1/5)
We appreciate the reviewer for the detailed and constructive feedback and comments, which are definitely helpful for us to improve the quality of the paper. We have uploaded a revised version to accommodate all your suggestions. Moreover, following your suggestion, we will publicly release our complete PlotTwin-Bench dataset and the codebase. Below, we further make detailed clarifications for each of your questions:
Thanks for raising these points. To clarify, our per-attribute decomposition shows that the Scode inversion in Table 2 of the paper came from a grid-parsing edge case in the original scorer (A1-a), and direct evaluation confirms that Grounded Measurement substantially reduces attribute-reading error toward the Oracle box (A1-b). For the harness comparison raised in W1, we evaluate FigMirror against a matched-instruction naked harness and execution-equipped baselines in A2 below.