acceptodds
Under review as a conference paper at ICLR 2027

Figures as Programs: Recursive Generation of Editable Scientific Figures

Abstract

Scientific methodology figures are essential for communicating complex methods clearly, yet creating them remains labor-intensive and typically requires multiple rounds of refinement. Recent image-generation models can synthesize visually appealing raster figures, but producing a human-satisfactory result in a single generation step remains difficult. Moreover, precise edits to raster figures are challenging for both humans and models. We formulate scientific figure generation as recursive SVG program construction and propose FIGTREE, a multi-agent system that automatically transforms a scientific paper into a structured vector figure. FIGTREE grounds figure content in the source paper, decomposes a figure into a hierarchy of local regions, generates each region as a short SVG program, and assembles the resulting fragments. A render-critic refinement loop jointly inspects the rendered figure and its underlying program, enabling visual defects to be traced to specific statements and accurately repaired. We conduct extensive evaluations of FIGTREE on figure quality and editability, showing that FIGTREE produces high-quality figures, while also enabling more effective editing than existing raster-based methods.

Primary area:
generative models
Keywords:
Scientific Figure Generation, Multi-agent System, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_H4HpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates scientific methodology figure generation as recursive SVG program construction. They propose a multi-agent system called FigTree, composed of a Grounder, a tree of Generators, and a Critic. The Grounder extracts a content plan and a global style specification from the paper. The Generator then expands a layout tree top-down: each node is either drawn as a leaf SVG fragment or decomposed into 2~4 children. Each child returns its fragment, bounding box, and named ports, and the parent places the children and draws all cross-boundary connections. The Critic inspects both the rendered image and the SVG code and returns element-anchored repair instructions, guided by a verification memory distilled offline on AutoFigure-Edit Bench. Experiments are run on PaperBananaBench (292 cases), with Claude Sonnet 4.6 as the generator and Gemini 3.1 Pro used as both the critic and the judge. FigTree achieves the best reference-based score and the best fine-grained diagnostic score among tested methods.

Critical strengths

The paper shifts scientific figure generation from raster images to editable SVG programs. The recursive decomposition, with child-reported ports and parent-owned connections, is a clean way to handle long-horizon coordinate generation. The comparison with Direct-SVG, which uses the same Claude Sonnet 4.6 backbone (3.60 vs. 59.59 overall in Table 1), shows that the gain comes from the system design rather than the backbone. Also, FigTree shows clear improvement in faithfulness and hallucination control, which is arguably the most important property for scientific figures.

Critical weaknesses

The evaluation is closely coupled to the method itself. Gemini 3.1 Pro is used both as the in-loop critic and as the only automatic judge, and the verification memory is distilled by the same critic. Moreover, the generator and critic prompts (flat fills, no gradients, thin strokes, compact canvas, Times New Roman) mirror the authors’ own diagnostic rubric, where Visual Polish penalizes gradients, Compactness penalizes blank margins, and Text/Glyph targets raster artifacts. The method is therefore largely optimized against its own evaluator. The sample size is also unclear. Table 2 is described as evaluated on PaperBananaBench, but its means are consistent with integer scores averaged over only about 20–22 cases, with different n across methods. For example, every entry for Crafter, PaperBanana, Direct-SVG and AutoFigure-Edit is a multiple of 1/22, such as Crafter’s 7.41 = 163/22. If so, the 8.39 vs. 8.28 margin over GPT-Image-2 is within noise. The human study does not solve this either. A Spearman ρ of 0.891 over 10 methods is dominated by the gap between strong and weak systems, and tells little about the ordering among the top methods.

The ablation results are hard to explain, and the editability claim is weakly supported. Removing verification memory drops the overall score to 4.43. This is far below removing the entire render-critic loop (7.86), and even below Direct-SVG (6.40), which has no grounding, recursion, or critic. Content grounding also drops by 5.23, although the memory is described as visual and structural repair rules. Yet the paper gives almost no information about this memory, such as the number of rules, examples, or whether AutoFigure-Edit Bench overlaps with PaperBananaBench. Moreover, GPT-Image-2 is already close to FigTree in Table 2 and even better on topology and flow, so editability becomes the main advantage of FigTree. However, editability is only evaluated on 5 figures against a single raster model, without a vector baseline such as AutoFigure-Edit and without any human editing study.

Questions for authors
  1. Please re-score Table 1 and Table 2 with a VLM judge from a model family not used in FigTree, and report whether the ranking and margins hold. For the human study, were the evaluators blinded, and are any of them authors? Please report inter-annotator agreement and the confidence intervals of HPS in Table 4.

  2. Why does removing verification memory (4.43) perform worse than removing the whole render-critic loop (7.86) and Direct-SVG? How many rules does the memory contain? Was AutoFigure-Edit Bench disjoint from PaperBananaBench at the paper level?

  3. Section 4.1 says each method receives only the method description and caption, but the grounder prompt in Appendix D.1 takes the full paper text. Which input did FigTree actually receive?

  4. In the iterative-editing evaluation, which model edits each branch, and how were the defects injected? Why are AutoFigure-Edit and Crafter not included, given that Crafter’s title also claims editable figure generation?

4 comments

W1. Clarify the independence between the evaluation and the method.

Thank you for raising this important concern. We address them below.

  • Shared judge. We re-scored Tables 1 and 2 with an independent VLM judge, doubao-seed-2-1-pro-260628, from a model family not used anywhere in FIGTREE (full results in our response to Q1). The method ranking is fully preserved: FIGTREE remains the best under both the reference-based protocol (56.65 vs. 48.74 for Crafter) and the fine-grained diagnostic protocol (8.11 vs. 7.90 for GPT-Image-2).

  • Verification memory distilled by the critic. The memory is distilled offline on AutoFigure-Edit Bench, which is disjoint from PaperBananaBench at the paper level, and is frozen before evaluation. If the memory merely encoded Gemini-specific preferences, FIGTREE’s advantage would diminish under a different judge. Instead, it persists under the independent Doubao judge, suggesting that the rules capture general defect-repair patterns.

  • Prompts aligned with the rubric. To verify that the effectiveness of our method does not depend on our own rubric, we further evaluate it with the official PaperBananaBench reference-based protocol, whose criteria were not designed by us. Our method achieves the best overall score as well. (59.59 vs. 50.00 for the strongest baseline, Crafter; Table 1).

W2. Clarify the sample size of Table 2 and the ranking agreement among top methods.

We thank the reviewer for these sharp observations.

On sample size: You are exactly right about the 1/221/22 multiples. We acknowledge that these entries may have been contaminated by an early 22-case debugging subset. We therefore re-ran the four affected methods, Crafter, PaperBanana, Direct-SVG, and AutoFigure-Edit, and re-checked the remaining methods; the remaining methods are confirmed to be correct. The updated scores for the four affected methods are:

MethodSem. Cov.Halluc. Ctrl.TopologyFlowText/GlyphPolishAcad. StyleCompactnessOverall
Crafter6.9096.2386.9698.2768.3647.9277.3046.9627.369
PaperBanana6.2274.8885.9976.3016.9695.7454.2766.9835.923
Direct-SVG6.1436.8535.9626.8746.4275.9836.4135.9906.331
AutoFigure-Edit2.0771.4931.6751.6852.3741.7061.3501.5771.742

This does not affect the conclusion. We assess the margin with the full-set Win/Tie/Lose comparison described in our response to PzaL W2: against GPT-Image-2, FIGTREE wins 148, ties 96, and loses 42 across 286 cases, with a weighted score of 68.53% and a two-sided exact binomial p<0.001 on 190 non-tied cases.

On the Spearman ρ\rho: We agree that the overall ρ=0.891\rho=0.891 is heavily driven by the gap between strong and weak systems. Following your suggestion, we split the methods into Top-5 and Bottom-5 groups to evaluate intra-group ranking. Within the Top-5 group (FIGTREE, GPT-Image-2, Crafter, Direct-SVG, PaperBanana), the Spearman ρ\rho is 0.90. Specifically, both the VLM and human judges agree perfectly on the top-3 (FIGTREE > GPT-Image-2 > Crafter), with only a minor swap between Direct-SVG and PaperBanana at ranks 4–5. This demonstrates that the VLM judge effectively distinguishes fine-grained ordering among strong methods. We will include this split-group analysis in the revision.

W3. Clarify the ablation results and strengthen the evidence for editability.

Please see Q2 and Q4.

Q1. Re-score with an independent VLM judge and report details of the human study.

Thank you for this helpful request. We re-scored Tables 1 and 2 with an independent VLM judge, doubao-seed-2-1-pro-260628. The method ranking is preserved under this judge. FIGTREE remains the best under the reference-based protocol (56.65 vs. 48.74 for Crafter) and the fine-grained diagnostic protocol (8.11 vs. 7.90 for GPT-Image-2).

Table 1

MethodFaithfulnessConcisenessReadabilityAestheticsOverall
Standalone generators (proprietary)
Wan2.6-Image3.282.482.8210.626.81
Nano Banana Pro23.1333.0936.4347.4122.05
GPT-Image-228.1248.4745.0060.0632.10
FLUX.2-Pro0.008.226.368.440.00
Qwen-Image-2.0-Pro4.0115.7018.446.340.00
Agentic raster frameworks
PaperBanana (w/ NB Pro)29.8450.8241.4257.4333.94
Crafter (w/ NB Pro)35.6153.2048.4862.3748.74
Editable and structured systems
AutoFigure-Edit (w/ NB2)13.017.414.458.510.96
Direct-SVG30.2626.285.1413.343.44
FIGTREE43.2661.0149.8061.8056.65

Table 2

MethodSem. Cov.Halluc. Ctrl.TopologyFlowText/GlyphPolishAcad. StyleCompactnessOverall
Wan2.6-Image3.703.653.845.553.414.243.116.734.28
Nano Banana Pro6.063.165.035.665.303.892.696.074.73
FLUX.2-Pro3.953.524.025.303.214.793.526.594.36
Qwen-Image-2.0-Pro3.774.793.896.497.546.274.607.415.60
GPT-Image-27.997.238.117.957.947.928.337.757.90
Crafter6.525.686.407.738.087.476.956.566.91
PaperBanana6.154.345.666.186.225.604.716.575.68
Direct-SVG6.186.345.736.355.945.965.925.776.02
AutoFigure-Edit1.741.371.451.452.151.731.051.551.56
FIGTREE8.198.357.847.808.248.398.607.748.11

For the human study, we sampled 50 cases from PaperBananaBench. All three PhD-level evaluators were blinded to method identity, and none of them are authors of this paper. Each evaluator independently ranked all outputs, with ties allowed. Inter-annotator agreement is Fleiss’ κ=0.37\kappa = 0.37 [0.31,0.43][0.31, 0.43] on pairwise preferences, indicating fair agreement, and Kendall’s W=0.66W = 0.66 [0.60,0.72][0.60, 0.72] on full rankings, indicating substantial agreement. We report 95% bootstrap confidence intervals for HPS below and will add them to our manuscript.

Table 3

MethodHPS95% CI
FIGTREE96.36[93.4, 99.0]
GPT-Image-292.45[88.7, 96.1]
Crafter74.17[66.9, 81.1]
PaperBanana67.22[59.2, 74.8]
Direct-SVG57.97[49.6, 66.0]
FLUX.2-Pro30.17[22.6, 38.2]
Wan2.6-Image30.14[22.5, 38.1]
Qwen-Image-2.0-Pro27.02[19.7, 34.9]
Nano Banana Pro24.49[17.5, 32.2]
AutoFigure-Edit0.00[0.0, 1.2]

Q2. Explain the verification-memory ablation and provide details of the memory.

Thank you for identifying an ambiguity in our explanation. We agree that the paper should describe the verification memory and this ablation in more detail.

Why removing the memory is worse than removing the loop. Removing the entire render–critic loop preserves the initial output of paper-grounded recursive construction. This output can retain the correct components and relationships despite unresolved visual defects. In contrast, retaining refinement without verification memory exposes that output to additional edits without the same rule-guided diagnosis and repair constraints. Such edits can improve a local appearance while damaging an otherwise correct fragment—for example, by merging distinct components to reduce crowding, changing a label’s meaning, or moving a block without consistently updating its connections. Repeated edits across the tree can compound these errors. Thus, the two ablations remove different safeguards: one avoids refinement altogether, whereas the other retains a refinement process that can introduce additional errors.

This distinction also addresses the 5.23-point drop in Content. The consequences of an incorrect repair are not limited to typography or spacing. Deleting, conflating, or relabeling a paper-supported component, or changing a semantic connection, directly reduces content fidelity. Verification memory therefore supports both defect correction and preservation of already-correct scientific content; it is more than an aesthetic checklist.

Memory size and content. The memory contains 65 reusable condition–repair rules, distilled offline and frozen during evaluation. Representative principles are:

  1. Paper Grounding: Preserve paper-supported modules, notation, dimensions, and connection directions. Repair visual defects without deleting scientific content or introducing unsupported elements.
  2. Composition Before Coordinates: Resolve the scientific relationships and visual structure before adjusting positions. Layout repairs must preserve those relationships rather than replace distinct concepts with uniform text boxes.
  3. Scientific Visual Primitives: Use tokens, patches, matrices, tensors, and distributions where they clarify the method. Preserve their ordering and dimensional correspondence, and distinguish schematic illustrations from experimental results.
  4. Validation and Visual Review: Correct overflow, overlaps, and arrows crossing unrelated elements; then check scientific accuracy, proportions, typography, and readability. Anchor each repair to specific elements and preserve unrelated content.

Dataset separation. AutoFigure-Edit Bench, used to construct the memory, is fully disjoint from PaperBananaBench at the paper level, not merely at the image level. The frozen memory transfers reusable defect–repair rules rather than paper-specific examples from the evaluation set.

Q3. Clarify the input provided to FIGTREE.

Thank you for pointing out this inconsistency. FIGTREE, like all baselines, receives only the method description and caption, as stated in Section 4.1. PaperBananaBench also provides the method description, instead of full paper. The references to the “full paper” in Appendix D.1 are wording problem. We will revise them to refer to the method-section text, and update the notation in Section 3.1 accordingly. This does not affect the experimental setup or the fairness of the comparisons.

Q4. Clarify the iterative-editing setup and the exclusion of AutoFigure-Edit and Crafter.

We thank the reviewer. The vector branch uses Claude Sonnet 4.6 to edit the SVG code, while the raster branch uses GPT-Image-2 to edit the image. Both branches start from corresponding versions of the same manually defect-injected figure and use the same defect-free gold-standard reference. Injected defects cover misaligned arrows, overlapping elements, text overflow, incorrect connections, and poor color harmony.

At each round, Gemini 3.1 Pro receives the gold-standard figure and the current rendering of the respective branch, then generates an edit request using the same prompt template. The resulting requests are branch-specific: once the outputs diverge, their remaining defects also differ. Thus, we control the initial defects, reference target, feedback model, and feedback-generation template, rather than requiring identical edit requests across subsequent rounds. We will revise the manuscript to make this distinction explicit.

Moreover, we would like to clarify that this experiment is designed to compare the editability of SVG output with that of its raster counterpart. The generation quality of FIGTREE relative to other baselines has already been evaluated in Tables 1 and 2 of our manuscript.

AuthorsOpenReview ↗

Follow-up discussion

Dear reviewer H4Hp,

We greatly appreciate your efforts to review our submission and your invaluable comments. We have tried our best to carefully consider each of your suggestions and conduct all required experiments.

As the discussion period will conclude today, Nov 18, we respectfully invite you to review our responses. We hope that our additional experiments and clarifications have adequately addressed your concerns. Nevertheless, we remain fully prepared to provide further clarification or discussion on any outstanding points. If you find that our responses have resolved your concerns, we would be deeply grateful if you would consider raising your score in support of our submission.

Thank you again for your thoughtful and valuable review.

Best regards,

Paper14643 Authors

Reviewer_PkEjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents FigTree, a multi-agent system for generating editable scientific methodology figures as SVG programs. The main idea is to break a complex diagram into smaller regions, generate them separately, and assemble them through a hierarchical layout. A grounding stage determines the scientific content and visual style, while a render-critic loop inspects both the rendered figure and its code to guide revisions. The authors evaluate generation quality on PaperBananaBench and include component ablations, human preference judgments, and an iterative-editing experiment. The reported results show improvements in hallucination control and iterative refinement over the evaluated baselines.

Critical strengths
  1. The paper addresses a practical need in scientific communication: generating figures that can be revised after their initial creation.

  2. The system design is coherent, combining source-grounded content planning, recursive decomposition, explicit connections between subfigures, and render-based feedback.

  3. The evaluation includes component ablations and human preference judgments, providing useful evidence beyond automated quality scores alone.

Critical weaknesses

1. Missing comparison with LiveFigure and limited evaluation of practical editability

A closely related recent work, LiveFigure (ICML 2026), is neither cited nor compared against. LiveFigure directly addresses editable scientific figure generation and includes human-in-the-loop experiments measuring the editing effort required to bring generated figures to publication-ready quality. Its experimental design is particularly relevant to the central claims of this paper and deserves careful consideration. Although FigTree includes a human preference study, its editability evaluation is limited to a VLM-driven refinement loop over only five figures. This provides relatively limited evidence of practical usability. A direct comparison with LiveFigure, together with measurements of human editing effort, would substantially strengthen the evaluation.

2. Insufficient qualitative evidence of a clear visual advantage

For a paper focused on scientific figure generation, the qualitative presentation is surprisingly limited. The examples in Figure 4 are not particularly compelling to me and do not, by themselves, demonstrate a clear advantage in visual quality or layout. They are also presented as compact previews without side-by-side comparisons against competing methods, making the claimed improvements difficult to assess visually. The paper would benefit from more representative, full-size examples, direct comparisons on identical inputs, and before-and-after editing sequences that clearly illustrate where the proposed approach makes a meaningful difference.

3. Questionable timeliness and missing comparisons with modern general-purpose coding agents

I am concerned that the practical motivation for this specialized workflow is being weakened by advances in general-purpose coding agents. In my view, a Codex +GPT+PowerPoint workflow is a potentially strong baseline for editable scientific figure generation, and I encourage the authors to test it directly. The existing one-shot Direct-SVG baseline does not represent an interactive coding agent that can execute code, inspect rendered outputs, and iteratively revise a figure. Without a comparison against such workflows under comparable inputs and iteration budgets, I am not convinced that FigTree establishes a substantial practical advantage over readily available general-purpose tools.

Questions for authors
  1. Comparison with LiveFigure and practical editability: How does FigTree compare with LiveFigure in generation quality and editability? Could the authors conduct a human-in-the-loop study measuring the number of editing steps and time required to reach an agreed publication-quality standard?

  2. Qualitative evidence: Could the authors provide more full-size examples, side-by-side comparisons with strong baselines on identical inputs, and before-and-after editing sequences? These would help readers assess the specific visual and editing advantages of the proposed approach.

  3. General-purpose coding-agent baselines: Have the authors evaluated an interactive coding-agent baseline, such as a Codex + GPT + PowerPoint workflow? Under comparable inputs and computational budgets, what advantages does the specialized recursive workflow offer over a general-purpose agent that generates, renders, and revises editable figures?

2 comments

W1&Q1. Compare with LiveFigure and evaluate practical editability through a human study.

Thank you for highlighting LiveFigure. We will cite and discuss it in the revision. Both methods produce editable vector figures: LiveFigure generates PowerPoint objects, whereas FigTree recursively constructs SVG subprograms and jointly reviews code and rendered output for localized repair. Our distinction lies in recursive construction and verification rather than editability alone. Our current experiments do not establish superiority over LiveFigure in generation quality or human editing effort. We agree that a controlled human-in-the-loop comparison would strengthen the evaluation, measuring editing steps and completion time on matched tasks until independent experts judge that a shared publication-quality rubric is met. We did not run this full human-in-the-loop study because, within the short revision timeframe, we could not recruit a sufficient number of real human participants and independent expert judges to make such a comparison meaningful. We will state this practical constraint explicitly and discuss the limitation and intended study design in the revision.

W2&Q2. Provide full-size examples, side-by-side comparisons with baselines, and before-and-after editing sequences.

Thank you for pointing this out. We present the full-size comparison in [https://anonymous.4open.science/r/ICLR-FIGTREE-DEMO-FIGURE-F34C/]; specifically, Crafter.png, CODEX.png, and FigTree.png in this anonymous repository correspond to the three compared figures here. This example illustrates several specific differences between FigTree and the two baselines. Compared with the GPT output, FigTree preserves all three role–scene groups, including the Patient group that is left blank in the GPT result, and presents a more complete end-to-end workflow. Compared with Crafter, FigTree places greater visual emphasis on the central methodological hierarchy: anatomical structures, clinical findings, and question types occupy distinct regions, while green arrows highlight a sampled path across these stages. The topic-finding and scene-driven branches are also spatially separated and visibly converge at the VQA generation stage, making their relationship easier to follow. These observations support FigTree’s advantages in structural clarity, preservation of key components, and traceability of the illustrated workflow.

We present the before-and-after editing sequence illustrations in [https://anonymous.4open.science/r/ICLR-FIGTREE-DEMO-FIGURE-F34C/] We constructed an editing example (Edit.png in the link) by injecting eight visual defects into a reference diagram, including misaligned blocks and labels, an oversized block, an inconsistent font, a heading overlapping a container boundary, and a convoluted connector. As shown in the figure ((a) gold standard, (b) defect-injected input, (c) PNG-based repair, and (d) FIGTREE repair), the PNG-based repair fixes five defects but leaves the oversized block, the misaligned label, and the inconsistent font unresolved, and it replaces the convoluted connector with an abrupt arc. In contrast, FIGTREE fixes seven of the eight defects and restores the original right-angle connector. Both methods fail to fix the inconsistent font. This example illustrates that the SVG representation enables more precise structural and geometric edits than pixel-level modifications.

W3&Q3. Compare with general-purpose coding-agent baselines.

Thank you for your constructive suggestion. We added an interactive coding-agent baseline, Codex + PowerPoint, which generates an editable slide, renders it, inspects the rendered output, and iteratively revises the figure. Under the same evaluation protocol and inputs, it obtains Overall 8.11 on the fine-grained diagnostic evaluation, compared with 8.39 for FIGTREE. We also evaluated Codex + Direct-SVG (6.62) and Codex + FIGTREE + PowerPoint (8.47). These results show that a general-purpose coding agent with PowerPoint is a strong baseline, but the specialized recursive SVG construction and render-critic repair still provide a consistent gain. The combination of FIGTREE with a general coding agent is numerically slightly higher (8.47 vs. 8.39), but the margin is small; we therefore report it as a complementary combination rather than a separate improvement. We will add these baselines to the manuscript and discuss them in the revision.

AuthorsOpenReview ↗

Follow-up discussion

Dear reviewer PkEj,

We greatly appreciate your efforts to review our submission and your invaluable comments. We have tried our best to carefully consider each of your suggestions and conduct all required experiments.

As the discussion period will conclude today, Nov 18, we respectfully invite you to review our responses. We hope that our additional experiments and clarifications have adequately addressed your concerns. Nevertheless, we remain fully prepared to provide further clarification or discussion on any outstanding points. If you find that our responses have resolved your concerns, we would be deeply grateful if you would consider raising your score in support of our submission.

Thank you again for your thoughtful and valuable review.

Best regards,

Paper14643 Authors

Reviewer_PzaLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FigTree, a multi-agent system that generates editable scientific figures by recursively constructing and assembling SVG fragments, with visual and code-based feedback for local repair. Experiments on PaperBananaBench show competitive figure quality, while a small iterative-editing study suggests more effective refinement than raster-based generation.

Critical strengths
  1. The paper addresses a practical need: scientific figures often require precise revisions, and SVG output makes individual elements directly editable.
  2. The method is clearly motivated. Recursive decomposition simplifies complex figures, while joint inspection of rendered images and SVG code supports targeted repairs.
Critical weaknesses
  1. The method relies mostly on prompting and strong closed-source models, without model parameter training. It is evaluated only with Sonnet 4.6 and Gemini 3.1 Pro. Performance with open-source models remains unclear, as does whether the gains depend on the capabilities of this specific model combination.
  2. The overall quality gain over the strongest baseline is limited (8.28 → 8.39), without reported statistical significance. The editing advantage is evaluated on only five figures, providing weak evidence for one of the paper’s central claims.
Questions for authors
  1. Direct-SVG uses a single model call, while FigTree uses multiple rounds of generation, checking, and repair. Is this comparison fair?
  2. What are FigTree’s average number of model calls, token consumption, and generation time per figure?
3 comments

Thank you for your constructive comments. We sincerely appreciate your recognition of the practical need our work addresses and your positive assessment of our method as clearly motivated. We highly value your review and have carefully addressed each of your comments below. We hope our clarifications and additional experiments help resolve your concerns.

W1. Clarify training-free design and evaluate performance on open-source models.

Thank you for pointing out this important concern. We first clarify that the absence of parameter training is a deliberate design choice: the contribution of this work is agentic system design, recursive SVG construction, dual-space render-critic refinement, and verification memory, instead of a new trained model. This is consistent with existing agentic figure-generation frameworks such as PaperBanana, which likewise build on frozen foundation models. This training-free design allows the backbone to be replaced without retraining, so FIGTREE can directly benefit from future advances in foundation models. To examine whether the gains depend on the specific Sonnet 4.6 + Gemini 3.1 Pro combination, we conducted additional experiments with the open-source Qwen 3.8 MAX, which replaces all roles in FIGTREE, including both the generator and the critic. The workflow, prompts, and iteration budget are kept unchanged. For a controlled comparison, Direct-SVG uses the same Qwen 3.8 MAX backbone. The results are shown in the following table.

BackboneDirect-SVGFIGTREEGain
Sonnet 4.6 + Gemini 3.1 Pro6.408.39+1.99
Qwen 3.8 MAX5.727.77+2.05

Although the open-source backbone yields lower absolute quality, FIGTREE achieves a comparable improvement over Direct-SVG (+2.05 vs. +1.99). This indicates that the gains stem primarily from recursive construction and render-critic refinement rather than from the specific Sonnet/Gemini combination. We will add these results to the revision.

W2. Report the statistical significance of the quality gain and expand the iterative-editing evaluation.

Thank you for your valuable comments. To assess whether the quality gap is reliable rather than noise, we ran a full-set pairwise preference comparison on PaperBananaBench, following the PaperBanana evaluation style. For each case, a judge (Gemini 3.1 Pro) compared FIGTREE and GPT-Image-2 side by side on identical inputs and returned Win / Tie / Lose. On the full set of cases, FIGTREE wins 148, ties 96, and loses 42 against GPT-Image-2, giving a weighted score of 68.53% (win=100, tie=50, lose=0). Excluding ties, a two-sided exact binomial test on 148 wins out of 190 non-tied cases gives p<0.001. We use this pairwise comparison as the primary evidence for the quality gap and will add the sampling procedure, judge prompt, and per-case outcomes to the revision.

On the editing evaluation, we agree that five figures were insufficient for a central claim. We have expanded the iterative-editing study from 5 to 25 figures and rerun all eight editing rounds under the same protocol. The advantage of FIGTREE becomes clearer and more stable as editing proceeds: at round 8, FIGTREE reaches 8.524 versus 7.918 for GPT-Image-2, and it consistently leads from round 3 onward. We will replace Figure 2 with the 25-figure results, report the per-round averages and variance, and state the sample size explicitly in the revision.

Q1. Clarify the fairness of the comparison between Direct-SVG and FIGTREE.

Thank you for your insightful question. Direct-SVG is designed as a backbone-control baseline rather than a compute-matched one: it uses the same SVG generation model and identical inputs as FIGTREE, isolating the contribution of our system design from the capability of the backbone. Multi-round generation, verification, and repair are integral to the proposed method, so restricting FIGTREE to a single call would remove the very mechanism under study.

We agree that the computational budgets differ, and our results show that the gains do not come merely from additional model calls:

  • Multi-round variant without recursion. In the ablation study (Table 3), the variant without recursive decomposition still uses multiple rounds of critic-guided repair, yet it reaches only 6.27, compared with 8.38 for the full FIGTREE.
  • Multi-round agentic baselines. PaperBanana and Crafter also perform iterative planning and refinement under their default budgets, and FIGTREE outperforms both.
  • Cost. FIGTREE achieves higher quality at a lower average cost per figure than GPT-Image-2 and Crafter (Appendix A.1).

Q2. Report the average number of model calls, token consumption, and generation time per figure.

Thank you for your good question. We randomly sampled 50 cases and profiled the computational cost of FIGTREE. On average, FIGTREE requires 24.7 model calls, 561.4K tokens, and 4.3 minutes per figure. Most tokens are input tokens, since each call includes the relevant paper content and the current SVG fragment, while the generated outputs are short local programs. Despite the multi-round design, FIGTREE achieves a lower average cost per figure than GPT-Image-2 and Crafter (Appendix A.1). We will add this analysis, including the input/output token breakdown and variance, to the revision.

AuthorsOpenReview ↗

Follow-up discussion

Dear reviewer PzaL,

We greatly appreciate your efforts to review our submission and your invaluable comments. We have tried our best to carefully consider each of your suggestions and conduct all required experiments.

As the discussion period will conclude today, Nov 18, we respectfully invite you to review our responses. We hope that our additional experiments and clarifications have adequately addressed your concerns. Nevertheless, we remain fully prepared to provide further clarification or discussion on any outstanding points. If you find that our responses have resolved your concerns, we would be deeply grateful if you would consider raising your score in support of our submission.

Thank you again for your thoughtful and valuable review.

Best regards,

Paper14643 Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and the AC

We truly appreciate all reviewers and the AC for their effort and time. We are encouraged to receive the following recognition:

  • Practical, clearly motivated research question: generating scientific figures as editable SVG programs rather than raster images (PzaL, PkEj, H4Hp)
  • Coherent system design: source-grounded content planning, recursive decomposition with child-reported ports and parent-owned connections, and joint render-code feedback (PkEj, H4Hp)
  • Clear gains in faithfulness and hallucination control, arguably the most important property for scientific figures (H4Hp)
  • Useful ablations and human preference judgments beyond automated scores (PkEj)
  • Clean control against Direct-SVG showing the gain comes from system design rather than the backbone (H4Hp)

The comments have largely improved our study. As suggested, we have made the following improvements:

  • Added an open-source backbone experiment (Qwen 3.8 MAX replacing all roles), showing a comparable gain over Direct-SVG (+2.05 vs. +1.99).
  • Added a full-set Win/Tie/Lose preference comparison with a two-sided exact binomial test (p < 0.001 on 190 non-tied cases).
  • Expanded the iterative-editing study from 5 to 25 figures and reran all eight rounds.
  • Reported cost profiling: 24.7 model calls, 561.4K tokens, and 4.3 minutes per figure on average.
  • Re-scored Tables 1 and 2 with an independent VLM judge (doubao-seed-2-1-pro-260628), preserving the ranking and margins.
  • Added a split-group Spearman analysis for the Top-5 group (ρ = 0.90).
  • Added an interactive coding-agent baseline (Codex + PowerPoint) and its combinations.
  • Added full-size side-by-side comparisons and a before-and-after editing sequence in an anonymous repository.
  • Clarified the verification memory (65 rules, dataset separation, and why removing it is worse than removing the loop).
  • Clarified the input provided to FIGTREE and the iterative-editing setup.

We thank again all reviewers and the AC for their invaluable service to our community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.