acceptodds
Under review as a conference paper at ICLR 2027

Planning Before Drawing: Dual-Expert On-Policy Distillation for Text-to-SVG

Abstract

Text-to-SVG generation aims to translate textual descriptions into executable SVG code that produces visually faithful and editable vector graphics. A key challenge is to preserve visual fidelity while organizing low-level drawing commands into meaningful semantic components that better reflect the compositional structure of the depicted content. Existing methods improve SVG generation through large scale data, SVG specific representations, and reinforcement learning, yet typically learn semantic organization and geometric realization within a single model. Since planning what to draw and realizing how to draw involve distinct objectives, joint optimization can cause competing supervision and limit both capabilities. To address these, we propose DEFT, a dual expert framework that separates planning from drawing and then integrates their complementary knowledge. We first construct SVGVerse, a semantically structured SVG dataset, and perform two stage supervised fine tuning to establish correspondences among textual descriptions, semantic components, and drawing commands. We then optimize a Planning Expert for semantic organization and a Drawing Expert for geometric realization, followed by Command Routed On-Policy Distillation that selectively transfers their expertise according to the semantic roles of SVG commands. Extensive experiments demonstrate that our method achieves superior performance over traditional SVG generation methods, zero-shot models, and specialized SVG-LLMs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-SVG Generation, Multimodal Large Language Models, Reinforcement Learning, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GhcfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies Text-to-SVG generation and argues that semantic component organization (planning) and geometric realization (drawing) are distinct capabilities that interfere under joint optimization. The authors first build SVGVerse, a 196,757-sample SVG dataset in which visually coherent elements are reorganized into explicit semantic groups via multi-LLM reconstruction filtered by a fine-tuned VLM evaluator and human review. They then propose DEFT: (i) two-stage SFT establishing correspondences among text, semantic components, and drawing commands; (ii) independent GRPO optimization of a Planning Expert (format × (recall + Jaccard + order) reward) and a Drawing Expert (format × (DreamSim + LPIPS) reward) from the shared SFT initialization; and (iii) Command-Routed On-Policy Distillation that integrates the two experts into a single model, applying forward KL on planning commands (<g>, <title>, <desc>) and reverse KL on drawing commands (<path>, <rect>, <circle>, <polygon>). Experiments on SVGVerse, SArena, and MMSVG-Bench show competitive or best results among 8B-scale and larger baselines, and ablations validate command routing, the planning→drawing distillation order, and the asymmetric KL design. The claimed conclusion is that separating and re-integrating planning and drawing yields both better semantic organization and better visual quality.

Critical strengths
  1. Well-motivated framework with direct evidence for the motivating premise: Table 6 shows joint multi-reward RL underperforms each specialized expert on its own axis, justifying the separate-then-integrate design rather than asserting it (Sec. 1, Sec. 5.4).
  2. SVGVerse is a substantive data contribution: 196k semantically grouped SVGs with a three-stage quality pipeline (multi-model reconstruction, VLM evaluator, human verification; Sec. 3, Fig. 1). If released, it has independent value to the community.
  3. Command-routed OPD is a concrete and well-ablated instantiation: token-level routing by the semantic role of SVG commands, plus the FKL-for-planning / RKL-for-drawing asymmetry, each supported by dedicated ablations (Table 5).
  4. Broad evaluation: three evaluation settings, 20+ baselines spanning optimization-based methods, general VLMs, proprietary models, and specialized SVG-LLMs (Tables 2–4). DEFT is best on all four SVGVerse drawing metrics and on all three SArena-Illustration metrics at 8B scale.
  5. The analysis section (Sec. 5.4) is denser than typical for this subfield, including reward-interference analysis (Table 6) and per-component reward ablations (Table 7).
Critical weaknesses
  1. The method is essentially an application of MOPD (Ma et al., 2026) to Text-to-SVG. MOPD which is listed in the paper's own bibliography but never discussed in the main text, proposes the same skeleton: specialized per-domain RL experts trained from a shared SFT initialization and integrated into a single student via on-policy distillation on the student's own rollouts. DEFT's deltas are (i) token-level routing by the semantic role of SVG commands instead of prompt-level routing, (ii) a two-stage sequential distillation with asymmetric KL objectives, and (iii) the SVG instantiation with specialized rewards and the SVGVerse dataset. These are competent, well-ablated engineering adaptations, but they amount to transferring an existing post-training framework to a new modality rather than contributing a new method. Command routing and the FKL/RKL choice are local design decisions, not a new methodological principle. A rebuttal showing DEFT beats an MOPD-style baseline would demonstrate good adaptation, not new methodology.
  2. The related work is placed in the supplementary material, so the main text is not self-contained. Related work is precisely where the novelty positioning belongs; moving it out of the main text means readers cannot assess the increment over the closest works (MOPD, SVGBuider) from the paper itself. Consequently, roughly ten bibliography entries (DeepSVG, ClipDraw, CLIPasso, SVGBuider, SVGenius, SVGEditBench v1/v2, FIGR, PPO, MOPD) are never cited in the main text. Whatever the motivation (e.g., page pressure), this is a structural compliance issue and has the practical effect of avoiding a head-on comparison with the most related method.
  3. In-domain evaluation is partly circular: SVGVerse test references are themselves LLM reconstructions, and planning metrics (F1/LCS) are computed against the reconstruction pipeline's own component sequences, on which the model is trained. No human evaluation anchors these proxies to actual human judgments of semantic organization or editability — the very motivation of the paper (Abstract).
  4. Reproducibility: no code/data release statement; GRPO hyperparameters, reward shaping details (sigmoid parameters, BERT model), and compute are absent from the main text (deferred to appendices).
  5. The abstract's blanket superiority claim is contradicted in places by the paper's own tables (SArena-Icon FID 10.723 vs. InternSVG 8.715; MMSVG-Illustration HPS 0.235 vs. Chat2SVG 0.250; Aesthetic 4.64 vs. VectorFusion 4.94). The more careful claims in Sec. 5.2 are the right caliber.
  6. Suspicious or unverifiable details: the CLIP cosine threshold of 0.95 for text–image filtering (Sec. 3.1) is implausibly high on the usual CLIP similarity scale; "Qwen3.8-Max" does not match its cited reference; the "plan-then-generate (Li et al., 2026)" citation appears mismatched with GeoSVG-RL; the SAgoge dataset is used without a citation.
  7. CTRL-S lacks SArena-Illustration results without explanation; there is no failure-case analysis and no limitations section.
Questions for authors
  1. Is DEFT's increment over MOPD (Ma et al., 2026) — which you cite but never discuss — limited to an SVG-domain adaptation? Do command-level routing and the asymmetric KL constitute a general methodological contribution transferable to other structured/code generation tasks? If so, please argue or demonstrate it; if not, please articulate the independent methodological value beyond MOPD.
  2. What exactly is the CLIP similarity threshold of 0.95 in Sec. 3.1 (encoder, normalization)? How many samples survive it?
  3. Will SVGVerse and DEFT weights be released, under what license and timeline?
  4. The reconstruction evaluator is Qwen3-VL-8B-based and the generator is Qwen3-VL-8B-Instruct; does this family overlap bias data selection? What fraction of reconstructions underwent human review, and what was the correction rate?
  5. Is there any human-annotated gold standard for semantic component organization against which the reconstruction references themselves are validated?
1 comment

Dear Reviewer Ghcf,

As the discussion deadline approaches, we thank you again for your constructive feedback on our methodological contribution, evaluation reliability, and baseline comparisons. We have clarified DEFT's distinction from MOPD through analysis and controlled comparisons, and illustrated potential extensions to Mermaid and TikZ. We also clarified the human validation process and downstream editing evidence, completed the CTRL-S results, and added complex illustration comparisons and failure analyses. We would appreciate your feedback on whether these additions and clarifications address your concerns. If you have any remaining questions, please let us know so that we can respond before the discussion closes.

Reviewer_9cnCOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DEFT separately trains planning and drawing experts and combines them through command-routed distillation for text-to-SVG generation. The paper introduces semantically structured SVGVerse data and evaluates visual quality, component organization, and editing performance.

Critical strengths
  1. Separating semantic organization and geometric realization is a coherent design, supported by ablations on expert distillation.

  2. Component-consistency and editing evaluations provide evidence for the practical value of semantically structured SVGs.

Critical weaknesses
  1. Limited comparison with contemporary strong models. Coverage of recent GPT, Claude, and DeepSeek models available before submission is limited. A newer GPT model constructs the editing instructions and targets but is omitted as a baseline (Appendix D, p. 19). The baseline selection is insufficiently justified.

  2. Limited evidence for planning in complex illustrations. Illustration benchmarks are evaluated, but qualitative examples mainly show simple icons. Planning metrics emphasize component matching and ordering, providing limited evidence about complex layouts and object relationships. The ablations do not fully separate planning supervision from additional training, while the editing study combines multiple representation changes.

Questions for authors
  1. Why is the GPT model used to construct editing targets omitted as a baseline, and how were contemporary GPT, Claude, and DeepSeek baselines selected?

  2. Can you show representative complex outputs and failures from both illustration benchmarks and clarify whether the full test sets were evaluated?

  3. Does planning improve object coverage, attribute binding, and spatial relations when compared with a control using the same training budget?

3 comments

Dear Reviewer 9cnC,

As the discussion deadline approaches, we thank you again for your constructive suggestions on stronger baselines, complex illustrations, and the contribution of planning supervision. We have expanded comparisons with GPT, Claude, and DeepSeek models, added complex examples and failure analyses from both illustration benchmarks, and introduced controls for additional distillation and planning supervision. The ablation with matched training budgets shows improvements in object coverage, attribute binding, and spatial relations. We also clarified full test-set coverage and the scope of the editing evaluation. We would appreciate your feedback on whether these additions address your concerns. If you have any remaining questions, please let us know so that we can respond before the discussion closes.

Reviewer_9cnCOpenReview ↗

Response to Rebuttal

Thank you for the response, I think my concerns are addressed, I will raise my rating. Thank you again for the authors.

Thank you for taking the time to read our rebuttal and for letting us know that it addressed your concerns.

Reviewer_qUPaOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies text-to-SVG generation with both visual fidelity and meaningful component organization. It introduces SVGVerse, a reconstruction of existing SVG data into named semantic groups, and DEFT, a training procedure that combines separate planning and drawing experts. Both experts generate complete SVGs, but are optimized using different rewards. They are then distilled into a single model, with planning supervision applied to semantic markup and drawing supervision applied to geometric commands. Experiments on SVGVerse and two external benchmarks, together with ablations and additional component and editing evaluations, support improvements in generation quality and the usefulness of the reconstructed representation.

Critical strengths
  • The paper makes a useful contribution to generating SVGs that are easier to understand and edit. Visual quality alone does not capture whether an SVG has a meaningful component structure. SVGVerse provides explicit supervision for this structure, and the editing results show that the reconstructed representation benefits both DEFT and the base model. The component-level inspection further supports that the generated groups correspond to their stated content. I find this a practically relevant contribution beyond improving the appearance of the rendered image.

  • DEFT offers an effective way to combine semantic organization and drawing quality in one model. Assigning the two teachers to different types of SVG commands is a natural use of the output structure. The method performs well on the authors’ dataset and on external benchmarks, while the routing and distillation ablations support the specific integration design. Taken together, the results make a convincing empirical case for the approach within SVG generation.

Critical weaknesses

A central motivation in the paper is that separate experts overcome competition between planning and drawing objectives. Table 6 shows limitations of the tested joint-RL policy, but the final DEFT model also receives additional distillation. Since distillation improves even the single-expert variants, the results do not establish how much of DEFT’s advantage comes from separating the objectives.

Questions for authors

Tables 5–7 report component F1 and LCS, but the relationship between their matching rules, normalization, and aggregation is unclear, how are the planning F1 and LCS scores in Tables 5–7 computed ?

1 comment

Dear Reviewer qUPa,

As the discussion deadline approaches, we thank you again for your constructive suggestions on distinguishing expert specialization from additional distillation and clarifying the planning metrics. In our responses, we added Joint-RL + OPD, showing that DEFT improves all six metrics under the same distillation procedure, and clarified the matching, normalization, and aggregation of F1 and LCS. We would appreciate your feedback on whether these additions address your concerns. If you have any remaining questions, please let us know so that we can respond before the discussion closes.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Brief Summary of Our Rebuttal

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for your time and constructive feedback. We appreciate Reviewers qUPa and 9cnC for recognizing the coherent design and practical value of separating planning and drawing objectives, as well as our experimental and ablation results; we also thank Reviewer Ghcf for acknowledging our well-motivated research, the substantive contribution of the SVGVerse dataset, and our broad evaluations, ablations, and in-depth analyses. To assist in facilitating the review process, we’d like to provide a brief summary of the rebuttal:

  1. Concerns on whether objective separation brings genuine gains Reviewers qUPa and 9cnC questioned whether DEFT's gains stem from the additional distillation introduced by expert separation rather than the separation of planning and drawing objectives itself. We conducted a controlled comparison between DEFT and a baseline with only additional distillation (see the table in our response to Reviewer qUPa). Under the exact same distillation pipeline, DEFT maintains a clear lead, demonstrating that our design of separating planning and drawing objectives brings substantial benefits.

  2. Concerns on whether the planning objective yields practical improvements Reviewer 9cnC asked whether the planning objective improves capabilities in object coverage, attribute binding, and spatial relations. We added an equal-budget ablation without the planning expert and independently evaluated these three capabilities (see the table in our response to Reviewer 9cnC). DEFT improves over this control by 15.6, 11.4, and 11.6 percentage points across the three dimensions, respectively, confirming the role of the planning expert in improving object recovery as well as attribute and spatial constraints.

  3. Lack of comparisons with frontier models and qualitative comparisons on complex illustrations Reviewers 9cnC and Ghcf raised concerns regarding the sufficiency of baseline comparisons and the lack of qualitative comparisons on complex images. In response, we added comparisons with frontier models from the GPT, Claude, and DeepSeek series, as well as CTRL-S [1] (see the tables in our responses to Reviewers 9cnC and Ghcf). DEFT remains competitive with frontier models while offering substantial advantages in inference latency and cost. We have also supplemented qualitative comparisons on complex illustrations and failure-case analyses in the revised manuscript (Tables 2-4, Figures 8-10).

  4. Methodological scalability and related work Reviewer Ghcf noted the lack of comparison and discussion of related work in the main text and raised concerns about the scalability of our method. Table 5 in our original manuscript already provided an ablation comparison against an MOPD [2] architectural baseline, demonstrating DEFT's superiority, with related work discussed in Appendix A. We have added citations to the related work in the main text of the revised manuscript. Additionally, we included examples in Figure 11 of the revised manuscript illustrating the separation of planning and drawing commands in Mermaid and TikZ code, discussing potential extension pathways of this mechanism to other structured languages.

  5. Evaluation metrics and dataset details Reviewer qUPa requested a detailed explanation of the evaluation metrics, and Reviewer Ghcf raised concerns regarding the data construction pipeline and whether there is a human-annotated gold standard for semantic components. We have elaborated on the evaluation metrics in our response and expanded the details in Appendix C of the revised manuscript. Appendices C and D of the original manuscript have already demonstrated the improvements brought by the SVGVerse dataset and the DEFT method to the semantics and editability of generated SVGs through human evaluations and editability experiments, and we have further supplemented the data construction details and training settings (see our response to Reviewer Ghcf for details).

We have addressed all reviewers' comments in our rebuttal and sincerely hope that the reviewers will take our responses into consideration. Thank you again for your attention and support.

[1] Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning, ECCV 2026.

[2] MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training, ICML 2026 Workshop on RL from World Feedback (RLxF).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.