acceptodds
Under review as a conference paper at ICLR 2027

VA-SVG: Improving SVG Code Prediction via Generative Visual Alignment

Abstract

Existing SVG generation methods typically adopt multimodal large language models (MLLMs) as their backbone and formulate SVG synthesis primarily as code sequence prediction. However, SVGs constitute a distinctive image–code multimodal representation, requiring the model to generate code that is visually consistent with the target image. This process demands not only a precise understanding of fine-grained visual attributes, such as layout, structure, and spatial relationships, but also accurate alignment among images, textual semantics, and SVG code. Without explicit visual guidance, conventional cross-entropy supervision over code sequences struggles to capture complex visual semantics, often producing SVGs with limited visual fidelity or poor geometric consistency. To address these limitations, we propose **VA-SVG, a visual alignment post-training framework that couples SVG code prediction with visual generation.** Specifically, we train an auxiliary diffusion model to map continuous SVG code representations into the visual domain, and align its intermediate representations with visual features extracted from rendered SVGs. This differentiable pathway transfers visual generative knowledge back to the code generation model without requiring gradients through the SVG renderer. Extensive experiments and ablation studies demonstrate the effectiveness of the proposed training paradigm.

Primary area:
generative models
Keywords:
SVG Generation, Unified Model, Generative Visual Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jhGrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes VA-SVG, a post-training framework that introduces generative visual alignment into autoregressive SVG code generation. Starting from a pretrained SVG generator, the method trains a diffusion branch conditioned on the MLLM’s continuous code representations to reconstruct corresponding raster images. It then freezes the diffusion branch and conditioning projector, and updates the MLLM through token-level cross-entropy and alignment between projected intermediate DiT features and DINO features of the model’s rendered SVG predictions. The auxiliary visual branch is used only during training, preserving standard autoregressive inference. Experiments on the Icon and Illustration subsets of MMSVG-Bench evaluate Text-to-SVG and Image-to-SVG generation, with improvements over CE-only post-training demonstrated on the Icon subset, particularly in reconstruction quality.

Critical strengths
  1. The proposed framework brings differentiable visual supervision into SVG code generation in the SFT stage. Because the auxiliary diffusion branch is used only during training, the model keeps standard autoregressive inference. This makes the approach practically appealing, as it improves visual fidelity without adding an inference-time feedback loop.

  2. Table 2 provides encouraging evidence that visual alignment improves over CE-only post-training. On MMSVG-Bench Icon, LPIPS drops from 0.205 to 0.101, and SSIM rises from 0.902 to 0.925. Notably, these gains appear on metrics other than DINO, which is used as the supervision signal.

Critical weaknesses
  1. Section 3.2 argues that diffusion provides a stronger visual prior, yet visual alignment training uses the MLLM’s own rendered predictions as alignment targets. It is unclear why the projected DiT features are not additionally aligned with features of the ground-truth images, which could provide direct corrective supervision to the MLLM. The paper lacks justification for this target choice and a comparison with ground-truth alignment, leaving the role of the claimed stronger visual prior insufficiently established.

  2. Evaluation is limited to MMSVG-Bench, leaving generalization to other benchmarks insufficiently established. Additional evaluation on SArena, UniSVG, or the SVG-Stack test set would strengthen the evidence. Moreover, the RL baseline’s training configuration and reward formulation are not specified, making the comparison in Table 2 difficult to assess or reproduce.

  3. Reinforcement learning methods can use reward signals beyond visual quality, such as image–text alignment and code efficiency. They can also use SVG validity checks to substantially reduce degenerate repetition in generated code, where SVG-specific models keep emitting the same digit or character. This makes RL-based approaches strong baselines for this task. Table 1 reports SVGen's performance on MMSVG-Bench, but this comparison is not entirely fair because SVGen was not trained on the same data. To show that VA-SVG is effective, the authors should retrain SVGen on the MMSVG dataset and compare the two methods under the same training data. The authors should also compare VA-SVG with CTRL-S on SArena, so the evaluation covers more than one benchmark.

Questions for authors
  1. Could the authors explain why only the MLLM's rendered predictions are used as alignment targets, rather than ground-truth images? Could the authors also report how performance changes when the projected DiT features are additionally aligned with ground-truth image features?

  2. Could the authors describe the algorithm, reward formulation, and training configuration of the RL baseline in Table 2? Could the authors also clarify whether its initialization, training data, and computational budget are comparable to those of VA-SVG?

  3. Could the authors clarify the size of the code-generation pretraining dataset? Section 4.1 reports approximately 1.3M training examples, whereas Appendix A and Table 3 report 1.67M samples. It would be helpful to know which number is correct, or whether the two numbers refer to different subsets.

  4. Could the authors report results on additional benchmarks, such as SArena, UniSVG, or the SVG-Stack test set, to assess generalization beyond MMSVG-Bench?

  5. Could the authors clarify whether the SVGen results in Table 1 were obtained from an existing checkpoint or from a retrained model? To better assess VA-SVG's advantages over RL-based methods, it would be helpful to retrain SVGen on the same training data as VA-SVG and compare the two methods under this setting. Alternatively, a comparison with CTRL-S on SArena under a consistent evaluation protocol would also be very helpful.

Reviewer_4tU2OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

To address the lack of direct visual supervision in SVG code generation, this paper proposes VA-SVG. It employs a frozen, pre-trained conditional Diffusion Transformer and enhances code generation quality by aligning the DiT’s visual representations with the DINO features of the rendered SVG images. Experiments demonstrate that this approach outperforms the CE baseline in terms of visual similarity and reconstruction quality.

Critical strengths
  1. The research problem is clearly defined: it tackles the disconnect between SVG code and its visual output by introducing a frozen visual generation branch that provides differentiable supervision, while remaining compatible with standard autoregressive inference workflows.

  2. The experiments cover both Text-to-SVG and Image-to-SVG tasks. Compared to CE-based training, VA-SVG shows improvements across multiple metrics. Ablation studies further validate the roles of the visual encoder, the alignment layer, and the training objectives.

Critical weaknesses
  1. The absence of a baseline involving direct visual feature supervision (bypassing the DiT) makes it difficult to definitively conclude that the DiT-based generative visual prior is the primary driver of performance gains, or to assess its necessity.

  2. The mechanism by which the visual objective based on predicting the rendered SVG corrects visually erroneous code is not fully explained. Furthermore, the construction of the image latent variables and their interaction with the code-based conditioning require further clarification.

  3. The experiments rely on a single model and dataset, with training data heavily skewed toward icons, this raises questions regarding the method's generalizability to complex SVGs. Additionally, details concerning sample size, computational overhead, and training budget are lacking.

Questions for authors

Could the authors include a baseline for direct visual feature alignment bypassing the DiT by aligning the MLLM’s internal SVG representations directly to the same DINO targets while ensuring that the training data and computational budget remain comparable?

Reviewer_tLjjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the problem of SVG generation from image or text with autoregressive models. The authors propose a post-training framework called VA-SVG that adds differentiable visual supervision without differentiating through the renderer. It first trains a DiT to map the MLLM's code hidden states to rasterised SVG images. It then freezes the DiT and aligns its intermediate features with DINO features of renderings of the MLLM's predicted SVGs. The MLLM is trained with cross-entropy and alignment losses. Their results show that on icon and illustraion datasets, VA-SVG is competitive with most models and strongest among autoregressive methods on image-to-svg.

Critical strengths

The proposed training idea is clever and practical, and the matched ablation shows large gains where they matter most. Using a learned generator as a differentiable path from code to image skips the need to backpropagate through a renderer and avoids the instabilities that RL training brings with rendering rewards. The authors'results show that Image-to-SVG reconstruction improves substantially (Table 2). Ablations show the necessity of their proposed alignment signal.

Critical weaknesses

There are no critical issues that I see. Some minor comments:

The paper does not explain (or I could not find any explanation) as to why self-consistency would improve fidelity to the target: the Dino target comes from rending the model's own preditions (not gt image). This rendered image also contributes to DiT's latent intput. If so, then the loss is rewarding consistency with what the model drew, not states that are correct.

Questions for authors

[clarification] What is the RL baseline exactly? Some more details would be helpful.

[clarification] Each alignment step requires decoding, rendering and a DINO pass. What is the training cost relative to CE-only SFT, and relative to RL with rendering rewards?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.