acceptodds
Under review as a conference paper at ICLR 2027

LITEGLYPH: A COMPACT AUTOREGRESSIVE FRAMEWORK FOR FEW-SHOT CHINESE VECTOR FONT GENERATION

Abstract

Few-shot Chinese vector font generation requires synthesizing editable outlines for unseen characters while preserving character structure and typeface style, given a standard-form structure reference and only a few cross-character style refer- ences. Existing raster methods require post-hoc vectorization, whereas direct vector approaches often rely on specialized multi-stage pipelines or substantially larger generative models. We present LITEGLYPH, a compact unified autore- gressive framework that directly generates complete SVG command sequences. LITEGLYPH represents target-character structure as a visual prefix and encodes reference glyphs into dense style tokens that provide query-adaptive modulation throughout the decoder, allowing different SVG commands and coordinates to retrieve relevant local style evidence. We additionally introduce Raster-to-SVG initialization to establish a transferable vector prior and loop-aware rollback to improve decoding reliability. We train and evaluate on a new large-scale Chinese raster–vector corpus containing approximately 16M glyph pairs, over 2,600 cu- rated fonts, and 6,763 characters. LITEGLYPH achieves the lowest FID among the evaluated external baselines across validation, unseen-font seen-character, and unseen-font unseen-character settings, while directly producing editable SVGs with a 350M-parameter decoder. Ablations further show that dense style modulation improves unseen-font generalization over global conditioning on most metrics, and that Raster-to-SVG initialization is particularly effective when downstream font diversity is limited.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Few-shot Chinese font generation, Vector font synthesis, Autoregressive SVG generation, Multimodal conditional generation, Structure and style conditioning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zpxyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LITEGLYPH is a compact autoregressive framework for few-shot Chinese vector font generation that directly produces complete, editable SVG glyphs. The method factorizes character structure and font style through a visual structure prefix and dense style tokens.

The model is trained on a newly constructed large-scale Chinese raster-vector corpus containing approximately 16M pairs across 2.6k fonts and 6,763 characters. Experiments on validation, unseen-font seen-character (UFSC), and unseen-font unseen-character (UFUC) splits show competitive rasterized glyph quality compared with evaluated baselines. Ablations examine dense versus global style conditioning, the number of reference glyphs, and the scale-dependent effect of Raster-to-SVG initialization.

Critical strengths
  1. LITEGLYPH separates character structure and font style by using a visual prefix for structure and dense style tokens derived from reference glyphs for style conditioning. The dense style tokens provide query-dependent modulation throughout the decoder. This design allows the model to directly generate complete SVG glyphs without an additional raster-to-vector conversion stage.
  2. The paper constructs a substantially larger Chinese raster-vector corpus and evaluates on validation, UFSC, and UFUC splits. Retraining raster baselines on the same dataset makes the main comparisons reasonably controlled, and LITEGLYPH achieves strong results, particularly in FID. The analysis of Raster-to-SVG initialization and other ablation results also make the work useful.
Critical weaknesses
  1. The paper motivates LITEGLYPH by stating that existing work has not established a unified framework that jointly handles complete SVG generation and factorized structure/style conditioning. However, prior work has already explored closely related problem formulations. However, as noted in the related work, prior studies have already explored closely related problem formulations. Aoki & Aizawa (2022) [1] generate Chinese SVG glyphs from content and style references, DeepVecFont-v2[2] addresses few-shot Chinese vector-font generation by conditioning on target-character information and reference glyphs with Transformer-based vector generation, and VecGlypher[3] formulates direct vector-glyph generation within a unified language-model framework. Therefore, the novelty appears to lie more specifically in LITEGLYPH's conditioning architecture; otherwise, the overall methodological contribution appears relatively incremental.
  2. While Table2 provides comparisons against raster baselines, it does not directly compare against some of the most relevant recent vector-generation approaches. The held-out fonts in Table1 is reported seen by LITEGLYPH during Raster-to-SVG initialization, which makes me difficult to disentangle the benefit. I am also unclear about why DeepVecFont-v2 and the other vector baselines are restricted to the reduced method-compatible subset. It would be helpful to report how the increased glyph complexity in the full benchmark affects these methods, for example through the distribution of command or sequence lengths and the fraction of glyphs that exceed each method's supported range. If the limitation arises from representation or sequence-length constraints, I would also like to know whether these constraints could be adjusted to enable a broader and more directly comparable evaluation.
  3. Figure 6 compares Raster-to-SVG initialization against training from scratch. However, the initialized model has already undergone a separate Raster-to-SVG pretraining stage before this matched downstream training. In particular, the strong results with only 250 downstream fonts may be interpreted in the context that the initialized model has already been exposed to 2,376 fonts during Raster-to-SVG pretraining. Reporting the total training data exposure and computational cost would provide a clearer picture of the practical efficiency of this approach.

[1] SVG vector font generation for Chinese Characters with Transformer [2] DeepVecFont-v2: Exploiting Transformers to Synthesize Vector Fonts with Higher Quality [3] VecGlypher: Unified Vector Glyph Generation with Language Models

Questions for authors
  1. Could the authors clarify the primary technical novelty of LITEGLYPH relative to prior vector-font generation methods? Is the main novelty specifically the causal structure prefix and query-adaptive dense style conditioning?
  2. Could the authors report the command/sequence-length distribution and coverage of these methods on the full benchmark, and clarify whether their constraints could be adjusted to enable a broader common evaluation?
  3. For the Raster-to-SVG initialization study, could the authors report the total training data exposure and computational cost including the pretraining stage? A controlled comparison that isolates the effect of the additional pretraining data would also help distinguish the contribution of Raster-to-SVG initialization from that of additional data and compute.

Addressing these concerns, particularly by clarifying the novelty positioning and providing more controlled evidence for the vector-method comparisons and initialization study, would positively affect my overall assessment of the paper.

4 comments

W1 and Q1. What is the technical novelty?

Yes. Our main technical contribution is a conditioning interface that combines a causal structure prefix with query-adaptive dense style conditioning at every decoder layer. Aoki and Aizawa already generated Chinese SVG glyphs from separate content and style references, pooling style into a single vector for AdaIN. In LiteGlyph, a standard-font rendering specifies the target character through the prefix. Raster images of eight other characters in the target font supply a separate memory of local style features. At each layer, the current SVG position queries that memory to predict the next command or coordinate. Aoki and Aizawa, Sec. 3.2

With this interface, a single 350M-parameter Transformer decoder directly generates complete, editable SVG outlines for complex Chinese characters. Figure 14(b) presents multiple examples of complex Chinese characters generated under the UFUC setting, where both the typefaces and characters are unseen during training. The full Dense model has 680.0M parameters, of which 375.6M are trainable (Table 4).

Raster-to-SVG initialization provides a transferable vector prior for few-shot font generation. We first train the same autoregressive generator to reconstruct a glyph's complete SVG from its raster image, learning drawing commands, contour organization and coordinate geometry. We then transfer the visual projection and SVG decoder to the few-shot font task and jointly train them with the style-conditioning branch. In our experiments with few downstream fonts, initialization yields lower training loss and better generation quality under the same downstream training budget (Sec. 3.4, Figs. 6 and 9).

We also contribute a large-scale paired Chinese raster–vector corpus. It contains approximately 16 million raster–SVG glyph pairs spanning over 2,600 typefaces and 6,763 GB2312 characters. Our construction pipeline combines deduplication, structural-consistency filtering, outline canonicalization, automated checks and manual audits to align standard-font structure references with target glyphs while retaining broad stylistic diversity. The corpus pairs complete SVG outlines with raster images, supporting end-to-end vector generation of complex Chinese characters and evaluation of generalization to unseen typefaces and characters through explicit font and character splits (Sec. 3.1 and Appendix A).

We are comparing a visual prefix with layer-wise cross-attention over the same Heiti image features, using the same style branch and training episodes.

Table 2 compares our style conditioning with the pooled Global variant.

Prior methodRelevant comparison with LiteGlyph
Aoki and AizawaAlready generated Chinese SVGs from separate vector content and style references. Their decoder uses a single pooled style vector for AdaIN. Sec. 3
DeepVecFont-v2Uses Transformers for few-shot vector font generation and reports failures on complex Chinese glyphs with long drawing paths. Sec. 4.6
VecGlypherGenerates SVGs from text or image exemplars. Its image-conditioned comparison on Latin letters and digits uses a 27B model, with a 4B variant in its ablation. Tables 7 and 9
VecFontLLMGenerates complex Chinese glyphs from vector style exemplars using a StarCoder-1B backbone and three separately fine-tuned stages for anchor construction, anchor refinement and curve completion. Sec. 3.2.2

To quantify performance on long outlines, we took every target whose ground-truth path contained more than 140 drawing commands (M, L, C) in the separately frozen rebuttal sets of 10,000 targets per split. We used the paper Dense checkpoint with eight references and rollback enabled. Mean LPIPS at 512 pixels includes every selected target, scoring invalid or blank outputs as white images.

LiteGlyph on long target outlinesUFSCUFUC
Outputs that reached EOS and rendered as valid, visible SVGs / targets1,697 / 1,703 (99.65%)1,748 / 1,752 (99.77%)
Mean LPIPS ↓0.242110.24347

W2 and Q2. Why use a smaller benchmark for the vector baselines?

Table 1 uses a common subset for downstream training and evaluation so that all methods are compared on the same complete glyphs. Many complex Chinese outlines exceed the 70-command limit in our original DeepVecFont and DeepVecFont-v2 configurations. DualVector represents each glyph with a fixed set of curve primitives. Citing these methods’ difficulty generating complex glyphs reliably, VecFontLLM trained the same three baselines on 495 simple Chinese characters and tested them on those characters in held-out fonts. VecFontLLM, Secs. 4.1.1 and 4.2

Target length under the DeepVecFont encoding (10,000 per split)UFSCUFUC
Drawing commands (excluding EOS), median / 95th percentile87 / 18688 / 188
Target coverage by length, limit of 70 → 140 commands35.43% → 83.60%34.44% → 82.94%

We raised the limit to 140 commands and retrained DeepVecFont-v2. This expanded the UFSC comparison with LiteGlyph from 62 to 333 targets with four matched references per target. The original 300-character comparison vocabulary covers no UFUC targets.

W2 (continued). Did initialization use the Table 1 test fonts?

Yes, the original Table 1 setting had font-level data leakage. We held out all 65 test fonts during downstream training, but all 65 were in the original Raster-to-SVG pretraining set. The separate 250-font subset used in Figure 6 also contained six of those fonts.

For a fair comparison, we retrained both the Raster-to-SVG initializer and the downstream Dense model on the vector baselines’ training split. Both stages used the same 71,796 glyphs from 260 training fonts and excluded all 65 test fonts. We verified that all 17,772 targets and their eight references matched the original evaluation. The retrained LiteGlyph model scored better than the published model on all five metrics below.

Table 1 comparisonFID ↓LPIPS ↓L1 ↓RMSE ↓SSIM ↑
DeepVecFont-v272.8760.2950.2040.4180.640
LiteGlyph, original (test fonts held out downstream only)3.6860.1730.1170.2780.606
LiteGlyph, fonts held out at both stages3.3030.1660.1100.2690.611

LiteGlyph retains its FID and LPIPS advantage over the vector baselines when the test fonts are held out of both stages. Pretraining corpus size and training budget also changed between the two LiteGlyph runs, so this comparison does not isolate the effect of removing the overlap.

W3 and Q3. How much data and compute did initialization add?

The Raster-to-SVG initializer used a pool of 13,476,734 glyphs from 2,376 fonts. Each training slot below is one example in a global batch of 32. The totals count repeated exposure across stages. The GPU-hours column covers downstream training only. Historical wall time for pretraining is unavailable.

Downstream runInitialization updatesDownstream updatesTotal updatesTotal training slotsDownstream H800 GPU-hoursDownstream training loss near epoch end ↓
250 fonts, scratch044,47844,4781,423,29657.730.971
250 fonts, Raster-to-SVG initialization421,14744,478465,62514,900,00055.330.767
2,376 fonts, Raster-to-SVG initialization421,147421,147842,29426,953,408≈5200.7215

Downstream GPU-hours for 2,376 fonts are estimated by linearly scaling historical job durations for the same Dense24 model on 250, 500, and 1,000 fonts by the number of training updates. These runs also used eight H800 GPUs, a global batch size of 32, and FP16. The estimate excludes Raster-to-SVG pretraining and does not adjust for differences in validation frequency or glyph-length distribution between the subset and full-data runs.

The 250-font losses are the endpoints shown in Fig. 9. For 2,376 fonts, we report the last training-log value, averaged over the 20 updates ending at update 421,140.

At 250 fonts, the initialized model reached lower training loss after the same number of downstream updates (Fig. 9) and lower FID on both UFSC and UFUC (Table 8). This suggests that learning to construct outlines from SVG commands and coordinates before adaptation helps optimize font generation when downstream examples cover a limited range of contours. The initialized model used additional pretraining data and updates. Isolating the effect of the Raster-to-SVG objective would require matching total data exposure and updates across both stages. The separate 260-font experiment above addresses prior exposure to the Table 1 test fonts.

Thank you for the response. I have read it carefully and will maintain my current score.

We appreciate the reviewer’s suggestions on the experimental setup, which prompted us to strengthen the controls for a fairer comparison.

Reviewer_57wkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LiteGlyph, an autoregressive framework for few-shot Chinese vector font generation, designed to overcome the limitations of existing methods that rely on raster image post-processing or complex, multi-stage models. The approach decouples the modeling of target character structure from reference font style; it employs DINOv2 to extract structural features and utilizes "Dense Style Modulation" to dynamically incorporate local style information during the autoregressive decoding process, enabling the direct generation of editable SVG sequences. Furthermore, the authors introduce a "Raster-to-SVG" pre-training phase and a cyclic fallback strategy to enhance the model's generative capability and stability. The study also contributes a dataset comprising approximately 16 million pairs of Chinese raster and vector data. Experimental results demonstrate that this method achieves superior generation performance under settings such as UFSC (Unseen Font, Seen Character) and UFUC (Unseen Font, Unseen Character).

Critical strengths
  1. Directly generating editable vector fonts eliminates the need for a vectorization step—a process typically required after generating raster-based fonts—offering significant value for both font design and practical applications.
  2. The authors have constructed a dataset comprising approximately 16 million raster–vector pairs, covering over 2,600 Chinese fonts, thereby providing a large-scale training resource for the generation of Chinese vector fonts.
  3. Compared to some multi-stage vector font generation methods, this approach employs a unified autoregressive framework to generate SVG output directly, achieving superior visual quality and structural consistency across various testing scenarios.
Critical weaknesses
  1. This paper primarily leverages existing autoregressive Transformer, cross-attention, and conditional modulation mechanisms for vector font generation. In particular, the "Dense Style Modulation"—presented as a core contribution—yields only a marginal performance improvement over global conditioning (reducing the UFSC FID from 2.802 to 2.759), which is insufficient to fully justify the necessity of this design. Furthermore, the paper would be strengthened by an analysis of these local features—specifically, by clarifying what information these tokens actually contribute during the generation process.
  2. Experiments indicate that this initialization strategy is effective primarily when the number of training fonts is small. As the training scale expands to the full dataset, its advantage diminishes significantly; for some metrics, it even underperforms compared to random initialization. Consequently, the necessity of this module as a primary contribution remains open to debate.
  3. Comparisons with other vector generation methods, such as DeepVecFont, are limited to a small subset of the data, and some of the test fonts were already included in the pre-training phase of the proposed model. Moreover, differences in output resolution and model configuration between this approach and methods for raster font generation may compromise the fairness of the performance comparisons.
Questions for authors

Will the data mentioned in this paper be made publicly available?

4 comments

W1. How much does Dense help, and what is in the style tokens?

Table 2 evaluates the published Global and Dense models with eight references and 10,000 targets per split.

SplitGlobal FID ↓Dense FID ↓Global LPIPS ↓Dense LPIPS ↓
Validation1.8411.8380.1750.174
UFSC2.8022.7590.2010.198
UFUC2.7432.7590.2040.201

The FID change is small and reverses direction on UFUC. Paired LPIPS improves on all three splits. FID compares image distributions, whereas LPIPS compares each generated character with the same character in the target font. In Figure 5, the ground truth for 途 has a short, flat upper contour. Global draws a point, while Dense comes closer to the flat top. For 曲, the right-hand protruding stroke at the top has a slanted terminal. Global makes this terminal flatter, while Dense better preserves its upward-right extension. For 仿, Dense retains the pronounced bend in the left stroke, which Global simplifies to a straight vertical with a slanted top.

We measured the contribution of local descriptors on all 10,000 targets from 268 held-out fonts per test split. We compared two ways of processing the reference features before style resampling. Dense24 keeps the original patch features. The control computes each channel's mean and variance across patches within each reference, reconstructs the feature map from these statistics, and feeds it into the same resampler. The two models had equal parameter counts, identical initial trainable weights, 64 memory tokens, 24 modulation layers and the same 640,000 training episodes. The LPIPS comparison includes invalid outputs.

MeasurementTargets per splitUFSCUFUC
LPIPS reduction from retaining original patch features (reconstructed features − original features)10,000+0.00257+0.00204

We fixed the Dense weights, target structure image and eight style references, and assigned equal attention to all 64 style tokens at each SVG position. For free generation, we used one target per prespecified font outside the pilot. Exactly one of its eight references shared an annotated component with the target. In a separate sample, we measured next-token loss given the ground-truth SVG history.

Change from the trained model under uniform attentionTargets (UFSC / UFUC)UFSCUFUC
Change in free-generation LPIPS195 / 196+0.0590+0.0599
Change in next-token loss given the ground-truth SVG history (nats/token)128 / 128−0.028−0.036

Assigning equal weights to the style tokens increased free-generation LPIPS on both splits.

W2. When is Raster-to-SVG initialization useful?

Raster-to-SVG initialization is most useful when downstream training covers few fonts. Figure 6 shows the largest gain at 250 fonts. Both models used the same Dense architecture and trained on the same glyphs for one epoch. The initialized model finished that epoch with lower training loss (Fig. 9) and generated glyphs with lower FID on both unseen-font splits (Table 8).

This suggests that a decoder’s limited ability to model SVG geometry can constrain optimization of the font-generation task when downstream fonts are scarce. A decoder trained from scratch must learn both how to construct contours from commands and coordinates and how to draw them in the reference style. During Raster-to-SVG pretraining (Sec. 3.4), it learns valid command sequences and control-point placement from paired glyph images and outlines. As downstream font diversity grows, the target SVGs show more ways to draw the same character, and the gap in end-of-epoch training loss narrows. This scale dependence is qualitatively consistent with observations in visual transfer learning (He et al., 2019). Rethinking ImageNet Pre-Training, Sec. 5.2

Each pair shows training from scratch → Raster-to-SVG initialization. Losses for 250, 500 and 1,000 fonts are the curve endpoints in Fig. 9. FID is from Table 8.

Downstream training fontsTraining loss near the end of one epoch ↓Validation FID ↓UFSC FID ↓UFUC FID ↓
2500.971 → 0.7678.74 → 2.788.74 → 4.149.05 → 4.19
5000.858 → 0.7373.78 → 1.914.33 → 3.604.60 → 3.63
1,0000.802 → 0.7322.38 → 1.873.05 → 3.083.17 → 3.10
2,3760.756 → 0.7221.86 → 1.842.43 → 2.762.46 → 2.76

With all 2,376 downstream fonts, training from scratch gives lower FID on both unseen-font splits. The initializer used at every scale had been pretrained on 2,376 fonts. Since training lasted one epoch on each subset, larger subsets also gave both models more downstream updates.

W3. Are the baseline comparisons fair?

The original vector-baseline comparison in Table 1 had font-level data leakage. We held out all 65 test fonts during downstream training, but all 65 were in the original Raster-to-SVG pretraining set. For a fair comparison, we retrained both the Raster-to-SVG initializer and the downstream Dense model on the vector baselines’ training split. Both stages used the same 71,796 glyphs from 260 training fonts and excluded all 65 test fonts. On the original 17,772 targets with the same eight references per target, the new LiteGlyph model achieved lower FID and LPIPS and higher SSIM than the published LiteGlyph model.

Table 1 comparisonFID ↓LPIPS ↓SSIM ↑
DeepVecFont-v272.8760.2950.640
LiteGlyph, original (test fonts held out downstream only)3.6860.1730.606
LiteGlyph, fonts held out at both stages3.3030.1660.611

DeepVecFont-v2 had the best FID and LPIPS among the published vector baselines and retains the highest SSIM in this comparison. Pretraining corpus size and training budget also changed between the two LiteGlyph runs, so this comparison does not isolate the effect of removing the overlap. The main UFSC/UFUC benchmark uses a separate set of 268 test fonts held out of both pretraining and downstream training.

Table 2 uses each method’s native output resolution and reference budget. For the common-canvas comparison, we evaluated LiteGlyph on the same 10,000 targets per split using VQ-Font’s first three reference glyphs. The VQ-Font scores come from its archived evaluation reports. Both columns for a given metric use the same canvas and resolution-matched ground truth.

SplitCanvasLiteGlyph FID ↓VQ-Font FID ↓LiteGlyph LPIPS ↓VQ-Font LPIPS ↓
UFSC128 px3.13210.2380.21800.1988
UFSC512 px3.71466.7150.21890.2642
UFUC128 px3.24510.4340.22150.2023
UFUC512 px3.80467.5020.22200.2668

LiteGlyph has lower FID at both sizes. VQ-Font has lower LPIPS at its native 128 pixels, and LiteGlyph has lower LPIPS at 512 pixels. We will report both resolutions.

For the vector comparison, we matched four reference identities and a 512-pixel canvas on all 333 eligible UFSC targets from 176 fonts. All means include every target, with a white image for an invalid or blank output. We trained both DeepVecFont-v2 variants with a 140-command decoder for 48,000 updates. The original training set contained only short glyphs. The expanded training set included longer examples and additional characters. LiteGlyph uses the paper checkpoint.

Result on the same 333 targetsLiteGlyphDeepVecFont-v2, original 300 charactersDeepVecFont-v2, 383 characters, mixed lengths
Paired LPIPS ↓0.162830.327460.25917
Valid, visible glyphs333 / 333323 / 333319 / 333
LPIPS difference from LiteGlyphReference+0.16463+0.09634

Question on data availability.

Yes, for the portion we can redistribute. The Reproducibility Statement on p. 10 gives the anonymous review link for corpus-construction and evaluation code, per-font sources and licenses, and the exact split lists. At camera-ready, we plan to distribute glyph-level assets and checkpoints for the license-permitted subset. For the remaining fonts, we will provide identifiers and source pointers for reconstruction. Anonymous repository listed in the submission

Thank you for the authors' response, which addressed some of my questions. Overall, this paper presents an interesting and technically sound approach to vector font generation; it surpasses traditional methods by producing SVG commands that can be directly edited. Furthermore, the authors contribute a large-scale paired dataset. In terms of innovation, however, the reusable technical contributions appear somewhat limited. Given the current state of the review, I encourage the authors to reorganize the manuscript and resubmit.

A few minor suggestions: Figure 1, intended as an example of the task, lacks clarity; it fails to provide an intuitive sense of the paper's specific contributions or how it differs from traditional font generation methods. Additionally, key definitions regarding the representation of SVG data should be included in the main text rather than relegated to the appendix.

We thank the reviewer for this constructive suggestion, which helped us clarify our design rationale and its relationship to prior work.

Reviewer_rJydOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses few-shot Chinese vector font generation by directly synthesizing editable SVG code from structural and style references, bypassing raster-to-vector post-processing. It targets the difficulty of direct SVG generation due to strict geometric dependencies, the challenge of disentangling character structure from font style, and the lack of a compact unified framework. The workflow involves Raster-to-SVG initialization followed by joint fine-tuning. Key modules include a factorized conditioning interface, a full-depth dense style modulation for query-adaptive injection at every decoder layer and Raster-to-SVG initialization for geometric priors. Experiments demonstrate that LiteGlyph outperforms traditional vector methods, raster baselines, and multimodal LLMs. Ablations confirm dense modulation surpasses global conditioning, and initialization benefits few-font fine-tuning but diminishes with more data. Loop-aware rollback significantly improves SVG validity, whereas LLMs struggle with format control and hallucination in SVG generation.

Critical strengths

LiteGlyph designs a factorized conditioning interface that injects style and structure through two separate pathways into the autoregressive generation process, dynamically and query-adaptively modulating the generation process without relying on vectorization post-processing steps or complex network computation.

Critical weaknesses
  1. The paper mentions cascaded diffusion-based vector generation in the related work, but does not include it as a baseline in the experimental section. Since current diffusion models have advantages in generating complex structures, excluding such methods weakens the comprehensiveness of the comparison.

  2. The proposed factorized conditioning interface and full-depth dense modulation lack theoretical justification. For instance, there is no theoretical proof explaining why this layer-wise injection can better preserve local style features than global conditioning.

Questions for authors
  1. Why is it necessary to adopt the architecture of autoregressive model with external memory, rather than directly fine-tuning off-the-shelf large multimodal models such as LLM4SVG or StarVector?

  2. Can the proposed rollback mechanism handle contour tearing or topological errors caused by gradual coordinate drift?

  3. Why does full-depth dense modulation, which retrieves style via cross-attention at every decoder layer, preserve local style features better than a single global vector?

3 comments

W2 and Q3. Why can Dense retain local style cues?

Both Global and Dense condition all 24 decoder layers. Global pools each reference's CNN features into per-channel spatial means and standard deviations. Within a layer, every SVG position then receives the same modulation parameters. For a fixed structure image and SVG prefix, two reference sets with identical per-reference moments therefore give Global exactly the same next-token distribution.

Consider two sets of patch descriptors, A={(1,1),(0,0)}A=\{(1,1),(0,0)\} and B={(1,0),(0,1)}B=\{(1,0),(0,1)\}. They have identical per-channel means and standard deviations. A softmax attention head using the first coordinate as its key and the second as its value returns 0.73 for AA and 0.27 for BB with a unit scalar query. This counterexample shows that per-channel statistics discard feature relationships that attention can distinguish; it does not by itself guarantee that a trained Dense model retains them.

Our visual conditioning interface draws on the design of Flamingo (Alayrac et al., 2022). Its Perceiver Resampler compresses visual features into 64 tokens, which are queried through cross-attention at multiple decoder depths using the current text hidden states. Flamingo Following this design principle, Dense resamples reference-glyph patch features into 64 style tokens. In Eq. (2), each SVG position queries this memory at every layer using its updated hidden state and converts the retrieved features into position-specific AdaRMSNorm style-modulation parameters. Positions drawing stroke terminals and stroke bodies can therefore receive different style conditions. The practical benefit for local style fidelity is supported by our ablations below and Fig. 5.

Figure 5 shows several local contour differences. For 途, Global draws a pointed upper contour where the target has a short, flat top. Dense comes closer to that contour. For 曲, the right-hand protruding stroke at the top has a slanted terminal. Global makes this terminal flatter, while Dense better preserves its upward-right extension. For 仿, Dense retains the pronounced bend in the left stroke, which Global simplifies to a straight vertical with a slanted top. Table 2 compares the published checkpoints with eight references on the same 10,000 target glyphs per split.

Published modelUFSC LPIPS ↓UFUC LPIPS ↓
Global0.2010.204
Dense0.1980.201

We changed one part of style conditioning at a time, holding the Dense weights, eight reference glyphs and per-target sampling settings fixed. For uniform attention, we used one target per eligible font outside the pilot, with exactly one reference sharing an annotated component with the target. For the patch-reconstruction probe, we used the original 32 fonts per split. A positive change means the generated glyph moved farther from its target.

Intervention on the trained Dense modelTargets (UFSC / UFUC)UFSC change in LPIPSUFUC change in LPIPS
Give all style tokens equal attention195 / 196+0.0590+0.0599
Reconstruct each reference's patch descriptors from its channel means and variances32 / 32+0.0921+0.1132

To control for Dense's 18.1M additional trainable parameters over Global, we retrained two equal-parameter Dense24 variants from identical trainable weights. One feeds each reference's original patch features into the style resampler. The other computes each channel's mean and variance across patches within each reference, reconstructs the feature map from these statistics, and feeds it into the same resampler. Both models use 64 memory tokens and apply style modulation at all 24 decoder layers. They processed the same 640,000 target and reference sets in the same order and applied 19,993 optimizer updates over 20,000 batches. LPIPS includes all 10,000 targets from 268 held-out fonts per split, using a white image for invalid or blank outputs.

Feature processing and metricUFSCUFUC
Keep original patch features (Dense24), LPIPS ↓0.227250.22840
Reconstruct patch features from channel means and variances, LPIPS ↓0.229820.23044
LPIPS reduction (reconstructed features − original features)+0.00257+0.00204

Retaining the local descriptors gave a modest perceptual gain on both splits at equal model size.

To compare style injection depth, we trained Dense8, which injects style only in layers 17–24 of the same 24-layer decoder. Dense8 and Dense24 saw the same 640,000 target/reference episodes in the same order. Each split contains 10,000 evaluation targets. This comparison changes both the depth of injection and the number of modulation parameters.

Style injection depthUFSC LPIPS ↓UFUC LPIPS ↓
Last 8 layers0.236600.23771
All 24 layers0.227250.22840
LPIPS reduction with full-depth injection+0.00936+0.00931

Q1. Why use a task-specific decoder when a general SVG model could be fine-tuned?

Fine-tuning a general SVG model is a viable alternative. Our task supplies two different kinds of evidence. A standard-font rendering specifies which character to draw, including characters unseen in training. Eight other characters show how the target typeface draws strokes and terminals. LiteGlyph encodes the first image as a causal prefix and lets each SVG decoding position query local style descriptors from the other eight. This conditioning design lets a 350M-class Transformer generate editable glyph outlines without a billion-parameter language backbone. The full Dense model has 680.0M parameters, of which 375.6M are trainable (Appendix D.1). LLM4SVG itself evaluates several decoder sizes, including GPT-2 small, on text-to-SVG generation. LLM4SVG, quantitative comparison

We fine-tuned StarVector-1B with the same structure image and eight style exemplars, withholding the requested glyph in the target font. LiteGlyph started from its Raster-to-SVG checkpoint (P0). We jointly fine-tuned the SVG decoder and Dense style branch with DINOv2 frozen. StarVector started from its public 1B checkpoint. Both processed the same 640,000 episodes over 20,000 global batches on eight RTX 5090 GPUs. We computed GPU-hours by multiplying the elapsed runtime of each fine-tuning command by eight. We verified all 10,000 target IDs and the order of their eight references on each split. LPIPS includes every target and uses a white image for an invalid or blank output. StarVector

Matched downstream comparisonStarVector-1B, fine-tunedLiteGlyph Dense24
Fine-tuning for 20,000 batches (RTX 5090 GPU-hours)138.428.6
UFSC LPIPS ↓0.246690.22725
UFUC LPIPS ↓0.246990.22840
UFSC valid, visible outputs7,130 / 10,000 (71.30%)9,836 / 10,000 (98.36%)
UFUC valid, visible outputs7,123 / 10,000 (71.23%)9,806 / 10,000 (98.06%)

Every StarVector output that failed the validity and visibility check reached its decoding-token limit. Under this fine-tuning protocol, Dense24 used fewer downstream GPU-hours and produced more valid, visible glyphs on both splits.

Q2. Can rollback repair contour tears or topological errors caused by gradual coordinate drift?

In general, no. If coordinates drift until a stroke disconnects and no exact loop occurs, rollback never triggers and cannot correct the break. Rollback requires a block of complete SVG commands, including their quantized coordinates, to occur three times consecutively. It resamples from the start of the third occurrence. Even if a tear coincides with a loop, the decoder does not check contour connectivity, so resampling cannot guarantee that the stroke reconnects.

We did not measure a repair rate for naturally occurring contour tears. To measure how often rollback suppresses exact loops during generation, we decoded 10,000 targets per split with rollback off and on, using the same references and initial sampling state for each pair.

Paired decoding resultUFSCUFUC
Targets with an exact loop, off → on254 → 7266 → 14
New exact loops introduced by rollback00
Untriggered targets with identical output tokens9,746 / 9,7469,734 / 9,734
Valid, visible SVGs, off → on99.64% → 99.86%99.54% → 99.79%
Mean LPIPS over all targets ↓, off → on0.200400 → 0.2002530.202830 → 0.202686

We will describe rollback in the paper as a safeguard against exact command repetition.

W1. Cascaded vector diffusion.

VecFusion is already discussed in Sec. 2.1. We did not include a quantitative comparison because its official repository has not released a training or inference implementation. VecFusion official repository

We chose direct autoregressive SVG generation to handle character structure, font style, and complete outlines within one generator. VecFusion first generates a raster glyph and auxiliary control-point fields, then uses a separate vector diffusion model to produce outlines. At inference, LiteGlyph uses a single decoder with a 350M Transformer backbone to predict complete SVG commands and coordinates for complex Chinese characters. Structure and style references directly condition this vector generation process, eliminating the intermediate target-raster synthesis and the second cascaded generator. VecFusion, Sec. 3

For conditioning, VecFusion specifies the target character through a learned character-ID embedding. LiteGlyph instead uses a standard-font structure image as a visual prefix and queries style information from eight other reference glyphs at every layer. This lets LiteGlyph receive structural guidance for characters unseen in training without learning new character-ID embeddings. Fig. 14(b) shows multiple complex Chinese glyphs generated under UFUC, where both fonts and characters are unseen. VecFusion, Sec. 3.1

For the high-resolution raster synthesis followed by vectorization used in HFH-Font, LiteGlyph's advantage is to generate SVG commands, contour organization, and curve coordinates directly, without an additional image-vectorization step. HFH-Font

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.