acceptodds
Withdrawn from ICLR 2027

Glyph-to-Image: Semantic Chinese Glyph Generation via Text-to-Image Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Chinese characters combine symbolic identity with highly structured visual forms, making them a natural medium for semantic visual expression. However, automatically transforming prescribed Chinese glyphs into concept-bearing images while preserving their character identity remains challenging, since unconstrained generative models provide strong semantics but limited structural control. We introduce Glyph-to-Image, a task that transforms editable Chinese glyphs according to a target concept. Starting from native cubic B\'ezier outlines, Glyph-to-Image jointly optimizes glyph geometry and a learnable appearance field through differentiable rendering. To stabilize semantic appearance formation, we introduce an Appearance Low-Pass Filter (ALPF) that constrains the bandwidth of both the rendered appearance and its updates. To preserve local glyph structure during semantic deformation, we further introduce Glyph Consistency Regularization (GCR), which regularizes neighboring B\'ezier segments with respect to local similarity transformations. Beyond single-character generation, Glyph-to-Image naturally extends to multi-character generation. We establish benchmarks, develop evaluation metrics, and reproduce representative methods from semantic and artistic typography as baselines. Extensive experiments show that the proposed Glyph-to-Image achieves a better balance between semantic expressiveness and character readability than existing approaches.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Semantic Typography, Character Image Generation, Artistic Glyph Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3abxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Glyph-to-Image: a model that turns Chinese characters into images of a target concept while keeping them readable. At test time, it optimizes the font’s own cubic Bézier outline together with a 64×64 RGB appearance map. A frozen text-to-image model guides the optimization through differentiable rendering. Two components control the trade-off between meaning and readability: an Appearance Low-Pass Filter (ALPF), which smooths the map and its gradients; Glyph Consistency Regularization (GCR), a margin-based penalty on deformations of neighbouring Bézier segments that go beyond rotation and uniform scaling.

Authors build single- and multi-character benchmarks (90 and 270 pairs) and propose the SGBS metric, which combines normalized CLIP-T with OCR confidence. They report the best overall SGBS against CLIPDraw, DS-Fusion, OracleFusion and VitaGlyph.

Critical strengths

I believe CLIPScore alone is insufficient even to capture semantic alignment, and I fully understand the need for a new metric: the work proposes SGBS, which jointly accounts for semantic alignment and glyph readability. It is a good addition.

Critical weaknesses
  1. I am mainly concerned about the novelty of the work. Diffusion-guided optimisation of vector font outlines has already been explored for semantic typography (e.g., Word-As-Image), yet the paper does not compare against it. With respect to this line of work, the main changes in the method seems to be a newer flow-based prior, a raster colour map, and GCR. GCR also seems a regularizer, similar to those used in prior work (e.g., Word-As-Image’s loss).
  2. Word-As-Image is a far more relevant baseline than CLIPDraw. A quantitative and qualitative comparison with it would help frame the contribution of this work.
  3. Minor: some relevant references are missing for the generative models: DreamFusion, vHector, Vector Grimoire and Text-to-Vector Generation with Neural Path Representation.
Questions for authors
  1. CLIPDraw mentioned in Table 1 seems to be an adapted version that starts from glyph outlines (l. 358). Please describe this adaptation and clarify it in the tables. How does the adapted version differ from Word-As-Image, which already deforms glyph outlines under a diffusion prior? Could you also compare your method directly with Word-As-Image?

  2. A human study with native readers would strengthen the evaluation, e.g., forced-choice concept recognition and character recognition, as in Word-As-Image. Reporting how well SGBS agrees with these human judgments would also help validate the metric.

  3. L_T2M is only described in words (l. 203–207). Please define it explicitly.

  4. How much does the appearance field contribute on its own? Could you add an outline-only variant (flat fill, no appearance field) and an appearance-only variant (outline frozen) to Table 4?

1 and 2 are most important.

Reviewer_tq6cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Glyph-to-Image, the task of transforming a prescribed editable Chinese glyph into a concept-bearing image while preserving character identity. A glyph is represented by native cubic Bézier control points and a learnable (64\times64) RGB appearance field; both are optimized for 500 iterations through differentiable rendering using guidance from a frozen SD3.5 rectified-flow text-to-image model. The Appearance Low-Pass Filter (ALPF) combines antialiased bicubic upsampling and forward Gaussian filtering with backward gradient filtering, while Glyph Consistency Regularization (GCR) penalizes excessive non-similarity distortion in adjacent seven-point, two-segment Bézier patches. Multi-character expressions are optimized independently per character under a shared concept and then arranged in reading order.

Critical strengths
  1. A well-motivated, editable formulation of semantic Chinese typography.
  2. The initial evidence covers both component behavior and breadth of use.
Critical weaknesses
  • The core semantic objective is not specified sufficiently to reproduce or assess the method. Section 3.1 introduces only a generic (L_{\mathrm{T2M}}(I,y)), and Figure 3 calls it an “image-text loss,” while Section 4.1 identifies the prior as SD3.5 rectified flow. The paper does not define the actual loss, latent noising or timestep sampling, text-conditioning/guidance procedure, or how gradients are obtained from this model. Since this objective drives both geometry and appearance, the omission undermines the reproducibility statement and makes it difficult to distinguish the technical contribution from prior diffusion-guided vector optimization methods.
  • The main quantitative evidence for a better semantic–glyph balance is internally inconsistent and insufficiently validated. Under Eqs. (10–11), (S) and (P_{ocr}) are in ([0,1]), so (SGBS=S\cdot P_{ocr}\leq S) for every sample and therefore also after averaging. Yet Raw Glyph in Table 1 has SGBS 0.1484 despite (S=0.0708), and in Table 3 has SGBS 0.0378 despite (S=0.0107); no alternative aggregation is described. This is consequential because the proposed method is not the leader on CLIP-T or HPSv2, and its complex-character SGBS in Table 2 trails VitaGlyph. Thus the headline superiority claim relies heavily on a new composite metric whose reported values need correction or explanation and whose OCR-based readability component lacks human validation.
Questions for authors
  1. What is the exact (L_{\mathrm{T2M}}) used with SD3.5 rectified flow? Please provide its equation, timestep/noising procedure, classifier-free or other guidance settings, and the path by which its gradients reach the rendered image.
  2. How precisely are (S) and SGBS normalized, clipped, and averaged? Under Eqs. (10–11), how can Raw Glyph's mean SGBS exceed its mean (S) in Tables 1 and 3? Can the authors correct or reconcile all affected tables using per-sample calculations?
  3. Do the authors have human-study or other independent evidence that PaddleOCR confidence and SGBS measure perceived character readability and semantic–glyph balance? In particular, how should the complex-character result in Table 2 (Ours 0.0963 versus VitaGlyph 0.1234 and Raw Glyph 0.1805) qualify the overall superiority claim?
  4. Why is Word-As-Image, a directly relevant vector semantic-typography method discussed in Section 2, absent from the baselines? Can a comparison be provided, together with enough backbone and optimization details for all baselines to establish that the gains are not primarily due to using SD3.5?
Reviewer_GdvgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Glyph-to-Image, an inference-time optimization framework for generating semantic visuals from Chinese glyphs. It starts with native cubic Bézier outlines and a learnable RGB appearance field, then uses a differentiable renderer to propagate semantic gradients from a frozen SD3.5 rectified-flow model.

Critical strengths
  1. Because the method operates directly on native Bézier outlines at test time, it generalizes well across eight distinct Chinese calligraphic and typographic fonts (Figures S2–S3) without retraining.
  2. The paper is well-written and the overall idea is easy-to-follow.
Critical weaknesses
  1. The core objective LT2M\mathcal{L}_{\text{T2M}} is not clearly defined.
  2. The comparison lacks more general T2I methods, such as Gemini, GPT-Image-2, Qwen-Image, and Hunyuan-Image. The baselines are from 2024 and are outdated.
  3. Several prior works have explored using SVGs to assist image generation, e.g., , making the overall approach less novel. In addition, the overall performance is difficult to evaluate objectively.
Questions for authors

Please refer to the weakness part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.