- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces TextVerifier, a validator designed to detect structural rendering errors in Chinese characters, conditioned on the target character. The method employs a dual-encoder architecture, processing both images and Ideographic Description Sequences (IDS), and is trained in three stages: using authentic character images, synthetic stroke-level perturbations, and pairs of samples with varying relative quality (collected from the training trajectories of an image inpainting model). The authors also constructed TextRenderBench-ZH, a dataset comprising 7,358 characters extracted from 308 generated images. Experimental results demonstrate that this method achieves an F1 score of 0.844, outperforming both TextPecker (0.590) and an OCR-confidence-based approach (0.526, using an optimal threshold). Furthermore, the authors integrated this validator as a reward signal into Flow-GRPO to enhance the quality of Chinese text rendering.
Critical strengths
- The proposed benchmark provides character-level annotations of location, identity, and correctness, enabling direct evaluation of structural errors that recognition-based metrics may overlook.
- Ablation experiments fully demonstrate the effectiveness of the three-stage training process.
- Compared to the previous related work, TextPecker, it offers significant advantages in terms of model parameters and inference speed.
Critical weaknesses
1.In Section 4.1 and Appendix E, the benchmark construction is insufficiently documented. The generating models and versions, sampling distribution, number of annotators, inter-annotator agreement, and disagreement-resolution procedure are not reported. Since distinguishing legitimate stylistic variation from rendering errors is central to the task, these omissions make it difficult to assess annotation reliability and generalization across generators and styles. 2.The comparison does not sufficiently characterize the difference in language coverage between TextVerifier and the baselines. TextVerifier is specifically designed around Chinese character structure, and its quantitative evaluation focuses on Chinese. However, TextPecker covers both English and Chinese. A Chinese-only comparison is relevant to the stated task, but provides an incomplete account of their respective applicability. More importantly, the paper does not evaluate whether Chinese-focused reward optimization preserves English or mixed Chinese–English rendering capabilities. 3.The evidence supporting TextVerifier as an optimization reward in Section 4.5 is insufficiently comprehensive. The experiments use only Qwen-Image and evaluate on oneID-ZH, leaving transferability to other generator families unverified. Although GlyphPrinter is briefly cited in Section 2, the paper lacks a substantive comparison with its region-level preference optimization approach or an experimental comparison under comparable conditions.
Questions for authors
1.How are English letters, digits, and mixed-language text handled by the verifier and the glyph reward? Could the authors report English and mixed Chinese–English rendering performance before and after post-training to assess whether these capabilities are preserved? 2.Could the authors evaluate the reward on another compatible generator family, such as FLUX, using controlled prompts and training budgets? This would help establish whether the improvement extends beyond Qwen-Image. 3.Could the authors provide a substantive comparison with GlyphPrinter and, where feasible, evaluate the approaches under comparable conditions? An independent human evaluation of glyph correctness would also help distinguish actual rendering improvements from improvements according to the optimized verifier.
6 comments
Response to Reviewer Uap7 (Part II)
Comment: Independent human evaluation and the shared reward model and evaluator
Reviewer concern. An independent human evaluation of glyph correctness would help distinguish actual rendering improvements from improvements according to the optimized verifier.
Response. We agree that using TextVerifier for both reward computation and automatic Glyph-Acc evaluation requires independent human evidence to validate actual improvements.
(1) Evaluation protocol. We randomly selected 100 prompts from OneIG-ZH and compared Ours against Base and the TextPecker-reward model. Five evaluators participated in a blind GSB (Good/Same/Bad) pairwise assessment: method names were hidden, and the left–right placement of outputs was varied. Judgments were based solely on Chinese glyph correctness, without considering overall aesthetics or layout preferences. The final judgment for each image pair was determined by majority vote.
Table 1. Blind human evaluation of Chinese glyph correctness on 100 matched prompts.
| Compared against | Ours wins (G) | Ties (S) | Ours loses (B) | Net win rate |
|---|---|---|---|---|
| Base (Qwen-Image) | 52 | 33 | 15 | +37.0% |
| TextPecker-reward | 27 | 44 | 29 | −2.0% |
(2) Results and interpretation. Against Base, Ours wins substantially more comparisons than it loses, achieving a 77.6% win rate among non-tied comparisons. Against TextPecker-reward, the win–loss distribution is broadly balanced. These results are consistent with the trend in Table 4 of the original manuscript, where Ours substantially outperforms Base and achieves performance close to TextPecker-reward. The human judgments provide evidence independent of TextVerifier’s automatic scores that Chinese rendering correctness improves over Base.
Response to Reviewer Uap7 (Part III)
Comment: Applicability of the reward to another generator family
Reviewer concern. Could the authors evaluate the reward on another compatible generator family, such as FLUX?
Response. To address cross-generator applicability, we added experiments on FLUX 3 Image Klein Base, using the same TextVerifier reward formulation for Flow-GRPO post-training.
(1) Automatic evaluation. The additional results are reported below.
Table 1. Automatic Glyph-Acc before and after Flow-GRPO post-training on FLUX.
| Model | Automatic Glyph-Acc ↑ |
|---|---|
| FLUX 3 Image Klein Base | 0.445 |
| FLUX 3 Image Klein, post-trained with TextVerifier reward | 0.501 |
Glyph-Acc increases from 0.445 to 0.501, an improvement of 5.6 percentage points.
(2) Independent human evaluation. Since this metric is still computed by TextVerifier, we additionally conducted a human GSB comparison between the post-trained model and Base.
Table 2. Human GSB evaluation of the post-trained FLUX model against Base.
| Comparison | Wins (G) | Ties (S) | Losses (B) | Net win rate | Non-tie win rate |
|---|---|---|---|---|---|
| Post-trained model vs. Base | 34 | 42 | 24 | +10.0% | 58.6% |
The human evaluation likewise shows more wins than losses, although the advantage is modest and many comparisons are tied. Together with the automatic results, this provides additional evidence that the TextVerifier reward can improve rendering in a generator family beyond Qwen-Image.
We also observe that the FLUX base model starts with lower performance than Qwen-Image on this Chinese rendering task, and the current post-training does not close the gap between their final performance levels. This is consistent with the dependence of RL post-training outcomes on the capabilities of the starting model.
Response to Reviewer Uap7 (Part I)
Comment: Chinese-specific verification and preservation of English rendering capabilities
Response. We thank the reviewer for recognizing our character-level benchmark, three-stage training ablations, and model efficiency. Your suggestions regarding language coverage, cross-generator applicability, and related-method comparisons have helped us strengthen the empirical evidence and clarify the scope of our work. We have added language-capability preservation experiments, independent human evaluation, and reward-based optimization experiments on a FLUX-family model.
(1) Why Chinese rendering requires additional structural verification. We appreciate the request to clarify language coverage. Our focus on Chinese structure is motivated by a mismatch between the granularity of recognition-based evaluation and the errors we seek to detect.
For English, character-level edit distance operates on the letters within a word. If a rendering error causes a letter to be recognized as a different letter, omitted, or added, the resulting OCR string differs from the target, and NED can capture that discrepancy. A capable OCR model therefore provides a useful signal for assessing letter-level content and spelling within English words.
For Chinese, the corresponding unit is an entire Chinese character. A glyph may contain a missing or extra stroke while still being recognized as the intended character. In this case, the OCR output matches the target, and NED does not penalize the internal structural error. Recognition-based metrics can detect character substitutions, insertions, and deletions, but they cannot reveal an internal glyph defect when the recognized character identity remains unchanged. Our examples and the limited glyph-error detection performance of the OCR baseline illustrate this limitation.
Therefore, it motivates a dedicated verifier for the fine-grained structural errors within Chinese characters that recognition-based evaluation may overlook. TextVerifier complements recognition-based evaluation rather than replacing it across all languages.
(2) English rendering after Chinese-focused post-training. To examine whether Chinese-focused post-training harms English generation, we evaluated the same checkpoint obtained through TextVerifier-guided Flow-GRPO on the English Text Rendering task of OneIG, following its official text evaluation protocol.
Table 1. English rendering performance before and after Chinese-focused post-training.
The English text metrics improve rather than decline, indicating no observed degradation in English rendering on this evaluation. One possible explanation for the improvement is the auxiliary NED term in our training reward. Besides providing Chinese content-alignment supervision, this term encourages more complete and accurate rendering of the text requested in the prompt, potentially producing positive transfer to English generation. We present this as a possible explanation rather than an established causal mechanism.