- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces ExtremeTextBench, a benchmark for evaluating image generation models on long-text rendering, small-text rendering, and adherence to specified typefaces. It contains 600 English and Chinese prompts, 3,695 text segments, 78 typeface families, and 24 design categories. Experiments on 16 models show that long and small text remain challenging, and accurate text rendering does not necessarily imply adherence to the requested typeface.
Critical strengths
- The paper is well written and easy to follow, with a clear presentation of the benchmark construction and evaluation protocol.
- ExtremeTextBench addresses a meaningful evaluation problem by assessing long-text rendering, small-text rendering, and typeface fidelity in practical text-rich design scenarios.
Critical weaknesses
-
The paper does not clearly show what makes the long-text setting challenging. The authors report length statistics and results across length ranges, but lacks examples showing how long the text is and where models fail. Longer text also tends to involve smaller fonts and more complex layouts, so it is difficult to tell which factor causes the performance drop.
-
The reason for combining font evaluation with long-text evaluation is not clear enough. Font control has practical uses, but the paper mostly evaluates text accuracy, without showing what new insights come from combining them. Moreover, font evaluation only includes sufficiently matched text regions of at least 50 pixels in height, leaving out the small-text setting that the paper emphasizes.
-
The added value over existing benchmarks remains unclear. The evaluation mainly uses OCR-based text scores and multimodal-model-based font scores, and the results mostly describe score differences between models. Since existing benchmarks use different metrics and evaluation procedures, a wider score spread does not by itself establish a better benchmark.
Questions for authors
see weakness above