InfoDiag-Bench: A Scenario-aware Diagnostic Benchmark for Deliverable Infographic Generation
Abstract
While modern text-to-image (T2I) models produce visually compelling natural images, generating infographics suitable for practical use remains challenging. Such images require accurate rendering of dense text and numerical content, and coherent topological structures, which vary across different tasks. Consequently, infographic evaluation also requires pronounced inter-domain diversity across application scenarios. Furthermore, assessing practical deliverability necessitates distinguishing tolerable deviations from delivery-blocking errors. To address these challenges, we propose InfoDiag-Bench, a scenario-aware infographic benchmark spanning 663 realistic test cases across 14 application scenarios. Central to our evaluation framework is a set of predefined, evidence-grounded error types designed to systematically separate tolerable flaws from delivery-blocking ones. Across 10 open-source and close-source models, we find that the inability to preserve topological relationships among elements is the primary cause of delivery failure. Human evaluation further demonstrates a strong alignment between our automated judgements with human anticipation.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.