acceptodds
Under review as a conference paper at ICLR 2027

From Prompts to Question Graphs: Structure-Aware Evaluation and Selection of Text-to-Image Models

Abstract

Text-to-image evaluation usually compresses the satisfaction of multiple, interacting requirements into a single score for each prompt. This flat treatment creates three ambiguities: a missing entity can make all of its attributes and relations unanswerable, prompts containing more checks can receive disproportionate influence, and the same error may reflect either a basic generation failure or an inability to preserve a concept under composition. We introduce QC-T2I-Bench, a framework that represents each prompt as a graph of attributed atomic questions. Built on Davidsonian Scene Graphs, this representation explicitly connects entities, attributes, relations, and higher-level semantics through their prerequisite structure. We develop a dependency-aware aggregation rule that evaluates a question only when its prerequisites are satisfied and balances contributions across prompts of different complexity. The resulting graph-structured evidence supports analyses unavailable from conventional prompt scores: it separates initial concept realization from conditional preservation, measures the probability of jointly satisfying connected requirements, and provides capability profiles for model selection. Experiments across diverse text-to-image generators and bilingual English–Chinese prompts reveal a sharp compositional bottleneck: joint completion decreases from 80.7% for components involving two capabilities to 37.2% for those involving seven or more. Finally, without training a routing model, question-level profiles select generators under quality–cost constraints, matching ERNIE's 89.51-point estimate while reducing inference cost by 21.3% in GPU-s/MP. These results show that question graphs provide a common representation for reliable comparison, interpretable failure analysis, and efficient deployment of text-to-image models.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.