Where Do Pixels Hide Their Secrets? Uncovering Model Internals for Creative Text-to-Image Evaluation

Text-to-image (T2I) models can produce visually convincing images while still failing to correctly realize the concepts and relationships expressed in a prompt. This challenge is particularly difficult for creative inputs such as poems and sto- ries, for which reference images are typically unavailable. Existing reference-free metrics primarily evaluate the relationship between the input text and the final gen- erated image, while recent interpretability studies suggest that intermediate repre- sentations contain information about the spatial organization of generated content during the denoising process. We ask whether such model-internal signals can instead be used to evaluate generation quality. We introduce MIRAGE (Model- Internal Representation Analysis for Generation Evaluation), a reference-free metric that evaluates spatial properties of internal representations during image generation. MIRAGE extracts image-token representations from intermediate diffusion-transformer blocks and decomposes them into sparse latent features. For each generation state, it identifies the dominant sparse feature and derives two complementary spatial signals: cross-seed spatial consistency, which mea- sures the stability of the feature’s spatial centroid across stochastic generations, and activation concentration, which measures how spatially localized the feature’s activation is around its centroid. These signals are aggregated into an interpretable score for each generated image. We evaluate MIRAGE on a curated collection of 6,000 poems and stories generated using three diffusion-transformer T2I models. Across human evaluations of semantic fidelity, attribute presence, compositional correctness, and narrative fidelity, MIRAGE shows agreement with human judg- ments and provides a complementary signal to existing reference-free evaluation metrics. These results suggest that model-internal spatial representations can pro- vide useful signals for reference-free evaluation of creative T2I generation.