- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how text-to-image generation degrades as the number of entities requested increases from 1 to 64. Across different models, it evaluates entity recall, face validity, identity duplication, CLIPScore, and density-matched KID, while additionally calibrating face-detector degradation on real images. The main findings are an approximately exponential decay in detector-corrected recall, increasing identity duplication at high density, and a mismatch between local structural fidelity and global image-quality metrics.
Critical strengths
-
The paper identifies that in crowded scenes, face detectors themselves degrade and they explicitly measure this effect on real images. This is an important evaluation confound that is often overlooked.
-
Studying entity counts up to 64 reveals failure modes that are not captured by standard compositional benchmarks or global metrics. The people/car comparison and local-vs-global evaluation are particularly useful.
Critical weaknesses
-
The main prompt requires the model to satisfy “exactly N people ... no other people in the scene.” However, even at N=1, the models generate 1.4–3.2 times the requested number of people. This is important because a model simply can have poor numerical understanding and tends to generate something like “a few people,” “a group,” or “a crowd,” rather than faithfully representing 4, 16, 32, or 64. In that case, N^/N would decrease strongly with N even without demonstrating a separate visual-capacity collapse. Prior work such as T2ICountBench [1] and LLM-Grounded Diffusion [2] already shows substantial difficulty with numerical constraints and negation.
-
The detector correction is calibrated on WIDER FACE using the actual number of faces present, whereas generated samples are grouped by the requested count N. These quantities may differ substantially. For example, an N=64 prompt that actually renders 20 people may not be as difficult for the detector as a real image containing 64 faces. Applying the 64-face correction could therefore over-correct recall at high N. An equivalent density-dependent calibration is also not provided for the YOLO person detector.
-
The identity measure asks whether an image contains at least one highly similar face pair. However, the number of possible pairs grows rapidly with the number of faces: 4 faces give 6 pairs, while 20 faces give 190. Thus, even if the similarity distribution of individual pairs stayed unchanged, the probability of finding at least one high-similarity pair would naturally increase with density. A related effect is acknowledged in the paper's birthday-collision analysis, but a fixed-comparison analysis is needed to show that identities themselves become less diverse. Recent work such as DisCo [3] also directly studies duplicate identities in multi-human T2I generation.
-
The empirical finding that an exponential fits the observed recall better than a power law is interesting. However, the paper further interprets this as evidence for an intrinsic finite entity budget and a constant per-entity survival hazard. The curve fit alone does not uniquely support these mechanisms: count saturation, worsening prompt adherence, spatial-resolution limitations, or other capacity effects could produce similar behavior. I think the paper should distinguish the empirical scaling result from the proposed mechanistic explanation more clearly.
[1] Cao et al., “Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help,” 2025.
[2] Lian et al., “LLM-Grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models,” TMLR 2024.
[3] Borse et al., “Resolving the Identity Crisis in Text-to-Image Generation,” CVPR 2026.
Questions for authors
Q1: Does the same scaling behavior remain if the prompt is simplified, e.g. comparing “N people,” “exactly N people,” and “exactly N people, no other people”? This would help determine whether the observed decay reflects density itself or difficulty following numerical/negative instructions. It is also well known that these models behave badly with negation words like "no", "not". How would you handle this?
Q2: How is the correction factor chosen when the requested and actually rendered numbers of faces differ?
Q3: Does the identity effect remain if the same number of faces or face pairs is sampled from every image?
Q4: Could the authors justify why the exponential specifically indicates a finite entity budget, or compare against other plausible saturation/capacity models?