- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces a dataset of 13.7M images with detailed structured annotations and compares generation performance with dense-caption baselines. They also compare pixel-space and latent-space generators and investigate training techniques to reduce their performance gap.
Critical strengths
They construct a large dataset with detailed annotations for 13.7M images, which could be useful for further research.
There are nice ablations both on the generative model side and on the text format (e.g. comparing JSON and prose while keeping the annotation content the same).
Critical weaknesses
-
Although the dataset could be useful, the method's novelty seems limited. The annotation scheme closely follows existing work, and the improvements to pixel-space generation combine known techniques. Structured captions being useful has also been studied before. Thus, the additional insights beyond existing work seem limited to me.
-
Table 1 reports low score for some aspects like object coverage. This could be concerning about the quality of the annotations. It would be useful to understand how this is affecting training and the resulting models.
Questions for authors
Aside from showing that richer annotations are helpful, which components contribute the most to the superior performance (bounding boxes, separate records for objects, ...)? Also, compared with the dense-caption version, what do the authors believe is the reason behind the boost in some aspects and not the others in Table 14?
3 comments
Response to Reviewer z8DU: Questions for Authors Q1–Q2
Dear Reviewer z8DU
Thank you for your questions about which annotation components drive the gains and why improvements vary across generation skills. We address Q1 and Q2 below through the field ablations and an analysis of the skill-specific results.
Q1. Contributions of individual components
Aside from showing that richer annotations are helpful, which components contribute the most to the superior performance (bounding boxes, separate records for objects, ...)
To identify which annotation components contribute most to the generation capability, we compare Full JSON with three field-ablation variants. Global-only removes all element records, retaining only global scene and style descriptions. w/o Box removes element coordinates. w/o Region-desc removes element descriptions while retaining boxes, types, colors, and literal text.
| Annotation | GenEval2 ↑ | PRISM Total ↑ | Alignment ↑ | Aesthetic ↑ |
|---|---|---|---|---|
| Full JSON | 62.98 | 6.29 | 7.17 | 5.42 |
| w/o Box | 50.20 | 6.12 | 6.87 | 5.37 |
| w/o Region-desc | 37.00 | 5.50 | 6.26 | 4.74 |
| Global-only | 31.94 | 5.37 | 6.05 | 4.69 |
Removing regional descriptions has a larger effect than removing boxes: both PRISM alignment and aesthetics decrease, and GenEval2 performance approaches Global-only. The retained boxes, types, colors, and literal text do not substitute for detailed local descriptions. This points to element descriptions as an important source of supervision for both content fidelity and visual quality.
Removing boxes has a more selective effect: alignment decreases while aesthetics remains close to Full JSON. The GenEval2 breakdown further shows a larger change in Count, from 78.95 to 65.78, than in Position, from 81.56 to 78.88, while object and attribute scores remain comparable. Our interpretation is that explicit boxes associate element records with distinct image regions, helping the model distinguish individual instances and render the requested number of objects.
Q2. Uneven gains across GenEval2 skills
Compared with the dense-caption version, what do the authors believe is the reason behind the boost in some aspects and not the others in Table 14?
Thank you for highlighting the uneven gains across the skills in Table 14. We believe these differences reflect the additional information each skill needs from structured annotations. Dense captions already convey object identities, attributes, and actions, whereas precise layout and counting benefit more from explicit instance-level detail.
| Training annotation | Evaluation input | Object ↑ | Attribute ↑ | Count ↑ | Position ↑ | Verb ↑ |
|---|---|---|---|---|---|---|
| Dense | Dense desc. | 93.78 | 90.58 | 56.14 | 69.31 | 32.87 |
| Dense | Record desc. | 92.21 | 91.26 | 61.51 | 81.57 | 38.96 |
| JSON | JSON | 90.05 | 91.24 | 78.95 | 81.56 | 35.78 |
| Prose | Prose | 90.12 | 91.43 | 78.76 | 83.23 | 34.58 |
Positioning benefits largely from richer inference descriptions. With the Dense model unchanged, providing record-based descriptions brings its Position performance close to that of JSON and prose. This suggests that clearer layout information in the input explains much of the positioning gain.
Counting, however, benefits additionally from training with structured annotations. Even with record-based inputs, the Dense model retains a substantial counting gap relative to models trained on JSON or prose. We interpret this as a benefit of learning from separate element records: each record associates an individual instance with its description and location, helping the model distinguish instances and render the requested number of objects.
I thank the authors for their comprehensive responses, especially the ablations. Although novelty remains a concern for me, the responses address my main questions and help clarify the value of the dataset for open research. I will therefore revise my score to weak acceptance.
Response to Reviewer z8DU: Critical Weaknesses W1–W2
Dear Reviewer z8DU
Thank you for your careful review and for recognizing the value of our dataset and controlled studies. Below, we clarify the contributions beyond prior work and discuss how annotation quality relates to generation performance.
W1. Novelty and additional insights
JIG's primary contribution lies in the open data foundation and empirical findings about structured image generation. Structured JSON conditioning has been widely adopted by leading image generators, including FLUX.3, MOSS-Image, FIBO2, etc. Yet few large-scale datasets with structured image annotations are publicly available, limiting the community’s ability to conduct further research under this paradigm. JIG-14M addresses this gap with high-quality structured annotations for 13.7M images, providing a shared data foundation for open research.
Beyond data release, we also provide a systematic empirical study of structured supervision. FIBO compares long structured captions with short captions, while ContextScaling examines caption content and organization jointly. Compared with these prior studies, we further distinguish annotation content from its serialization, a distinction that a dense-versus-structured comparison alone cannot establish. We vary the format and ordering of the same finalized records while preserving their information and element associations. As shown below, the gains over dense captions persist across formats and record orders. In addition, the field-level ablations in Q1 further examine the roles of different components, including element records, bounding boxes, and regional descriptions.
Regarding the pixel-space improvements, we acknowledge that they are built on established techniques. Our contribution here lies in evaluating these techniques under structured supervision using four generator baselines spanning pixel and latent representations and diffusion and Transfusion architectures. The baseline pixel–latent gap exceeds the gap between the two architectures, and pixel-adapted training narrows it in both. These results show that pixel-space training strategies developed for text-to-image generation remain effective in the JSON-conditioned setting. Together with the latent baselines, these models and their training recipes provide a common reference for future JSON-to-image research.
W2. Annotation quality and training impact
Thank you for highlighting the room for improvement in annotation quality, particularly object coverage. Our assessment shows that Ideogram 4-style structured annotations provide broader object coverage and more accurate descriptions than dense captions. Models trained with these annotations also yield substantially better generation performance compared with their dense-caption counterparts. We additionally provide Reve 2-style annotations with better quality, especially higher object coverage. Across these annotation variants, higher annotation quality is accompanied by stronger generation performance, supporting the value of improving structured supervision.