- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CGTR aligns image-text data to a concept hierarchy built from Wikipedia and Wikidata. It estimates coverage and flags concepts that are missing or underrepresented. Using an audit of around 370 million annotations, the authors construct KnowGapBench, which covers 3,286 sparse concepts; identity and compositional understanding get scored separately. To boost Fine-T2I, they generate extra training pairs targeted at dataset-specific gaps, drawing on scene graphs, reference appearances, and filtering rules. They then resample data with concept-aware weighting and fine-tune UniWorld-V1. On KnowGapBench, its knowledge score rises from 15.3 to 22.5, while performance on CommonConceptBench holds steady without degradation. Thirteen existing T2I models are also evaluated on this benchmark.
Critical strengths
Coverage here is actionable, not just for analysis. These coverage numbers directly decide which concepts get augmented data and adjust the sampling weights. That makes this work more practical than many papers that merely demonstrate poor performance on rare concepts.
KnowGapBench shares the same measurement framework used to set training targets. Identity assessments can rely on references, while compositional evaluation remains decoupled. The UniWorld result, paired with the check showing no collapse on unrelated concepts, gives solid grounds to trust their approach.
Critical weaknesses
“Uncovered in annotations” does not equal a true lack of visual supervision. Aliasing errors, misassigned category links, or text-only mentions can create false gaps. On the flip side, a handful of nearly identical images might register as covered yet contribute almost no useful learning signal. Without human auditing to measure the precision and recall of these detected gaps, the headline coverage numbers are less robust than they appear.
The training evidence also feels lightweight relative to the scope of their claim. They only test one backbone, swap in roughly 5% of the data, and run no ablations isolating sampling weighting alone, synthetic data alone, or filtering. It is impossible to tell which component delivered that +7.2 gain on the Knowledge metric.
Questions for authors
We ask for four additional results and clarifications. First, for a random sample of 200 concepts, report human agreement against the automatically assigned Uncovered and Underrepresented labels. Second, using the same UniWorld setup, provide KnowGapBench and CommonConceptBench scores for three separate variants: concept-aware sampling alone, synthetic augmented data alone, and the full CGTR pipeline. Third, describe the safeguards used to prevent synthetic images from closely replicating the reference appearances in KnowGapBench, such as perceptual hashing or retrieval-based filtering. Fourth, frame the related-work discussion to distinguish this data-gap completion approach from methods that use fine-grained feedback on the reward side; Unicorn, SRUM and UniRL are good points of comparison.
3 comments
Thanks for fast reply, I recommand you to show these changes and additional discussions in paper using different color. It will be clearer for AC and us to check!
I think knowledge is an important topic in UMMs, if you authors can give us revisied manuscript with exact improvements, I tend to accept this paper.
We sincerely thank the reviewer for the thoughtful questions and constructive feedback on our work. We have carefully addressed the concerns raised and provided corresponding explanations and additional clarifications in our response. We have also incorporated the relevant revisions and supplementary details into the revised manuscript. For ease of review, the corresponding changes are highlighted in different colors to help identify the revisions and clarify the points raised in the original review. We hope that our responses and revisions adequately address your concerns and provide further clarity on our work. We would greatly appreciate any further feedback.
Response to Reviewer Ldx4
We thank the reviewer for constructive feedback and encouraging comments on our work. Below, we address the questions and suggestions raised in the review.
CGTR provides data infrastructure for large scale T2I training. It audits concept coverage, supplements missing visual supervision, and adjusts sampling to improve basic knowledge before post training.
Q1. Reliability of Coverage Gaps
Our coverage labels measure support in the audited annotation pool, rather than proving that visual evidence is absent from every image. We use hierarchical concept extraction with structured outputs and allow multiple concepts per image. KNN and BM25 link concepts to the knowledge graph, and a VLM checks uncertain matches. We manually examined 200 low frequency training samples to assess extraction and linking:
We also sampled 100 concepts, 50 labeled Uncovered and 50 Underrepresented, and checked caption retrieval results manually:
Many retrieved captions contain related terms or mention the target only as a secondary subject. Such matches do not necessarily provide useful visual supervision. The two audits assess different issues: the 200 examples test extraction and linking, while the 100 concepts test gap labels. Together, they support our concept coverage and support estimates for the audited corpus.
Q2. Training Evidence and Component Contributions
We use about 6M Fine-T2I samples to study data refinement at a scale relevant to T2I pretraining. The original Baseline and CGTR process the same number of training samples, but use global batch sizes of 64 and 256. A full training run costs about 627 GPU-hours, so repeating full runs with matched batch sizes is costly. We instead ran four controlled experiments at batch size 64, with matched optimization settings and about one quarter of the original training volume per run.
Table. Controlled ablations on KnowGapBench and CommonConceptBench.
With matched settings, sampling alone raises KnowGapBench Knowledge by 0.4 points, synthesis alone by 2.1, and full CGTR by 4.2. Sampling has limited effect when long tail concepts have few examples, and cannot supply missing visual supervision. Synthetic data provides this supervision, while concept aware sampling increases exposure to these concepts. On CommonConceptBench, the four settings remain close (62.1 to 64.2). Full CGTR scores 63.6 against the baseline's 62.3, showing that performance on familiar concepts is preserved while KnowGapBench improves. In the original full runs, KnowGapBench improves from 15.3 to 22.5 (+7.2), and CommonConceptBench from 62.1 to 65.9 (+3.8).
Q3. Similarity to Benchmark References
We already applied hash deduplication during reference collection and selection. Synthesis reference images were collected from different sources than KnowGapBench evaluation references. Our scene graph prompts vary auxiliary objects, scenes, and styles while keeping the target concept fixed.
These measures reduce near duplicate risk. We will add a cross-set pHash check between synthesis references, final synthetic images, and KnowGapBench references, with manual review of potential matches, and report the results in the appendix.
Q4. Related Work
We will expand the Related Work section in the main paper. UniCorn uses self generated supervision through Proposer, Solver, and Judge roles. SRUM uses global and local reward feedback from a unified model's understanding component. UniRL combines generated data with SFT and reinforcement learning to improve generation.
CGTR instead focuses on training data. It identifies missing and underrepresented concepts, guides targeted supplementation, and adjusts sampling during training. When visual evidence for a concept is scarce, feedback alone may not provide sufficient supervision for its appearance. CGTR is therefore complementary to these methods.