acceptodds
Withdrawn from ICLR 2027

GTS: Gap-Targeted Synthesis for Budget-Efficient Synthetic Data Scaling

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion models have made synthetic images a practical additional source of training data for visual recognition. In this scaling context, dataset distillation compresses a real training set into a small synthetic replacement, an objective misaligned with settings where the complete real dataset is already available. We instead ask how much accuracy a minimal addition of synthetic images can gain when added on top of that complete real set. We introduce Gap-Targeted Synthesis (GTS), a data-centric framework that diagnoses where synthetic data are needed before generating them. GTS constructs two complementary structural references solely from real features: a mode gap is an observed class mode whose coverage is insufficient; a variation gap is a class-specific interval missing along variation directions shared across classes. GTS converts these deficits into class- and target-level generation quotas and guides a pre-trained diffusion model toward the assigned feature targets during sampling. Gap diagnosis and synthesis require no downstream student classifier, so the same synthetic set can be reused across training architectures. Neither stage draws from an oversized candidate pool. On ImageWoof2, the reported fixed-budget comparisons show higher mean top-1 accuracy than unguided sampling across three student architectures, while paired-noise measurements show that guidance reaches the assigned mode and variation targets more often.

Primary area:
generative models
Keywords:
Diffusion Models, Data Synthetic, Data Scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xUb2OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how to use a limited synthetic-data budget to improve image classification when the complete real training set is retained. The authors propose Gap-Targeted Synthesis (GTS), which identifies two types of coverage gaps in frozen real-data features: mode gaps, representing relatively underrepresented observed modes within a class, and variation gaps, representing undercovered intervals along variation directions shared across classes. These gaps determine generation quotas across classes and targets, and a pretrained diffusion model is guided toward the corresponding feature targets during sampling. The resulting synthetic dataset is then combined with the full real training set for classifier training. The authors evaluate GTS on ImageWoof2 across three classifier architectures and multiple synthetic-data budgets, comparing it with controlled baselines and prior diffusion-based dataset-distillation methods. Additional experiments cover ImageNette and ImageIDC, evaluation under distribution shifts, paired-noise target-attainment measurements, ablations of the two gap components, guidance-strength sensitivity, controlled mode deletion, and generation cost.

However, the proposed gap definitions lack sufficient justification. The uniform mode reference may incorrectly treat naturally rare modes as coverage gaps, while the cross-class variation reference may mistake legitimate class-specific differences for insufficient coverage. The experiments are also limited to ten-class datasets, leaving the method's applicability to larger-scale classification tasks unclear. In addition, the ablations do not adequately examine how key design choices affect gap identification.

Overall, I recommend Clear Rejection (1/4).

Critical strengths

The paper addresses an interesting and practically important question: how to allocate a limited synthetic-data budget to improve classification when the full real training set is already available. Focusing on what synthetic samples can add beyond the existing data provides a useful perspective on making synthetic augmentation more effective.

Critical weaknesses
  1. The distributional rationale for targeted augmentation remains unclear. GTS selectively adds synthetic samples to regions identified as undercovered, thereby changing the effective training distribution. However, it is unclear whether these regions are underrepresented relative to the downstream test distribution or merely relative to the method’s chosen structural references. If the real training and test data follow the same distribution, naturally unequal mode frequencies or class-specific variation distributions do not necessarily indicate deficiencies that should be corrected. If a distribution shift is assumed, the paper does not establish why the proposed references approximate the coverage needed under that shift. A more explicit distributional formulation and controlled analysis of this mechanism would strengthen the motivation for targeted synthesis.

  2. The uniform reference for mode gaps is insufficiently justified. Eq. (4) assumes equal mass across observed clusters, although naturally occurring modes need not be equally frequent. Consequently, a small cluster may represent a genuinely rare variation rather than missing coverage. The diagnosis also depends on clustering granularity and cannot independently identify completely unobserved modes.

  3. Variation gaps rely on an insufficiently validated cross-class assumption. Class centering does not ensure that shared PCA axes capture semantically equivalent variations across classes. Differences from the other-class reference may therefore reflect legitimate class-specific structure rather than undercoverage. More direct validation of the detected target-class gaps is needed.

  4. The evaluation is limited to small-scale datasets. GTS is evaluated on three ten-class ImageNet subsets, leaving its scalability to larger and more heterogeneous class collections unverified. Several directly related methods provide such evaluations: MGD³ [1] reports ImageNet-100 and ImageNet-1K experiments in Sec. 5.1; CaO² [2] reports both in Table 2; and IMS³ [3] includes them in Appendix Secs. 8.1 and 8.11. This matters particularly for GTS because its variation-gap diagnostic relies on shared cross-class structure, whose reliability may change substantially as class diversity increases. Evaluation on ImageNet-100 or ImageNet-1K under the proposed additive protocol would strengthen the evidence for general applicability.

  5. Sensitivity of the gap diagnosis is insufficiently examined.. The reported sensitivity analysis focuses on guidance strengths, while the cluster count, PCA dimensionality, gap thresholds, and diagnostic calibration also determine the selected targets and quotas.

  6. The presentation is difficult to follow. Key methodological details are scattered between the main text and appendix, and some notation is ambiguous. Clearer explanations and more consistent notation would improve readability.

[1] Chan-Santiago, J. A., Tirupattur, P., Nayak, G. K., Liu, G., & Shah, M. (2025). MGD 3^ 3: Mode-Guided Dataset Distillation using Diffusion Models. arXiv preprint arXiv:2505.18963.

[2] Wang, H., Zhao, Z., Wu, J., Shang, Y., Liu, G., & Yan, Y. (2025, October). Cao 2: Rectifying inconsistencies in diffusion-based dataset distillation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 4722-4731). IEEE.

[3] Wang, C., Chen, Y., Yang, Z., Zhou, J. T., & Zhang, C. (2026). IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation. arXiv preprint arXiv:2603.13960.

Questions for authors
  1. Could the authors explain the distributional rationale for GTS more explicitly? Does the method assume a shift between the real training and test distributions, and why should the uniform mode reference and cross-class variation reference identify deficiencies relevant to the test distribution rather than natural distributional differences?

  2. Could the authors provide experiments on ImageNet-100 or ImageNet-1K under the same additive protocol to assess scalability and the reliability of the shared-variation reference with more heterogeneous classes?

  3. Could the authors provide additional ablations or sensitivity analyses for the cluster count, number of retained PCA axes, gap-detection thresholds, and relative weighting of the two diagnostics, and examine how these choices affect the detected gaps and downstream accuracy?

Reviewer_vGDdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper writing needs to be signicantly improved, now looks like a memo not a scientific paper that constructs what is already there, what is the problem and why we are intersted in solving it. e.g., there is no single mention that what is the generator that the authors used or on what dataset it have been trained on? is it the original dit model xie and pebles?
  • The related works section must be improved, there are lots of relevant works that the authors have not mentioend.
  • The scope of experiments are super limited, authors should consider additional datasets, benchmarks … generator sampling configuration e.g., number os steps, will i t work with a distilled generator? is it the property of the specific ddim sampler they are using? can it be applied to stochastic samplers that may produce more diverse samples with more compute expense cost?
  • The improvements are statistically insignicant, even by considering the reported variances.
  • So authors method are using a much stronger dinov2 right? how this method’s performace (zero shot or tuned are missing from the reports), if the either the tuned or zero-shot are lower? have the uathors considered using their own models trained on the ensemble of the datasets?
  • Comparing Convet6, resnet-10 and 18 reported in table 1, the gains seems to be lowered down, what are the advantage in terms of training/ total cost of simply scaling the model e.g., resnet50 or scalable architectures like vits for this in comparison to costs took for generating more additional samples and retraining on the combination? this cost analysis mus tbe reported.
Critical strengths

Please see the Paper Summary

Critical weaknesses

Please see the paper Summary

Questions for authors

Please see the paper summary

Reviewer_cdydOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a gap-targeted synthesis (GTS) framework to improve the utility of synthetic data when the complete real training dataset is already available. Instead of unguidedly generating synthetic images across classes, the paper first identify coverage gaps in the real dataset along two dimensions: (i) underrepresented modes within a class, where some observed variations have fewer real samples (images in this case), and (ii) missing shared variations, where a variation prevalent in other classes is underrepresented or absent in the target class. To identify these gaps, GTS first use a frozen DINOv2 encoder to extract real-image features and construct structural references based on equal mass across observed modes and shared within-class variation directions (obtained using PCA). Based on the measured deficits in the coverage and a fixed synthetic-data budget, the proposed framework allocates generation quotas across classes and target regions in proportion to their calibrated gap masses (outlined in Figure 1). A pretrained class-conditional diffusion model is then guided toward these identified feature-space targets during sampling, and the generated images are added to the complete real training dataset. The synthetic dataset is generated independently of the downstream classifier and can therefore be reused across different student architectures.

The GTS framework is mainly on ImageWoof2 (dog-images centric dataset) across 5 synthetic-data budgets and 3 CNN-based student architectures, and further extend their experiments to ImageNette and Image-IDC (subsets of ImageNet). They compare against several controlled baselines, including random generation, class-level rebalancing and target-based selection, as well as the prior diffusion-based dataset-distillation methods under matched synthetic-data budgets. The GTS achieves the highest mean classification accuracy across all reported main comparisons. The paper also includes distribution-shift evaluations experiments to verify whether guided generations reach the intended feature-space regions. The ablation studies demonstrates the complementary contributions of the two proposed gap diagnostics. The key takeaway is that identifying coverage gaps in the real data and allocating synthetic images to those gaps can be more beneficial than simply increasing the number of synthetic images.

Critical strengths
  1. The problem formulation in this work is well-motivated. It addresses a practical problem of allocated a limited budget of synthetic-images to improve classification when a complete real training dataset exists and is readily available. Also, the proposed mode and variation gap diagnosis seems like an intuitive and structured approach to address this stated problem.

  2. On empirical front (main results Table1), the GTS (gap targeted synthesis) framework achieves the highest mean top1 accuracy across all five synthetic budgets and three student model architectures on ImageWoof2 consistently. Additional experiments also support these results moderately, and distribution-shift experiments show some transferability beyond standard evaluation.

  3. Ablation is well conducted. The ablations in Table 4 showed that the mode-gap and variation-gap diagnostics provide complementary benefits. Also, Appendix G showed that adding targeted synthetic data can partially recover the accuracy lost when certain groups of real training images are artificially removed.

  4. The generated synthetic dataset is independent of the downstream student (model) architecture and can be reused across different learners.

Minor Strength: The supplementary material provides more details on guidance schedules, quota-allocation procedures and generation-costs.

Critical weaknesses

Major Weakness

  1. The biggest drawback of this work is that the evaluation is restricted to three ten-class ImageNet subsets, with ImageWoof2 as the primary benchmark. Although the results are consistently better, this setting seems narrow for a method that relies on shared within-class variations across categories. The class-centered PCA reference may behave differently on larger and more diverse datasets, such as ImageNet or iNaturalist. Appendix I provides some useful qualitative evidence, but only on the dog-centric dataset. Moreover, prior diffusion-based augmentation methods and feedback-guided synthesis (cited in the introduction), have evaluated their approaches on larger benchmarks like ImageNet. Evaluating GTS on such datasets would provide a stronger comparison with prior work and support its generality claims.

  2. The introduction and related-work section currently overlook relevant published approaches, including Diffusion Curriculum (DisCL; ICCV 2025) and TADA (ICLR 2026). DisCL identifies difficult real training examples and adaptively controls image-guided diffusion through curriculum learning, while TADA selectively augments real samples based on learning dynamics. This manuscript's introduction states that synthetic-data budget efficiency has not previously been treated as an objective.

However, both TADA and DisCL have explored selective, real-data aware synthetic augmentation, so this broad claim needs qualification. Discussing these works would help clarify how GTS differs from prior difficulty-based approaches through its structural coverage-gap estimation and budget allocation.

  1. One of the main contributions of this paper is allocating synthetic images based on the measured coverage gaps -- also highlighted in the Figure 2. Although Appendix C clearly explains the allocation procedure, it is unclear to me why the synthetic-image quotas should be linearly proportional to the calibrated gap masses, particularly when the calibration weights λmode=1,λvar=2\lambda_{\mathrm{mode}}=1, \lambda_{\mathrm{var}}=2 are fixed without sufficient empirical justification. Also, ablations showed that both gap diagnostics are useful, they do not directly evaluate whether the proposed budget allocation is better than simpler alternatives, such as uniform allocation across the discovered targets in step one.

Moderate Weakness:

  1. The GTS method assumes equal mass across observed modes and uses variations from other classes as a reference to identify coverage gaps. However, naturally imbalanced modes may not indicate missing data, and variations observed in other classes may not always be valid for the target class. Appendix provides some useful supporting evidence but further analysis of these assumptions or sensitivity to the diagnostic thresholds would strengthen the justification for the identified coverage gaps.

Minor Weakness

  1. Algorithm-1 uses k as the image repetition index but k is already used for indexing PCA variation axes.

  2. Algorithm-1 : line 1 is confusing in notations. Section 4.1 defines zi=ϕ(xi)z_i=\phi(x_i), whereas algorithm 1 defines zi=norm⁡(ϕ(xi))z_i=\operatorname{norm}(\phi(x_i)). can you clarify and correct the text?

  3. All three evaluated student architectures are CNN-based. Could the authors justify the absence of a ViT-based classifier, given that GTS claims student-independent generation and uses frozen DINOv2 ViT features for gap detection? Including a ViT-based student would strengthen the evidence of cross-architecture generalization.

References

TADA: DO WE NEED ALL THE SYNTHETIC DATA? TARGETED IMAGE AUGMENTATION VIA DIFFUSION MODELS. ICLR 2026.

DisCL : Diffusion Curriculum: Synthetic-to-Real Data Curriculum via Image-Guided Diffusion. ICCV 2025.

Questions for authors

Q1. Generalization to larger datasets: Could the authors evaluate GTS on a larger and more diverse dataset, such as ImageNet or iNaturalist, to demonstrate whether the proposed gap detection remains effective across hundreds of semantically diverse classes, as evaluated in prior diffusion-based augmentation works?

Q2. Missing baseline: The paper includes a classifier-feedback baseline on the additional datasets but not in the main ImageWoof2 comparison (Table 1). Could the authors clarify this choice?

Q3. Budget allocation: What motivates the choice of λmode=1\lambda_{\mathrm{mode}}=1 and λvar=2\lambda_{\mathrm{var}}=2, and how sensitive is performance to these values? Could the authors compare the proposed gap-proportional allocation with uniform allocation across the same identified targets, keeping the total synthetic budget KtotK_{\mathrm{tot}} fixed? This would help establish whether allocating more images to larger gaps improves classification accuracy.

Q4. Structural coverage assumptions:

(a) What motivates the equal-mass reference across observed modes? Could the authors analyze whether naturally imbalanced modes are incorrectly identified as coverage gaps?

(b) How do the authors verify that the shared variations identified from other classes correspond to valid variations in the target class, beyond the qualitative evidence in Appendix I?

(c) How sensitive is GTS to the variation-gap thresholds τ\tau and ρ\rho, and the no. of retained PCA axes?

Q5. Clarify the concerns asked in Minor Weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.