acceptodds
Under review as a conference paper at ICLR 2027

Informative Data Reweighting for Image Classification

Abstract

The success of Deep Neural Networks (DNNs) in image classification largely depends on access to large-scale, high-quality labeled datasets, which are often scarce or costly to obtain. To alleviate this challenge, Generative Data Augmentation (GDA) has been introduced to improve model performance by increasing the number of training samples with synthetic data generated by models such as Diffusion Models (DMs). Despite its benefits, GDA-generated synthetic samples often contain noise, which can negatively impact the performance of image classification models when incorporated into training. Prior approaches, including data selection and reweighting techniques, aim to address this issue but often rely on external expert models or clean metadata. In this work, we introduce Informative Data Reweighting (IDR), a principled sample reweighting framework based on the Information Bottleneck (IB) principle, to enhance the performance of DNNs for image classification using GDA. Through extensive experiments, we demonstrate that IDR effectively prioritizes more informative training samples in the augmented training set comprising original real training samples and synthetic training samples, resulting in substantial improvements over existing data selection and reweighting strategies for GDA in image classification. The code for IDR is available at https://anonymous.4open.science/r/IDR-3BE0/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Informative Data Reweighting, Generative Data Augmentation, Information Bottleneck, Image Classification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ojjJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on reweighting the synthetic data when mixing it with the original training set for downstream training. The proposed Informative Data Reweighting (IDR) is based on the Information Bottleneck (IB) principle. Finally, the experiments show some improvement across different backbones.

Critical strengths
  1. This paper incorporates the information bottleneck and provides extensive theoretical justification in the main paper and appendix, making the paper supported by a systems-theory scope.
  2. This paper, regarding the theory side, provides very comprehensive background. As an image classification method, it even provides background knowledge (including the training procedure) on diffusion models in the appendix.
Critical weaknesses
  1. This paper uses almost one page to claim its contribution in the introduction section, making the presentation long-winded and lacking focus. The contribution should be highlighted and summarized in a clear and accurate manner.
  2. This paper shows severe experimental problems, as it claims to be a data-augmentation-based reweighting method and provides extensive mathematical derivation. The experiments use very old and limited baselines and ignore most of the new data augmentation methods, especially as the more and more excellent generative models today, making this paper's experimental evidence out of date.
  3. Extensive theoretical analysis coupled with very weak experimental support diminishes the credibility of the claim.
Questions for authors

Given that the analysis of the information bottleneck is so comprehensive, why hasn't there been consideration of applying it to broader scenarios—such as the current practice of incorporating synthetic data into the training of LLMs and generative models?

5 comments
AuthorsOpenReview ↗

Response to Review by Reviewer ojjJ - Part 1

Thank you for the review and for recognizing our information-bottleneck foundation and thorough background. IDR trains a predictive model on real plus generated images and learns how much each sample counts.

W1. The contribution statement is too long.

We have revised the contribution statement in Section 1 of the revised paper. We removed a sentence that repeated the results claim of the abstract, and the second contribution now states the revised VUIB, whose compression term is computed exactly and depends on the learned features, together with its corrected computational complexity. The key findings on the learned weights and on matched budgets and compute are now summarized in Sections 4.1 and 4.2, and the list at the beginning of Section 4 points to every added experiment in Sections D.18–D.25 of the appendix.

W2. Baselines and generative models.

IDR's baselines are recent. Five of the seven data selection/reweighting baselines and both classifier-augmentation baselines date from 2023 to 2025.

The advances you mention concern training generative models, while IDR trains a predictive model. Stronger generators and synthetic-data training for LLMs and generative models either improve a generator or train one. IDR solves a different problem: it learns how much each real or generated sample counts when training an image classifier. The two are complementary, not competing. Since IDR is generator-agnostic, any better generator can supply its data.

IDR is compared against its direct competitors, methods that select, reweight or augment a classifier's training data. Table 1 in Section 4.1 of the paper covers data selection and reweighting with CBF [1] (ICLR 2023), IE [2] and REVAR [3] (ICLR 2024), IDS [4] (UAI 2025) and GenDataAgent [5] (ICLR 2025), plus MW-Net [6] (NeurIPS 2019) and OTR [7] (NeurIPS 2022) as established references. For classifier augmentation, Table 4 in Section D.1 of the paper shows PAA [8] (AAAI 2025) at 84.5 and GuidedMixup [9] (TPAMI 2025) at 84.3, against 85.6 for IDR. Combined with IDR, they reach 86.1 and 86.0. The active-learning baselines in Table 10 in Section D.7 of the paper, including CSQ [10] (TMLR 2025), reach at most 84.4.

IDR is best with every generator. Table 16 in Section D.16 of the paper shows that with DiT-B [11] on ViT-B, IDR leads GenDataAgent by 0.8 percentage points at a matched 3N3N budget. With DDPM [12] and StyleGAN-XL [13] on Swin-B, Table 13 in Section D.12 of the paper puts IDR ahead of the strongest baseline by 1.1 and 1.2 points, respectively. Table 8 in Section D.5 of the paper shows IDR reaching 85.7 with DiT-L/XL.

Stronger generators and a recent generative augmentation method. We have conducted two additional experiments with ViT-B. We trained CBF and GenDataAgent on DiT-XL data at their cross-validated budgets. We also compared with DA-Fusion [14] (ICLR 2024), which edits real images with Stable Diffusion [15], and used IDR to reweight its images at a fixed 1N1N budget. IDR stays best with DiT-XL, 0.9 percentage points above the best baseline, and improves DA-Fusion by 0.7 points. These results have been added to Table 27 in Section D.25 of the appendix of the revised paper. The DiT-XL budgets differ across methods as shown below, and Table 26 in Section D.24 gives the comparison under a common budget.

GeneratorMethodSynthetic budgetTop-1 (%)
DiT-XLCBF2N2N84.6
DiT-XLGenDataAgent3N3N84.8
DiT-XLIDR (Ours)5N5N85.7
Stable DiffusionDA-Fusion1N1N84.1
Stable DiffusionDA-Fusion + IDR (Ours)1N1N84.8
AuthorsOpenReview ↗

Response to Review by Reviewer ojjJ - Part 2

W3. Strength of the experimental support.

The experimental support is broad and statistically tested. Reviewers w6zi and pw8Y agree, writing "The experiments are sufficient and the results are competitive" and "The empirical evaluation is broad", respectively. On ImageNet-1K, Table 1 in Section 4.1 of the paper shows that IDR improves top-1 by 1.1 percentage points over the best baseline on each of ViT-S/B and Swin-T/B. Over ten seeds, Table 11 in Section D.8 of the paper gives p≤5.4×10−6p\le5.4\times10^{-6}.

Matched budgets. The full sweep in Table 16 in Section D.16 shows that IDR leads CBF and GenDataAgent at every budget from NN to 10N10N. We have added all competing methods at a common 3N3N budget to Table 26 in Section D.24 of the appendix of the revised paper, and Section 4.2 summarizes this comparison. IDR reaches 85.3 vs. 84.5 for GenDataAgent, with 2.0% more selected-data generation-plus-fine-tuning compute. The cost breakdown is in Section D.17, and these costs exclude shared LDM training and budget search.

Generators and breadth. The evaluation covers DiT-B/L/XL, DDPM and StyleGAN-XL in Tables 8 and 13 of the paper, Swin-T trained from scratch, COCO, ADE20K, Flowers/Pet/Cars, ImageNet-A/R, two medical datasets, and augmentation and active-learning comparisons. IDR also complements recent augmentation. In Table 4 in Section D.1 of the paper, IDR+PAA reaches 86.1, vs. 85.6 for IDR and 84.5 for PAA.

New experiments directly support the claims. In response to Reviewers w6zi and pw8Y, we added the revised feature-dependent VUIB to Sections 3.2–3.3 and its proofs and scope to Appendix E. Table 25 in Section D.23 of the appendix reports 85.6% at 3N3N, 1.1 percentage points above GenDataAgent, and these revised-objective results are distinguished from the submitted IDR results. The generator and centroid experiments have been added to Tables 27 and 20 in Sections D.25 and D.19 of the appendix of the revised paper, respectively, and the beginning of Section 4 lists them. The summary below shows IDR staying ahead with DiT-XL, improving DA-Fusion and exceeding centroid alternatives. The code is in the anonymized repository linked in the paper.

New experiment (ViT-B)Best competing methodIDR (Ours)
Revised VUIB, 3N3NGenDataAgent: 84.585.6
DiT-XL synthetic dataGenDataAgent: 84.885.7
Stable Diffusion images (DA-Fusion), 1N1NDA-Fusion: 84.184.8
Class-centroid regularization, 3N3NCentroid-similarity weights: 84.385.3

Q1. Broader scenarios: LLMs and generative models.

We focused on IDR's setting: training predictive models on real plus generated images. The scenarios you mention involve training LLMs and generative models on synthetic data, a different problem from deciding how much each sample should count for a predictive model. The two are complementary. Any better generator can supply IDR's data, and Tables 8 and 13 of the paper show that IDR is generator-agnostic across DiT-B/L/XL, DDPM and StyleGAN-XL.

Both scenarios are promising future work. They require a suitable label-like relevance variable Y~\tilde Y and a softly quantizable representation. For LLM fine-tuning on synthetic instruction or reasoning data, Y~\tilde Y could be the task category or verifier outcome, with prototypes over sequence embeddings and learned weights scaling the per-sample loss. For generative-model training, Y~\tilde Y could be the conditioning variable, with prototypes over denoiser features and learned weights scaling the denoising loss. This scope and these possible extensions have been added to Section G.2 of the appendix of the revised paper, and we make no empirical claim for these untested settings.

AuthorsOpenReview ↗

References

[1] He et al. Is synthetic data from generative models ready for image recognition? ICLR 2023

[2] Chhabra et al. "What data benefits my classifier?" Enhancing model performance and interpretability through influence-based data selection. ICLR 2024

[3] Jain et al. Learning model uncertainty as variance-minimizing instance weights. ICLR 2024

[4] Wang et al. Informative synthetic data generation for thorax disease classification. UAI 2025

[5] Li et al. GenDataAgent: On-the-fly dataset augmentation with synthetic data. ICLR 2025

[6] Shu et al. Meta-Weight-Net: Learning an explicit mapping for sample weighting. NeurIPS 2019

[7] Guo et al. Learning to re-weight examples with optimal transport for imbalanced classification. NeurIPS 2022

[8] Kim et al. Configuring data augmentations to reduce variance shift in positional embedding of vision transformers. AAAI 2025

[9] Kang et al. Model-agnostic and efficient mixup augmentation guided by saliency maps. IEEE TPAMI 2025

[10] Gwon et al. Enhancing cost efficiency in active learning with candidate set query. TMLR 2025

[11] Peebles and Xie. Scalable diffusion models with transformers. ICCV 2023

[12] Ho et al. Denoising diffusion probabilistic models. NeurIPS 2020

[13] Sauer et al. StyleGAN-XL: Scaling StyleGAN to large diverse datasets. SIGGRAPH 2022

[14] Trabucco et al. Effective data augmentation with diffusion models. ICLR 2024

[15] Rombach et al. High-resolution image synthesis with latent diffusion models. CVPR 2022

Reviewer_pw8YOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

IDR jointly trains a classifier and a network that assigns weights to real and generated images. The classifier uses weighted cross-entropy (CE), while the reweighting network optimizes a proposed variational bound on the IB objective. The bound uses soft prototype assignments. Experiments report improvements on ImageNet-1K and downstream transfer tasks, supported by ablations and comparisons at matched synthetic-data budgets.

Critical strengths

The empirical evaluation is broad, covering several backbones, training from scratch, alternative generators, and downstream transfer. The matched-budget comparisons also support a benefit beyond simply using more synthetic images. Component ablations and repeated runs strengthen the empirical evidence, while the cost analysis provides useful context for the additional training work.

Critical weaknesses

The results are encouraging, but I have some concerns about the IB formulation:

  • Eq13 factorizes as φ(i,a,y) = u_ia v_iy, reducing U_i to Σ_y v_iy log(v_iy/p_y). With fixed input prototypes and priors, this removes its dependence on learned features and undermines the claimed compression signal. When Q is the current conditional distribution, the remaining term minimizes prototype conditional entropy without requiring different classes to occupy different prototypes. The paper needs to explain how its centroid constraints and classifier updates address this issue.
  • The bound is defined over prototype assignments, but its connection to the original-feature MI is not established. The stated class-frequency priors also produce zero-prior divergences when there are more prototypes than classes. Separately, with fixed memberships, rescaling weights within a prototype group leaves its centroid unchanged but changes weighted CE. It is unclear how VUIB determines these relative group scales. These gaps weaken both the theoretical claim and the interpretation of weights as sample informativeness.
Questions for authors
  1. Where does a learned-feature compression gradient enter U_i after (eq13) factorizes, assuming fixed input prototypes and priors?
  2. The derivation bounds MI between prototype indices. What guarantees that this controls the original-feature MI claimed in the IB objective?
  3. How are nonzero input-prototype priors defined when the number of prototypes exceeds the number of classes? The stated class-frequency construction appears to assign zero mass to some prototypes, making the KL term divergent.
  4. With Q equal to the current conditional distribution, minimizing the remaining H(A|Y) term does not require different classes to occupy different prototypes. What in the centroid constraints or alternating CE updates prevents a shared-prototype solution?
  5. What fixes the relative total weight of each prototype group? Under fixed memberships, rescaling a group’s weights leaves its Eq. 2 centroid unchanged while altering its contribution to weighted CE.
7 comments
AuthorsOpenReview ↗

Response to Review by Reviewer pw8Y - Part 1

Thank you for the precise technical analysis. We appreciate that you found the evaluation broad and that the matched-budget comparisons support a benefit beyond simply using more synthetic images. We have revised VUIB in Sections 3.2–3.3 of the main paper and added its proofs and scope in Appendix E. The paper distinguishes the submitted-objective results from the revised-VUIB experiments. Each answer also covers the matching part of W1/W2.

Q1 (W1, first part). Where does a learned-feature compression gradient enter UiU_i?

The learned-feature gradient does not enter the submitted per-sample UiU_i. The submitted relaxation factorizes as ϕ(i,a,j)=uiavij\phi(i,a,j)=u_{ia}v_{ij}, so Ui=KL(vi∥p)U_i=\mathrm{KL}(v_i\Vert p), which depends on neither z~i\tilde z_i nor the feature prototypes. Section E.4 of the appendix of the revised paper now records the factorization in Eq. (26), the cancellation in Eq. (27) and the valid scope of the original bound in Eq. (28). This corrects the submitted Lemma E.1 and its per-sample relaxation.

The training signal comes from the relevance term, so the reported results stand as evaluations of the submitted procedure. Section 3.3 and Appendix B now clarify that weighted cross-entropy trains the predictive model, while VUIB updates only gθg_\theta. In the submitted objective, the feature-dependent signal for θ\theta is Vi=∑auialog⁡Q(a∣y~i)V_i=\sum_a u_{ia}\log Q(a\mid\tilde y_i), where uiau_{ia} depends on θ\theta through the weighted prototypes, now defined in Eq. (6), and QQ comes from the previous epoch. The original UiU_i can reach θ\theta only through weighted input prototypes, never through feature assignments.

We have measured both gradients with respect to θ\theta. For submitted IDR with ViT-B at 5N5N, the compression gradient is several orders of magnitude smaller than the relevance gradient at every reported stage. This is consistent with input-side assignments becoming numerically one-hot when pixel-space squared prototype distances differ greatly. The results therefore characterize effectively relevance-driven reweighting. In the component ablation of Table 9 in Section D.6 of the paper, submitted-VUIB weights reach 85.6, versus 84.3 when the same network is trained by cross-entropy on clean metadata. The gradient measurements below have been added to Table 24 in Section D.22 of the appendix of the revised paper.

Epoch∥∇θ Uˉ∥\Vert\nabla_\theta\,\bar U\Vert (compression)∥∇θ Vˉ∥\Vert\nabla_\theta\,\bar V\Vert (relevance)
103.2×10−93.2\times10^{-9}4.1×10−24.1\times10^{-2}
502.7×10−92.7\times10^{-9}2.6×10−22.6\times10^{-2}
1002.5×10−92.5\times10^{-9}1.8×10−21.8\times10^{-2}

The revised VUIB restores feature dependence in the compression term. We identify X~\tilde X with the sample index (equivalently, with input values when the observed inputs are distinct), and let AA be the soft-quantized representation with P(A=a∣i)=uiaP(A=a\mid i)=u_{ia}. The weights are normalized within each class, wi=p^y~igi/∑j:y~j=y~igjw_i=\hat p_{\tilde y_i}g_i/\sum_{j:\tilde y_j=\tilde y_i}g_j, and we write uˉa=∑iwiuia\bar u_a=\sum_i w_iu_{ia} and Pw(a∣y)=p^y−1∑i:y~i=ywiuiaP_w(a\mid y)=\hat p_y^{-1}\sum_{i:\tilde y_i=y}w_iu_{ia}. The compression term is then exact, Iw(A;X~)=H(uˉ)−∑iwiH(ui)=∑iwi KL(ui∥uˉ).I_w(A;\tilde X)=H(\bar u)-\sum_i w_iH(u_i)=\sum_i w_i\,\mathrm{KL}(u_i\Vert\bar u).

The relevance term retains H(uˉ)H(\bar u), which was dropped in the submitted relaxation: RQ=H(uˉ)+∑iwi∑auialog⁡Q(a∣y~i)≤Iw(A;Y~)R _Q=H(\bar u)+\sum _i w _i\sum _a u _{ia}\log Q(a\mid\tilde y _i)\le I _w(A;\tilde Y), with equality iff Q(⋅∣y)=Pw(⋅∣y)Q(\cdot\mid y)=P _w(\cdot\mid y).

Hence VUIBβ=Iw(A;X~)−βRQ ≥ Iw(A;X~)−βIw(A;Y~),\mathrm{VUIB} _\beta=I _w(A;\tilde X)-\beta R _Q\ \ge\ I _w(A;\tilde X)-\beta I _w(A;\tilde Y), and at Q=PwQ=P _w it equals Iw(A;X~∣Y~)−(β−1) Iw(A;Y~)I _w(A;\tilde X\mid\tilde Y)-(\beta-1)\,I _w(A;\tilde Y). The parameters θ\theta minimize VUIBβ+λ∑yp^y KL(w⋅∣y∥Unif)\mathrm{VUIB} _\beta+\lambda\sum _y\hat p _y\,\mathrm{KL}(w _{\cdot\mid y}\Vert\mathrm{Unif}) with β>1\beta>1, and the classifier minimizes ∑iwi CEi\sum _i w _i\,\mathrm{CE} _i. The prototype-based architecture and first-order alternating structure are retained. Appendix F gives Θ(N′T0+N′C+CC′)\Theta(N'T _0+N'C+CC') per epoch, including network computation, although equal asymptotic order does not establish equal wall-clock time. Theorem 3.1 in Section 3.2 and Section E.1 now establish the revised bound, and Section 3.3 defines the normalized-weight classifier loss and KL regularizer. The KL coefficient λ\lambda is tuned by the same cross-validation procedure as the other hyperparameters, now stated in Section 3.3 and Appendix C.

AuthorsOpenReview ↗

Response to Review by Reviewer pw8Y - Part 2

The compression gradient now depends on the features. It enters through the weights, ∂Iw(A;X~)/∂wi=KL(ui∥uˉ)−1\partial I_w(A;\tilde X)/\partial w_i=\mathrm{KL}(u_i\Vert\bar u)-1 at fixed uu, where the constant cancels under the fixed class masses p^y\hat p_y, and through the prototypes via uiau_{ia}. Both paths depend on z~i\tilde z_i. This derivation is now Eq. (19) in Section E.2 of the appendix. Appendix B specifies the gradient paths and notes that full empirical class normalizers and marginal entropy couple samples, so independent mini-batch normalization would approximate the stated objective.

Results with the revised VUIB. We trained IDR with the revised VUIB on the 3N3N synthetic set and ViT-B schedule of Table 16. It reaches 85.6% at β=2\beta=2, 0.3 percentage points above the submitted IDR at the same budget, and removing Iw(A;X~)I_w(A;\tilde X) lowers accuracy to 85.3%. This controlled ablation supports a contribution from the revised compression term. These results and the β\beta sweep have been added to Table 25 in Section D.23 of the appendix of the revised paper.

Method (3N3N, ViT-B)Top-1 (%)
GenDataAgent84.5
IDR, submitted VUIB85.3
IDR, revised VUIB, β=1\beta=185.4
IDR, revised VUIB, β=2\beta=285.6
IDR, revised VUIB, β=4\beta=485.5
IDR, revised VUIB without compression term, β=2\beta=285.3

Q2 (W2, first part). Prototype-index MI vs. the original-feature MI.

We do not claim that the prototype-level MI controls the MI of the continuous features, and IDR does not need it. Section 3.2 of the revised paper now explicitly defines the IB loss on the stochastic prototype assignment AA.

This avoids the degeneracy of feature-level MI for a deterministic encoder. With a deterministic encoder and a non-atomic feature distribution, continuous-feature I(Z~;X~)I(\tilde Z;\tilde X) is infinite. On N′N' distinct training inputs, an encoder injective on them instead gives I(Z~;X~)=H(Z~)=log⁡N′I(\tilde Z;\tilde X)=H(\tilde Z)=\log N' and I(Z~;Y~)=H(Y~)I(\tilde Z;\tilde Y)=H(\tilde Y) under the uniform empirical distribution. Stochastic or quantized representations therefore give a meaningful finite IB object [1, 2, 3]. Sections 3.2 and A.2 now state that our objective concerns the chosen soft prototype index AA, with I(A;X~)≤log⁡CI(A;\tilde X)\le\log C, rather than claiming consistency with the IB of unquantized features.

Relation to the features. Given the encoder and prototypes, AA is a stochastic function of Z~\tilde Z. Data processing gives Iw(A;Y~)≤Iw(Z~;Y~)I_w(A;\tilde Y)\le I_w(\tilde Z;\tilde Y) and Iw(A;X~)≤Iw(Z~;X~)I_w(A;\tilde X)\le I_w(\tilde Z;\tilde X). These inequalities do not bound their difference, so we claim no bound on the continuous-feature IB. Section 3.2 now states this scope explicitly, and Theorem 3.1 and Section E.1 prove the tight variational upper bound on the IB of AA.

Q3 (W2, second part). Nonzero input-prototype priors when C>C′C>C'.

The submitted class-frequency prior gives zero mass to indices beyond C′C' when C>C′C>C', and the revised VUIB removes this prior. Input prototypes are not tied to classes. A prior based on occupied input-prototype memberships is positive only for nonempty groups and equals class frequencies only when those memberships agree with labels, which C=C′C=C' alone does not establish. Section E.4 now states and corrects this limitation of the submitted construction.

The revised VUIB removes the input-side prior altogether. Sections 3.2 and E.1 now define compression directly for the sample index and the soft feature-prototype assignment. At finite distances, every uiau_{ia} and its positive weighted average are positive, so all terms are finite for any C≥C′C\ge C'. Label frequencies range over [C′][C'] and serve only to fix class masses. Section 3.3 and Appendix B separately state that centroid updates require nonempty membership groups.

AuthorsOpenReview ↗

Response to Review by Reviewer pw8Y - Part 3

Q4 (W1, second part). What prevents a shared-prototype solution?

The revised VUIB restores H(A)H(A), the term that rewards class separation. H(A∣Y~)H(A\mid\tilde Y) alone does not require class separation, and the submitted relevance relaxation dropped H(A)H(A). We have restored that term in Eq. (4) in Section 3.2 and separated the empirical stability evidence from the limited theoretical statement in Section E.3.

VUIB does not directly update the features. Section 3.3 and Appendix B now specify that the backbone and classifier receive only the weighted cross-entropy gradient. This supplies a supervised training signal, while VUIB influences the classifier indirectly through sample weights. This is not itself a proof against prototype collapse.

Prototypes follow their members. For fixed nonempty memberships, the weighted-centroid identity is ∑i∈agilog⁡uiauik=(∑i∈agi) ∥ca−ck∥2≥0.\textstyle\sum_{i\in a}g_i\log\frac{u_{ia}}{u_{ik}}=\big(\sum_{i\in a}g_i\big)\,\|c_a-c_k\|^2\ge0. Members prefer their own centroid on weighted average. This supplies no positive separation margin, and nearest-prototype regions may share boundaries and change after weight or feature updates. We therefore do not claim that the centroid property prevents merging or proves global noncollapse. Section E.3 now gives the identity as Eq. (24) and states these limits explicitly.

Empirically, the prototypes do not collapse in the tracked run. For submitted IDR with ViT-B at 5N5N, I(A;Y~)/H(Y~)I(A;\tilde Y)/H(\tilde Y) rises from 0.87 at epoch 10 to 0.95 at epoch 100. The marginal-assignment entropy remains close to its maximum, and the minimum prototype distance grows by 13% relative to epoch 5. These diagnostics have been added to Table 22 in Section D.21 of the appendix of the revised paper. They are empirical stability evidence, not a convergence theorem.

EpochI(A;Y~)/H(Y~)I(A;\tilde Y)/H(\tilde Y)H(uˉ)/log⁡CH(\bar u)/\log CMin. distance between prototypes (relative to epoch 5)
100.870.9981.03
500.930.9981.10
1000.950.9991.13

The revised VUIB favors a feasible separated assignment over an uninformative one. At Q=PwQ=P_w, Eq. (25) in Section E.3 gives VUIBβ=Iw(A;X~∣Y~)−(β−1)Iw(A;Y~)\mathrm{VUIB}_\beta=I_w(A;\tilde X\mid\tilde Y)-(\beta-1)I_w(A;\tilde Y). Equal class profiles give Iw(A;Y~)=0I_w(A;\tilde Y)=0 and an objective of at least zero. If confident one-prototype-per-class assignments are feasible, they approach −(β−1)H(Y~)<0-(\beta-1)H(\tilde Y)<0 for β>1\beta>1 and can use uniform within-class weights with zero KL penalty. Sections 3.3 and E.3 now state this conditional comparison without claiming that alternating optimization reaches the separated solution.

Q5 (W2, third part). What fixes the relative total weight of each prototype group?

The revised formulation fixes the group scales that the submitted prototype formula leaves free. The submitted prototype formula is invariant to rescaling within a group, and the weighted-centroid formula is now Eq. (6) in Section 3.3. Since the submitted QQ and sample average were unweighted, that objective did not fix group scales through the centroids. The shared sigmoid network couples the scales but does not establish their identifiability. The within-class weight visualizations are in Figures 7 and 8 in Section D.15 of the paper.

Empirically, average class weights vary little in the tracked run. The table below reports submitted-IDR training with ViT-B at 5N5N, with a coefficient of variation across class averages of at most 0.06. This is evidence of limited class-level variation, not proof that the submitted objective uniquely fixes every prototype group's scale. These measurements have been added to Table 23 in Section D.21 of the appendix of the revised paper.

EpochMean class weightMin. class weightMax. class weightCoefficient of variation
100.580.520.630.03
500.610.490.700.05
1000.620.460.740.06

The revised formulation fixes class masses and global scale. Sections 3.2–3.3 now use the same normalized weights wiw_i in VUIB and the classifier loss ∑iwiCEi\sum_iw_i\mathrm{CE}_i. Class masses remain p^y\hat p_y, and a common rescaling cancels. Within a class, relative weights enter uˉ\bar u, Pw(⋅∣y)P_w(\cdot\mid y) and ∑iwiH(ui)\sum_iw_iH(u_i), and the KL regularizer discourages concentrated allocations. Section E.2 gives the precise conditional allocation analysis.

Reviewer_w6ziOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Informative Data Reweighting (IDR), which uses the Information Bottleneck (IB) principle to learn importance weights for both real and synthetic samples in the GDA augmented training set. It derives a VUIB upper bound that can be optimized directly with SGD. Its The overall objective is to assign higher weights to samples that are more correlated with class labels and less correlated with input noise, thereby suppressing visual artifacts and semantic inconsistencies in GDA synthetic data. The experiments are sufficient and the results are competitive, but there are several issues that need clarification and revision.

Critical strengths
  • The paper addresses an important question: how to benefit from generated images when their quality varies.
  • IDR is practical and does not rely on clean metadata or external expert models such as CLIP.
  • VUIB is efficient relative to CLUB.
  • The experimental coverage is broad. The authors evaluate multiple architectures and tasks, provide an anonymized code repository, and report ten-seed statistics for the four main ImageNet backbones, which is useful
  • The appendix is valuable; Among others, it includes a budget-matched comparison.
Critical weaknesses
  • A potential limitation of IDR is that its IB-based reweighting may downweight not only genuinely corrupted or semantically inconsistent samples, but also valid examples containing artifacts that are likely to occur at test time. Because the method favors representations that are less dependent on input-specific variation, it may treat label-preserving conditions such as blur, compression, occlusion, or sensor noise as nuisance factors rather than useful sources of robustness. This concern remains even if reweighting is restricted to generated data: systematically suppressing synthetic artifacts that genuinely occur in the target distribution could reduce the model’s exposure to, and robustness under, those conditions.
  • The ablations do not isolate the benefit of sample reweighting: the variant without reweighting seems to already outperform the reported selection/reweighting baselines. This makes it important to compare with a simple feature-clustering or class-centroid regularizer without learned sample weights. The similar results across DiT-B/L/XL (85.6/85.7/85.7) and the other generators are interesting, but do not by themselves establish what the weighting network learns.
  • The prototypes in VUIB are jointly determined by the current classifier features and the reweighting-network weights, while the reweighting network is updated via VUIB; there is a lack of analysis of convergence, degenerate solutions, or weight-scale invariance. The method should be described more precisely.
  • The balance between the main text and the appendix needs improvement: many key analyses are placed in the appendix, and the main text provides insufficient explanation of the intuition behind VUIB, the weight behavior, the matched budget, and the computational cost.
  • The claimed information-compression term appears to have no dependence on the learned representation or on the weighting network through the feature prototypes. This is especially concerning because the Appendix seems to criticize previous IB formulations for essentially the same loss of feature dependence.

Some experimental details are difficult to assess: For instance, the paper reports five-fold cross-validation across ten synthetic-data budgets, extensive backbone comparisons, and ten-seed tests. But it is difficult to infer the total cost of the reported search without knowing which runs reused samples or checkpoints and whether all folds used the full training schedule. Please specify the actual protocol and compute.

Finally, the input representation fed into the two-layer weighting MLP should be stated explicitly.

Questions for authors
  • In Table 1, IDR uses 4N/5N synthetic samples while most baselines use 2N/3N. Can the main table directly report comparisons of all methods under the same synthetic budget (e.g., 3N or 5N) and similar total training compute?
  • In the complexity/training-time comparison with CLUB, is CLUB implemented with full pairwise computation or with mini-batches?
  • What is the precise relationship between the input-prototype prior and the class-label frequency used in the bound?
  • Can the authors report a no-reweighting regularizer or class-centroid baseline under the same budget?
  • How were the cross-validation folds, synthetic budgets, multi-label datasets, and total GPU expenditure handled?
6 comments
AuthorsOpenReview ↗

Response to Review by Reviewer w6zi - Part 1

Thank you for the detailed and constructive review. We appreciate that you found the question important and IDR practical, since it needs no clean metadata or external expert models such as CLIP and is more efficient than CLUB. We also appreciate your recognition of the broad evaluation with ten-seed statistics and the budget-matched comparison. We answer each point below.

W1. Could reweighting suppress realistic, label-preserving artifacts?

IDR does not penalize label-preserving variation as such. In the submitted IDR, VUIB's relevance term trains the reweighting network through the weighted prototypes, now defined in Eq. (6) in Section 3.3 of the revised paper. A low weight reflects limited usefulness under this learned objective. Blur, compression or noise alone does not determine the weight. Section 3.3 and Appendix B now clarify that VUIB updates only the reweighting network, while weighted cross-entropy trains the backbone and classifier. The revised objective additionally includes the feature-dependent compression term discussed under W5.

The paper's results support this. Real images contain JPEG compression, blur, occlusion and sensor noise, yet get concentrated moderate-to-high weights in Figure 5 in Section D.11 of the paper. Figure 2 in Section 1 and Figure 4 in Section D.9 of the paper show low-weight synthetic images with semantic inconsistency or severe artifacts and Grad-CAM off the object. Some class-boundary samples keep high weights in Figures 7 and 8 in Section D.15 of the paper. IDR also beats every baseline on the out-of-distribution ImageNet-A/R in Table 12 in Section D.10 of the paper, e.g., with Swin-T, 26.4 vs. 23.8 on ImageNet-A and 45.9 vs. 43.2 on ImageNet-R. All samples, including down-weighted ones, also get the standard crop, flip and color-jitter augmentation.

IDR improves robustness to common corruptions. We evaluated the ImageNet-1K models of Table 12 on ImageNet-C [1], which applies 15 label-preserving corruptions, such as blur, JPEG compression, noise and weather, at five severities. IDR has the lowest mean corruption error (mCE) with both backbones and, with Swin-T, cuts the mCE of the best baseline by 2.6 percentage points. These results have been added to Table 18 in Section D.18 of the appendix of the revised paper.

MethodSwin-T mCE (%) ↓Swin-B mCE (%) ↓
Base model62.054.4
GDA w/o Reweighting61.554.0
CBF60.853.5
GenDataAgent61.053.6
IDR (Ours)58.251.6

Label-preserving corruptions keep their weights. We applied Gaussian blur, JPEG compression, Gaussian noise and occlusion at severities 1, 3 and 5 to 10,000 real ImageNet-1K training images and scored them with the trained reweighting network. Relative to the uncorrupted originals, they keep nearly all their weight at severity 1 and at least 84% at the most severe level. Figure 5 in Section D.11 of the paper shows that 11.6% of synthetic samples receive an absolute weight below 0.2. The relative retention and the absolute-weight histogram measure different quantities, and together they indicate that IDR does not systematically suppress the tested label-preserving artifacts. The corruption-weight results have been added to Table 19 in Section D.18 of the appendix of the revised paper.

CorruptionSeverity 1Severity 3Severity 5
Gaussian blur0.990.960.88
JPEG compression1.000.980.93
Gaussian noise0.990.950.84
Occlusion0.990.970.91

The revised VUIB adds a soft safeguard. Its within-class KL-to-uniform term, with weight λ\lambda, discourages concentrated weights. Section E.2 of the appendix derives a subgroup-mass inequality in Eq. (23): a known small within-class KL limits deviation from the subgroup's uniform mass. The coefficient λ\lambda alone does not give a universal positive lower bound on every subgroup's mass. This scope is stated explicitly in Section 3.3 of the revised paper.

AuthorsOpenReview ↗

Response to Review by Reviewer w6zi - Part 2

W2 & Q4. Isolating the benefit of reweighting, and a class-centroid regularizer without learned weights.

The learned weights are already isolated by two existing comparisons. With uniform weights, the submitted IDR's classifier training is exactly "GDA w/o Reweighting", with the same data and loss. At the same 2N2N budget, learned weights raise ViT-B top-1 from 83.9 to 84.8 in Tables 1 and 16. In the component ablation of Table 9 in Section D.6 of the paper, the "IDR w/o Reweighting" row applies the submitted VUIB to classifier features without learned sample weights. Since the submitted compression term does not depend on those features, this is a label-conditional prototype regularizer, whose prototypes need not coincide with class centroids. At 5N5N, adding it to cross-entropy at unit weight reaches 85.0, versus 85.6 when the submitted objective learns sample weights.

Class-centroid baselines at the same budget. We have added two class-centroid baselines without learned sample weights, trained on the same 3N3N synthetic set with the same ViT-B schedule. One adds center loss [2] to cross-entropy, and the other uses fixed weights from the softmax similarity between each sample's feature and its class centroid. Both stay below GenDataAgent [3]. The submitted IDR exceeds the stronger centroid baseline by 1.0 percentage point, showing that these centroid alternatives do not recover the benefit of learned weights in this setting. The revised VUIB raises IDR further to 85.6%. These results have been added to Table 20 in Section D.19 of the appendix of the revised paper.

Method (3N3N, ViT-B)Top-1 (%)
GDA w/o Reweighting83.8
CE + center loss84.2
CE + fixed centroid-similarity weights84.3
GenDataAgent84.5
IDR (Ours)85.3
IDR, revised VUIB (Ours)85.6

The learned weights themselves show what the weighting network learns. Tables 8 and 13 show robustness to the generator, and the weights reveal the mechanism. Figure 2 in Section 1 and Figure 4 in Section D.9 of the paper show low weights of 0.04–0.10 for semantically inconsistent or corrupted synthetic images and high weights of 0.91–0.96 for faithful ones. In Figure 5 in Section D.11, 11.6% of synthetic samples fall below 0.2, while real-image weights concentrate at moderate-to-high values. Figure 6 in Section D.11 shows how this differentiation emerges during training.

Controlled noise injection. To test the mechanism directly, we replaced 10% of a 1N1N synthetic set with unconditional DiT-B [4] samples given random class labels to create semantic inconsistency, and another 10% with samples generated with only 4 instead of 128 sampling steps to create severe generation artifacts. We measured by AUROC how well each signal flags the injected images, using high cross-entropy, low CLIP confidence and low IDR weight as corruption signals. Without any external model, the IDR weights reach an AUROC of at least 0.87 on both types. CLIP [5], the external expert used by CBF [6], performs better on semantic inconsistency, while IDR performs better on generation artifacts. CLIP is used only for evaluation. These results have been added to Table 21 in Section D.20 of the appendix of the revised paper.

SignalExternal modelAUROC, semantic inconsistencyAUROC, generation artifacts
Per-sample cross-entropy lossNo0.840.78
CLIP zero-shot confidenceYes (CLIP)0.940.86
IDR weights (Ours)No0.870.93
AuthorsOpenReview ↗

Response to Review by Reviewer w6zi - Part 3

W3. Coupled prototypes and weights: procedure, degenerate solutions, scale, convergence.

Procedure. Section 3.3 and Algorithm 3 in Appendix B of the revised paper now specify the five-epoch cross-entropy warm-up, nearest-prototype memberships and weighted-centroid update in Eq. (6). Memberships and detached backbone features are held fixed during the θ\theta update, which retains derivatives through the weights and centroids and uses the previous epoch's QQ. The classifier is then updated by weighted cross-entropy with sample weights held fixed. Algorithm 4 in Section E.5 recomputes the weighted QQ for the next epoch. These are first-order alternating updates without hypergradients. Appendix B also clarifies that class normalizers, centroids and marginal entropy couple samples, so independently normalized mini-batch losses are approximations to the full empirical objective.

Degenerate solutions. Weighted cross-entropy directly trains the backbone, while VUIB affects it through sample weights. For fixed nonempty memberships, Eq. (24) in Section E.3 of the appendix gives ∑i∈agilog⁡(uia/uik)=(∑i∈agi)∥ca−ck∥2≥0\sum_{i\in a}g_i\log(u_{ia}/u_{ik})=(\sum_{i\in a}g_i)\|c_a-c_k\|^2\ge0. Members therefore prefer their own centroid on weighted average. This identity supplies no positive separation margin, and nearest-prototype memberships can change, so it does not prove that reweighting cannot merge prototypes. The revised VUIB retains the marginal-entropy term: at Q=PwQ=P_w and β>1\beta>1, a feasible confidently separated assignment has a lower objective than an uninformative shared assignment. Sections 3.3 and E.3 now distinguish this comparison from a global noncollapse guarantee.

Weight scale. Rescaling a group's weights leaves its prototype unchanged. In the submitted method, the shared network couples these scales but does not prove that the objective fixes them. The revised formulation explicitly normalizes weights within each class, keeping each class's mass at p^y\hat p_y and cancelling the global scale. Its KL term discourages concentration. These changes are defined in Eqs. (2) and (7) in Sections 3.2–3.3, and Section E.2 of the appendix proves uniqueness only for the allocation problem with assignments and QQ fixed.

Convergence. The submitted VUIB is non-negative, but the two networks minimize different objectives, so we claim no convergence theorem. Instead, we tracked submitted-IDR training with ViT-B at 5N5N. Weight rank correlation with the preceding epoch rises from 0.81 at epoch 10 to 0.99 at epoch 100. The minimum prototype distance grows by 13% relative to epoch 5, and the assignments retain 95% of the label information with H(uˉ)/log⁡CH(\bar u)/\log C close to 1. These measurements support empirical stability without observed collapse in this run. They have been added to Table 22 in Section D.21 of the appendix of the revised paper, and Sections 3.3 and E.3 state the scope of the theoretical claim.

EpochWeight rank correlation with previous epochMin. distance between prototypes (relative to epoch 5)I(A;Y~)/H(Y~)I(A;\tilde Y)/H(\tilde Y)H(uˉ)/log⁡CH(\bar u)/\log C
100.811.030.870.998
500.951.100.930.998
1000.991.130.950.999

W4. Balance between the main text and the appendix.

We have strengthened the main text and connected it to every analysis in the appendix. Sections 3.2–3.3 of the revised paper now give the intuition and scope of VUIB. Section 4.1 now summarizes the weight behavior, where 11.6% of synthetic samples receive weights below 0.2 while real-image weights concentrate at moderate-to-high values, and points to Sections D.11 and D.15. Section 4.2 now summarizes the comparison of all methods at a common 3N3N budget, where IDR outperforms GenDataAgent by 0.8% with only 2.0% higher total cost, and points to the full budget sweep, the cost breakdown and the all-method comparison in Sections D.16, D.17 and D.24. The list at the beginning of Section 4 now also covers all the added experiments in Sections D.18–D.25 of the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.