acceptodds
Under review as a conference paper at ICLR 2027

How Many, Not Which: Automatic Data Budgets for Conditional GANs Using Gradient Spectra

Abstract

Data subset selection promises cheaper training. While the coreset literature focuses on deciding which samples to keep at a practitioner specified size, our evidence suggests that, for conditional GANs, the predominant factor is how many samples. At equal budgets, uniform random selections during training runs tie or outperform gradient matching, coverage matching, and difficulty scoring. Meanwhile, changing data quantity and allocation for each condition moves quality well outside between-seed variation. To address this gap in determining a subset’s quantity, we introduce FLARE (Fast Learning via Adaptive Rank Estimation), which derives the data budget from the effective rank of each condition's centered per-sample critic gradient spectrum, i.e., the number of independent directions the gradient signal spans. That budget is then split across conditions proportionate to those ranks, tilted toward the conditions the generator currently worst fits as ordered by per-condition sliced Wasserstein distance. Because the budget is an output of that spectrum rather than a required input, there is no size parameter to tune. We apply FLARE to optiGAN—a conditional Wasserstein GAN with Gradient Penalty (WGAN-GP) that stands in for state-of-the-art Monte Carlo simulation of optical photon transport in radiation detectors—to determine a coreset size under 2% of a 513k-sample pool. With random subsets at this coreset size, optiGAN trains 13x faster (including incurred selection costs, excluding evaluation) for the same number of optimizer steps, matching full-data training on global sliced-Wasserstein similarity at a minimal quantified cost in per-condition fidelity. Because the criterion is solely based on the critic's own gradients, it transfers across domains without re-tuning. As such, we apply FLARE unchanged to calorimeter shower simulation, and to tabular synthesis on a network-intrusion benchmark. Because FLARE’s selected fraction differs substantially across these domains, no single ratio fits them all; deriving the budget instead of tuning it removes the sweep that data reduction normally demands.

Primary area:
generative models
Keywords:
data subset selection, coreset selection, data-efficient training, conditional generative adversarial networks, effective rank, gradient spectrum, hyperparameter-free methods, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uhSMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Fast Learning via Adaptive Rank Estimation (FLARE), which is an adaptive data-budget estimation method for conditional GANs. FLARE estimates the required sample count for each condition using the effective rank of centered per-sample critic gradients rather than selecting which samples to retain under a predefined budget. In addition, FLARE reallocates the total budget toward poorly modeled conditions based on global sliced-Wasserstein similarity. On optiGAN, FLARE uses about 1% of the optical photon simulation data and achieves a 19x training speedup while maintaining comparable generation quality. On CaloShowerGAN, χ²/NDF increasing from 1.56 to 5.39. On CTGAN, macro F1 improves from 0.518 to 0.570.

Critical strengths
  1. The paper addresses the practically relevant question of automatically determining data budgets for conditional generative models. It is interesting about the observation that uniform random sampling can match or outperform several sophisticated membership-selection methods.

  2. The paper provides multiple random-seed experiments, statistical comparisons, and detailed computational cost analyses.

Critical weaknesses
  1. Most components of FLARE are based on established methods.
    (1) automatic coreset sizing has been investigated by Xia et al. [1]
    (2) the spectral energy criterion (Eq. 2) follows standard PCA explained-variance selection, while the entropy-based effective rank (Eq. 3) is directly from Roy and Vetterli [2];
    (3) performance-aware, class-wise data allocation and random sampling have been explored by DRoP [3] and RCAP [4].
    Therefore, FLARE appears primarily to combine existing techniques rather than introduce fundamentally new mechanisms.

  2. The main novel component is mapping the effective rank of critic gradients directly to the required number of training samples. However, the paper provides no convincing mathematical justification for why the number of dominant gradient directions should determine the sample complexity of learning a conditional distribution.

  3. On optiGAN, FLARE uses smaller batch sizes than full-data training, but a full-data baseline with matched small batches is missing. Fixed random sampling across different budget fractions is also insufficiently explored. Furthermore, CaloShowerGAN shows substantial quality degradation (chi-squared/NDF: 1.56 to 5.39), while CTGAN's macro-F1 improvement may primarily result from class rebalancing, with worse sliced-Wasserstein distance.

References

[1] Xia et al., Refined Coreset Selection, ICML 2024.

[2] Roy and Vetterli, The Effective Rank: A Measure of Effective Dimensionality, EUSIPCO 2007.

[3] Vysogorets et al., DRoP: Distributionally Robust Data Pruning, ICLR 2025.

[4] Hassan et al., RCAP: Robust, Class-Aware, Probabilistic Dynamic Dataset Pruning, UAI 2025.

Questions for authors
  1. Could the authors clarify how FLARE differs fundamentally from existing coreset sizing methods [1] and performance-aware class-wise allocation methods [4, 5], beyond combining these techniques with spectral rank estimation?

  2. What theoretical or empirical evidence supports using the effective rank of critic gradients as the required number of training samples?

  3. Could the authors compare FLARE against full-data training with matched small batch sizes and fixed random sampling at different budget fractions?

  4. Could the authors clarify whether the CTGAN improvements come from spectral sizing rather than simple class rebalancing, and explain the substantial quality degradation on CaloShowerGAN?

3 comments

Thank you. We agree with most of the component list and have narrowed our claims to match.

The energy rule is PCA's explained variance and the entropy rank is Roy and Vetterli's, and we credit both as standard. RCS (Xia et al.) makes the size an output, so that idea is theirs. It refines a size the user provides, by a search that retrains for each candidate. FLARE needs no starting size and no search. It reads the total training budget from the critic's gradient spectrum during one training run. That is our main claim. The closed-form per-sample gradient with the penalty term, which makes the sizing cheap, is our second. Classifier pruning already uses last-layer per-sample gradients, but its losses have no gradient penalty, so the penalty term's closed form is the new part. The paper times it only against a per-sample loop, and a comparison with automatic per-sample differentiation is in progress.

DRoP and RCAP take the pruning rate as given and split it across classes. Our reallocation toward poorly fit conditions (the tilt) was a version of the same idea, so the split is not our contribution. We ran their allocation rules with each condition's sliced-Wasserstein error in place of class accuracy, and they do as well or better. Given FLARE's total on optiGAN, the DRoP-style split is equivalent to FLARE on the worst quartile, and the RCAP-style comparison is inconclusive. Both cost less recall than our tilt. On CTGAN the RCAP-style split of random rows sits 0.04 to 0.06 macro F1 above FLARE's own split. The revision therefore pairs FLARE's total with these splits (general response). FLARE supplies the rate they take as given. Two experiments in progress test that directly. One runs these splits at totals below and above FLARE's on optiGAN. We expect them to need a total as our own rule did, but that is untested. The other runs FLARE's sizer and the splits together, since the runs above used a fixed total equal to FLARE's average budget. If there is a comparison with these methods that you would like to see in the revision, we are open to running it.

Not citing RCAP was our oversight. This project began as an attempt to adapt GradMatch, a subset-selection method built for classifiers, to optiGAN. It did no better than random subsets, and the other classifier-derived selection rules we tried skewed each condition's distribution. We then assumed that class-wise pruning rules such as DRoP and RCAP would not carry over to generators either, and we never tested them. The runs above show that assumption was wrong.

We have no bound linking the rank to a sample count, and we drop "the required number". The GAN sample-complexity bounds we know of hold over a whole critic class and do not use a measured rank. So the revision presents the rank as a size rule with evidence. A uniform subset of FLARE's size keeps 0.996 of the spectral energy of the pool's per-sample gradients, against 0.999 at best. FLARE's budget is also the smallest multiple we tested that is equivalent to full data on the worst quartile. An experiment in progress changes the pool size. If the size that works is a row count read from the model, it should stay put while a fixed ratio moves with the pool.

We ran both baselines you asked for. Full-pool training at FLARE's batch and step count is as fast as FLARE and about 0.016 lower on the worst quartile. Random subsets from 0.25% to 4% never reach full data on that endpoint. FLARE beats random at 1, 1.47, and 2%, and its budget never goes above 2%. A guessed 1% that keeps FLARE's tilt ties FLARE, and we report that. Run at six sizes, that guess falls short of full data at 0.5% and below and reaches it from 1% up. It still needs a total, and FLARE's lands in that range.

On CTGAN the gain is class rebalancing, as you suspected. Random rows at FLARE's total in the pool's proportions show no detectable difference from full data, and halving or doubling that total made none either. Runs at smaller totals are in progress. Random rows at the same total under a DRoP- or RCAP-style split keep the sliced-Wasserstein distance at full data's level, so the worse distance comes from our split and not from the small budget. On CaloShowerGAN, FLARE's subset at full data's batch did not close the gap, so the batch alone is not the cause. It stays a stated limitation.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address the concerns regarding novelty, mathematical justification, and experimental fairness.

First, the clarification of FLARE's relationship to prior work is helpful. The authors acknowledge that the spectral estimators and performance-aware allocation strategies are established techniques, and appropriately narrow their novelty claims. The new comparisons with matched small-batch training and fixed-budget random sampling also address important experimental concerns.

However, my main concerns remain unresolved. The new experiments show that a fixed 1% budget with the same allocation strategy can match FLARE on optiGAN, while DRoP- and RCAP-style allocation achieves comparable or better results. This suggests that the benefits of the proposed spectral sizing mechanism remain insufficiently isolated from existing allocation strategies. Moreover, preserving 99.6% of gradient spectral energy does not establish why effective rank should determine the training sample budget. The reported sensitivity to critic width further raises concerns about the robustness of this heuristic.

Finally, the cross-domain evidence remains limited. CaloShowerGAN still exhibits substantial quality degradation, while the authors acknowledge that CTGAN's improvements mainly result from class rebalancing rather than spectral sizing.

Overall, the central methodological contribution and its advantages over simpler alternatives are still not sufficiently established. Therefore, I maintain my original rating of 2 (Weak Rejection).

Thank you for reading the new results. You noted that a fixed 1% budget with the same allocation matches FLARE on optiGAN. It does, but only within one pool. The same 1% fell short of full data on a quarter of the pool and beat it on five times the pool. FLARE's row count stayed within about 5% of its value on the original pool (general follow-up).

The DRoP- and RCAP-style splits did about as well as our tilt. We no longer claim the split, and the revision credits it to DRoP and RCAP. With FLARE setting the total in the same run, both matched what they did at a fixed total.

We agree that keeping 99.6% of the gradient energy does not explain why the effective rank should set the budget. The revision presents the rank as a size rule backed by experiments, not a derived count. Its sensitivity to critic width, the CaloShowerGAN gap, and CTGAN's rebalancing stay as stated limitations.

Our first reply kept the closed-form per-sample gradient as a second claim while a comparison with automatic per-sample differentiation was in progress. That comparison went against it. We withdraw the claim (general follow-up). We also called the paper's baseline a per-sample loop, but it looped over micro-batches of 8 rows.

Reviewer_n1pbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes FLARE, which reduces the training sample size for conditional GANs based on the effective rank of per-sample discriminator gradients. FLARE randomly draws training subsets from the entire dataset, with their sizes determined by the estimated rank. The budget and subsets are periodically updated during training. On optiGAN, FLARE reduces training time while achieving performance comparable to full-data training on the primary evaluation metric.

Critical strengths
  • Idea of effective rank of discriminator signal is interesting.
Critical weaknesses
  • The experiments cover a limited range of datasets. Although FLARE is evaluated on three domains, these focus on physical simulation and tabular synthesis without image generation benchmarks (Section 2 and Appendix B). The generality to larger models and more complex generation tasks is not sufficiently clear for me. I believe image generation experiments would help support its broader applicability.

  • The justification of using effective rank as the sample budget is not sufficiently clear for me. As I understand, FLARE periodically updates the rank, so changes during training are considered. However, a low-dimensional gradient subspace does not necessarily mean that the selected number of random samples preserves its important directions. I believe this relation needs further analysis beyond comparing resizing intervals.

  • The most critical weakness is that the benefit beyond reducing batch size is unclear. In the optiGAN experiments, the batch size is reduced while the optimization steps remain unchanged (Section 3). I suspect that much of the speedup could also be obtained by directly sampling smaller batches from the entire dataset. Without a comparison using the same batch sizes and condition allocation, the benefit of constructing and reusing a subset is not sufficiently established. The authors also acknowledge this limitation (Section 4.1).

  • The memory cost of budget recomputation is unclear. FLARE repeatedly draws subsets from the entire dataset, and recomputation requires gathering output-layer gradients over the full pool. Thus, reducing the active subset does not imply that the remaining data can be discarded. I believe peak memory usage, including gradient gathering and SVD, should be reported to clarify its practical scalability.

Questions for authors
  • Can you evaluate FLARE on image generation benchmarks and larger models?

  • Can you analyze whether the sampled subsets preserve the important gradient directions of the full pool, and how this changes during training and across resizing intervals?

  • Can you compare FLARE with direct random sampling from the entire dataset using the same batch sizes, condition allocation, and optimization steps?

  • Can you report peak CPU/GPU memory usage during training and budget recomputation, and clarify whether FLARE provides storage savings or only reduces the active subset?

2 comments

Thank you for the review.

We ran the comparison you asked for. Sampling from the full pool at FLARE's batch, step count, and per-epoch budget trains as fast as FLARE. It sits about 0.016 below FLARE and full data on the worst quartile of conditions, outside the equivalence margin, and it is equivalent to FLARE on typical conditions. Sampling with FLARE's per-condition allocation, redrawn every epoch, is equivalent to FLARE on every condition-level and global endpoint. So reuse is not the source of the gain, as you suspected. At the same batch the benefit comes from the allocation across conditions. The speed comes from the batch.

We also measured, on saved checkpoints, whether that many random rows keep the main gradient directions. A uniform subset of FLARE's per-condition size keeps 0.996 of the spectral energy of the pool's per-sample gradients, against 0.999 for the best subspace of that size, and its mean gradient lands about 3% from the pool's. A budget computed at one point in training still works 500 epochs later, as well as a freshly computed one. An experiment in progress changes the pool size, to test whether the number of rows that works stays fixed as the pool grows or shrinks. Coverage drops to 0.905, against 0.965 at best, on the easy conditions that FLARE's tilt toward poorly fit conditions leaves with 20 rows or fewer, about a sixth of the cases. Without the tilt, nine in ten cases keep at least 0.995.

On optiGAN, the recomputation step (the gradient gather plus the SVD) peaks at about 0.4 GB of GPU memory. That gather uses our closed-form per-sample gradient with the penalty term, which we keep as our second contribution beside the total (general response). FLARE's overall peak is about 1.1 GB, against 4.0 GB for full-data training. The two peaks are the same on CaloShowerGAN. On CTGAN, FLARE's recomputation step peaked a little above full data (2.1 against 1.8 GB) in a test run made before the penalty term was added. Host memory does not shrink. The whole pool stays loaded for recomputation, and the revision says FLARE reduces the working set, not storage.

We cannot add image benchmarks within the discussion period. The sizer needs per-sample critic gradients, which any GAN gives, so we scope the claims to conditional GANs and list image models, larger models, and diffusion as untested.

For readability, the revision adds an overview figure, an algorithm box, and a numbered contribution list, and it splits the main table by question.

Our first reply said we were testing whether the number of rows that works stays fixed as the pool grows or shrinks. It does. On a quarter of the pool and on five times the pool, FLARE's row count stayed within about 5% of its value on the original pool (general follow-up).

On memory, PyTorch's torch.func used about the same GPU memory for the per-sample gradients as our closed form. We no longer count the closed form as a contribution.

For readability, the revision adds the overview figure, algorithm box, numbered contributions, and split main table we described.

Reviewer_jcszOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies training conditional GANs on a subset of the data, motivated by GAN surrogates for physics simulators that must be retrained whenever the detector setup changes. The authors argue that what matters is how many samples each condition gets, not which ones, since uniform random subsets tie or beat gradient matching, coverage and difficulty-based selection in their experiments. FLARE sets each condition's budget from the effective rank of the per-sample gradients of the critic's output layer, tilts the budget toward the conditions the generator fits worst, and draws rows uniformly at random, redrawing them periodically. On optiGAN, FLARE uses under 2% of the pool, is equivalent to full data on the worst-quartile metric, and reports a large speedup. CaloShowerGAN and CTGAN serve as additional domains.

Critical strengths
  • The question is practical for the surrogate setting, where a model has to be retrained many times.
  • The evaluation is unusually careful: repeated runs, bootstrap intervals, equivalence tests, and contrasts registered before the cross-domain sweeps.
  • The evidence that membership rules don't beat uniform random sampling is solid, and the density diagnostics explain why.
  • Negative results and limitations are reported openly.
Critical weaknesses
  1. The main experiments don't isolate what the rank-based sizing adds. Appendix A shows that fixed random subsets of 1–5% already match full data on optiGAN, but the main table has no fixed-fraction random baseline, with or without the tilt. In Table 1, the untilted configurations are never equivalent to full data while the tilted ones are, so the benefit seems to come mostly from the allocation across conditions rather than from the computed total. This sits awkwardly with the "how many, not which" message.
  2. The speedup mostly comes from the smaller batch, as the authors acknowledge. A reduced-data run with the full-data batch matched or beat full data on quality but was slightly slower. A full-pool baseline with the same small batch and step count is needed. The two extra domains show essentially no GPU speedup.
  3. "Under 2% of the pool" may overstate the reduction. Membership is redrawn at every selection event, so a run probably touches a much larger share of the pool (roughly a third for optiGAN by my estimate). Please report the number of unique rows used, and whether results hold with a single fixed draw.
  4. The sizing rule is a heuristic with real knobs. The budget depends on θ (two values give the same quality at twice the data), the estimator, the subbatch size, the cap, and the critic's output width, which also bounds the rank. So "no size parameter to tune" is too strong. There is also no argument for why the effective rank should equal the number of samples needed. Without the gradient penalty it is just the number of principal components of the critic's penultimate features, and spanning a subspace is not the same as estimating a distribution. The budget ablation supports this concern: quality keeps improving at two and four times the rank budget, and at four times it is significantly better than full data.
  5. The evidence beyond optiGAN is weak. On CaloShowerGAN every reduced-data configuration is several times worse than full data on χ². On CTGAN the gains sit in a few rare classes, which the authors can't separate from class rebalancing. All three models are small GANs, the main dataset isn't public, and the method needs a per-sample Wasserstein critic, so it's unclear how it would extend to diffusion or flow models.
  6. Novelty is overstated and related work is missing. Periodic random resampling (RS2), coreset-accelerated GAN training (Small-GAN), and the finding that random sampling is hard to beat at high pruning rates (DeepCore, coverage-centric coreset selection) are not cited. The building blocks are existing tools, so the new part is reading the budget off the gradient spectrum, which is the least supported part (point 1).
  7. Minor: the worst-quartile endpoint naturally favors the tilt. In the budget ablation, FLARE falls short of full data on most other metrics. Please report all metrics for the headline configuration.
Questions for authors
  1. Under the same 5000-epoch protocol, how do fixed ~1% random subsets do, with proportional allocation and with the tilt? How does full-pool training with the same small batch do?
  2. How many unique rows does a default run touch, and does a single fixed draw remain equivalent to full data?
  3. Why should the effective rank equal the needed sample count? Does quality versus budget show a knee near the FLARE budget?
  4. How were θ, the CTGAN subbatch size and the 25% cap chosen, and how sensitive is the budget to the critic's output width?
2 comments

Thank you for a careful review.

You were right that the main table did not show what the computed size adds. With fresh draws per repeat, FLARE was 0.019 above random on the worst quartile at 1% and 0.017 above it at 2%, both outside the equivalence margin. It ties random on typical conditions and trails it on recall by about 0.02. You were also right about the tilt. Without it FLARE sits below full data on the worst quartile, and a fixed 1% that keeps the tilt ties FLARE. So we claim the total, not the tilt. A guessed ratio with the tilt still needs a total (general response). An experiment in progress repeats that comparison on smaller and larger pools, to test whether a ratio that works here carries to another pool size.

Full-pool training at FLARE's batch (general response) has 0.02 higher recall than FLARE. The other two domains get no GPU speedup, and we list that as a limitation. Our claim there is the budget, and a speedup on CPU-only runs.

A default run touches about half the pool, and the revision reports this beside the budget. Our single-draw test used a random 1% subset, not FLARE's own subset frozen. Against a 1% subset redrawn every 100 epochs, it sits about 0.01 lower on the worst quartile, global sliced-Wasserstein (SW) similarity, and typical conditions, and 0.03 lower on recall.

We agree that "no size parameter to tune" goes. We fixed theta = 0.999 once on optiGAN pilot checkpoints, before any sizing sweep. We set the CTGAN subbatch size s = 8 before any FLARE run there, from an analytic count of the largest rank each class can reach. At s = 1 the budget falls about 7x. The submitted version then lost its gain over full data, while with the penalty term in the gradients it still beats full data at 0.2% of the pool. On optiGAN the 25% cap is never reached, since the critic's width bounds the rank and so limits the total to about 2.7%. The budget follows critic width, at 0.83, 1.47, and 2.36% of the pool for widths 64, 128, and 256. Against full data trained at the same width, FLARE asks for too little at 64 and sits below it on the worst quartile and on typical conditions. At 256 the worst-quartile comparison is inconclusive, and FLARE trails on precision and recall.

We have no bound linking the rank to a sample count, so we drop "the samples needed" and present the rank as a size rule with evidence. Rows are drawn uniformly within each condition and redrawn over a run, so no single subset has to estimate the distribution. A uniform subset of FLARE's size keeps 0.996 of the spectral energy of the pool's per-sample gradients, against 0.999 for the best subspace of that size. On the penalty term you are right. Without it the count is a PCA count of penultimate features, and that is how CaloShowerGAN and CTGAN ran in the submission (general response).

Quality shows no sharp plateau. Half of FLARE's budget sits below full data on the worst quartile, FLARE's own budget and twice it are equivalent, and four times it is above full data.

CaloShowerGAN stays a stated limitation. Training on FLARE's subset at full data's batch of 1024 did not close the gap (chi2/NDF 5.5, against 1.6 for full data). This may be due to the gap in higher and lower dimensional data/models.

You are right about CTGAN. The gain is rebalancing, and the revision says so. An RCAP-style split of random rows does better there than FLARE's own split, and halving or doubling FLARE's total made no detectable difference (general response). Runs at smaller totals are in progress, to find where the score drops.

The sizer needs per-sample critic gradients, not a Wasserstein critic. Our code computes them in closed form for Wasserstein, cross-entropy, hinge, and least-squares losses, and our tests match each to automatic differentiation. Our three critics are all Wasserstein with a gradient penalty, and other GAN losses, larger models, diffusion, and flows are untested. The optiGAN data are available from the authors on reasonable request, and the code for all three domains will be released.

We read all four papers you named and now cite them. Small-GAN used coresets in GAN training before us. Inside FLARE's per-condition budgets, k-center and facility location did not beat uniform random rows on optiGAN (general response). Our finding that choosing rows does not beat random covers only the rules we tested. The novelty claims are now the total and the closed-form per-sample gradient with the penalty term.

The revision reports every endpoint for the headline configuration. In our rerun it is equivalent to full data on the worst quartile, the worst condition, and global SW, and inconclusive on 99th-percentile SW. It trails full data by 1.2 to 2.7% of full data's value on mean and median condition, precision, recall, and realism, outside the equivalence margin. We registered the worst quartile as the primary endpoint before the headline experiments, because a surrogate has to hold at every condition.

You asked what the computed size adds over a guessed ratio. A ratio that ties FLARE on one pool does not carry to a smaller or a larger pool (general follow-up). FLARE's row count does.

You also noted that quality kept rising at twice and four times the budget. We have since run the guess that keeps FLARE's tilt at sizes up to 32% of the pool. Quality on the worst quartile rises slowly and shows no sharp plateau. FLARE's default trails the 32% guess on the worst quartile and on typical conditions. In the revision we plot the whole curve beside random subsets and FLARE's budget.

On CTGAN, macro F1 did not change detectably with the total from a quarter to twice FLARE's, while the split moved it (general follow-up).

We also withdraw the closed-form gradient as a contribution.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response to all reviewers

We thank the three reviewers. Their questions led to new experiments, and while some of which are in progress, I have enough to start the discussion.

All three reviewers asked whether the speedup is just the smaller batch. It is. We trained on the full pool at FLARE's batch and step count, with a fresh uniform subset of FLARE's size every epoch, and it ran as fast as FLARE. But it sat about 0.016 below both FLARE and full data on the worst quartile of conditions, outside the equivalence margin. At the same batch, every allocation toward the worst-fit conditions that we tested sat above it there. So the speed comes from the batch that the budget implies, and the allocation keeps the hardest conditions at full-data quality.

Two reviewers asked what the computed budget adds over a guessed ratio. On optiGAN, uniform random subsets from 0.25% to 4% of the pool never reached full data on the worst quartile, and FLARE beat random at 1, 1.47, and 2%, the range its own budget covers. A guess that keeps FLARE's tilt toward the worst-fit conditions does better. At 1% it ties FLARE. Run at six sizes, it falls short of full data at 0.25 and 0.5%, is equivalent to it at 1, 1.47, and 2%, and sits above it at 4%. So the guess still needs a total, and FLARE's 1.47% lands in the range that works without a sweep. That range is wide, and every size we tested inside it did as well.

We no longer claim the split of that total. We ran DRoP- and RCAP-style splits, meaning their allocation rules with each condition's sliced-Wasserstein (SW) error in place of class accuracy and uniform random rows inside each condition. Given FLARE's total on optiGAN, the DRoP-style split is equivalent to FLARE on the worst quartile. The RCAP-style comparison with FLARE is inconclusive, and that split is equivalent to full data when its error is the critic's real-minus-fake gap. Both lose about half as much recall as our tilt. On CTGAN, random rows in the pool's proportions at FLARE's total show no detectable difference from full data. An equal split across classes gains about 0.08 macro F1, the DRoP-style split about 0.10, and the RCAP-style split about 0.13, which is 0.04 to 0.06 above FLARE's own split. Both keep SW at full data's level, and our split does not. For the proportional and equal splits, halving or doubling the total moved macro F1 by less than the spread between evaluations of a full-data run (0.033), so CTGAN does not test the total over that range. Runs at smaller totals are in progress.

So FLARE becomes the sizing step, and the revision pairs its total with a DRoP- or RCAP-style split. This changes the method as submitted. It takes in prior work that we had wrongly set aside (reply to uhSM). The main contribution is the same as before, a total read from the critic's gradient spectrum in one training run. The split runs above used a fixed total, set to the average budget FLARE chose over a run (1.47% of the pool on optiGAN and 1.33% on CTGAN). A run with the sizer and these splits together is in progress, as are the splits at other totals on optiGAN. We welcome suggestions for further experiments on this pairing.

Two reviewers found the novelty thin, and we agree with most of the component list. The energy and entropy ranks are standard estimators, size as an optimization output comes from Xia et al., and uniform redraws within each class are RS2's rule. None of the papers we read takes a training-data budget from the model's own gradients, and that is the main claim we keep. The second is the closed-form per-sample gradient with the penalty term (reply to uhSM). K-center and facility location (small-GAN and DeepCore coverage-centric coreset selection respectively) were both mentioned as other methods that find random is hard to beat, however both performed well below uniform random rows. The rows they chose drifted away from each condition's own distribution. Furthermore, RS2 resamples every epoch, where as we reselect a subset much less frequently, which asks the question of how frequently recomputing is necessary.

In the revision, the abstract says FLARE trains 19x faster, is equivalent to full data on the worst quartile of conditions, and sits about 1 to 3% behind it on typical conditions. "No size parameter to tune" goes, and the methods will name the settings that move the budget. The paper also states that the budget is largest early in training, which spends about 47% more rows than a flat 1% for no gain we could measure.

Our own review found that the per-sample gradients for CaloShowerGAN and CTGAN left out the gradient-penalty term the paper describes. With it, every FLARE configuration on CTGAN still beats full data, now at a 1.33% budget instead of 0.77%, and CaloShowerGAN does not change beyond its spread between repeats. Our random baselines also reused the same draws in every repeat. With fresh draws per repeat at 1% and 2%, every worst-quartile verdict for FLARE held.

Every experiment that our first reply listed as in progress has finished. Before the discussion closes we will upload a revised PDF that reports each one in full.

We now run FLARE's sizer and the DRoP- and RCAP-style splits together, and FLARE sets the total each time it recomputes the budget. On optiGAN the results matched those of the same splits at a fixed total. The DRoP-style split was equivalent to full data on the worst quartile, and the RCAP-style split was inconclusive. On CTGAN both splits again beat full data on macro F1. Given FLARE's total, these splits do about as well as our own split on optiGAN and better on CTGAN, so the revised method pairs the total with one of them.

Both splits need a total. At 0.5% of the pool both fell short of full data on the worst quartile, and at 4% both reached it. Within one pool, though, every total from 1 to 4% reached full data under at least one split, our tilt included, so those runs alone do not separate FLARE's total from a good guess. Changing the pool size does. On a quarter of the pool and on five times the pool, FLARE's row count stayed within about 5% of its value on the original pool. A fixed 1% did not carry over. It fell short of full data on the quarter pool and beat it on the five-fold pool. So the size that works here is a row count, and FLARE reads it from the critic in one run. More rows still help on the large pool, and we do not claim the count is an optimum.

CTGAN's macro F1 showed no detectable change from twice FLARE's total down to a quarter of it. The total shows up in the sliced-Wasserstein distance instead. Under the RCAP-style split that distance grows once the total drops below FLARE's.

We withdraw the closed-form per-sample gradient as a contribution. PyTorch's torch.func computes the per-sample gradients in one vectorized call, and on optiGAN it matched our closed form in time and memory. The paper's 30-fold speedup was against a loop over micro-batches of 8 rows. We keep the derivation in the appendix as the way we compute the gradients.

The revision leads with what these runs show for conditional GANs: the split of rows across conditions moves quality, the batch sets the speed, and none of the rules we tested for choosing which rows to keep improves on uniform random rows across the endpoints. The sizing rule comes second, with its limits beside it.

We know that these follow-ups and the revision do not resolve every concern you raised. Why the rank should set the budget is still an open question, and so is the gap on CaloShowerGAN. This is as much as we could run and write up in the time the discussion allowed. Thank you all for the careful reviews.

AuthorsOpenReview ↗

Closing Comments

We have uploaded the revised PDF. Our follow-ups already give the new results, so this note only lists what changed in the paper.

The paper now opens with the split across conditions and the batch. Sizing comes second. The main text adds a table that splits the gap to full data into batch, split, and reuse, and a table of selection rules inside FLARE's budgets. It also adds quality against the share of the pool, the adapted DRoP and RCAP splits, and the pool-size test. Every optiGAN number in the main text comes from a rerun with our released code. The submitted sweeps are in an appendix.

The closed-form gradient is no longer a contribution, and "no size parameter to tune" is gone. PyTorch's torch.func matched the closed form in time and memory, so its appendix is gone too. The methods name the four settings that change the budget. A new paragraph says what the rank does and does not give. Limitations now cover memory, critic width, and the wide range of budgets that work within one pool. Related work adds RS2, Small-GAN, DeepCore, CCS, and RCAP.

We also fixed our own errors. The CaloShowerGAN and CTGAN gradients now include the penalty term. Random baselines at 1% and 2% draw new rows in each repeat, and some rank and timing numbers are corrected. The overview figure, algorithm box, and numbered contributions are in, as we said they would be.

Thank you all for your time and helpful feedback throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.