acceptodds
Under review as a conference paper at ICLR 2027

Stage-Adaptive Submodular Objective Selection for Data-Efficient Training

Abstract

The most useful data-selection objective can change as a model learns, yet existing bandit-based selectors rely on historical rewards that can lag the model's current needs. We introduce DriftBandit, which scores candidate objectives before updating the model using gradient-validation alignment. By comparing candidate-subset gradients with a held-out validation gradient, it estimates objective utility at the current parameters. A complementary Page-Hinkley trigger temporarily increases exploration after validation deterioration. An arm-sampling variant reduces per-decision scoring cost from O(K) to O(m) by evaluating only m of K objectives. Across three vision and six medical-imaging benchmarks, each at three per-batch update budgets, full-information DriftBandit outperforms OnlineSubmod in all 27 settings. It also surpasses sample-level dynamic selectors and non-stationary bandits in the vision experiments, demonstrating gains beyond the closest objective-selection baseline. Fixed-objective comparisons establish the benefit of adapting the selection criterion during training, while ablations identify immediate gradient alignment as the principal source of improvement and deterioration-triggered exploration as a complementary gain.

Primary area:
reinforcement learning
Keywords:
data-efficient training, data subset selection, submodular optimization, adaptive objective selection, multi-armed bandits

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YaGsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to choose among four existing data-selection objectives during training. Building on OnlineSubmod, DriftBandit replaces historical reward estimates with gradient–validation alignment computed at the current model state. It selects an objective once per epoch and uses that objective to construct subsets throughout the epoch. A Page–Hinkley detector temporarily increases exploration when validation accuracy deteriorates. Experiments report consistent improvements over OnlineSubmod on image-classification tasks.

Critical strengths

None

Critical weaknesses
  • The incremental contribution is limited, and the practical reason for using this procedure remains unclear. The four selection objectives already exist, and OnlineSubmod already considers dynamically choosing among them. The main change is to choose using current gradient alignment rather than historical rewards, with an additional exploration adjustment after validation deterioration. This is a fairly direct modification. A simple modification can still be valuable, but here I would need either a compelling practical benefit or a clearer new finding to justify the contribution. The practical benefit is not established by the comparison to other selection methods alone. In Table 4, full-information DriftBandit reaches 74.81% accuracy on CIFAR-100 and takes 50.1 seconds per epoch, while ordinary full-data training reaches 76.04% and takes 45.2 seconds. The two-arm variant is faster at 41.5 seconds, but its accuracy is 74.60%. These results leave open why one should use the proposed selection procedure instead of ordinary training, especially under an equal wall-clock budget.
  • The meaning of the data budget also matters. Selecting 30% of each batch for updates does not mean that only 30% of the data need to be collected or labeled. The method still uses labeled candidate data to construct and score subsets. I therefore do not see how the reported update-fraction experiments establish the claimed annotation-efficiency benefit. The paper needs a clearer account of the actual resource constraint this procedure addresses.
  • The paper does not sufficiently establish a transferable explanation for the gains. The potentially interesting question is whether different selection objectives have different useful operating regimes as a model learns. If this relationship can be characterized and transferred, it could be a more substantial contribution than the selection rule itself. The paper describes an early Graph Cut, middle Facility Location, and late Log Determinant pattern. However, the supporting stage analysis is concentrated on CIFAR-100 at a 30% update fraction. Moreover, the “oracle” objective is defined using gradient alignment, rather than by measuring which objective actually produces better subsequent training. Agreement with this oracle shows agreement with the scoring criterion. it does not establish that each objective is genuinely preferable in the corresponding stage.
  • I acknowledge the results on TinyImageNet and medical-imaging datasets, as well as the five-backbone comparison. These provide evidence beyond a single experiment. However, the key fixed-objective comparisons, ablations, and stage analyses remain concentrated on CIFAR-100. Final-accuracy improvements elsewhere do not show that the same mechanism explains those improvements. The medical experiments also involve fine-tuning pretrained models, which is a different setting from learning representations from scratch. I am therefore unsure whether the paper has identified a general property of training, a broadly useful adaptive heuristic with task-specific behavior, or a benefit limited to the evaluated selection setups. The current analysis does not distinguish these possibilities.
Questions for authors
  1. What practical constraint makes a fixed per-batch update fraction the appropriate budget here? Under equal total training time, how does DriftBandit compare with ordinary full-data training? Please also clarify how the method reduces annotation requirements when labeled candidate examples are needed for selection.

  2. Does the reported stage-dependent ordering reflect actual training utility? For example, starting from the same early, middle, and late checkpoints, does continuing with each objective for an equal budget reproduce the ordering suggested by gradient alignment?

  3. How much of the stage pattern transfers across datasets and architectures? A useful comparison would be a predetermined switching schedule derived from one setting and evaluated in another. This would help distinguish a transferable stage pattern from the benefit of making task-specific online decisions.

Reviewer_HdYnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DriftBandit for adaptively choosing data-selection objectives during training. Once per epoch, the method constructs candidate subsets using several objectives, scores their last-layer gradients by cosine similarity with a validation gradient, and uses the selected objective throughout the epoch. A Page–Hinkley-style detector temporarily increases exploration after validation deterioration. The paper also introduces variants that score only a uniformly sampled subset of objectives.

Critical strengths
  1. The paper addresses a practical question: whether the criterion for selecting training data should change as the model learns. Evaluating candidate objectives using the model’s current gradients is a reasonable way to make this choice, especially when past rewards no longer reflect their usefulness. This is a quite interesting perspective.

  2. The experiments cover natural-image and medical-image classification, multiple update fractions, ten seeds, and several architectures. Fixed-objective and component ablations are useful additions.

  3. The paper is explicit about several gaps between the analysis and the implementation, including dot-product versus cosine scoring and single-step analysis versus epoch-level decisions. These qualifications help readers understand what the theoretical results are intended to establish.

Critical weaknesses
  1. Section 2.1 claims that the four objectives admit the standard 1−1/e1-1/e greedy approximation. However, Table 5 defines Disparity Sum as f(S)=∑i,j∈Sdijf(S)=\sum_{i,j\in S}d_{ij}. For nonnegative symmetric distances, the marginal gain of adding ee is 2∑i∈Sdie2\sum_{i\in S}d_{ie}, which increases with SS. This is a supermodular function, rather than a submodular one. The specified Graph Cut objective is submodular but generally non-monotone. The stated Log Determinant objective also does not automatically satisfy the monotonicity and nonnegativity conditions required by the claimed guarantee. Consequently, the blanket approximation statement is invalid.

  2. Under the stated uniform-sampling policy, exploitation selects the sole sampled arm regardless of its score, making the policy equivalent to uniform random objective selection. Its performance therefore cannot be attributed to gradient alignment without an additional mechanism.

  3. Some experimental interpretations need stronger support. With the exploration boost set to zero, the PH threshold should not affect the selection policy, but Figure 6 shows noticeable variation. The runtime explanation also attributes a 1.8-second overhead to scoring described elsewhere as taking 1–2 milliseconds per epoch.

Questions for authors
  1. What exact objective functions and greedy algorithms are implemented? How should the submodularity and approximation claims be corrected?

  2. Eqs. (7)–(8) imply PHt≤1PH_t\le1 when accuracy lies in [0,1][0,1], yet Table 10 reports detections at thresholds of 1 and 2. What accuracy scale is used by PH, and can the reported detection counts be reproduced from the stated equations?

  3. What makes the (m=1) policy different from uniform random objective selection? From the paper, I can't tell the difference from those two.

Reviewer_fMD5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DriftBandit, which picks one of four submodular objectives per epoch for per-batch subset selection. Objectives are ranked by cosine similarity between each candidate subset's last-layer gradient and a 32-sample validation gradient, and a Page-Hinkley (PH) test on validation accuracy triggers a one-epoch exploration boost. Experiments on nine vision and medical datasets show consistent improvements over OnlineSubmod.

Critical strengths
  1. The motivation is clear: delayed EMA rewards can lag a changing model, and scoring objectives at the current parameters is a sensible fix.
  2. The method is simple and adds little to OnlineSubmod's pipeline.
Critical weaknesses
  1. Several results contradict Algorithm 1. With α=0, PH has no effect on selection, so α=0 should match GA-only and should not vary with γ. Instead GA-only is 74.44 (Fig. 10), α=0 is 74.63 (Table 11), and the α=0 row of Fig. 6 ranges from 73.73 to 74.80 across γ. Fig. 6 also disagrees with Tables 6 and 11 on overlapping cells.
  2. In Fig. 1, accuracy rises monotonically while the PH statistic repeatedly reaches γ, which Eqs. (7)-(9) do not allow. The units of v_t are also inconsistent: the proof assumes v_t ∈ [0,1], under which γ=0.5 would need a 50-point drop to fire.
  3. Fig. 4 shows Disparity Sum selected in 20% of epochs, though Fig. 2 gives it the lowest alignment throughout and the schedule allows only about three random epochs per run. This also conflicts with the 19% oracle mismatch.
  4. Timings in Table 4 do not match the described compute (K subsets on one batch per epoch, 1 to 2 ms scoring), and Appendix F.16 contradicts Algorithm 1 on per-batch computation.
  5. Smaller mismatches: TinyImageNet 10% gain is +0.85 in Table 1 but +0.65 in Fig. 8; InfoBatch and MOSO are said to be run at one setting but appear at nine; the LR window in Table 13 is off by about 65 epochs; §5.1 misstates which gains are largest.
  6. A random-subset baseline is missing, as are simple alternatives such as a random arm per epoch or a fixed Graph Cut, Facility Location, Log Determinant schedule (the phase order the paper itself reports).
  7. The practical benefit is unclear. Selection needs labels for every sample, yet the paper claims label savings for clinical settings. The main configuration is slower than full training and 1.2 points less accurate.
  8. Related work on validation-gradient alignment for data selection is missing, including GREATS (Wang et al., 2024), DoGE (Fan et al., 2024), and bandit-based online data mixing (Albalak et al., 2023).
  9. Reported p-values (e.g. p < 10⁻¹² with 10 seeds) imply implausibly correlated seeds, and the hyperparameter selection procedure for DriftBandit is not described.
Questions for authors
  1. How do the α=0 results and the Fig. 1 PH trace follow from Algorithm 1?
  2. Why is Disparity Sum selected in 20% of epochs?
  3. How do random-subset and fixed-schedule baselines perform?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.