acceptodds
Under review as a conference paper at ICLR 2027

Useful to Whom? The Easy-versus-Coverage Boundary Moves with the Learner

Abstract

Is the same data equally useful to every learner? The question is whether the value of a training sample belongs to the sample or arises in its interaction with the learner. We test it with a selection experiment. Under a per-class budget, two kinds of samples compete for the slots: easy samples, which a short training run learns first, and coverage samples, which spread over the class in feature space. Easy samples win at small budgets and coverage samples at large ones, and the budget at which the winner flips, the crossover m0, can be measured. We ask whether m0 is set by the data or by the learner. To separate the two, we freeze the selected subsets and change only the learner that trains on them. On low-resolution ImageNet-100, doubling the width of a ResNet-18 moves the crossover from 57 to 85 samples per class on the same subsets. Enlarging the input grid from 32 to 64 pixels at fixed image information moves it from 85 to 57, and a single stride change reproduces or removes this shift. The grid shift shrinks as width grows. Under the native ImageNet-1k protocol, a 4× range of width moves the boundary by at most a few samples per class. On a Vision Transformer, coverage wins at every measured budget, even with easy subsets that work on the convolutional learner. The budget at which one kind of sample stops being the more useful kind therefore belongs to the learner, and the preferred regime of a selection strategy has to be read against the learner that will use the data.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
coreset selection, data selection, learner dependence, model capacity, inductive bias, vision transformers, sample utility, regime boundary

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_43GfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to explore the utility of training samples. Specifically, whether this utility is an inherent property of the data itself or dependent on the model. The paper uses the crossover point between simple samples and coverage-based samples as a metric. They verify assumptions by keeping the selected subset fixed while varying the downstream learner. Experiments demonstrate that model width, input resolution, and the dimensions of feature maps can shift the range of budget allocations for which one strategy outperforms the other. However, these effects are highly conditional; For instance, the influence of model width on the crossover point diminishes when the information content is richer. Furthermore, within the ViT-Tiny configurations and budget constraints, the coverage-based strategy consistently shows superiority. Based on these experimental results, the data selection strategy must be evaluated for the target model and training protocol, rather than inferred only from parameter numbers. This paper mainly focuses on empirical study and does not propose a new method.

Critical strengths
  1. I think the research problem is interesting. By fixing the selected subset across key comparisons, this paper effectively decouples the impact of the selection strategy from the variations in the downstream learner. The experiments cover different model widths, input resolution, and step size, providing a coherent line of reasoning.

  2. The paper is easy to follow.

Critical weaknesses
  1. My primary concern is about the practical implications of this work. While it deepens our understanding of data utility, it offers no guidance or specific methodology for how to actually select data in practice. It remains unclear how a practitioner should determine which strategy is better suited for their target learner, especially with limited resources. According to the paper, if applied directly to real-world decision-making, the proposed crossover-point metric requires training models on both types of subsets across multiple budget levels, which could incur high additional costs. It would be beneficial if the authors could more clearly highlight the practical utility of their findings.

  2. Additionally, the core findings are based on a low-resolution (32*32) version of the ImageNet-100 dataset and models on the scale of ResNet-18. At the more information-rich 64x64 resolution, the width effect essentially disappears. Even on the full-resolution ImageNet-1k dataset, increasing the number of parameters results in only minor changes to the selection outcomes for each class. This raises questions about whether the observed phenomenon still holds any practical significance in real-world, large-scale data and model scenarios.

  3. Figures 1 and 3 should be provided in a vector graphics format.

Questions for authors

Please see the weakness.

Reviewer_Akn8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper aims to determine whether the usefulness of a training sample depends on the learner and empirically shows relationships between sample value and architectural features. Empirical assessment is based on training different models with a fixed budget under two sample selections critiria: easy-first or coverage. Subsets are selected once with ResNet-18 and frozen, and only the learner that trains on them changes. The main conclusions are that sample usefulness: i) depends on the learner; ii) changes with the budget; iii) seems to be “controllable” by some architectural features, namely size of the feature maps and width (although impact is dependent on input size), while parameter count does not seem to be predictive. However, none of these conclusions hold for the ViT family. The authors conclude that the preferred budget regime of a selection strategy depends on the learner that will use the data.

Critical strengths
  1. The question address in the paper is clearly posed.

  2. The evaluation protocol, with a frozen-subset, is an interesting tool to remove the value scorer from the comparison.

  3. The empirical analysis performed provides some useful insights for future work on model characteristics that act as knobs to determine which data selection regime is more useful for each model.

Critical weaknesses
  1. The general insight that data usefulness depends on the learner is hardly new or unexpected. And the authors should not suggest that previous works have all treated usefulness as a property of the sample and not dependent on the model. Several lines of work in curriculum and self-paced learning have addressed this previously. To name a few: • Importance sampling (Katharopoulos & Fleuret). • Teacher–student weighting (MentorNet). • Sample weighting (LOW) • Data valuation (Ghorbani & Zou). For instance, in MentorNet, a teacher pretrained on a different student performs poorly, which showed weighting is student-specific. Importance sampling is also architecture-sensitive, since it worked with Wide ResNet but failed with DenseNet. And LOW weights samples according to per-sample gradient norms, which depend on the learner by design. It also showed that sample usefulness changes throughout the learning process (easy or prototypical samples are more useful early on and more ambiguous samples are later stages). Despite the above and the fact that the main conclusion of the paper is hardly a contribution, the empirical analysis performed does provide some useful insights for future work.
Questions for authors

Additional concerns and suggestions for future work:

  1. One critical observation that undermines the conclusions drawn in the paper is that the performances achieved with the range budgets tested are all very low and may not be representative of actual sample usefulness. The fact that W2 is less accurate than R18 on all settings, except when trained with the full dataset, also suggests that the wider network (or all the networks) may just be under-trained, which does not constitute an interesting scenario to draw any conclusions. Additionally, it seems that, for reasonably higher budgets, herding would always be better and sample value could potentially stabilize across learners.

  2. The informative contribution is the structure of the dependence (feature map size, model capacity), and framing the paper in that direction would make its main claims more novel.

  3. Results with ViT are disappointing but are also confounded by other changes (different optimizer, schedule), and test accuracy is very low compared to full data training, despite memorizing the training set. This further suggests that the models are just under-trained, which weakens the support that the experimental setup is really measuring sample usefulness.

  4. Considering how this paper is built on prior work from the authors (the evaluation design and data-side stability) and the small added value here, this paper feels like a minor/incremental outcome that could have been included in the previous publication.

Reviewer_DXHXOpenReview ↗
Rating
1
Confidence
3
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies whether the budget at which easy-first selection becomes less effective than coverage-based selection depends on the downstream learner. It measures the accuracy crossover between LFrac/EL2N and Herding, freezes the selected subsets, and varies network width, input grid, stride, and architecture family. The results show learner-dependent shifts, with substantially weaker width effects under the native ImageNet-1k protocol.

Critical strengths

None

Critical weaknesses

1. Limited conceptual novelty.

The dependence of data utility on the learner, including the transferability of selected subsets across architectures, has already been investigated in dataset pruning research [1], [2]. Existing cross-architecture evaluations also reuse selected subsets across downstream models. The paper does not sufficiently establish what additional insight follows from expressing these effects through a crossover budget.

2. Insufficient justification for the boundary’s interpretation.

Equation (1) defines a reasonable pairwise accuracy difference, but its zero crossing is specific to the selected methods, feature extractor, and training protocol. The interpretation as a general easy-versus-coverage boundary requires stronger evidence. Difficulty and diversity are complementary considerations already combined by methods such as CCS [3], D² [4], InfoMax [5], and DUAL with Beta sampling [6].

3. Unclear explanatory and practical significance.

Changes in relative performance across budgets and learners are foreseeable. Although their magnitude is worth measuring, the paper provides limited guidance for predicting these changes or improving selection and training. Table 1’s verdict cost restates the observed accuracy gap when the two learners prefer different endpoints; it does not demonstrate an actionable selection procedure.

4. Limited evidence for generality.

The headline width effect occurs on low-resolution ImageNet-100, particularly the 32@64 setting. The substantially smaller, probe-dependent response under native ImageNet-1k leaves the broader significance unclear. The paper needs stronger justification for emphasizing this particular experimental regime.

5. Unclear independent contribution relative to the companion manuscript.

Both papers share the LFrac-Herding comparison, crossover definition, budget-regime framing, and experimental infrastructure [7]. Their interventions differ, examining data properties and learner properties respectively, but the submission does not sufficiently establish why the learner-side extension constitutes a substantial standalone contribution.

References

[1] Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep Learning on a Data Diet: Finding Important Examples Early in Training. NeurIPS, 2021.

[2] Chengcheng Guo, Bo Zhao, and Yanbing Bai. DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning. DEXA, 2022.

[3] Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centric Coreset Selection for High Pruning Rates. ICLR, 2023.

[4] Adyasha Maharana, Prateek Yadav, and Mohit Bansal. D² Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning. ICLR, 2024.

[5] Haoru Tan, Sitong Wu, Wei Huang, Shizhen Zhao, and Xiaojuan Qi. Data Pruning by Information Maximization. ICLR, 2025.

[6] Yeseul Cho, Baekrok Shin, Changmin Kang, and Chulhee Yun. Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty. ICML, 2025.

[7] Anonymous. What Does 5% Mean? The Coreset Regime Boundary Is Pinned in Samples per Class. Manuscript under review, 2026.

Questions for authors
  1. What evidence establishes that the boundary defined through Equation (1) generalizes beyond the specific selector pairs evaluated?
  2. How do competitive hybrid selectors perform in the same frozen-subset learner comparisons?
  3. What explains the much weaker width response under the native ImageNet-1k protocol?
  4. Which substantive findings establish this submission’s independent contribution beyond What Does 5% Mean?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.