acceptodds
Under review as a conference paper at ICLR 2027

When Does a Samples-per-Class Choice Transfer in Supervised Contrastive Learning?

Abstract

Class-balanced contrastive batches trade class coverage for more samples per class. We examine when a preference between K=2 and K=4 remains valid across training budgets, objectives, and evaluation methods. In single-view CIFAR-100, native supervised contrastive learning favors K=4 early and K=2 late, including at a prespecified lower learning rate. On identical K=4 batches, SupCon's existing L_in removes a within-positive KL term while retaining the shared denominator and improves late retrieval. Reused-seed extensions give both L_in and an averaged per-target objective their own learning-rate selection: they improve over native K=4 by 4.11 and 3.64 Recall@1 points, but differ from K=2 by only +0.08 and -0.40 points at the late hold-out head endpoint. The same alternatives differ by 2.44 points at a shorter budget. Paired head–encoder analyses and a post-hoc linear probe further separate objective improvement from the remaining K preference. Pretrained CUB and SOP extend the per-target objective improvement to class-disjoint retrieval, including development-selected stopping on CUB. These results show that the objective gain at a fixed batch composition and the evidence for choosing that composition are distinct, and must be evaluated at the intended budget and readout.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
supervised contrastive learning, deep metric learning, class-balanced sampling, batch composition, training budget

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_YayzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper investigates how the preferred number of samples per class in supervised contrastive learning depends on training budget, objective formulation, and evaluation protocol. The authors first compare class-balanced batches with K=2 and K=4 under standard SupCon, observing that K=4 performs better at early training stages while K=2 becomes preferable after longer training in their single-view CIFAR-100 setting. They then examine SupCon's existing inside-log and outside-log objectives, showing that their difference corresponds to a within-positive KL term, and compare these formulations with an averaged per-target objective on identical K=4 batches. Experiments across multiple learning rates and training seeds demonstrate that changing the objective substantially improves K=4 performance relative to native SupCon, but often produces only small differences relative to K=2 at later training stages. Additional evaluations include encoder versus projection-head retrieval, frozen linear probing, class-disjoint retrieval on CUB and SOP, and experiments with stronger augmentations and multiple views.
Critical strengths
  • The paper addresses a practically relevant question about the interaction between batch composition, training budget, and contrastive objective design. I appreciate the distinction between improving an objective under a fixed K and demonstrating that the resulting K is preferable to another batch composition. The matched comparisons and decomposition of the SupCon objectives provide a clear basis for interpreting these differences.
  • The empirical investigation is extensive, with multiple training seeds, learning-rate comparisons, different training budgets, and evaluations using both projection-head and encoder representations. The additional class-disjoint retrieval experiments and frozen-feature linear probes help clarify the scope of the findings. The authors also explicitly report unsuccessful predictions, inconclusive comparisons, and limitations of their statistical analyses, which is valuable for interpreting the evidence.
Critical weaknesses
  • The technical novelty and significance are somewhat limited. The inside-log and outside-log formulations are already present in SupCon, and the averaged per-target objective is closely related to existing methods such as UniMoCo and SINCERE. The KL decomposition is useful for interpretation but follows directly from these established formulations. More importantly, the substantial improvements over native K=4 largely compensate for its performance gap relative to K=2. After learning-rate selection, the late differences from K=2 are only +0.08 and −0.40 Recall@1 points for the two alternative objectives. The paper therefore provides careful empirical observations, but relatively little new understanding that could predict or guide the optimal batch composition beyond the evaluated configurations.

  • The generality and reproducibility of the conclusions remain concerns. The most extensive controlled experiments use a relatively small convolutional encoder with single-view, weakly augmented CIFAR-100 training. Stronger augmentations and two-view training produce different K preferences, and the two-view experiments lack a same-objective comparison between K=2 and K=4 for the averaged per-target loss. In addition, several learning-rate-selected analyses rely on overlapping seed folds and repeatedly used evaluation data, while the complete historical training code, executed configurations, and raw experimental outputs are not provided. Although these limitations are acknowledged, they make it difficult to independently verify the broader conclusions or assess their robustness in more representative supervised contrastive learning settings.

Questions for authors
  • The main K=4 versus K=2 reversal is established in a single-view setting, while the two-view experiments show substantially different behavior. How does the preference change when the same alternative objective is applied to both K values under standard multi-view SupCon training? In particular, the missing averaged per-target K=2 versus K=4 comparison would help isolate the effect of batch composition from the objective change.
  • The early-to-late reversal is empirically consistent in the main configuration, but its underlying mechanism remains unclear. What primarily drives this transition: the number of positive samples, reduced class coverage, within-positive interactions, or optimization dynamics? A more explicit analysis of these factors would strengthen the contribution beyond the reported empirical comparisons.
  • The alternative objectives substantially improve projection-head retrieval over native K=4, but their advantages over K=2 become small at longer budgets. How much of this improvement transfers to the backbone representations used for downstream tasks after learning-rate selection? The current linear-probe analysis uses only the reference-rate checkpoints, making it difficult to determine the practical benefit of the selected procedures.
  • Several comparisons use overlapping learning-rate selection folds, previously examined evaluation data, and post-hoc analyses. How robust are the main conclusions to an independent evaluation protocol with fresh seeds and an untouched validation/test split? Please also clarify whether the complete executed training configurations and raw per-run results can be released, since the supplied companion currently supports only partial reproduction of the reported experiments.
Reviewer_NYgMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when a samples-per-class choice transfers across training budgets, objectives, and evaluation settings in supervised contrastive learning. In the single-view CIFAR-100 setting, native SupCon favors K=4 early and K=2 late. The authors compare native SupCon with two alternative objectives. Both improve over native K=4, but their performance is close to K=2 at the late endpoint. The paper also examines learning-rate selection, head and encoder retrieval, and frozen-feature linear classification. Additional experiments are conducted on the CUB and SOP datasets using pretrained models for class-disjoint retrieval. The results highlight the distinction between objective improvement and K preference, as well as the effects of training budget and evaluation readout.

Critical strengths

The paper follows a relatively clear line of investigation, starting from the observed K preference, then examining alternative loss functions and learning rate selection. The authors also conduct extensive experiments across different training budgets, objectives, evaluation readouts, and datasets to examine these effects.

Critical weaknesses

1: Limited novelty: The paper mainly studies existing supervised contrastive learning objectives under different training settings. Although the experiments are extensive, the main contribution is empirical. The paper does not propose a new method. While it provides some mathematical analysis of the objectives, it lacks an in-depth explanation or rigorous theoretical results for why the relative performance of K=2 and K=4 changes across training conditions.

2: Limited practical significance: The alternative objectives improve native K=4, but their final performance is often close to K=2 in the main experiments. The study also focuses mainly on K=2 and K=4, without systematically examining larger values of K. This limits the practical guidance for choosing K in other settings.

3: Limited generality: The main findings are largely based on a single-view CIFAR-100 setting. The relative performance of K=2 and K=4 changes with training budget, learning rate, augmentation, and view count. The K comparisons also yield different results across evaluation readouts.

Questions for authors

1: Beyond the empirical observations, what new insights does this work provide into why the preferred K changes across training conditions or how to choose K in practice across different hyperparameter settings?

2: If K=2 can achieve similar final performance, why should I use K=4 with an alternative objective? What is the practical benefit?

3: Many supervised contrastive learning methods use two views during training. How do those conclusions extend to the two-view setting?

Reviewer_NfQkOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how K, the number of samples per class in class-balanced batches, affects supervised contrastive learning, mainly in a single-view CIFAR-100 setting. With native SupCon, K = 4 beats K = 2 early in training but loses late. Switching the K = 4 objective to L_in or to a SINCERE-like averaged loss recovers most of the late gap. Using this result together with head vs. encoder readouts, linear probes and CUB/SOP retrieval, the authors argue that an objective improvement and a preferred batch composition are separate questions, and that both must be judged at the intended training budget and readout.

Critical strengths
  • The experiments are careful and honest. Results are replicated across independent seed blocks, intervals are paired, and failed predictions and adverse results are kept in the paper. This level of transparency is rare, and it makes the findings credible within their setting.

  • The decomposition is a useful way to think about the problem. Since all objectives coincide at K = 2 in the single-view setting, the difference from K = 2 splits cleanly into a native K effect and an objective effect. The finding that the late K = 2 advantage mostly comes from the within-positive KL term of L_out is a concrete and interesting point about SupCon.

Critical weaknesses
  • The setting is narrow and the general take-away is thin. Most evidence comes from single-view SupCon with a small conv net on 64-pixel CIFAR-100, which is not how SupCon is usually used. The paper itself shows that the pattern changes with two views or strong augmentation. The final message, to evaluate at the intended budget and readout, is reasonable but fairly generic, and it does not help a practitioner choose K.

  • Reproducibility is limited. The paper says that the original trainer, the executed configs and the raw per-run outputs are not included, and some training details cannot be recovered. The official CIFAR test set was also used earlier in the project. Together with many post-hoc analyses, readers have to trust summary tables they cannot reproduce, which is a real problem for a purely empirical paper.

Questions for authors

Clarifications:

  • Do the main findings hold in the standard two-view SupCon setup with a ResNet at normal resolution, comparing K under the same objective?
  • Khosla et al. found L_out better than L_in on ImageNet, but here L_in is clearly better. Is this caused by the single-view setup?
  • Can you release the training code and per-run outputs so that the main numbers can be reproduced?

Suggestions:

  • Restructure the main text around two or three clear findings, and move the statistical bookkeeping to the appendix.
  • Check whether the conclusions hold for larger K and larger batch sizes, since practical SupCon batches are much bigger than 96.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.