acceptodds
Under review as a conference paper at ICLR 2027

When Does a Samples-per-Class Choice Transfer in Supervised Contrastive Learning?

Abstract

Class-balanced contrastive batches trade class coverage for more samples per class. We examine when a preference between K=2 and K=4 remains valid across training budgets, objectives, and evaluation methods. In single-view CIFAR-100, native supervised contrastive learning favors K=4 early and K=2 late, including at a prespecified lower learning rate. On identical K=4 batches, SupCon's existing L_in removes a within-positive KL term while retaining the shared denominator and improves late retrieval. Reused-seed extensions give both L_in and an averaged per-target objective their own learning-rate selection: they improve over native K=4 by 4.11 and 3.64 Recall@1 points, but differ from K=2 by only +0.08 and -0.40 points at the late hold-out head endpoint. The same alternatives differ by 2.44 points at a shorter budget. Paired head–encoder analyses and a post-hoc linear probe further separate objective improvement from the remaining K preference. Pretrained CUB and SOP extend the per-target objective improvement to class-disjoint retrieval, including development-selected stopping on CUB. These results show that the objective gain at a fixed batch composition and the evidence for choosing that composition are distinct, and must be evaluated at the intended budget and readout.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.