- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes Distributional Contrastive Learning (DistCLR), which models pairwise similarity uncertainty through an adaptive temperature derived from probabilistic representations. The method combines uncertainty-aware InfoNCE with a directional prior regularizer and can be integrated into existing contrastive learning frameworks. Experiments on CIFAR-10, CIFAR-100, ImageNet-100, and ImageNet-1k evaluate classification, transfer learning, representation quality, open-set recognition, and robustness under distribution shifts.
Critical strengths
- The paper introduces pairwise similarity uncertainty into contrastive learning through an adaptive temperature formulation. The method can be integrated into existing contrastive frameworks with limited changes and low computational overhead, making it practically useful.
- The experiments cover several contrastive frameworks and datasets, including ImageNet-1k. The evaluation also considers transfer learning, open-set recognition, and corruption robustness, providing evidence of the method's potential benefits beyond classification accuracy.
Critical weaknesses
-
The theoretical foundation of the proposed uncertainty formulation is not sufficiently convincing. The similarity variance approximation in Proposition 1 relies on concentrated Gaussian posteriors, but it is unclear whether this assumption holds in practice. For example, the learned scale reported in Figure 3 appears inconsistent with the concentrated-posterior regime at the given representation dimension. This raises concerns about whether the derived similarity uncertainty accurately reflects the underlying representation uncertainty.
-
The experiments do not clearly establish the benefit of learning similarity uncertainty over simpler temperature adaptations. In Table 5(b), the constant-scale variant with the same geometric factor achieves nearly the same ARI as the full DistCLR (18.16 vs. 18.21). While the learned scales improve error detection, their contribution to representation quality remains less clear. This weakens the evidence that the proposed uncertainty modeling, rather than the geometric temperature adjustment, drives the reported representation improvements.
Questions for authors
- Can the authors provide empirical evidence that the first-order similarity variance approximation in Proposition 1 remains accurate under the learned scales? In particular, how well does Eq. (11) match the actual similarity variance estimated through Gaussian sampling at the operating point reported in Figure 3?
- Could the authors compare the full DistCLR with the constant-scale-plus-geometry variant on linear-probe accuracy and corruption robustness under the same training settings? Given their nearly identical ARI results in Table 5(b), this would help clarify whether the learned uncertainty provides additional benefits for representation learning beyond the geometric temperature adjustment.
5 comments
Response to Reviewer 5v1w [Part 2/3]
Q2 in one line. The geometric factor (1 − s²) is DistCLR's similarity-uncertainty temperature (its exact variance form, Proposition 2), so the shared-scale arm is DistCLR with one tied scale. With either scale, DistCLR is more robust than InfoNCE and the cosine temperature schedules and keeps more structure than a learned per-view temperature without the variance form. Learning the scale per view adds an uncertainty signal that neither a shared scale nor a head trained afterwards reaches.
Q2. Learned scale vs shared scale + geometry under identical code and settings (Table 5c: CIFAR-100 SimCLR, ResNet-18, all at τ_A = 0.185, 3 seeds per arm, all arms in one code path). This experiment is separate from Table 1 and not comparable with it in absolute values. The shared arm uses one scale for all views and pairs (App. B.11). Online: linear classifier trained during pretraining. Linear: 100-epoch linear evaluation. CIFAR-100-C and AUROC: frozen probe.
| Online | Linear | CIFAR-100-C | Error AUROC of U, clean / sev. 5 | |
|---|---|---|---|---|
| InfoNCE (τ = τ_A) | 66.26 ± 0.21 | 66.78 ± 0.32 | 48.52 ± 0.15 | – |
| DistCLR, shared scale | 66.17 ± 0.18 | 66.76 ± 0.20 | 48.91 ± 0.10 | .481 / .479 |
| DistCLR, learned scales | 66.49 ± 0.21 | 67.04 ± 0.16 | 49.52 ± 0.11 | .756 / .743 |
-
Accuracy (linear evaluation, the primary metric): 67.04 with learned scales, 66.76 with the shared scale, 66.78 for InfoNCE. On the frozen probe the shared scale is higher (59.18 vs 58.78, InfoNCE 58.16; Table 13).
-
Corruption robustness: both DistCLR arms are more accurate than InfoNCE on CIFAR-100-C (48.91 and 49.52 vs 48.52, Welch p = .03 and .001), the learned scales most.
-
Uncertainty: the shared arm's scale head is trained by the prior alone, so this control isolates what learning the per-view scale adds: an error-ranking signal (AUROC .756, still .743 at severity 5, vs ~.5). U also outperforms every other label-free score we tested (embedding and feature norms, k-NN distance, augmentation disagreement, TaU and MCInfoNCE, App. B.2 and B.14):
- covariate-shift detection on CIFAR-100-C: .565 → .690 from severity 1 to 5, vs at most .545 → .629 for the norms and the k-NN distance (ahead at every severity) and at most .55 at any severity for TaU and MCInfoNCE.
- accuracy under shift, with no labels of the target set and no classifier: over the 96 clean and CIFAR-100-C test sets, the mean U of a set predicts the probe's accuracy on it (Spearman −.96, held-out-corruption error 3.2 points vs 3.7 to 7.3 for the feature norms and k-NN distances of DistCLR, shared-scale and InfoNCE encoders, and 2.1 to 2.3 for the probe's own average confidence, which needs the classifier; App. B.14).
- unseen classes (CIFAR-10 images against CIFAR-100): .690 vs .649 for the k-NN distance, on every seed (App. B.14).
- a head trained afterwards on the frozen InfoNCE encoder to predict each view's agreement with its positive ranks errors at .707, below U (.756, every seed): the signal comes from training with DistCLR's objective (App. B.14).
- error ranking: .756 vs at most .733 for the norms and the k-NN distance, at most .61 for TaU and MCInfoNCE, and .749 vs .681 for augmentation disagreement under the protocol of App. B.2.
Fused with the probe's MSP, U also gives the best open-set recognition (Table 4).
Table 5c and Table 1. Table 5c is a separate experiment (three pretraining seeds, the revision's code path, every arm at τ_A), which the Table 5c caption now states, so each table is compared within itself. At the same base temperature, DistCLR is ahead of InfoNCE in online, linear and CIFAR-100-C accuracy (+0.23, +0.26, +1.00).
Comment 3/3 repeats this comparison in NNCLR, All4One, ViT-Small and on CIFAR-10.
W2. Simpler temperature adaptations (Table 5b, matched optimization, 3 runs each):
| Variant | ARI | EffRank | Error AUROC |
|---|---|---|---|
| Fixed InfoNCE | 16.87 ± 0.81 | 191.67 ± 0.69 | – |
| Learned per-view temperature, no geometry ("pointwise") | 17.34 ± 0.06 | 185.29 ± 0.05 | .540 ± .001 |
| Shared scale + geometry | 18.16 ± 1.03 | 194.86 ± 0.17 | .502 ± .013 |
| Learned scale, shuffled across samples | 16.80 ± 0.27 | 193.56 ± 1.24 | .553 ± .042 |
| DistCLR | 18.21 ± 0.61 | 195.49 ± 0.68 | .749 ± .002 |
A learned per-view temperature without the geometry loses structure (EffRank 185.3) and the error signal (.540). Only the combination of the geometry with correctly assigned learned scales gives both.
What this establishes. Both parts of DistCLR's temperature contribute, as Contribution 4 now states:
"Matched ablations separate the two parts of the temperature. Both contribute: the similarity-uncertainty geometry (1−s²) improves semantic organization (clustering and effective rank) and robustness over InfoNCE, and learned per-view scales add a signal that ranks errors and unknown inputs."
Response to Reviewer 5v1w [Part 3/3]
Adapting the global temperature instead (App. B.12, Table 5c recipe and evaluation, 3 seeds each):
| Temperature | Online | Linear | CIFAR-100-C | Error AUROC of U |
|---|---|---|---|---|
| Cosine schedule centred on the tuned τ (τ 0.1–0.3) | 65.86 ± 0.13 | 66.26 ± 0.10 | 47.70 ± 0.36 | – |
| DistCLR | 66.49 ± 0.21 | 67.04 ± 0.16 | 49.52 ± 0.11 | .756 ± .003 |
The strongest adaptive baseline, the cosine schedule centred on the tuned τ, stays below DistCLR by 0.63 online, 0.78 linear and 1.82 points on CIFAR-100-C, and no global temperature gives a per-sample signal.
A hard-pair baseline at the matched temperature (App. B.12). HCL (Robinson et al., 2021; published β = 1, τ⁺ = 0.1), run in the same pipeline at τ = 0.2, reaches 58.0 online with β = 1 and 56.86 ± 0.44 with the milder β = 0.5 (3 seeds). It needs its published τ = 0.5 (66.01 online, 66.07 linear), and there it remains below DistCLR (66.49, 67.04).
The same comparison in other frameworks, on a transformer backbone and on a second dataset (App. B.11, Table 13, CIFAR-100 unless stated, 3 seeds each, deterministic arms of NNCLR and All4One at their released τ_d = 0.2). Linear: standard linear evaluation. Probe, corrupted-data accuracy and AUROC: frozen probe. Shared scale → learned scale:
| Setting | Linear acc. | Probe acc. | CIFAR-100-C / CIFAR-10-C | Error AUROC of U |
|---|---|---|---|---|
| NNCLR, ResNet-18 | 67.35 → 67.80 | 61.81 → 62.66 | 42.10 → 42.36 | .505 → .748 |
| All4One, ResNet-18 | 70.54 → 70.72 | 65.22 → 65.31 | 42.46 → 42.65 | .496 → .784 |
| SimCLR, ViT-Small (800 ep) | 67.19 → 67.55 | 60.45 → 61.03 | 52.50 → 52.93 | .488 → .726 |
| SimCLR, ResNet-18, CIFAR-10 (CIFAR-10-C) | 90.58 → 90.81 | 89.25 → 89.12 | 82.22 → 82.12 | .508 → .780 |
In NNCLR, All4One and ViT-Small the learned scale is also slightly more accurate on every metric, and only it ranks errors. On CIFAR-10 both DistCLR variants improve on InfoNCE (learned scale: probe 89.12 vs 88.62, Welch p = .02, and CIFAR-10-C 82.12 vs 81.55, p = .04). Only the learned scale ranks errors, in every setting. DistCLR also matches or exceeds each deterministic objective (NNCLR linear 67.80 vs 67.54 and CIFAR-100-C 42.36 vs 42.02, All4One probe 65.31 vs 64.99, CIFAR-100-C 42.65 vs 42.13 and linear 70.72 vs 70.42).
The shared scale sits where the prior puts it (u = 2σ_p² = 0.04, vs a learned mean ≈ 0.031). Doubling it (σ_p² = 0.04, App. B.10) keeps its error AUROC near chance (.511), so the error-ranking signal does not come from the scale's magnitude.
Over the tested hyperparameter ranges (τ_A 3×, λ_prior 10×, σ_p² 4×), its error AUROC stays within .733–.762 (App. B.10, 3 seeds per setting).
All additions are in the appendix. In the main text, §3.2 states Proposition 2 and Table 5c reports the comparison you proposed.
Thanks to your questions, the temperature now rests on an exact result at the learned operating point, and the comparison you proposed holds in SimCLR, NNCLR and All4One, on a ViT-Small backbone and on CIFAR-10: learned scales alone rank errors and detect shift. If these answers resolve your soundness concern, we would be grateful if you would consider updating your score. If anything still stands in the way, we would be glad to address it within the discussion period.
Response to Reviewer 5v1w [Part 1/3]
Thank you for the two precise questions. We ran both checks. They led to an exact result that now anchors §3.2 (Proposition 2) and to the comparison you proposed in six settings (all changes in blue).
Q1. Is Eq. (11) accurate under the learned scales? We sampled it at the operating point of Fig. 3 (App. B.8, three checkpoints, 256 test pairs × 16,384 Gaussian samples each). There (L − 1)u ≈ 7.9, outside the condition (L − 1)u ≪ 1 of Proposition 1, and Eq. (11) overstates the sampled variance about 16×. Two properties do hold at this operating point. The dependence of the variance on the pair is exactly (1 − s²) at any concentration (Proposition 2 below), and within each anchor's softmax row Eq. (11) orders the candidates as sampling does (median Spearman .96 in the subspace the embeddings occupy, App. B.8). The form that DistCLR's temperature uses is therefore exact, and its magnitude is learned. Where Proposition 1 does hold (a 16-dimensional projector output, (L − 1)u ≈ 0.43), DistCLR improves on InfoNCE at the same temperature on every metric and every seed (+0.76 linear, +1.17 probe, +1.31 CIFAR-100-C, p ≤ .014), also against InfoNCE at DistCLR's mean temperature τ = 0.20 (App. B.8).
Proposition 2 (exact similarity variance). For z_i = normalize(ẑ_i + ε_i) with independent isotropic Gaussian noise of any variance, write a_i = ⟨z_i, ẑ_i⟩, μ_i = E[a_i], m_i = E[a_i²], c_i = (1 − m_i)/(L − 1). Rotational symmetry (Mardia & Jupp, 2000) gives
Var⟨z_1, z_2⟩ = (B + C) + (A − B)(1 − s²), with A = m_1c_2 + c_1m_2 − c_1c_2, B = m_1m_2 − μ_1²μ_2², C = (L − 1)c_1c_2.
Eq. (11) is its first-order limit (A − B → u_1 + u_2, B + C → 0), and direct sampling matches the identity within 0.5%. DistCLR's temperature τ_A + (u_r + u_t)(1 − s²) has this form at every concentration: τ_A carries the pair-independent part and u the slope, learned per view. At any operating point, (1 − s²) is also the squared spherical gradient norm of s, so each view's scale is weighted by how strongly the pair's similarity responds to that view (App. B.15).
What u is in the trained model. As the submission states (§4.5), the learned scale ranks uncertainty rather than estimating a calibrated variance: harder views receive smaller u, and U = 1 − u ranks each image's spread over 16 training augmentations (Spearman .69), errors (AUROC .73–.78 in every framework and backbone), shifted inputs and unseen classes (comment 2/3).
Framing and title. Following your review, the abstract, Fig. 1, the introduction, §3 and the Conclusion present DistCLR as (i) a geometric per-pair temperature with the functional form of the similarity variance (the form is exact at any concentration, and its magnitude is learned) and (ii) a learned per-view scale that scores view hardness. The word "exactly" now refers only to this form. For the camera-ready we can retitle the paper accordingly: "Embrace Ambiguity: Rethinking Contrastive Learning through the Geometry of Similarity Uncertainty".