- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes D4C, a framework for long-tailed clustering that integrates optimal transport, layer-wise density modeling, geometric repulsion, and prototype merging.
Critical strengths
-
The proposed method jointly addresses class imbalance, local density variations, and prototype separation.
-
The experiments cover multiple benchmark settings and evaluate performance using complementary metrics.
-
The ablation studies, training analyses, and theoretical discussions provide useful insights into the proposed framework.
Critical weaknesses
-
The methodological novelty needs to be more clearly articulated. Sinkhorn assignments, vMF density modeling, geometric repulsion, and prototype merging are widely adopted techniques. While the coverage-risk perspective offers a coherent integration of these modules, the paper has not sufficiently demonstrated what algorithmic properties or theoretical conclusions this formal framework brings that differ from existing methods. Therefore, the reviewer has concerns regarding the contributions of this work.
-
The coverage-risk theory does not establish a guarantee for the implemented method. The bound does not account for learning the representation from the same data, and the paper does not show that its Sinkhorn-target cross-entropy reduces the bounded nearest-center cost. Center separation also does not generally guarantee a smaller worst-case covering radius.
-
The fine-grained long-tailed claim lacks targeted validation. The paper reports overall results on CUB, Dogs, and Flowers, but its key component ablations are conducted on CIFAR-10. Without fine-grained ablations and tail-class analyses, it is unclear which components improve this specific setting.
-
The benefit of layer-wise control is not isolated. The paper does not directly show that fine-grained information is progressively lost across layers, or compare its layer-wise design with a final-layer-only alternative of matched capacity.
-
The baseline comparisons lack a clear common protocol. The tables do not identify which results come from published sources and which are reruns, or consistently clarify the encoders and evaluation settings used for each method.
-
The reference list contains minor inaccuracies. For example, “Self-labelling via Simultaneous Clustering and Representation Learning” is cited as a 2019 arXiv preprint, although the work was subsequently published at ICLR 2020.
Questions for authors
-
Could the authors clarify the sources and experimental settings of the MiniClustering results reported in Tables 3 and 4? For example, the reported ACC on CIFAR-10 at an imbalance ratio of 5 is 62.3, compared with 74.3 in the published MiniClustering paper. This discrepancy changes the relative ranking between MiniClustering and D4C, which achieves an ACC of 68.0. If the reported MiniClustering results were obtained through independent reruns, why was this baseline rerun while other baselines appear to rely on previously published results? Please clarify the pretrained encoder, data split, and evaluation protocol used, and explicitly distinguish published results from independently reproduced results.
-
Could the authors provide controlled ablation studies of the three key components on one or two fine-grained long-tailed benchmarks to better substantiate their individual contributions in the target setting?
6 comments
W2 : The Theory Covers Coverage Risk in a Fixed Representation and Nothing Stronger
The coverage-risk theory gives a per-layer bound on a fixed representation, and this reply states exactly what it proves and what it does not.
The bound characterizes coverage risk in a given representation space, not an end-to-end guarantee for joint training. Appendix B (Eq. 17) upgrades the fixed-center bound (Eq. 10) to a uniform bound over all center sets . It holds for any data-dependent center set , adds only an complexity term, and keeps as the optimizable leading term. Both bounds are per-layer statements conditional on . They cover the features the losses observe, not the dependence of on the same training sample, which would require uniform control over the forward-stack parameter class that we do not claim. The revision has made this conditioning explicit.
For the Sinkhorn cross-entropy we separate three statements of distinct logical status.
- Provable. The plan is a feasible coupling with row marginal and column marginal , so by Eq. 2 the teacher-plan cost ||$f_i-w_k||$$^{2}\ge\Psi(W)$ upper bounds the nearest-center cost, and on the sphere |||| it is affinely equivalent to the inner-product cost of .
- Not claimed. We do not claim that minimizing monotonically decreases this bound. Its gradient involves , which is not the gradient of the fixed-plan transport cost, so the two descent directions are not interchangeable.
- Experimental. Fig. 7 shows , and decreasing in both layers throughout training, and the revision has been added the teacher-plan cost curve alongside .
On separation versus covering radius, the revision has stated three claims of distinct logical type. OT provides an optimizable upper bound on the average nearest-center cost (Eq. 2). The geometric repulsive potential provides a deterministic lower bound (Appendix D), transferred to the output prototypes via Lemma E. This non-degeneracy is necessary, not sufficient, for improving the worst-case covering radius, and the equatorial-ring counterexample in Appendix E documents the lack of a monotone equivalence.
Finally, the entropic OT equivalence of Appendix C holds for the full-dataset EM posterior at convergence, where . The mini-batch assignment used in the training regularizer is a detached approximation that does not satisfy the same global marginal constraints, and the revision has stated this boundary explicitly.
W3 & Q2 : All Components Contribute on Oxford Flowers and Tail-Sensitive Metrics Suffer the Most
We ran the controlled single-variable ablation of every component on Oxford Flowers (imb_ratio = 10, frozen MoCo v3, the exact protocol of Table 1), where ConMix leads in ARI (49.3 vs. 47.2), so it is the least favorable test environment for our components.
All three key components contribute, consistent with CIFAR-10 (Table 5). Removing the density-aware forward layers costs the most at ACC and ARI, so fine-grained long-tailed data depends more, not less, on layer-wise geometry control, and final-layer rebalancing alone cannot recover structure lost earlier in the forward pass.
The tail evidence is directly readable from CAA, the unweighted per-class recall. Removing costs 5.3 CAA, prototype anchoring 6.5, LTAM () 6.0, and density-aware forward layers 8.7, while head-dominated ACC stays flat or even rises for w/o EMA (53.5) and w/o (53.4). This is the head-masks-tail failure mode the long-tailed design targets, and these components contribute through class-balanced tail-sensitive performance rather than head-dominated ACC.
The components are complementary rather than redundant. Each controls a different term of the coverage-risk bound, and each removal produces a distinct measurable loss, mirroring the collapse on CIFAR-10 when both and are removed (Table 5).
| Hyperparameter | ACC | CAA | NMI | ARI |
|---|---|---|---|---|
| Full D4C | 53.0 | 51.2 | 73.2 | 47.2 |
| w/o prototype anchoring | 49.8 | 44.7 | 69.6 | 40.9 |
| w/o long-tail marginal() | 50.4 | 45.2 | 69.9 | 40.2 |
| w/o density-aware forward layers | 45.9 | 42.5 | 64.9 | 28.9 |
| w/o EMA | 53.5 | 48.9 | 70.2 | 42.7 |
| w/o only | 51.3 | 47.7 | 70.5 | 43.0 |
| w/o only | 53.4 | 45.9 | 70.4 | 43.2 |
This table has been added to the revision.
W1 : The Coverage-Risk Framework Licenses Predictions the Modules Alone Do Not
The novelty is not Sinkhorn, vMF, or the determinant kernel themselves, but how constraints at different granularities jointly construct the hierarchical coverage geometry.
First, a risk decomposition assigns each module a place in the bound. For any layer the coverage risk is Rτ(C)=Prx∼P[minm||x−cm||>τ]. The finite-sample bound (Eq. 10, adaptive version Eq. 17 in Appendix B) decomposes it into an optimizable average term Ψ(C)=N1∑iminm||xi−cm||2 and a geometric bottleneck governed by the worst-case covering radius rmax(C) (Eq. 11). Optimal transport minimizes Ψ, the geometric repulsive potential keeps the center set non-degenerate, and the vMF pull improves local density geometry, which empirically lowers Ψ and rmax. A list of tricks description provides no such assignment and offers no prediction about which module can be removed.
Second, the theoretical statements are ones the components alone do not yield, and they differ from existing mechanisms. Any global-assignment cost is a differentiable upper bound on the average coverage risk (Eq. 2). The log-det repulsion yields a quantitative lower bound on the minimum inter-center distance dmin, transferred to the output prototypes via Lemma E (Appendix D). The long-tail-aware marginal mk∝πkα is derived rather than assumed, because under long tails the uniform marginal is statistically mismatched with the class support. Existing deep clustering confines rebalancing and separation to the final feature space. D4C instead places a class-level long-tail marginal at the output layer, uses peak-level masses in the hidden layers so that class priors and local density structures are never conflated, and ties prototype separation to the coverage-radius analysis through the log-det bound. The logical status of each statement is given in our reply to W2.
Third, the framework predicts algorithmic properties that the experiments confirm. Because each objective controls a different term of the bound, the framework predicts complementarity rather than redundancy, and Table 5 confirms it. Removing Lgeo(l) or Lvmf(l) collapses training to 32.7 and 38.2 ACC even though over-segmentation remains active, so no other module can substitute. It also predicts that coverage risk is necessary but not sufficient for clustering accuracy, a falsifiable statement verified in Appendix L and Fig. 7, where Ψ, rmax and Rτ keep improving while ACC saturates. This statement justifies reporting checkpoints from the stable window rather than the final epoch.