acceptodds
Withdrawn from ICLR 2027

Dynamic Positive CLIP Distillation for Multi-Labeled Complementary Label Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-labeled complementary label learning (MLCLL) trains multi-label classifiers from complementary labels, which indicate classes absent from a sample rather than its positive labels. Since verifying absent classes can be easier than exhaustively annotating all positive labels, MLCLL provides a cost-effective solution for multi-label learning (MLL). Existing methods mainly learn from complementary labels, which reveal only a subset of negative classes and provide no positive supervision. Our analysis shows that such non-exhaustive complementary supervision may leave residual uncertainty about the positive-label set, while only negatives labels may result in a degenerate solution that assigns low probabilities to all classes. Pretrained vision-language models offer a promising way to recover the missing positive semantics, but existing approaches typically require observed positive labels as semantic anchors and therefore cannot be directly applied to MLCLL. To address this limitation, we propose CoDi, a CLIP-based framework for MLCLL without positive labels. CoDi first employs a frozen CLIP teacher with positive-only prompts to construct positive evidence from cached global predictions and dynamically updated random-local and class-aware views guided by an exponential moving average (EMA) student. It then introduces reliability aware pseudo-positive learning to select, weight, and temporally stabilize pseudo-positive labels while preserving observed complementary labels as reliable negative supervision. The resulting positive knowledge is distilled into a standalone student model, enabling CLIP-free inference. Experiments on four benchmark datasets demonstrate the effectiveness of CoDi.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Machine learning, weakly supervised learning, multi-label learning, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vcGsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-labeled complementary label learning, where training samples are provided with labels known to be absent rather than positive labels. The authors argue that non-exhaustive complementary labels leave substantial uncertainty regarding the underlying positive-label set, while purely negative supervision admits a degenerate all-zero prediction solution. To supplement the missing positive supervision, the paper proposes CoDi, a CLIP-based distillation framework that combines complementary-label supervision with positive semantic evidence extracted from a frozen CLIP teacher. CoDi constructs positive evidence using global views, random local views, and EMA-student-guided class-aware views. These CLIP-based predictions are class-wise calibrated and fused with EMA student predictions. The resulting evidence is further filtered through a reliability-aware pseudo-positive learning mechanism that incorporates confidence, cross-view stability, agreement with the EMA student, and thresholds derived in part from observed complementary labels. A pseudo-positive bank temporally smooths accepted pseudo-labels. Experiments are conducted on VOC07, COCO14, NUS-WIDE, and CUB200 under four complementary annotation ratios. CoDi achieves the best reported mAP in 13 of 16 settings.

Critical strengths
  1. The overall motivation and high-level pipeline are reasonably clear, and Figure 1 provides a useful overview of how CLIP-based positive evidence, complementary labels, and the student interact.
  2. The combination of reliable negative complementary supervision with VLM-derived positive semantic knowledge is sensible and potentially useful.
  3. CoDi achieves the highest mAP in 13/16 settings.
Critical weaknesses
  1. The method is highly complex, but the ablation study does not isolate many of its central design choices.
  2. Many individual components of CoDi—multi-view CLIP prediction, classifier-guided local crops, EMA teachers, pseudo-label banks, consistency training, and CLIP distillation—are already established techniques. The novelty therefore appears to lie primarily in how these components are adapted and combined with complementary-label constraints rather than in a fundamentally new learning mechanism.
  3. The experimental protocol uses ground-truth labels on a 20% validation split, despite the paper repeatedly motivating the problem as learning without observed positive labels. These labels are apparently used for model/hyperparameter selection. This weakens the practical claim of positive-label-free learning and raises the question of how much fully annotated validation supervision is needed in practice.
  4. The experimental comparisons do not sufficiently establish novelty or superiority over the closest prior work. The paper itself identifies CCD (Kim & Shim, 2025) as an annotation-free multi-label CLIP distillation method that addresses prediction bias and view sensitivity through classifier-guided local views and calibrated distillation. This appears particularly close to CoDi’s use of global/local/class-aware views, classifier-guided crops, calibration, and student distillation.
Questions for authors
  1. There is a notation inconsistency around Eq. (13): the text states that student predictions on a strong view are aligned with EMA predictions on a weak view, while the sentence immediately after the equation states that predictions on x_i use the weak view.
  2. Why is CCD not included as a baseline? Given that CCD already uses classifier-guided local views and calibrated CLIP distillation for annotation-free multi-label learning, what are the quantitative gains of CoDi specifically from incorporating complementary-label constraints?
  3. Why are recent MLCLL methods such as ComRank and the 2026 complementary-label methods discussed in Related Work but omitted from the experimental comparison? Please compare against the strongest current MLCLL baselines.
  4. The paper states that 20% of each original training set is reserved for validation and that ground-truth labels are used for validation. How many fully annotated positive labels are therefore required to deploy CoDi? Can the method and all hyperparameters be selected without access to exhaustive positive-label validation annotations?
  5. How are VLPL and AEVLP adapted to MLCLL? Both were originally developed for settings with positive-label information. Please specify exactly what supervision and pseudo-label initialization they receive in your implementation.
  6. Please provide a more complete component ablation.
Reviewer_6tnnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In order to supplement some of the missing category labels in multi-label data, this paper proposes CoDi. It is a CLIP-based label complementation framework that first uses a frozen CLIP teacher model and is guided by exponential moving average (EMA) students to build positive evidence through global predictions, stochastic local and class-aware views. Then, it introduces reliability-aware false positive learning to stabilize false positive labels while retaining the observed complementary labels as negative supervision. The resulting active knowledge is distilled into independent student models, enabling CLIP-free reasoning. Experiments on four benchmark datasets demonstrate the effectiveness of CoDi.

Critical strengths

S1: Exquisitely designed modules.

S2: Sound Motivation.

S3: Clear Figures.

Critical weaknesses

W1: The presentation can be further improved.

This paper only has a method diagram and lacks a motivation diagram to highlight the difference between this method and other methods (for example, other methods that only have negative labels will leave residual uncertainty in the positive label set and may lead to assigning low probabilities to all categories).

W2: The method proposed in this paper is based on pre-trained vlm, which is clip. However, this brings a paradox:(1) If clip can correctly predict all positive labels, why don't we use clip directly;(2) If clip cannot accurately predict all positive labels (this is more likely to happen, because the classification effect of clip is affected by the clip pre-training data set. If the clip pre-training set does not have data for some labels of MLCLL, clip should not be able to supplement those labels), how does this method improve the accuracy of clip's pseudo-labels?

Point 2 actually converts the MLCLL problem into a traditional pseudo-label training problem. Although this method seems to use many techniques to optimize the pseudo-positive labels of clips, it seems to only compare other MLCLL methods and not the current sota method of pseudo-label training. And it lacks an overall design idea that is different from other pseudo-label training methods (this leads to insufficient insight provided, which is just a pile-up of other existing research),

Questions for authors

The question in W2.

Reviewer_nDH2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies multi-labeled complementary label learning, where each training image is only annotated with some classes that are absent. The authors argue that such labels leave the positive set undetermined and that a negative-only loss admits an all-zero solution, and propose CoDi to recover positives from a frozen CLIP ViT-B/16. CLIP scores are computed on the full image, two random crops and CAM-guided crops from an EMA student, calibrated per class by the global median, and fused. Pseudo-positives are then selected with per-class thresholds anchored on complementary-labeled samples and a reliability score, kept in an EMA bank, and used with five other losses to train a ResNet-50 student that runs without CLIP at test time. CoDi is best in 13 of 16 settings across VOC07, COCO14, NUS-WIDE and CUB200.

Critical strengths
  1. Using VLM priors for MLCLL is a reasonable idea, and the motivation that complementary labels carry no positive signal is easy to follow.
  2. Inference only needs the student network.
  3. Results are averaged over five seeds with significance tests and several metrics. The gains on NUS-WIDE and CUB200 hold at every ratio.
Critical weaknesses
  1. Theorem 1 and Proposition 1 are close to trivial and are not connected to the proposed method. Theorem 1 follows directly from the definition of conditional mutual information, since the information YY carries about itself given XX is just its conditional entropy. Proposition 1 holds by construction, because a loss defined only on negatives is always minimized by predicting zero. Neither result tells us anything about CoDi, and the paper does not show that existing MLCLL objectives such as the unbiased risk estimator of Gao et al. 2023 are negative-only in this sense. I would treat them as motivation rather than a contribution.

  2. Most components of CoDi come from prior CLIP-based multi-label work, and the part specific to MLCLL is small. Global and local CLIP views follow CDUL, classifier-guided crops follow CCD, the pseudo-positive loss is taken from AEVLP, and the EMA teacher, consistency loss and label bank are standard. What remains is the negative anchor computed from complementary-labeled samples, and this component is never ablated.

  3. The experiments do not show how much of the gain comes from the complementary labels rather than from CLIP. GDF and CTL use no CLIP, so the gap of over 50 points on VOC07 at ρ=0.1\rho = 0.1 mainly shows that CLIP knows the VOC classes. AEVLP, VLPL and GR+DAMP are single-positive methods, and the paper does not say how they are run without any positive label. Their numbers suggest they may not use the complementary labels at all. VLPL drops from 75.60 to 73.19 on VOC07 and from 13.56 to 12.47 on CUB200 as ρ\rho grows, and AEVLP stays flat on VOC07. To answer this question, the paper needs the zero-shot CLIP teacher, the annotation-free methods CDUL and CCD, ComRank, Gao et al. 2026b whose protocol the paper follows, and CoDi with ρ=0\rho = 0.

  4. CoDi is highly sensitive to the calibration scale wwidew_{\text{wide}}, and all hyperparameters are tuned with full labels. On VOC07, mAP jumps from about 10 to about 80 when wwidew_{\text{wide}} moves from 0.45 to 0.48, with similar cliffs on COCO14 and NUS-WIDE. This looks like the student falls into the all-zero solution of Proposition 1 when too few candidates pass the thresholds, so avoiding the failure the paper describes depends on one hand-picked value. Calling this a stable operating range is not accurate. The method also has more than twenty hyperparameters, including a dataset-specific ζ\zeta, and they are selected on a fully labeled validation split equal to 20% of the training data, which is over 16k images for COCO14. That sits uneasily with the low annotation cost motivation.

  5. CoDi does not win in the sparsest setting, which is the case that motivates the paper. At ρ=0.1\rho = 0.1 it is behind GR+DAMP on VOC07 and COCO14, and Table 4 shows losses on one error and coverage. At the other end, ρ=0.4\rho = 0.4 means about seven verified negatives per VOC07 image and around thirty per COCO14 image, which is not obviously cheaper than one positive label. A short discussion of annotation cost would help.

  6. The ablation is too limited to justify the design. Only three components are removed, all at ρ=0.1\rho = 0.1. The six losses, the reliability score, the bank and the negative anchor are not ablated, and training cost is not reported even though CLIP runs on several crops per image during training.

  7. The submission does not follow the official ICLR 2027 template. The page header reads "Under review as a conference paper at ICLR 2026", while the reviewer notice on the first page refers to ICLR 2027, so the authors appear to have reused last year's style file. The body font is also changed to a Computer Modern style font instead of the Times font defined by the template. Both go against the submission instructions, and I would ask the AC to check the formatting.

Questions for authors

How are AEVLP, VLPL and GR+DAMP run without any positive label, and do they use the complementary labels? Please also report the zero-shot CLIP teacher and CoDi with ρ=0\rho = 0, so it is clear how much of the gain comes from the complementary labels rather than from CLIP.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.