acceptodds
Withdrawn from ICLR 2027

Structured Active Learning under Correlated Outputs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-label active learning often scores examples by summing labelwise uncertainties. Even when a predictor uses conditionally independent output heads, averaging over uncertain model parameters can induce dependencies in its posterior-predictive distribution. The sum of marginal entropies then exceeds joint entropy by the total correlation. We use this identity to motivate Structured Active Learning under Correlated outputs (SALC), which subtracts a pairwise mutual-information correction from marginal predictive entropy and combines the result with labelwise Bayesian disagreement. With exact pairwise estimates and unit weights, the correction leaves only a higher-order interaction remainder. The practical rule estimates pairwise dependence from MC-dropout samples and uses empirical co-occurrence and frozen text-encoder similarities as reliability gates; these gates introduce an additional approximation error. On MS-COCO, Pascal VOC, and CheXpert, SALC improves the reported label-efficiency metrics over the tested uncertainty, diversity, and graph-based baselines. Controlled synthetic experiments and component ablations support the role of the pairwise correction, while identifying higher-order dependence and finite-sample estimation as limitations. The results motivate accounting for posterior-predictive output dependence when designing multi-label acquisition criteria.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
active learning, structured prediction, vision language models, multi label classification, label dependencies

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vJjcOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SALC, an active learning method for multi-label prediction that accounts for dependencies between predicted labels. It uses the total-correlation gap between summed marginal uncertainty and joint uncertainty to motivate a pairwise mutual-information correction, estimated from MC-dropout predictions. The method weights this correction using label co-occurrence and semantic similarity, then combines it with labelwise BALD. The theoretical analysis describes the remaining approximation error, while experiments on MS-COCO, Pascal VOC, CheXpert, and synthetic data report improved label efficiency and examine the contribution of each component.

Critical strengths
  1. The paper clearly explains how conditionally independent output heads can still produce dependent posterior predictions. The total-correlation identity in Eq. (2) gives a useful explanation of why summed labelwise uncertainty can overcount information.

  2. The method estimates pairwise dependence from shared MC-dropout predictions without requiring a new prediction architecture (Eqs. (11)–(14)). This makes the approach practical to implement and test in existing multi-label active learning systems.

Critical weaknesses
  1. The reported confidence intervals are inconsistent with the standard deviations in Table 1 under the stated paired t-test procedure (Section 4.1; Appendix C.5). For VOC mAP, the standard deviations are 0.36 and 0.54 across five seeds, so the standard deviation of paired differences must be at least 0.18. The 95% paired t-interval must therefore have a half-width of at least 0.223, but the reported interval [0.62, 0.78] has a half-width of only 0.08. This discrepancy requires corrected statistics and seed-level results before the empirical evidence can be trusted.

  2. The ablation results do not consistently match the defined acquisition rule. When λ = 0, the score is independent of α (Eqs. (6), (8), and (14)), yet Figure 13 shows different results across α without explaining whether different random seeds were used. Its caption also describes mAP, while the colorbar shows AULC; interpreting it as AULC still leaves a mismatch with the no-correction result in Table 2 (74.8 versus 73.6 at α = 0.5). Appendix D.2 additionally reports different effects from removing the correction alone and removing both the correction and semantic information, although these should define the same score. These inconsistencies weaken the evidence for the proposed mechanism.

Questions for authors
  1. Could the authors provide the five paired seed-level results and the calculation used for the confidence intervals in Section 4.1? How do these intervals agree with the standard deviations in Table 1, particularly for VOC mAP?

  2. Does Figure 13 report mAP or AULC? Please explain why its λ = 0 results vary with α, whether the runs use matched seeds, and why the value at (λ, α) = (0, 0.5) differs from the no-correction result in Table 2.

  3. In Appendix D.2, why does removing both the correction and semantic information produce a different result from removing the correction alone? Since semantic information enters only through the correction weights, were these experiments run with different implementations or protocols?

Reviewer_DwTsOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a challenge in multi-label active learning where acquisition scores just sum over the predictive uncertainties of individual labels without accounting for dependencies among them. The authors propose to estimate pairwise label mutual information using multiple Monte Carlo dropout passes. These estimated pairwise label mutual information are then weighted by cosine similarity between label name embeddings. The estimated joint label uncertainty is estimated by subtracting the total sum of the marginal label predicted entropy from the weighted pairwise mutual information. This corrected entropy is then combined with label-wise Bayesian disagreement to form the acquisition score. Experiments on MS-COCO, Pascal VOC, and CheXpert show improvements in label efficiency over the evaluated baselines, including Bayesian and graph-based active learning methods.

Critical strengths

(S1) Clear motivation -- The paper addresses a clear gap in multi-label active learning: capturing the multi-label correlations.

(S2) Meaningful experimental evaluation -- The experiments evaluate label efficiency, use synthetic data to assess the accuracy of the pairwise mutual-information approximation against exact joint entropy, and include ablation studies examining the contribution of the finite-sample reliability gates used to weight the pairwise label mutual information.

(S3) Clear acknowledgment of limitations -- The paper explicitly acknowledges that the practical reliability gates can introduce bias that is irrelevant to the actual active learning context and that the pairwise approximation may deteriorate when higher-order posterior-predictive interactions are substantial.

Critical weaknesses

(W1) Potentially unfair comparisons -- The proposed approach incorporates external semantic information for multiple labels, which actually applies additional data modality for method improvement, where baseline methods only use multi-labels with numerical encoding.

(W2) Presentation -- The presentation of the method lacks clarity. Several equations and figures are referenced before they are adequately introduced or explained. The theoretical exposition is difficult to follow, and its connection to the practical implementation is insufficiently developed. In addition, Figure 1 would benefit from a more detailed explanation of its components and their interactions.

(W3) Novelty -- The methodological novelty appears limited. The combination of pairwise label-dependence estimation through Monte Carlo dropout and weighting based on rescaled cosine similarities between frozen label-name embeddings does not, by itself, establish a sufficiently substantial contribution.

Questions for authors

None

Reviewer_p7XqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an active learning method for multi-label classification. The proposed method estimates pairwise mutual information from MC-dropout predictions and subtracts weighted redundancy correction from marginal predictive entropy. The weights combine empirical label co-occurrence and semantic similarity, and the BALD term encourages epistemic exploration. The paper reports experiments on MS-COCO, Pascal VOC, and CheXpert that show improvements over baselines.

Critical strengths

1.Summing marginal uncertainties when labels are correlated is a meaningful issue in multi-label active learning, and the proposed correction is intuitive. 2.The method estimates pairwise dependencies without modeling the full joint label distribution. Overall, the design is simple and efficient. Also, the experiments show consistent improvement over baselines.

Critical weaknesses

1.The proposed method is closely related to Large BatchBALD (Rubashevskii et al., 2023), which also approximates the total correlation with pairwise mutual information. I encourage the authors to clarify the differences between SALC and Large BatchBALD and include Large BatchBALD as an experimental baseline. 2.At a glance, in Figure 2, the proposed method seems to maintain a gap between 5% to 50% consistently. Is this because the initial 5% labeled set is different? 3.Eqs. (1)–(2) show correcting redundancy in marginal information gain, but the final score in Eq. (14) applies the correction only to predictive entropy. It is unclear why this combination is used over directly correcting joint information gain. 4.Table 1 is incomplete. The Class-balanced entropy row contains XX.

Questions for authors

See weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.