- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SALC, an active learning method for multi-label prediction that accounts for dependencies between predicted labels. It uses the total-correlation gap between summed marginal uncertainty and joint uncertainty to motivate a pairwise mutual-information correction, estimated from MC-dropout predictions. The method weights this correction using label co-occurrence and semantic similarity, then combines it with labelwise BALD. The theoretical analysis describes the remaining approximation error, while experiments on MS-COCO, Pascal VOC, CheXpert, and synthetic data report improved label efficiency and examine the contribution of each component.
Critical strengths
-
The paper clearly explains how conditionally independent output heads can still produce dependent posterior predictions. The total-correlation identity in Eq. (2) gives a useful explanation of why summed labelwise uncertainty can overcount information.
-
The method estimates pairwise dependence from shared MC-dropout predictions without requiring a new prediction architecture (Eqs. (11)–(14)). This makes the approach practical to implement and test in existing multi-label active learning systems.
Critical weaknesses
-
The reported confidence intervals are inconsistent with the standard deviations in Table 1 under the stated paired t-test procedure (Section 4.1; Appendix C.5). For VOC mAP, the standard deviations are 0.36 and 0.54 across five seeds, so the standard deviation of paired differences must be at least 0.18. The 95% paired t-interval must therefore have a half-width of at least 0.223, but the reported interval [0.62, 0.78] has a half-width of only 0.08. This discrepancy requires corrected statistics and seed-level results before the empirical evidence can be trusted.
-
The ablation results do not consistently match the defined acquisition rule. When λ = 0, the score is independent of α (Eqs. (6), (8), and (14)), yet Figure 13 shows different results across α without explaining whether different random seeds were used. Its caption also describes mAP, while the colorbar shows AULC; interpreting it as AULC still leaves a mismatch with the no-correction result in Table 2 (74.8 versus 73.6 at α = 0.5). Appendix D.2 additionally reports different effects from removing the correction alone and removing both the correction and semantic information, although these should define the same score. These inconsistencies weaken the evidence for the proposed mechanism.
Questions for authors
-
Could the authors provide the five paired seed-level results and the calculation used for the confidence intervals in Section 4.1? How do these intervals agree with the standard deviations in Table 1, particularly for VOC mAP?
-
Does Figure 13 report mAP or AULC? Please explain why its λ = 0 results vary with α, whether the runs use matched seeds, and why the value at (λ, α) = (0, 0.5) differs from the no-correction result in Table 2.
-
In Appendix D.2, why does removing both the correction and semantic information produce a different result from removing the correction alone? Since semantic information enters only through the correction weights, were these experiments run with different implementations or protocols?