Structured Active Learning under Correlated Outputs

Multi-label active learning often scores examples by summing labelwise uncertainties. Even when a predictor uses conditionally independent output heads, averaging over uncertain model parameters can induce dependencies in its posterior-predictive distribution. The sum of marginal entropies then exceeds joint entropy by the total correlation. We use this identity to motivate Structured Active Learning under Correlated outputs (SALC), which subtracts a pairwise mutual-information correction from marginal predictive entropy and combines the result with labelwise Bayesian disagreement. With exact pairwise estimates and unit weights, the correction leaves only a higher-order interaction remainder. The practical rule estimates pairwise dependence from MC-dropout samples and uses empirical co-occurrence and frozen text-encoder similarities as reliability gates; these gates introduce an additional approximation error. On MS-COCO, Pascal VOC, and CheXpert, SALC improves the reported label-efficiency metrics over the tested uncertainty, diversity, and graph-based baselines. Controlled synthetic experiments and component ablations support the role of the pairwise correction, while identifying higher-order dependence and finite-sample estimation as limitations. The results motivate accounting for posterior-predictive output dependence when designing multi-label acquisition criteria.