Cost-aware Active Local Learning

Active learning (AL) reduces labeling effort by selecting informative queries, but the labeling costs usually vary substantially in real applications. Existing cost-aware active learning methods usually skip costly points, discount acquisition scores by cost, or query from a cheaper but noisy oracle. We study a complementary setting in which the label of an expensive point can be estimated from labels of cheaper points. We formalize this setting as cost-aware active local learning and develop two solutions. After a base learner selects a target, the Lipschitz solution compares the cost of local labels with a direct query, while the disagreement solution queries cheap inputs until the surviving hypotheses agree on its response. For disagreement search, our theory characterizes which labels suffice and bounds total expected cost under realizability and a condition relating agreement on candidates to accuracy at the selected point, including unsuccessful exploration and direct fallback. We conducted experiments on 18 molecular datasets. The results show that adding the proposed method to existing AL methods improved molecular predictions over the evaluated labeling budgets in 289 of 351 comparisons (82%) against the same methods without adding our approach.