Beyond the Initial Ranking: How Feedback Shapes Active Search

Active search aims to discover as many targets as possible within a limited measurement budget. Initial ranking quality captures only part of this task: observations change future choices, and planning acts through those changes. We separate initial predictions, feedback updates, and acquisition rules to evaluate their distinct roles in discovery. In a coherent probability model, we characterize when observing a fixed query has strictly positive value for a subsequent batch selection: no single continuation set is optimal across all possible outcomes. An exact correlated-group model expresses this value through dependence strength, selection margins, and the remaining capacity to exchange candidates. We then investigate these distinctions in retrospective protein-variant searches on Domainome and ProteinGym, with hits defined by within-pool top-5% fitness. In nested comparisons holding initial predictions and greedy acquisition fixed, adding sequence-position feedback improves discovery by approximately 0.4–1.0 hits at budgets of 12 and 24 measurements. A separate common-prior evaluation extends positive feedback gains to ridge and Gaussian-process updates on both corpora. Under its matched selected configurations, however, neither position-planning gains nor update-by-acquisition interactions pass multiplicity correction. Terminal-set comparisons distinguish changing query order, exchanging candidates, and improving yield. Together, these comparisons ask which relations carry useful feedback, whether candidate exchanges improve yield, and when planning adds value. Ranking quality, feedback utility, and planning value are distinct evaluation targets: a useful update can improve discovery without guaranteeing an additional benefit from planning.