Foveal Search: Learning Proxy Combinations for Certified Corpus Decisions

LLMs make it possible to draw nuanced conclusions from large collections of unstructured records. But using an LLM to judge every record is costly, especially when the goal is only an overall conclusion. Cheaper signals can be obtained for the entire collection, yet relying on them alone risks drawing the wrong conclusion. Foveal Search addresses this gap by acquiring a limited number of reference judgments and making each one serve two purposes: it contributes immediately to the decision and helps determine how to combine the cheaper signals for future judgments. The method guarantees a prescribed level of decision error even when those signals are poor. We also show when the information gained from learning their combination can outweigh the judgments spent learning it, with bounds that count every acquired judgment. Across evaluated settings, Foveal Search often uses fewer judgments than methods that ignore the cheaper signals while maintaining high observed correct completion. On held-out data, learning separate weights for the signals saves 3.0% and 4.6% of judgments compared with learning one weight for their average. Source Code: https://anonymous.4open.science/r/30228