In-Context Pure Exploration in Continuous Decision Spaces

In this work, we develop a practical and reusable framework for active sequential testing, also termed pure exploration, with continuous recommendations. In this setting, a learner adaptively acquires information to identify an unknown hypothesis with as few queries as possible while satisfying a prescribed prior-averaged accuracy level. Existing methods are predominantly problem-specific, while learned approaches have been limited to finite recommendation spaces. We formulate this objective as Bayesian fixed-confidence pure exploration and characterize the Bellman structure of an ideal learner. Guided by this characterization, we introduce C-ICPE, which meta-trains sequential neural architectures over a task prior to jointly learn exploration, stopping, and recommendation. At inference, C-ICPE gathers evidence and returns a recommendation without parameter updates. We conclude by analyzing the method with experiments across localization, identification and value estimation problems.