- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces C-ICPE, a meta-learning framework for Bayesian fixed-confidence pure exploration with continuous recommendation spaces. It jointly learn exploration, stopping, and inference policies through sequential neural architectures. The theoretical contributions include a Bellman characterization, zero-duality-gap results, and Bayesian -correctness under regularity conditions. Experiments on synthetic benchmarks and a geochemical exploration task demonstrate promising accuracy and sample efficiency.
Critical strengths
-
Meaningful theoretical and methodological contributions. the extension of Bayesian fixed-confidence pure exploration to continuous recommendations is significant. The Bellman characterization and correctness analysis under a weaker local closedness condition are technically interesting.
-
Broad and promising empirical evaluation. The experiments cover several distinct continuous exploration problems, including decoupled query / recommendation spaces. Results demonstrate competitive sample efficiency and successful adaptation across different tasks.
Critical weaknesses
Limitations in empirical validation and generality. Most comparisons are against fixed-budget optimization methods rather than directly comparable fixed-confidence algorithms; the specialized Lazy Track-and-Stop baseline is more sample-efficient on the linear benchmark. The geochemical experiment uses GP-fitted surrogate functions rather than direct sequential field measurements. Furthermore, reliance on task-prior simulators and known optimal targets limits applicability, while robustness under substantial prior shift remains uncertain.
Questions for authors
Could the authors strengthen the comparison with fixed confidence methods, particularly BO equipped with probabilistic stopping rules, to better isolate the benefits of learned exploration?
5 comments
Response (2/2)
"Most comparisons are against fixed-budget optimization methods rather than directly comparable fixed-confidence algorithms; [...]"
Most comparisons are against fixed-budget methods because no widely available fixed-confidence method handles these task families, and providing one is the main contribution of the paper. The few that exist for continuous problems each rely on one structure, such as Jedra and Proutiere (2020), our reference on the sphere.
The comparison with fixed-budget methods is: do these methods reach the target accuracy given the same average number of queries of C-ICPE? If so, they would be themselves fixed-confidence learners (Sec. 5, the new paragraph "Fixed budget and fixed confidence"). We have also added probabilistic bisection for binary search (see our answer to reviewer CqnG) and Wilson's stopping rule (Wilson, 2024) to strenghten the comparison
C-ICPE is a general practical method that can cover these tasks, and to our knowledge it is the only fixed-confidence method that does: its purpose is a practical method for task families where deriving dedicated exploration and stopping rules is difficult (Sec. 6).
"[...] the specialized Lazy Track-and-Stop baseline is more sample-efficient on the linear benchmark."
We agree, and the paper says so (Sec. 5.1). LT&S exploits the known linear observation model and a likelihood-ratio stopping rule. It is however interesting to note that, when using the same round-robin exploration strategy, the learned critic/inference models match the performance of LT&S, stopping after 14.8 queries at with accuracy about 0.96. This seems to indicate that the gap lies mainly in the exploration strategy.
"The geochemical experiment uses GP-fitted surrogate functions rather than direct sequential field measurements." "Furthermore, reliance on task-prior simulators [...] limits applicability" "[...] and known optimal targets limits applicability,"
We agree, and the last two points are shared with the in-context methods C-ICPE builds on:
- The geochem tasks are surrogates built from real USGS data. Generally, field measurements are costly to obtain, and a field campaign needs dedicated funding, which in turn requires the kind of evidence this study provides.
- C-ICPE relies on a simulator of the task prior, as do DPT (Lee et al., 2023), ICPE and Deep Adaptive Design, which also learn from simulated tasks before deployment (Sec. 4, App. B); at deployment it only uses its own queries and observations.
- C-ICPE needs the target of each task in simulation, as DPT and ICPE do: DPT is trained on the optimal action of each task (Lee et al., 2023), and ICPE on its true hypothesis (Russo et al., 2026). The target is used only for training and certification, never at deployment, and reducing this dependence on supervised targets is one of the directions named in Sec. 6.
"[...] while robustness under substantial prior shift remains uncertain."
App. E.4 tests this directly: frozen models trained under a uniform prior are deployed under Beta priors with , and the paper reports where accuracy holds and where it drops:
- on geochem, accuracy stays at about 0.90-0.93 over the whole grid (App. E.5);
- on binary search, TS, TTPS and the mixture stay above 0.85, while TD3 drops to 0.370-0.487 at the most skewed priors;
- on GP max-value, TD3 drops to 0.770 at the most skewed prior, ;
- on Ackley, accuracy degrades to 0.521-0.656 at the most skewed prior.
App. C.10 bounds the accuracy and the stopping time of a frozen learner in terms of the shift between the training and deployment priors, through their density ratio, chi-square divergence or total variation (Props. 11-12).
We thank the reviewer again for the positive evaluation and for these suggestions, which strengthen the comparison with fixed-confidence methods.
Dear Reviewer, Thank you again for your feedback and time spent reviewing our paper. We would like to know whether our response adequately addresses your concerns or if any further clarification is needed, so we can properly address it before the end of the rebuttal period.
Thank you very much again for your time! Best
Response (1/2)
We thank the reviewer for the positive assessment, and in particular for finding the continuous extension significant, and the analysis technically interesting. Below, we address your concerns. We have uploaded a revised version of the paper, where the changes are in blue, and each answer points to the changes it makes; this reply cites the revised version.
We thank the reviewer for this suggestion. We implemented Wilson's stopping rule, and the revised version adds BO with this probabilistic stopping rule on Ackley, GP max-value and geochem, with GP-UCB and GP-logEI. Furthermore, we also added probabilistic bisection to binary search (Horstein, 1963).
Wilson stopping rule. For each function g sampled from the GP posterior, Wilson's rule checks if the recommendation x^ has a small value regret, maxag(a)−g(x^)≤ϵ. This is the loss of geochem, so on geochem we use the rule as published. On Ackley and GP max-value the loss is different, and a small value regret does not imply a small loss (new, App. C.12, Prop. 14). So on these two tasks we change the check. On Ackley we check ∥x^−xg⋆∥2≤ϵ, with xg⋆ the optimum of g. On GP max-value the recommendation x^ is an estimate of the max value, and we check ∣x^−maxag(a)∣≤ϵ.
In general, this rule misses the target accuracy in every configuration (ranges are min-max over configurations and both acquisitions gp-ucb & gp-logEI; new Table 2 and App. E):
We find that the GP posterior is overconfident on geochem and GP max-value. The rule computes its probability with a GP whose settings, the lengthscales (how fast the function changes) and the scale (how much it varies), are fitted to the first observations and then used as if they were exact. With so few observations these settings can be wrong, and the GP is then too sure about the regions it has not observed. We believe this is why the rule stops too early. On geochem it stops only when it is at least 95% sure, but it is correct on only 35-41% of the tasks. Wilson (2024) also reports this problem when the settings are fitted (88-92% correct against a 95% target).
We also tried to calibrate the threshold used by the stopping rule. A natural fix is to learn the threshold from data, as C-ICPE learns its stopping rule from tasks of the prior. On geochem we stop at the first t at which pt, the GP's probability that its recommendation is correct, reaches λ, and choose the smallest λ that reaches C-ICPE's training target of 0.918 correct on 200 training tasks (new, App. E.5). No λ∈[0,1] works: the GP often gives pt=1 to a wrong recommendation, so even λ=1 is correct on only 38-50% of 200 held-out tasks.
In general, this points out that there is indeed a benefit in learning when to stop, and that the exploration strategy also matters where stopping cannot explain the gap: on Ackley no black-box baseline reaches the target even with 100 queries, and the C-ICPE-Uniform ablation, which keeps C-ICPE's stopping rule and changes only the queries, fails on binary search and Ackley at ϵ=0.1. On GP max-value, whose prior is itself a GP, the fixed-budget GP-logEI and TPE do reach the target at the budget of C-ICPE-TD3 (Sec. 5.1; Table 20, Table 15 in the submission).
Probabilistic bisection. The revised version adds probabilistic bisection (Horstein, 1963) as an exact-posterior, fixed-confidence reference for binary search (rows PBA-median and PBA-TS of the binary-search tables, App. E.3.1). Recall that this is a noisy environment, where each query returns one corrupted sign per coordinate (Eq. 52). Bisection knows the probability p with which each sign is flipped, and keeps the exact posterior and stops once its posterior success reaches 1−δ, therefore this is a reference that exploits the structure of the problem. We test two exploration approaches: (i) PBA-median, which queries the median of the posterior, while (ii) PBA-TS queries a sample from this posterior (which is the exploration strategy of C-ICPE-TS applied to the exact posterior).
-With median queries, it meets the accuracy target with fewer queries in every configuration (C-ICPE-TS needs 16-48% more). -With Thompson queries, the exploration strategy of C-ICPE-TS, the gap mostly disappears: C-ICPE-TS uses between 8% fewer and 3% more queries for d≤10, and up to 21% more at d=20 (23.0 against 19.0).
The revised version reports this in Sec. 5.1 and App. E.3.1