acceptodds
Under review as a conference paper at ICLR 2027

Multi-stage Bayesian Optimisation under Unknown Stopping Constraints

33 views

Abstract

We consider Bayesian optimisation over multi-stage decision processes with unknown stage-wise transitions and stopping constraints, where a single violation aborts the current run and prevents all downstream observations. Such problems are common in sequential experimental and manufacturing settings, where failed runs are acceptable during learning but remain costly. To explore uncertain but potentially high-value trajectories under this tolerance while limiting the cumulative number of aborted runs, we formulate the problem as a feasibility-indicated Markov decision process and propose \fiovi, a value iteration method for optimistic lookahead planning that uses an annealed feasibility margin to provide fine-grained control over the exploration-failure trade off. Under standard RKHS regularity and feasibility assumptions, we derive sublinear rates for both cumulative abort count and cumulative value regret. We demonstrate competitive performance against existing baselines on synthetic benchmarks and real-world problems.

Primary area:
optimization
Keywords:
Bayesian optimization, Multi-stage Bayesian optimization, Constrained Bayesian optimization, Gaussian processes

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 4.00 over 3 reviews, above 99% of the others.
Reviewer_PH2TOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies multi-stage optimization where unknown constraints can terminate an experiment and interrupt subsequent observations. It models unknown dynamics and constraints with GPs, then performs optimistic value iteration using a soft feasibility weight computed from constraint UCBs. An annealed threshold controls this weight, and the paper derives bounds on cumulative aborted episodes and realized cumulative regret for an idealized planner. Experiments compare this method, FI-O-VI, with a probability-of-feasibility variant and adapted BO baselines on eight synthetic cascades and four application-inspired simulation problems.

Critical strengths

S1. A meaningful problem formulation with joint theoretical guarantees. The paper formalizes multi-stage optimization with unknown transitions and unknown stopping constraints, whose violations censor the next transition and all downstream observations. Its FIMDP formulation enables a joint analysis of optimization progress and failures that restrict further learning. For the idealized planner, the paper establishes a pathwise cumulative-abort bound and a high-probability value-regret bound under noiseless constraint observations and the stated RKHS and feasibility assumptions, with the regret analysis additionally requiring nonzero Gaussian transition noise. These finite-time bounds yield simultaneous sublinear rates when topt=o(T)t_{\mathrm{opt}}=o(T) and the remaining bound terms are sublinear.

S2. An informative separation between plug-in and optimistic feasibility. Theorem 3 presents a single-stage, multiple-constraint construction in which FI-PoF-VI can incur linear regret while FI-O-VI achieves sublinear regret on the same instance under the required assumptions. The mechanism is concrete: posterior feasibility weighting suppresses uncertain alternatives and traps the plug-in variant at a feasible but suboptimal action. Figure 9 illustrates analogous behavior in the multi-stage GMM problem, supported by the seed-level results in Figure 7. This combination of theoretical separation and empirical diagnosis clarifies when optimistic feasibility can improve exploration.

Critical weaknesses

W1. The theoretical guarantees leave key implementation choices unresolved. The implementation applies the annealed threshold only to the executed stage, while future-stage backups use a fixed threshold of 0.01. This can downweight or exclude strictly feasible optimal continuations with small margins. With valid constraint upper confidence bounds, ξopt≥0.01\xi_{\mathrm{opt}}\ge0.01 suffices to resolve this particular obstruction to Lemma 8's optimism argument; it is not necessary and does not establish guarantees for the entire implementation. Separately, choosing ttr=⌊T/4⌋t_{\mathrm{tr}}=\lfloor T/4\rfloor can yield topt=Θ(T)t_{\mathrm{opt}}=\Theta(T). The finite-time bound for the corresponding idealized planner remains valid, but need not imply sublinear regret; this does not establish linear regret in practice. The authors should clarify these conditions and provide a targeted ablation applying threshold annealing at every planning stage, with a schedule satisfying the stated sublinearity requirements.

W2. Additional aborts bring no clear return gains in the application tasks. Tables 3 and 5 report approximately 15×, 10×, 1.5×, and 3.4× the median abort counts for FI-O-VI compared with FI-PoF-VI on DNS3, DNS4, QT, and NQT, respectively, while returns remain similar. QT shows a small increase in median return, but the reported results do not establish a clear return benefit that compensates for the additional aborts. Although synthetic experiments demonstrate benefits from optimistic feasibility, its practical return–abort tradeoff remains insufficiently established in the applications. A paired analysis of return and abort differences using the existing runs would help determine whether the extra exploration produces meaningful gains.

Questions for authors

Q1. What justifies using a fixed threshold of 0.01 in future-stage backups when the assumptions permit smaller optimal feasibility margins? Under valid constraint UCBs, ξopt≥0.01\xi_{\mathrm{opt}}\ge 0.01 suffices to resolve this particular optimism issue. Is this additional condition intended, and does applying threshold annealing at every planning stage materially change the results?

Q2. With ttr=⌊T/4⌋t_{\mathrm{tr}}=\lfloor T/4\rfloor, toptt_{\mathrm{opt}} can be Θ(T)\Theta(T), so the stated bound need not imply sublinear regret. Can you clarify the guarantee claimed for this configuration and whether the empirical conclusions persist with a TT-independent choice of ttrt_{\mathrm{tr}}, or more generally one satisfying ttr=o(T)t_{\mathrm{tr}}=o(T)?

Q3. FI-O-VI incurs approximately 15×, 10×, 1.5×, and 3.4× the median abort counts of FI-PoF-VI on DNS3, DNS4, QT, and NQT, respectively, with similar returns. What do paired comparisons across the existing seeds show about the benefit obtained from these additional aborts? Please report uncertainty for the return and abort differences, particularly whether QT’s small return increase is consistent across seeds.

3 comments

We thank you for the positive reviews, we are glad that you found the FIMDP formulation meaningful and consider the theoretical analysis, the theoretical demonstration of FI-PoF-VI in Theorem 3 as well as the empirical illustration through the GMM case as strengths. Together with the updated manuscript, we address the weaknesses and questions below.

  • W1: Please see our responses to Q1 (lookahead threshold, including the ablation with annealing at every planning stage) and Q2 (choice of ttrt_{\mathrm{tr}}).
  • W2: Please see the shared response, point 3.
  • Q1: We thank you for these detailed questions.
    • As stated in the shared response, point 2, as long as ξopt≥0.01\xi_{\mathrm{opt}}\ge 0.01 there is no optimality issue, and we now state this condition explicitly in the updated manuscript.
    • Applying the annealed threshold at every planning stage makes the planner more conservative before the transition episode (t≤ttrt \leq t_{tr}) since the future action space has been reduced by this conservative ξt\xi_t; this is why we use the fixed lookahead threshold, for which Theorems 1 and 2 still apply.
    • We have also run this variant: applying the annealed threshold at every planning stage does not materially change the results (Appendix H.4.5, Figures 13 and 14).
  • Q2:
    • We set ttr=⌊T/4⌋t_{\mathrm{tr}}=\lfloor T/4\rfloor in the experiments only so as not to introduce a further hyperparameter for each problem, which can be treated as a constant rather than as theoretically linked with TT (e.g., for GMM ttr=50t_{\mathrm{tr}}=50 when T=200T=200, and one can keep ttr=50t_{\mathrm{tr}}=50 when T=2000T=2000). We have added a footnote to clarify the setting.
    • Empirically, the sensitivity study on ttrt_{\mathrm{tr}} (Appendix H.4.5, Figures 11 and 12) shows that the conclusions do not hinge on this choice: with ttr=T/8t_{\mathrm{tr}}=T/8 or T/2T/2 the final rewards are similar, and an earlier transition mainly increases the aborts.
  • Q3: The paired per-seed comparison over the existing 2020 seeds is given below (FI-O-VI minus FI-PoF-VI; median and 95%95\% bootstrap interval). The reward increase on QT is small, and positive in 1616 of 2020 seeds; on the other three problems the additional aborts do not yield a higher reward. We discuss when this is to be expected in the shared response, point 3.
reward differenceabort differenceseeds with higher reward
DNS3−0.002-0.002 [−0.008, +0.002][-0.008,\,+0.002]+41+41 [+37, +46][+37,\,+46]8/208/20
DNS4−0.011-0.011 [−0.023, −0.004][-0.023,\,-0.004]+65+65 [+63, +68][+63,\,+68]5/205/20
QT+0.008+0.008 [+0.001, +0.011][+0.001,\,+0.011]+7+7 [+4, +12][+4,\,+12]16/2016/20
NQT−0.004-0.004 [−0.008, −0.001][-0.008,\,-0.001]+41+41 [+34, +51][+34,\,+51]5/205/20

We thank you again for the thorough review, in particular the suggested ablation and paired analysis. We are happy to address any further comments that you may have.

Hi PH2T, friendly reminder to please read the authors' response, acknowledge that you've done so, whether that changes your assessment of the paper, and if you have additional questions. Thanks!

I thank the authors for the responses. My concerns have been fully resolved and thus I raised the score to accept.

Reviewer_eBzuOpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines how to learn to operate a multi-stage process, where a single infeasible decision can stop the entire experimental sequence, resulting in the loss of all potential information from subsequent stages. The algorithm have to simultaneously identify a high-quality process and learn to avoid preventable failures.

Most notably, the authors address the uncertainty surrounding feasibility: being overly cautious prevents the algorithm from exploring promising yet untested options, while being overly optimistic frequently leads to failure.

The proposed method, FI-O-VI, optimizes a multi-stage process, where violating a constraint at any stage terminates the entire trial and results in the loss of data for subsequent steps. The algorithm employs a Gaussian process to learn the system's dynamics and the conditions required for each stage to proceed. It then uses dynamic programming to select actions based on the expected utility of the entire process, while simultaneously encouraging the exploration of potentially promising options for which data is currently lacking. A key feature is its optimistic assessment of feasibility: the algorithm does not prematurely discard an option simply because its feasibility remains uncertain. The level of caution is adjusted over time, and data from both successful and failed trials are used to refine future decision-making. The objective is to derive a high-quality policy while minimizing the rate of trial failures during the learning phase.

Critical strengths
  • This problem addresses a practical scenario: handling multi-stage processes where a failure results in the loss of data for subsequent steps.
  • The proposed method combines a Gaussian Process (GP) model with optimistic planning; it provides theoretical guarantees for controlling both cumulative regret and the number of failures, under the stated assumptions.
  • The analysis highlights the limitations of using standard feasibility probabilities, while the empirical evaluation, which includes comparisons against baselines and a sensitivity analysis of parameters, demonstrates the method's performance.
Critical weaknesses
  • The theory does not fully cover implementation details, particularly regarding fixed lookahead horizons and value function approximation.
  • Empirical benefits of the optimism mechanism are inconsistent: in some problems, there are significantly more failures, yet the rewards show no improvement over FI-PoF-VI.
  • Assumptions regarding precise observation constraints and a positive lower bound on the performance of the optimal policy limit the scope of applicability.
Questions for authors
  • According to Appendix D.2 and H.2, future stages in the lookahead process employ a fixed feasibility threshold of 0.01. Does the no-regret guarantee provided by Theorem 2 hold for this implementation? What conditions are required to ensure that a fixed threshold does not diminish or negate the value of optimal actions that possess a small feasibility margin?

  • In DNS4, FI-O-VI exhibits a median of 72 constraint violations, compared to 7 for FI-PoF-VI, with corresponding rewards of 0.778 and 0.790. Could the authors explain this trade-off and identify the conditions under which the optimism-based mechanism yields sufficient benefits to outweigh the increased cost of constraint violations?

  • Which benchmarks were designed to satisfy the feasibility assumptions outlined in Section 2.2- specifically, the existence of an optimal policy with a uniformly positive feasibility margin? Beyond the state-transition noise already considered, how sensitive is the method to noise in the measurement of constraint values?

2 comments

We thank you for the accurate summary, we are glad that you found this problem practically relevant, the analysis theoretically sound, and the analysis of the limitations of the standard probability of feasibility as well as the empirical evaluation useful. Together with the updated manuscript, we address the weaknesses and questions below.

  • W1: If we understand you correctly, "fixed lookahead horizons" refers either to the number of lookahead stages or to the fixed feasibility threshold used at the lookahead stages (as in your Q1); we address both readings.
    • Lookahead steps. Our algorithm always uses the full remaining horizon, i.e., planning at stage hh incorporates all H−hH-h remaining stages (see Algorithm 1), which is consistent with our theory.
    • Lookahead threshold. It is used only at the lookahead stages of the Bellman recursion (see the shared response, point 2).
    • Value function approximation. We have added an analysis in the updated manuscript (Appendix F.4); see the shared response, point 1, for a summary.
  • W2: Please see the shared response, point 3.
  • W3:
    • Exact constraint observations. After Cascade BO (which does not assume process noise in its main analysis), we provide a theoretical analysis for MSBO under stopping constraints in the presence of process noise (Torresi et al., although considering process noise, only propose a heuristic approach). Although theoretically plausible, observation noise in the constraint is in general not sensible in our scenario, as in practice the abort is decided from the constraint observation itself. We nevertheless find this setting theoretically interesting and have noted it as future work in the updated manuscript (Section 6).
    • Positive margin of the optimal policy. If we understand you correctly, you find the assumption that the optimal policy does not lie on the constraint boundary restrictive. It is needed in our analysis to maintain the value regret guarantee against v⋆v^\star under our ξt\xi_t schedule. In practice the margin can be very small without affecting the empirical performance, e.g., 0.0030.003 and 0.0090.009 on SynCompose-1 and SynCompose-2 (see our response to Q3).
  • Q1: Please see the shared response, point 2 (the fixed feasibility threshold in lookahead-stage planning): Theorem 1 is unaffected, Theorem 2 holds as stated when 0.01≤ξopt0.01\le\xi_{\mathrm{opt}}, and the ablation over the threshold value reported there shows the choice is not critical.
  • Q2: Please see the shared response, point 3. The additional aborts on DNS4 are the cost of the exploration that lets FI-O-VI not rule out the optimal policy. We conjecture that this exploration pays off when the value function is multi-modal and no initial trajectory is close to those of the optimal policy, where FI-PoF-VI can be trapped at a local optimum (e.g., GMM and GP prior); otherwise FI-PoF-VI remains competitive with fewer aborts (end of Section 5 and Appendix H.4.6 of the updated manuscript).
  • Q3: Note that since these are continuous state-action problems, we are in general able neither to solve them to exact optimality nor to design them so that they satisfy all the assumptions. We therefore check the assumptions approximately: similar to how we compute the approximate optimal values, for the eight synthetic problems (where the true process is known) we solve the Bellman optimality equation on a dense grid and read off, for the optimal policy, the margin ξopth\xi^{h}_{\mathrm{opt}} it keeps at each constrained stage hh and the overall margin ξopt\xi_{\mathrm{opt}} (Assumption 2), which is computed jointly over the stages and can therefore be slightly below the smallest stage-wise value; a negative value means that the optimal trajectory can reach a state with no feasible action (a dead end, violating Assumption 1). The margins are reported in Table 3 of the updated manuscript and summarised below.
    • Positive margin: MVN and the three SynCompose problems have a strictly positive margin (SynCompose-1 and SynCompose-2 only barely).
    • Optimum on the constraint boundary: BMR and GMM have ξopt=0\xi_{\mathrm{opt}}=0 at one or two stages.
    • Dead ends: GP prior and BMRCRSHMM contain dead-end states.
    • Real-life problems: since no closed-form process is available; the best trajectories found keep margins 0.220.22 (DNS3), 0.0380.038 (DNS4), 0.0100.010 (QT) and 0.0420.042 (NQT).
    • Constraint measurement noise: please see our response to W3.

We thank you again for the careful reading and the detailed questions. We are happy to address any further comments that you may have.

Reviewer_eBzuOpenReview ↗

After Rebuttal

I thank the authors for the substantive responses in such limited time frame. Their answer really clear out my initial judgments about the gaps between theory and empirical practise of the paper. The new theories added at appendix F.4 for Value function approximation and the ablations studies do address my original questions. I will raise my scores to Accept for this paper.

Reviewer_YDp9OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-stage Bayesian optimization with unknown transitions and stopping constraints, where a single violation aborts the run and censors all later observations. It formulates the problem as a feasibility-indicated MDP and proposes FI-O-VI, which combines Gaussian process models, optimistic value iteration, and an annealed feasibility threshold. The paper proves sublinear bounds on cumulative aborts and regret, and evaluates the method on synthetic and application-driven benchmarks.

Critical strengths
  • The problem is well motivated. A constraint violation terminates the process and prevents downstream observations.
  • The paper establishes sublinear bounds on both cumulative aborts and value regret under RKHS and feasibility assumptions.
  • The experiments cover eight synthetic benchmarks and two application settings. FI-O-VI and FI-PoF-VI generally outperform several baselines.
Critical weaknesses
  • Table 3 shows that FI-O-VI achieves slightly lower rewards than FI-PoF-VI on DNS3 (0.688 vs. 0.689) and DNS4 (0.778 vs. 0.790), while causing substantially more failures (44 vs. 3 and 72 vs. 7). This raises questions about the practical benefit of its optimistic exploration strategy.

  • The theoretical guarantees assume specific confidence parameters and Bellman updates. The implementation uses approximate value functions, empirical parameters, and fallback action selection. The effects of these differences are not analyzed.

  • Cascade BO (Kusakawa et al., 2022) addresses multi-stage optimization, while failure-aware GP optimization (Iwazaki et al., 2023) provides regret guarantees under observation failures. The paper could better distinguish its theoretical contributions from these approaches.

  • The Bellman update for post-transition constraints separates feasibility weighting from expected future value. It is unclear how dependence between next-state feasibility and future reward is handled.

  • Line 15 of Algorithm 1 appears to append observations only after successful episodes. It is unclear whether data from aborted episodes are retained. If these observations are discarded, the proof of Theorem 1 may no longer apply, since it relies on updating the constraint posterior after each evaluation.

Questions for authors
  • Why does FI-O-VI incur more failures than FI-PoF-VI on DNS3 and DNS4 without improving rewards?
  • How do the value-function approximation and fallback action selection affect the theoretical guarantees?
  • What are the main technical differences from combining Cascade BO with existing failure-aware GP-UCB methods?
  • How does the post-transition Bellman update account for dependence between feasibility and next-state value?
  • Does Algorithm 1 retain observations from aborted episodes?
2 comments

We thank you for the positive reviews, we are glad that you found this problem well motivated, the analysis sound and the two proposed approaches empirically competitive. Together with the updated manuscript, we address the weaknesses and questions below.

  • W1/Q1: Please see the shared response, point 3. In short, FI-O-VI is by design more explorative so as not to rule out the optimal policy, which costs additional aborts. On DNS3 and DNS4 this exploration does not translate into a higher reward within the given budget, as FI-PoF-VI already attains a comparable reward there. We discuss when FI-PoF-VI remains competitive at the end of Section 5 of the updated manuscript.
  • W2/Q2:
    • Value approximation. Please see the shared response, point 1.
    • Fallback selection. We note that both fallback steps are empirically motivated and do not affect Theorem 1 or 2. More specifically, when t>treacht > t_{reach}, there exists an action with p+>0p^+ >0 (actually, there exists an action such that p+=1p^+ = 1 according to Assumption 1) hence will not trigger fallback. Before treacht_{reach}, since both the abort count and the value regret treat these initial episodes pessimistically (assuming that every episode fails and incurs the maximum regret), neither theorem is affected by this fallback mechanism.
    • Confidence parameters. We note that a practical specification of the confidence parameters is the standard empirical practice for optimistic methods (e.g., in Casecade BO (Kusakawa et al., 2022) or RL (Sukhija et al., 2025)).
  • W3/Q3: Please refer to the updated related work section in Appendix B (in pink), in summary:
    • Transition error. Cascade BO propagates the transition uncertainty through stage-wise Lipschitz constants, which grow multiplicatively with the horizon and lead to a simple-regret guarantee. We instead bound the value error by the total-variation distance between the predicted and true next-state distributions, which avoids this term and gives a cumulative regret bound over the executed episodes.
    • Failure control. F-GP-UCB excludes a region around each failure whose radius is agnostic to the constraint value. We gate actions by the GP upper bound of the observed constraint value against an annealed threshold, so the abort count is governed by the information gain of the constraint model, and the same feasibility term enters the Bellman recursion.
  • W4/Q4: Thank you for this observation. We consider (as well in experiments) only the deterministic setting for post-transition constraints, ϵh≡0\boldsymbol\epsilon_h\equiv\boldsymbol 0, in both the appendix discussion and the experiments. We have revised the description in the main paper and the appendix to make this explicit, as the feasibility term in the Bellman optimality equation then takes the product form of Eq. 18.
  • W5/Q5: Thank you for spotting this inaccuracy in the algorithm description. As described in Algorithm 1 of the updated manuscript (in pink), observations from aborted episodes are retained including the violating constraint value. Only the unrealised next state and the terminal objective are discarded. The proof of Theorem 1 relies on the violating observation entering the constraint posterior, which is indeed what the actual algorithm (and code) does, so the guarantee is unaffected.

We thank you again for all the comments, which helped improve the quality of the submission significantly. We are happy to address any further comments that you may have.

Reviewer_YDp9OpenReview ↗

After Rebuttal

I thank the authors for addressing my concerns. I will raise my score to Accept.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Shared Responses

We thank all the reviewers for their comments. We are glad that they unanimously found the problem well motivated and the theoretical analysis sound. We address the shared comments below. All changes in the updated manuscript are marked in pink.

  • The effect of value-function approximation error on the value regret (YDp9 W2/Q2; eBzu W1) While our methodology relies on an additional value function to avoid repeatedly maximising the state-action value function over actions, which imposes another approximation error as mentioned in the paper, we show (in Appendix F.4 of the updated manuscript) that, under the assumption that each approximate backward step is within ϵtV\epsilon^{V}_t of the exact Bellman update in the supremum norm for every stage hh, its impact on the value regret can be decomposed into the original value regret of FI-O-VI plus one additive term imposed by the per-iteration approximation error. Consequently, under the value function approximation the abort bound of Theorem 1 is unchanged (it only requires the approximate value function to be nonnegative), and the regret bound of Theorem 2 becomes the original bound plus O(H∑t=1TϵtV)O(H\sum_{t=1}^T\epsilon^{V}_t). General analysis of per-iteration error ϵtV\epsilon^{V}_t has been studied extensively in the literature. We include the relevant references for finite sample error bounds in Appendix F.4.
  • The fixed feasibility threshold in lookahead-stage planning (eBzu Q1; PH2T W1/Q1)
    • Theoretically. The fixed threshold 0.010.01 is used only at the lookahead stages of the Bellman recursion; the executed stage always uses the annealed threshold ξt\xi_t, so the abort bound of Theorem 1 is unaffected. For the value regret, Theorem 2 holds as stated when 0.01≤ξopt0.01\le\xi_{\mathrm{opt}}.
    • Empirically. We additionally conduct a sensitivity analysis regarding this threshold (Appendix H.4.5, Figures 13 and 14 of the updated manuscript) with different lookahead thresholds (00, 0.0050.005, 0.050.05, 0.150.15, and the annealed ξt\xi_t applied at every planning stage), on the eight synthetic instances. The results are in general not sensitive to the value of the threshold (except for a zero threshold on GMM).
  • The empirical trade-off between FI-O-VI and FI-PoF-VI (YDp9 W1/Q1; eBzu W2/Q2; PH2T W2/Q3)
    • We first note that FI-PoF-VI, the combination of the probability of feasibility with optimistic value iteration, is also part of our contribution. As shown in the additional experiment in Table 1 below, the optimistic value iteration is what makes it competitive with FI-O-VI: without the exploration bonus FI-PoF-VI is clearly behind FI-O-VI on several problems, suggesting that the exploration bonus may alleviate the issue of PoF.
    • FI-O-VI is by design more explorative than FI-PoF-VI: it does not rule out the optimal policy and thereby attains sublinear regret (Theorem 2), at the empirical cost of additional aborts. Whether this benefit can be empirically observed in a finite-episode run depends on the problem, as discussed below.
    • In the updated manuscript (end of Section 5) we additionally discuss when FI-PoF-VI nevertheless remains competitive: we conjecture that this is the case when (i) the value function is unimodal, or (ii) some initial trajectories are close to those induced by the optimal policy.
    • We provide empirical support for this conjecture in Appendix H.4.6: on the two multi-modal problems where FI-PoF-VI underperforms (GMM and GP prior), adding one initial trajectory close to the optimal one raises the number of seeds reaching a near-optimal reward from 2 of 20 to 18 and 19 of 20.
    • Since the real-life problems were not tailored to the setting in which FI-O-VI is needed, it is understandable that FI-PoF-VI performs well on them. Competitive heuristics are also common, e.g., the EI-based acquisition in Cascade BO outperforms the theoretically guaranteed CI-based one.

Table 1. Median final reward with and without the exploration bonus (rows marked "no exp reward"). With bonus: as reported in the paper (2020 seeds); without bonus: 1010 seeds.

BMRSynC-1GP priorMVNGMMSynC-2BMRCRSHMMSynC-3DNS3DNS4
FI-PoF-VI, no exp reward0.5480.4640.6990.9160.5350.8250.4120.9370.6430.739
FI-O-VI, no exp reward0.6090.8140.6890.9160.4070.7670.4980.9400.6730.772
FI-PoF-VI0.6320.8920.6940.9270.7360.8490.4540.9740.6890.790
FI-O-VI0.6330.9050.9430.9300.9400.8460.5090.9730.6880.778

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.