- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies multi-stage optimization where unknown constraints can terminate an experiment and interrupt subsequent observations. It models unknown dynamics and constraints with GPs, then performs optimistic value iteration using a soft feasibility weight computed from constraint UCBs. An annealed threshold controls this weight, and the paper derives bounds on cumulative aborted episodes and realized cumulative regret for an idealized planner. Experiments compare this method, FI-O-VI, with a probability-of-feasibility variant and adapted BO baselines on eight synthetic cascades and four application-inspired simulation problems.
Critical strengths
S1. A meaningful problem formulation with joint theoretical guarantees. The paper formalizes multi-stage optimization with unknown transitions and unknown stopping constraints, whose violations censor the next transition and all downstream observations. Its FIMDP formulation enables a joint analysis of optimization progress and failures that restrict further learning. For the idealized planner, the paper establishes a pathwise cumulative-abort bound and a high-probability value-regret bound under noiseless constraint observations and the stated RKHS and feasibility assumptions, with the regret analysis additionally requiring nonzero Gaussian transition noise. These finite-time bounds yield simultaneous sublinear rates when and the remaining bound terms are sublinear.
S2. An informative separation between plug-in and optimistic feasibility. Theorem 3 presents a single-stage, multiple-constraint construction in which FI-PoF-VI can incur linear regret while FI-O-VI achieves sublinear regret on the same instance under the required assumptions. The mechanism is concrete: posterior feasibility weighting suppresses uncertain alternatives and traps the plug-in variant at a feasible but suboptimal action. Figure 9 illustrates analogous behavior in the multi-stage GMM problem, supported by the seed-level results in Figure 7. This combination of theoretical separation and empirical diagnosis clarifies when optimistic feasibility can improve exploration.
Critical weaknesses
W1. The theoretical guarantees leave key implementation choices unresolved. The implementation applies the annealed threshold only to the executed stage, while future-stage backups use a fixed threshold of 0.01. This can downweight or exclude strictly feasible optimal continuations with small margins. With valid constraint upper confidence bounds, suffices to resolve this particular obstruction to Lemma 8's optimism argument; it is not necessary and does not establish guarantees for the entire implementation. Separately, choosing can yield . The finite-time bound for the corresponding idealized planner remains valid, but need not imply sublinear regret; this does not establish linear regret in practice. The authors should clarify these conditions and provide a targeted ablation applying threshold annealing at every planning stage, with a schedule satisfying the stated sublinearity requirements.
W2. Additional aborts bring no clear return gains in the application tasks. Tables 3 and 5 report approximately 15×, 10×, 1.5×, and 3.4× the median abort counts for FI-O-VI compared with FI-PoF-VI on DNS3, DNS4, QT, and NQT, respectively, while returns remain similar. QT shows a small increase in median return, but the reported results do not establish a clear return benefit that compensates for the additional aborts. Although synthetic experiments demonstrate benefits from optimistic feasibility, its practical return–abort tradeoff remains insufficiently established in the applications. A paired analysis of return and abort differences using the existing runs would help determine whether the extra exploration produces meaningful gains.
Questions for authors
Q1. What justifies using a fixed threshold of 0.01 in future-stage backups when the assumptions permit smaller optimal feasibility margins? Under valid constraint UCBs, suffices to resolve this particular optimism issue. Is this additional condition intended, and does applying threshold annealing at every planning stage materially change the results?
Q2. With , can be , so the stated bound need not imply sublinear regret. Can you clarify the guarantee claimed for this configuration and whether the empirical conclusions persist with a -independent choice of , or more generally one satisfying ?
Q3. FI-O-VI incurs approximately 15×, 10×, 1.5×, and 3.4× the median abort counts of FI-PoF-VI on DNS3, DNS4, QT, and NQT, respectively, with similar returns. What do paired comparisons across the existing seeds show about the benefit obtained from these additional aborts? Please report uncertainty for the return and abort differences, particularly whether QT’s small return increase is consistent across seeds.
3 comments
Hi PH2T, friendly reminder to please read the authors' response, acknowledge that you've done so, whether that changes your assessment of the paper, and if you have additional questions. Thanks!
I thank the authors for the responses. My concerns have been fully resolved and thus I raised the score to accept.
We thank you for the positive reviews, we are glad that you found the FIMDP formulation meaningful and consider the theoretical analysis, the theoretical demonstration of FI-PoF-VI in Theorem 3 as well as the empirical illustration through the GMM case as strengths. Together with the updated manuscript, we address the weaknesses and questions below.
We thank you again for the thorough review, in particular the suggested ablation and paired analysis. We are happy to address any further comments that you may have.