acceptodds
Under review as a conference paper at ICLR 2027

PAG-CBO: Causal Bayesian Optimization under Partial Identification

Abstract

Causal Bayesian optimization uses causal structure and observational data to reduce costly experimentation. With hidden confounding and uncertain structure, however, intervention effects may remain only partially identified. We introduce PAG-CBO for systems whose structural uncertainty is represented by a partial ancestral graph (PAG). Our method combines structural pruning with bounds on intervention-response means, accounts for observational sampling uncertainty, and updates these bounds using interventional data. It eliminates actions that cannot be near-optimal and stops when the remaining uncertainty permits an -optimality certificate. Under a correct PAG and simultaneous confidence-band coverage, we prove certificate validity and localize a standard kernel-bandit simple-regret bound to the region surviving observational pruning. In discrete benchmarks, informative positive-width causal bounds can certify a recommendation before any intervention; when these bounds do not settle the decision, sequential experiments reduce the remaining uncertainty. An empirical calibration study records no false recommendations among 4,835 stopping claims under correctly specified PAGs. When informative causal bounds are unavailable, PAG-CBO uses the known outcome range and relies on experimental learning.

Primary area:
causal reasoning
Keywords:
Causality, Bayesian Optimization, Partial Ancestral Graphs, Intervention, Hidden Confounders, Uncertainty

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ETGuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduce PAG-CBO for intervention optimization when causal structure is represented by a partial ancestral graph and intervention responses may be only partially identified. It uses observational bounds and confidence intervals from intervention data to decide which experiment to run next or stop, once it can certify that its recommendation is within ϵ\epsilon of the optimum. The theoretical results concern safe pruning and localization of a standard kernel-bandit regret bound. Authors use synthetic causal models for experiments to certify proposed theorems.

Critical strengths

I think the motivation is reasonable: identifying every intervention response is unnecessary if the available information already determines a sufficiently good decision. The C1 example illustrates how informative partial-identification bounds can help beyond point identification alone.

Critical weaknesses

My main concern is that the experiments do not evaluate the procedure covered by the theoretical guarantees. Appendix D.2 acknowledges a missing 1/λ1/\sqrt{\lambda} factor in the intervention confidence coefficient, an unverified RKHS norm bound, and missing global error allocation across targets for the continuous observational bands. These differences directly affect certificate validity. I appreciate the empirical calibration study, but it does not establish whether the theoretically justified version remains useful at a reasonable intervention cost.

I think the novelty of theoretical contribution is limited. The identification and certificate results follow from existing identification guarantees and interval coverage, while the regret theorem adapts standard kernel-bandit analysis. In E0 to E4, roughly 95-99.9% of actions receive the uninformative outcome-range fallback, leaving little scope for observational pruning. The clearest success is the small discrete C1 example, where existing bound-aware methods can also use the same intervals.

I therefore remain unconvinced that this method offers a substantial improvement over simpler approaches.

Questions for authors

The authors should prioritize addressing the concerns raised in weaknesses.

Questions:
How does certification performance change when the implementation uses the confidence bounds and norm assumptions required by the theory?

Reviewer_8AgnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper presents a framework for intervention selection on a partial ancestral graph with partially identified effects, called PAG-CBO. It performs structural pruning of intervention targets and uses bounds on the intervention means and Gaussian process bands from the interventional data, eliminating uncompetitive actions.

The work establishes conditional certification and regret guarantees with evaluations on synthetic causal benchmarks.

Critical strengths

The authors clearly show the value of partial identification bounds by comparing their method to the IDP-only ablation. These bounds allow the stopping criterion to be met before any intervention, while point identification alone is insufficient in C1.

Critical weaknesses

The paper has narrow theoretical and methodological novelty. Structural pruning comes from PAG-POMIS (Park et al., 2025), and the causal response bounds come from Bellot (2024), which is acknowledged by the authors (main, lines 130–143 and 198–215). Using response bounds to clip exploration indices and eliminate actions already appears in B-kl-UCB (Zhang and Bareinboim, 2017) and UCB-SI (Sharma et al., 2020). I believe the proofs offer modest technical content. For example, Th. 1 follows directly by taking the expectation of an identified distribution, Proposition 1 and Theorem 2 are elementary interval-ordering arguments under simultaneous coverage, and Theorem 3 applies standard kernel-bandit analysis, as developed by Chowdhury and Gopalan (2017), to the surviving action domain (please see the supplement, lines 740–823). The paper states that the latter does not provide a new GP-UCB rate (main, lines 386–395). Hence, the guarantees depend on simultaneously valid confidence bands, without such coverage being established for the proposed experimental construction. Given these limited theoretical additions and the reliance on established algorithmic components, a convincing practical advantage is paramount for acceptance.

However, the empirical evaluations do not reliably establish the certification or optimization advantage necessary to support the proposed method. The observational bands have not been shown to provide the simultaneous coverage required by the theory, while the experimental confidence coefficient in Equation (25) omits the required 1/λ1/\sqrt{\lambda} factor from its noise term despite using λ=0.05\lambda=0.05. The appendix acknowledges that the implemented construction is not justified by the theoretical guarantee (supplement, lines 1234–1274). Also, there is an inconsistency in E5b. The specified bounds imply a population certificate gap of approximately 0.4020.402 when the no-intervention action is included, but Figure 27 reports a gap of approximately 0.03010.0301 and certification before any intervention (supplement, lines 1109–1119). Simultaneous outer confidence bands covering those stated population bounds cannot produce this certificate, and the paper does not reconcile the discrepancy. Moreover, on several continuous benchmarks, PAG-CBO’s recommendation regret remains almost flat while GP-UCB baselines improve and finish with lower regret (supplement, Figures 21–23 and 25). So the experiments neither offer the certification guarantees for the evaluated procedure nor demonstrate a consistent practical improvement sufficient to compensate for the limited novelty.

Questions for authors

Could the authors consolidate the supplementary material and the main paper into a single PDF?

Reviewer_QVoYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies causal Bayesian optimization when latent confounding leaves only a partial ancestral graph and intervention means may not be point identified. PAG-CBO combines PAG-based intervention-target pruning with observational response bounds and interventional confidence intervals. It removes actions that cannot be epsilon-optimal and stops when the remaining bounds certify a recommendation. The theoretical results give conditional certificate validity and a localization of a kernel-bandit regret bound. In selected discrete benchmarks, informative partial-identification intervals allow certification before any intervention; other cases require sequential experiments or fall back to the known outcome range.

Critical strengths
  1. The paper makes a useful distinction between identifying an intervention effect and identifying a good decision. The C1 example illustrates this clearly: positive-width bounds can rule out actions and give a certificate even though their response means are not point identified.

  2. The certificate is stated with explicit conditions on PAG correctness and simultaneous band coverage, rather than being presented as unconditional safety. It is also useful that the authors report both the fallback behavior and violations under deliberately misspecified PAGs.

Critical weaknesses
  1. The practical certification results depend heavily on uncertainty bounds whose simultaneous coverage is assumed in the theorem but not demonstrated by the reported experiments. Section 4 explicitly states that the observational inference is approximate and that the intervention radius differs from the sufficient theoretical radius. The 4,835 successful stopping claims are encouraging but do not establish the finite-sample validity needed for a reliable certificate. This gap affects the central claim rather than just the optimization rate.

  2. The tangible benefit of partial identification appears concentrated in a small number of discrete examples. C1 has only six actions after structural pruning, while the continuous benchmarks often use the uninformative [0,1] bound and retain almost the full contender region. The paper does not yet show when the extra PAG/PI machinery offers a meaningful benefit over an optimizer given the same valid response intervals.

Questions for authors
  1. For C1 and C2, what exact finite-sample procedure produces the observational confidence intervals for all retained actions simultaneously? Could the authors report empirical familywise coverage of the entire band, rather than only cell-wise marginal checks? This is the event required by Assumption 3 and the stopping guarantee.

  2. Could the authors give a matched comparison in C1 and C2 where a standard finite-arm bound-aware optimizer is supplied with exactly the same observational intervals, confidence budgets, and intervention batch sizes? The existing IDP-only ablation isolates the value of positive-width bounds, but not the additional value of PAG-CBO's search and stopping rule once those bounds are available.

  3. How many experimental settings obtain informative positive-width PAG bounds rather than the [0,1] fallback, and what fraction of actions are pruned by these bounds before intervention? A compact per-benchmark table would make the practical scope of the method much clearer.

  4. The supplied PDF ends after the references, but the main text points repeatedly to Appendices B and D for proofs, confidence-band construction, and calibration details. Are these available in a separate supplement? If not, these details are needed to assess the main guarantee and its empirical implementation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.