- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
DASCBO is a framework selecting effective interventions under uncertain causal structure and safety constrains. Its main contributions is the Structure Value of Information, which learned the expected improvements in the intervention sets. The proposed approach leverages this feature with grouping similar decision consequences to reduce the computation costs and increase efficiency. Across the four benchmarks, DASCBO claims to achieve the best final optimal object value and perfect empirical feasibility.
Critical strengths
-
Decision-relevant structural exploration. SVI provides a principled criterion for valuing causal information through its expected effect on intervention decisions. The result that SVI vanishes when all graph hypotheses share an optimal action makes this distinction precise by showing that informative experiments need not be useful experiments.
-
Principled integration of efficiency and safety. The framework compresses decision maps while retaining full likelihood evidence, and independently certifies risk using fresh samples from the full posterior predictive model. Explicit approximation bounds and model-discrepancy conditions clarify what compression preserves and when safety guarantees transfer to the physical system.
Critical weaknesses
-
Substantive discrepancies between the stated SCM and safety formulation and the released implementation.
Section 2.1 defines interventions as replacing only the targeted structural equations. However, the Health implementation introduces intervention-family-specific offsets that also change nonintervened variables. Within the same fitted hypothesis, the expected panel mean of root node B changes from 1502.852 under
do(C)to 1501.404 underdo(C,G), although B has no parents and is not intervened upon. This change arises from the offsets, rather than sampling noise or posterior updating. Unless an additional intervention-family context is explicitly modeled, this behavior conflicts with the stated fixed-SCM semantics and undermines the causal interpretation of the graph hypotheses.Furthermore, the implementation certifies violations of a ten-sample mean, whereas the paper formulates risk over SCM constraint variables without specifying this aggregation. The released defaults also did not reproduce the claimed 100% Health feasibility: one complete run achieved 98/99 feasible interventions, and two of ten configurations violated the constraint on their first intervention. These findings question whether the experiments instantiate the stated formulation and reproduce the reported empirical result; they do not, by themselves, disprove the posterior-predictive certification theorem.
-
Incomplete experimental reproducibility and insufficient evidence for the proposed components’ added value.
The paper does not fully specify the SCMs underlying all four benchmarks, including their structural equations, noise distributions, and fitted mechanisms. The supplied code covers only Healthcare and omits the baseline implementations. It therefore remains difficult to verify how shared initialization data and evaluation protocols were enforced across separate implementations or to reproduce the full experimental comparison.
Additionally, comparisons between complete algorithms do not isolate the contributions of SVI and decision-map compression. Controlled ablations are needed to establish whether these components account for the reported improvements; cumulative posterior movement alone does not demonstrate decision-relevant learning. Numerical summaries at matched budgets, with uncertainty estimates—including final feasible objectives and complementary measures such as GAP [1] and PA-GAP [2]—would strengthen the assessment. Together, the missing experimental artifacts and component-level evidence materially weaken confidence in the claimed advantages.
[1] Virginia Aglietti, Neil Dhir, Javier González, and Theodoros Damoulas. Dynamic Causal Bayesian Optimization. NeurIPS, 2021.
[2] Chenfeng Huang, Thuy T. Le, Zixuan Ma, and Hien Tran. Causal Bayesian Optimization: Foundations, Methods, and Applications. Transactions on Machine Learning Research, 2026.
Questions for authors
-
Benchmark specification, especially Protein-Signaling: Can the authors provide the complete SCM for each benchmark, including its graph, structural equations, noise distributions, intervention domains, constraint thresholds, and fitting procedures? The fitted SCM used in the original cCBO Protein-Signaling experiments is reported as unreleased. Did the authors obtain that model, reconstruct an SCM from the Sachs data, or use a different model?
-
Consistency with fixed-SCM intervention semantics: In the released Health implementation, intervention-family-specific offsets change nonintervened variables. Within the same fitted hypothesis, root B’s expected panel mean changes from 1502.852 under
do(C)to 1501.404 underdo(C,G), although B has no parents and is not intervened upon. How is this consistent with Section 2.1’s intervention semantics? Is an additional context variable intended, and if so, how is it represented in the graph and theoretical formulation? -
Safety definition and reported feasibility: Does the method certify violations of individual SCM constraint variables or violations of a ten-sample mean, as implemented in the Health code? Please clarify how the implemented event corresponds to Section 2.1. Furthermore, a released-default run with initial design 1 and algorithm seed 1001 achieved 98/99 feasible interventions, and two of ten configurations violated the constraint on their first intervention. Which configurations produced the reported 100% Health feasibility, and what explains this discrepancy?
-
Contribution of SVI and compression: Can the authors provide controlled ablations with SVI disabled, SVI replaced by an information-gain criterion, and compression disabled, while keeping the remaining experimental settings fixed? How much does each component contribute to feasible-objective improvement, empirical feasibility, and runtime?
-
Quantitative performance comparison: Can the authors report numerical summaries at matched budgets, with uncertainty estimates, alongside the existing curves—for example, final best-feasible objectives and complementary GAP/PA-GAP scores? If normalized metrics are used, how are reference values, infeasible evaluations, abstentions, and runs without an initially feasible point handled?
13 comments
2. Q2 / W1: Consistency with Fixed-SCM Intervention Semantics
We thank the reviewer for carefully inspecting the Healthcare implementation and identifying a potential inconsistency with the fixed structural causal model (SCM) intervention semantics in Section 2.1. We agree that, under a fixed SCM hypothesis , a standard surgical intervention replaces only the structural equations of the intervened variables; neither the mechanisms nor the exogenous noise distributions of non-intervened nodes should depend on the chosen intervention set.
Our inspection found that the original Healthcare surrogate model introduced intervention-family-specific residual anchoring offsets when fitting finite observation batches. Some offsets were applied to the predictive mechanisms of non-intervened nodes, causing their predictive distributions to depend on the intervention set even within the same fitted hypothesis. We thank the reviewer for identifying this issue. We have removed these offsets from the revised code, reconstructed the Healthcare SCM hypothesis library with fixed structural mechanisms, and re-evaluated performance under the revised setup. We emphasize that this was an implementation-specific deviation in the Healthcare surrogate, not an issue with the mathematical definitions or conditional theoretical results for SVI, decision-map compression, or posterior-predictive certification. We have also conducted preliminary validation across all four benchmark tasks, including Healthcare; the results indicate that this correction does not alter the effectiveness of the proposed method or the paper's central conclusions.
Specifically, in the revised Healthcare generative environment, the non-intervened root node follows the fixed structural equation
\varepsilon_B\sim\mathcal N(0,1) \text{ truncated to }[-1,2].$$ Neither this mechanism nor its exogenous noise distribution depends on $C$, $G$, or the intervention set. Thus, in the true generative environment, $$P(B\mid \mathrm{do}(C=-80)) = P(B\mid \mathrm{do}(C=-80,G=1))$$ and both interventions have the same theoretical expectation: $$\mathbb E[B]\approx1502.296372.$$ For any fixed fitted hypothesis $h$ in the revised model library, the non-intervened variable $B$ must likewise satisfy $$P(B\mid \mathrm{do}(C=-80)) = P(B\mid \mathrm{do}(C=-80,G=1))$$ Importantly, 1502.296372 is the theoretical expectation under the true Healthcare generative environment, not a parameter estimate shared by every fitted hypothesis. We inspected the SCM hypotheses constructed under the revision: in each, $B$ has no parents, and its simulated mechanism no longer includes offsets depending on the intervention family. We also examined the values 1502.852 and 1501.404 reported by the reviewer. In the revised fixed-SCM generative environment, using the original sequential random-number interface and generating two separate ten-sample batches with the same integer seed (`seed=1`) indeed reproduces - $\operatorname{do}(C=-80)$: sample mean of $B$ = 1502.851866; - $\operatorname{do}(C=-80,G=1)$: sample mean of $B$ = 1501.403855. However, these are empirical means from separately generated finite batches, not analytical expectations under a fixed SCM. Under sequential random-number sampling, intervening on $G$ removes the need to draw its random value, shifting subsequent consumption of the random stream. Consequently, even with the same integer seed, the two interventions need not share identical exogenous noise values node by node. The finite-batch difference in the revised environment can therefore be explained by unpaired random-number streams. To verify structural invariance more directly, we performed a counterfactual paired test using shared node-specific exogenous noise. Holding the same ten draws of the exogenous noise for $B$ fixed under both interventions yielded exactly identical values for all ten observations of $B$, with a sample mean of 1502.548502 in both cases. This directly verifies that, in the examined generative mechanism, the non-intervened root node $B$ is invariant to the intervention set. We have also rerun performance comparisons on all four benchmarks under the revised experimental settings. DASCBO achieves favorable empirical feasibility and nPA-GAP results across these tasks. The numerical results, uncertainty estimates, and comparisons are presented in the quantitative-performance response below. We again thank the reviewer for this careful examination and constructive recommendation. We believe these revisions provide a stronger basis for reproducing the empirical findings and clarify the conditions under which the method and its safety certificates should be interpreted.3. Q3 / W1: Safety Definitions and Feasibility Reporting
We thank the reviewer for highlighting ambiguities concerning safety events and batched observations in the original code and manuscript. We agree that the safety definition must be unambiguous in both theory and implementation. DASCBO certifies the joint-constraint violation event for a single SCM outcome, and we have revised the manuscript and relevant code accordingly to align certification with the chance-constraint formulation. We will further distinguish single-outcome safety events, batched feedback, and batch-mean evaluation and specify a unified observation protocol in the revision.
Each intervention returns a batch of predetermined size ; for example, in Healthcare:
V_t^{(k)}\mid(h,x_t)\overset{\mathrm{i.i.d.}}{\sim}P_h(\cdot\mid\operatorname{do}(x_t)).$$ The objective feedback is $\bar Y_t=m^{-1}\sum_kY_t^{(k)}$, which estimates the expected objective. For a fixed hypothesis $h$, the full-batch likelihood is $L_h(O_t^{(m)}\mid x_t)=\prod_kp_h(V_t^{(k)}\mid x_t)$, and the posterior-predictive batch likelihood is $$\sum_h w_{t,h}\prod_{k=1}^m p_h(V_t^{(k)}\mid x_t).$$ Thus, the entire batch shares the same hypothesis. The current Health posterior update accumulates log likelihoods over the complete batch, and hypothetical observations for SVI use the same batch unit. The safety event is defined separately as $$Z(V)=\mathbf1\\{\exists j:C_j(V)\notin\mathcal A_j\\},\qquad r_t(x)=\mathbb E_{P_t(\cdot\mid x)}[Z(V)].$$ For Health, the admissible set is $\mathrm{BMI}<25$, so $Z(V)=\mathbf1\\{\mathrm{BMI}\ge25\\}$. The certifier generates independent, single-outcome SCM rollouts from the full posterior-predictive mixture over causal graphs and mechanisms, estimates the violation rate, and uses a Hoeffding upper confidence bound to assess the risk budget. The certification sample count $N_t^{\mathrm{cert}}$ and feedback batch size $m$ are independent parameters. Decision-map compression does not replace the certification distribution; the risk pilot likewise uses the violation indicator for a single outcome. We acknowledge that the original description of "100% empirical feasibility" on Health could conflate feasibility of observed batch means with joint feasibility of individual SCM outcomes, potentially suggesting that no individual violations occurred or that all interventions satisfied the true chance constraint. In the revision, we no longer use that insufficiently qualified statement and instead consistently report the sample-level joint feasibility rate $FR_{ind}$, with corresponding manuscript and code updates. Let $\mathcal T_{\mathrm{exec}}\subseteq\\{1,\ldots,T\\}$ denote the rounds in which an intervention was actually executed, and let $m_s$ be the number of SCM outcome rows received in round $s$. We define $$FR_{ind} = \frac{\sum_{s\in T_{exec}} \sum_{k=1}^{m_s} 1\\{ \text{for all } j\in\{1,\ldots,q\}:C_j(V_s^{(k)})\in A_j\\}} {\sum_{s\in T_{exec}}m_s} = 1- \frac{\sum_{s\in T_{exec}} \sum_{k=1}^{m_s}Z(V_s^{(k)})} {\sum_{s\in T_{exec}}m_s}.$$ If every executed online batch has the same size $m$, and $N_{exec}=|T_{exec}|$, this simplifies to $$FR_{ind} = \frac{1}{mN_{exec}} \sum_{s\in T_{exec}} \sum_{k=1}^{m}\big[1-Z(V_s^{(k)})\big].$$
We sincerely thank the reviewer for recognizing the novelty and readability of our work and for carefully examining both its theoretical development and implementation. Your comments are valuable for strengthening the theory and making the method more readily reproducible.
We thank the reviewer for emphasizing the importance of providing complete SCMs, causal graphs, and benchmark settings for accuracy and reproducibility. In response, we have compiled the specifications of all four benchmarks, including their causal graphs, structural equations, noise distributions, intervention domains, constraint thresholds, and experimental settings in the revised manuscript.
For Protein-Signaling, we constructed the SCM using the publicly available protein-signaling data of Sachs et al. (2005), with reference to Huang et al., (2026), to reflect the underlying pathway structure.
Here, we present the detailed SCM definitions and experimental settings for the Health and Protein-Signaling benchmarks. The corresponding specifications for the Synthetic and Honda Energy benchmarks will be provided in the appendix of the revised manuscript. We write σ(z)=(1+e−z)−1, [z]+=max(z,0), and TN(a,b) for a standard normal distribution truncated to [a,b].
A. Health
The ten nodes are Age, BMR, Cal, Height, Weight, BMI, Statin, Aspirin, Cancer, and PSA. Their non-root parent sets are
\mathrm{Pa}(\mathrm{Weight})&=\{\mathrm{Age,BMR,Cal,Height}\},\\ \mathrm{Pa}(\mathrm{BMI})&=\{\mathrm{Weight,Height}\},\\ \mathrm{Pa}(\mathrm{Aspirin})&=\{\mathrm{Age,BMI}\},\\ \mathrm{Pa}(\mathrm{Cancer})&=\{\mathrm{Age,BMI,Statin,Aspirin}\},\\ \mathrm{Pa}(\mathrm{PSA})&=\{\mathrm{Age,BMI,Statin,Aspirin,Cancer}\}. \end{aligned}$$ The graph contains ten nodes and seventeen edges. The root nodes are generated independently: $\mathrm{Age}\sim U(55,75)$, $\mathrm{BMR}=1500+10\mathrm{TN}(-1,2)$, $\mathrm{Cal}\sim U(-100,100)$, $\mathrm{Height}=175+10\mathrm{TN}(-0.5,0.5)$, and $\mathrm{Statin}\sim U(0,1)$. The non-root mechanisms are $$\begin{aligned} \mathrm{Weight}&=(\mathrm{BMR}+6.8\mathrm{Age}-5\mathrm{Height})/13.7+150\mathrm{Cal}/7716,\\ \mathrm{BMI}&=\mathrm{Weight}/(\mathrm{Height}/100)^2,\\ \mathrm{Aspirin}&=\sigma(-8+0.10\mathrm{Age}+0.03\mathrm{BMI}),\\ \mathrm{Cancer}&=\sigma(2.2-0.05\mathrm{Age}+0.01\mathrm{BMI}-0.04\mathrm{Statin}+0.02\mathrm{Aspirin}),\\ \mathrm{PSA}&=6.8+0.04\mathrm{Age}-0.15\mathrm{BMI}-0.60\mathrm{Statin} +0.55\mathrm{Aspirin}+\mathrm{Cancer}+\varepsilon_{\mathrm{PSA}}, \end{aligned}$$ Here $\varepsilon_{\mathrm{PSA}}\sim\mathcal N(0,0.4^2)$ and is independent of the root disturbances. The objective is to minimize PSA subject to $\mathrm{BMI}<25$. The intervention domains are $\mathrm{Cal}\in[-400,400]$ and $\mathrm{Statin},\mathrm{Aspirin}\in[0,1]$. The permitted intervention sets are $\{\mathrm{Cal}\}$, $\{\mathrm{Cal,Statin}\}$, $\{\mathrm{Cal,Aspirin}\}$, and $\{\mathrm{Cal,Statin,Aspirin}\}$. **B. Protein-Signaling** The eight nodes are PKC, PKA, Jnk, P38, Akt, Raf, Mek, and Erk. Their parent sets are $$\begin{aligned} \mathrm{Pa}(\mathrm{PKC})&=\varnothing,&\mathrm{Pa}(\mathrm{PKA})&=\{\mathrm{PKC}\},\\ \mathrm{Pa}(\mathrm{Jnk})&=\mathrm{Pa}(\mathrm{P38})=\mathrm{Pa}(\mathrm{Raf})=\{\mathrm{PKC,PKA}\},\\ \mathrm{Pa}(\mathrm{Akt})&=\{\mathrm{PKA}\},&\mathrm{Pa}(\mathrm{Mek})&=\{\mathrm{PKC,PKA,Raf}\},\\ \mathrm{Pa}(\mathrm{Erk})&=\{\mathrm{PKA,Mek,Akt,Raf}\}. \end{aligned}$$ The graph has fifteen edges. The root variable is $\mathrm{PKC}\sim U(1,106)$, and the intermediate structural mechanisms are $$\begin{aligned} \mathrm{PKA}&=554.390731+0.841153\mathrm{PKC},\\ \mathrm{Jnk}&=52.953603-0.764801\mathrm{PKC}-0.005306\mathrm{PKA},\\ \mathrm{P38}&=15.144328+1.234783\mathrm{PKC}+0.000591\mathrm{PKA},\\ \mathrm{Akt}&=-31.110747+0.128905\mathrm{PKA},\\ \mathrm{Raf}&=62.199046-0.177745\mathrm{PKC}-0.000379\mathrm{PKA},\\ \mathrm{Mek}&=-1.090275+0.039662\mathrm{PKC}-0.000652\mathrm{PKA}+0.520845\mathrm{Raf}. \end{aligned}$$ The objective mechanism is $$\begin{aligned} \mathrm{Erk}={}&15.4025+3.0\times10^{-5}(\mathrm{PKA}-700)^2 +3.0\times10^{-4}(\mathrm{Mek}-44.098)^2\\ &+1.0\times10^{-5}(\mathrm{Akt}-100)^2 +5.0\times10^{-2}(\mathrm{Raf}-44)^2\\ &+3.0\times10^{-5}(\mathrm{PKA}-700)(\mathrm{Mek}-44.098) +\varepsilon_{\mathrm{Erk}}, \end{aligned}$$ where $\varepsilon_{\mathrm{Erk}}\sim\mathcal N(0,0.4^2)$. The objective is to minimize Erk subject to $\mathrm{Raf}\ge45$ and $\mathrm{PKA}>255.4$. The intervention domains are $\mathrm{PKC}\in[1,105]$, $\mathrm{PKA}\in[439.2449,4491.5]$, $\mathrm{Mek}\in[0.5,389.5]$, and $\mathrm{Akt}\in[1.2,3555.5]$. The permitted intervention sets are $\{\mathrm{PKC}\}$, $\{\mathrm{PKA}\}$, $\{\mathrm{Mek}\}$, $\{\mathrm{PKC,PKA}\}$, $\{\mathrm{PKC,Mek}\}$, and $\{\mathrm{PKA,Mek}\}$.