acceptodds
Under review as a conference paper at ICLR 2027

Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors

Abstract

Multi-fidelity Bayesian optimization (MFBO) accelerates the search for the global optimum of black-box functions by integrating inexpensive, low-fidelity approximations. The central task of an MFBO policy is to balance the cost-efficiency of low-fidelity proxies against their reduced accuracy to ensure effective progression toward the high-fidelity optimum. Existing MFBO methods primarily capture associational dependencies between inputs, fidelities, and objectives instead of causal mechanisms, and can perform poorly when lower-fidelity proxies are poorly aligned with the target fidelity. We propose RESCUE (REducing Sampling cost with Causal Understanding and Estimation), a multi-objective MFBO method that incorporates causal calculus to systematically address this challenge. RESCUE learns a structural causal model capturing causal relationships between inputs, fidelities, objectives, and constraints, and uses it to construct a probabilistic multi-fidelity (MF) surrogate that encodes intervention effects. Exploiting the causal structure, we introduce a value-of-causal-information hypervolume knowledge-gradient acquisition strategy to select input–fidelity pairs that balance expected feasible multi-objective improvement and cost. We show that RESCUE improves sample efficiency over state-of-the-art MF optimization methods on synthetic and real-world problems in robotics, and machine learning (AutoML).

Primary area:
optimization
Keywords:
Multi-fidelity optimization, Multi-objective Bayesian optimization, Causality

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_at34OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This article proposes RESCUE, a constrained multi-objective and multi fidelity Bayesian optimization method. It learns a structural causal model from observed data or a given DAG using PC or DirectLiNGAM, and uses the intervention mean and variance obtained by do calculation as the prior mean and low rank covariance term of the multi output multi fidelity Gaussian process. The collection function adopts cost normalized causal information hypervolume knowledge gradient (VoCI HVKG), where each output reliability weight is estimated by leaving one prediction likelihood. The theory provides an arbitrary time hypervolume regret bound that does not increase with the magnitude of cross fidelity deviation when the causal prior residuals are bounded; The variant with forced target fidelity exploration further provides a sublinear convergence rate. The experiment was compared with qEHVI, MOMF, and MF HVKG on three synthesis problems and two real tasks (robot navigation, constrained XGBoost hyperparameter optimization).

Critical strengths

Introducing learnable reliability weights into the hypervolume knowledge gradient is a clean and practical design that enables the optimizer to automatically reduce weights without discarding multi fidelity structures when causal priors are lost. The improvement in sample efficiency relative to the strongest baseline HVKG on the synthetic benchmark is consistent among independent seeds, and is still partially retained in ablation after removing causal terms, indicating that modeling selection has contributed at least some of the gain.

Critical weaknesses

This paper should be rejected.

W1. Causal discovery is considered a reliable black box, but relies on PC and DirectLiNGAM under the assumption of causal sufficiency. The robot task (25 dimensional continuous parameters) only uses 1000 observation samples, and the frequency of edge occurrence fluctuates greatly during 20 runs, with a large number of edge occurrence rates below 50%. Without conducting sensitivity analysis of significance level alpha or alternative discovery algorithms, the causal prior quality supporting theory and experiments cannot be verified.

W2. The arbitrary time bound of Theorem 5.8 is represented by the posterior standard deviation of the implementation at the target fidelity. The main algorithm (used in the experiment) does not have a mandatory exploration step and cannot guarantee the shrinkage of the quantity, so it only inherits a non vanishing residual that depends on the causal prior quality. The mismatch constant xi is assumed to be bounded but has never been estimated or defined from data, making it theoretically impossible to operate.

W3. The robot navigation agent itself is relatively weak. The energy per meter test set R ² is only 0.21, and the average R ² of the four indicators is only 0.61. All methods (including baselines) are evaluated on this proxy, and systematic biases can directly affect sorting. The paper claims that residuals propagate uniformly, but does not provide quantitative verification of sorting stability under proxy noise.

W4. The absolute gain on two real problems is limited. The AUR improvement relative to HVKG is about 5% in robot navigation, but statistically insignificant in constrained XGBoost. HVKG has shared the same multi output multi fidelity kernel (only lacking causal terms), but once the acquisition function itself is complex enough, the incremental value of the causal mechanism appears limited.

W5. Intentional destruction of observation data (result - configuration for random shuffling) can still achieve comparable performance to HVKG in ablation. This weakens the core proposition that 'causal priors are the source of advantage'; A more likely explanation is that the additional flexibility or reliability of the mean function itself is at work.

W6. The computational cost is underestimated. Causal discovery, do algorithm agent training, and reliability mixing are executed every round or iteration; The wall clock only reports the collection steps. The cubic complexity of multiple output Gram matrices in high-dimensional scenarios, coupled with repeated residual likelihood calculations, makes the relatively simpler multi fidelity baseline of this method impractical.

W7. There are multiple confounding factors in the experimental design. All methods share the same initial design; The robot cost model itself is a GP fitted on the same proxy data, forming a cyclic dependency; Only investigate a single YAHPO instance; Not compared with single fidelity methods under intentional mismatch fidelity. These choices are difficult to isolate the contribution of causal components.

W8. Several values exhibit suspicious regularity (initial cost intervals between seeds are completely consistent, AUR values are close to arithmetic sequences). Although it cannot be concluded that there is fraud, it reduces the credibility of the statistical report and requires the disclosure of the original trajectories for each sub track.

Questions for authors

Can a quantitative bound or data-driven estimate of the causal residual xi be provided on each benchmark, and how the hypervolume regret changes when the residual is artificially amplified?

Why was this scheduling never used in the experimental section of the main theoretical rate dependent forced exploration (Algorithm 3)?

What is the rank correlation of the recommended Pareto set when the robot agent is replaced by a more noisy or differently trained model?

3 comments

We thank the reviewer for the detailed review, and for recognizing the causal trust as a clean and practical design. Several concerns rest on statements that the paper contradicts, in particular on the runtime (W6), the cost model (W7) and the reported values (W8). We address each point below with references to the relevant sections, tables and figures, so that each clarification can be checked directly. Table, figure and equation numbers refer to the revised version. These clarifications do not fit within the character limit of a single comment, and our response spans three comments.

(1/3)

W1.

  • Causal discovery as a black box. RESCUE does not rely on the learned CPM being correct. Section 3.2 states that PC and DirectLiNGAM assume causal sufficiency, and that RESCUE takes the CPM as an input, so methods that relax this assumption, such as FCI, can be used instead. We added directions for relaxing causal sufficiency to the Discussion (Section 7) in the revision (see also our response to Reviewer VJrL). VoCI-HVKG weights the causal prior of each output by a causal trust ω^\hat{\boldsymbol{\omega}}, computed from its leave-one-out predictive likelihood (Eq. 2). An inaccurate prior predicts the left-out observations poorly, and VoCI-HVKG lowers its weight. Theorem 5.8 includes the causal-model error ξ\xi of Assumption 5.2 in the regret bound.
  • Robot edges. The edge frequencies in Figure 15 vary because each run learns the CPM from its own sample D^\hat{\mathcal{D}}, and the results in Table 6 average over this variation. The causal trust ω^\hat{\boldsymbol{\omega}} on robot navigation stays well above its value under the misspecified prior throughout optimization (Figure 5), so the learned priors predict the interventional data well despite this variation.
  • Prior quality. The theory does not require a correct prior, since Theorem 5.8 holds for any ξ\xi. The experiments test a wrong prior directly (Section 6.2, Appendix A.5). Our ablation builds a misspecified prior by shuffling the outcomes across configurations and fidelities, which removes most causal edges (Figure 10). The causal trust ω^\hat{\boldsymbol{\omega}} is lowest under this prior, and RESCUE remains competitive with HVKG on all problems (Figure 5). We also learn the CPM from 50% and 10% of D^\hat{\mathcal{D}} (Section 6.2, Figures 5 and 9). Where the true graph is known, we verify the discovered graph directly. On Synthetic SCM, DirectLiNGAM recovers every true edge into the objectives in all runs, even though the nonlinear mechanisms violate its assumptions (Appendix A.3.1).
  • Significance level and discovery algorithm. We did not vary alpha or apply a second discovery algorithm to the same problem. The ablations above vary the quality of the learned CPM itself, which is what alpha and the choice of algorithm would change.

W2.

  • Shrinkage of the posterior uncertainty. Section 5 states this distinction. Theorem 5.8 is an anytime guarantee for RESCUE, and the convergence guarantee with a sublinear rate is established only for a variant with periodic target-fidelity exploration (Section 5, Appendix B.6). Theorem 5.8 bounds the regret by the realized uncertainty σˉT\bar{\sigma}_T without asserting its decay (Remark B.4), and holds for every query sequence, so it applies to Algorithm 1 as used in the experiments. Its purpose is Corollary 5.9: the bound does not depend on the magnitude of the cross-fidelity bias. We added a statement that Algorithm 1 does not guarantee the decay of σˉT\bar{\sigma}_T to the Discussion (Section 7) in the revision.
  • The constant ξ\xi. ξ\xi is defined in Assumption 5.2 as a bound on the error of the causal prior. Like the RKHS bound B, it is a constant of the analysis and not an input of the algorithm, so RESCUE runs without knowing it. The misspecification constant of Bogunovic & Krause (2021) plays the same role. Within the algorithm, the causal trust ω^\hat{\boldsymbol{\omega}} assesses the causal prior from data (Eq. 2). In the revision, we also estimate ξ\xi on every benchmark (see Q1, Appendix A.5, Table 8).
  • Algorithm 3. We use Algorithm 3 only to prove the rate. Every NeN_{\mathrm{e}}-th query goes to the target fidelity at the point of maximum posterior uncertainty. These queries incur the cost of the target fidelity independent of their expected hypervolume gain. The cost-normalized VoCI-HVKG acquisition of Algorithm 1 weighs this gain against the cost of each query. Setting Ne=∞N_{\mathrm{e}} = \infty recovers Algorithm 1 (Appendix B.6). We added this to the Discussion (Section 7) in the revision.

(2/3)

W3 and Q3. We added this test to the revision (Appendix A.7, "Ranking Stability", Table 11). We keep the runs of Table 6 and recompute the regret of every recommended Pareto set on two kinds of alternative benchmark models. Five retrained models are trained on the same 300 robot runs with other random states. These are the differently trained models of Q3. Five biased models add a smooth random bias to each output of the benchmark model, sized to its test RMSE. These are the more noisy models of Q3. They treat the full test error as systematic, which is the scenario of W3.

Benchmark modelSpearmanKendallAUR qEHVIAUR MOMFAUR HVKGAUR RESCUE
Benchmark––−4025.31−3961.45−4019.72−4237.67
Retrained0.975 ± 0.0040.867 ± 0.183−3812.50 ± 106.03−3762.12 ± 101.22−3811.39 ± 109.56−4040.53 ± 97.12
Biased0.791 ± 0.1550.800 ± 0.183−3313.28 ± 30.11−3267.80 ± 15.70−3323.58 ± 42.80−3563.93 ± 276.87

Spearman is the rank correlation of the AUR of all runs with the benchmark model, and Kendall is the rank correlation of the ranking of the methods. RESCUE has the lowest AUR on every one of the ten alternative models. The only change in the ranking is between qEHVI and HVKG, which differ by 0.1% in Table 6. We revised the statement on surrogate error in Appendix A.7. It now refers to Table 11, which shows that the ranking of the methods is stable under alternative benchmark models.

W4. The gain over HVKG on robot navigation is not limited. The 5% figure is a percentage of AUR, which integrates the log HV regret, so it is not a percentage of regret. RESCUE has about 18% lower HV regret than HVKG on average over the budget, a significant and large effect (Table 6). On constrained XGBoost HPO, RESCUE also has lower AUR than HVKG, although the difference is not significant (Table 6). To isolate the causal mechanism itself, our ablation sets ω=0\boldsymbol{\omega} = \mathbf{0} within RESCUE and keeps every other component fixed (Section 6.2). Removing the causal prior increases the regret of RESCUE on every problem, including both real ones (Figures 5, 9 and 13).

W5. We believe the review reads this ablation in reverse. As Section 6.2 and Appendix A.5 state, when the causal information is destroyed, RESCUE falls to the level of HVKG, and VoCI-HVKG lowers its trust in the causal prior (Figure 5). Thus the advantage of RESCUE comes from the causal prior. This ablation keeps the causal prior mean, the causal covariance term and the causal trust of RESCUE, and shuffles the outcomes in D^\hat{\mathcal{D}} to obtain a misspecified causal prior (Table 7). Its regret is higher than that of RESCUE on every problem (Figures 5 and 9). So the gain comes from the causal relationships that RESCUE learns from D^\hat{\mathcal{D}}, which the shuffling destroys. Falling back toward HVKG is the intended robustness, as Section 6.2 states. With low trust, RESCUE adapts accordingly and relies on M0\mathcal{M}_{\mathbf{0}}, the MF-CGP without the causal prior (Eq. 2). The review's strengths describe this same mechanism.

W6. We believe the review misreads when each step runs and what the runtime figure reports. As Algorithm 1 and Appendix A.4 state, the CPM is constructed once, in Line 2, before the optimization loop begins in Line 3. This covers causal discovery and the proxy network distilled from the CPM (Appendix A.2). We now state this explicitly in the Discussion (Section 7) of the revision. The reliability mixture is part of the VoCI-HVKG acquisition (Section 4.2), which Line 6 optimizes inside the loop. Figure 8 reports "the mean wall-clock seconds per iteration" of every component and not only the acquisition, and notes that "the CPM of RESCUE is learned once per run". It lists CPM learning, GP fitting, the reliability mixture and acquisition optimization separately. The cubic complexity is not specific to RESCUE. Section 4 states the O((NJ)3)\mathcal{O}((NJ)^3) complexity of MF-CGP inference. The baselines also use a multi-output GP over all outputs (Appendix A.2).

(3/3)

W7.

  • Shared initial design. Sharing the initial design is not a confound. It is what makes the comparison fair. As Section 6.1 and Appendix A.1 state, all methods start from the same initial trials in each run, so differences in results come from the methods.
  • Robot cost model. The cost model does not form a cyclic dependency. Modeling an unknown evaluation cost with a GP on the log cost and weighting the acquisition by its inverse is standard practice (Snoek et al., NeurIPS 2012, Section 3.2). BoTorch provides this as its cost-aware utility, which we use unchanged (Balandat et al., NeurIPS 2020). Appendix A.3.2 states the setup. The cost of a robot run is known only after the run, so a GP on the log cost predicts it for the acquisition. After each evaluation, the budget uses its true cost in place of the prediction. We fit this GP once on evaluations of the benchmark model before optimization starts. We never refit it during optimization. No method's queries change its cost predictions, so no loop exists. RESCUE and HVKG use the same cost GP. We added this detail to Appendix A.3.2 in the revision.
  • Single YAHPO instance. The number of YAHPO instances does not change how we isolate the causal contribution. Our ablation isolates it within each problem by setting ω=0\boldsymbol{\omega} = \mathbf{0} and keeping every other component fixed (see W4). Constrained XGBoost HPO is one of five problems, and RESCUE has lower AUR than HVKG on all five (Table 6).
  • Single-fidelity methods. Appendix A.2 already explains why a single-fidelity method reveals little here. Such a method queries only the target fidelity and is unaffected by misleading lower fidelities by construction. Each of its queries on Branin-Currin and Park costs about 121 times a query at the lowest fidelity. This cost ratio alone determines its regret over cost from the first query. For this reason, we add the fidelity to the design variables of qEHVI. Corollary 5.9 addresses robustness to mismatched fidelities directly. Its bound for RESCUE does not depend on the magnitude of the cross-fidelity bias.

W8.

  • Initial cost. The initial costs differ between seeds. Table 2 reports initial cost as a range over 20 seeds: 6.5–13.1 on Branin-Currin, 11.9–20.8 on Park, 4.7–6.9 on Synthetic SCM, 1440–2152 on robot navigation, and 66–120 on constrained XGBoost HPO. Costs match only across methods within a seed, because all methods start from the same initial trials in each run, as Section 6.1 and Appendix A.1 state. This is what makes the comparison fair: every method begins from the same data and the same spent cost, so differences in results come from the methods.
  • AUR values. The AUR values of Table 6 do not form arithmetic sequences, within a problem or across problems. For example, the three baseline AUR values for robot navigation are −3961.45, −4019.72 and −4025.31, which differ by 58.27 and 5.59. The only near-even values in Table 6 are the gains over MOMF on the three synthetic problems (67.31, 112.77, 158.66). Each gain is computed from the AUR of RESCUE and of MOMF on the same problem as (RESCUE − MOMF) / |MOMF|. On Park, for example, it is (22.11 − 10.39) / 10.39 ≈ 112.8%.
  • Trajectories. The per-run trajectories are already public. The code linked in our Reproducibility Statement includes the run histories of every method and seed (experiments/results/data/run_histories) and the recommended Pareto sets (experiments/results/data/fronts).

Q1. We added this estimate to Appendix A.5 in the revision (Eqs. 7 and 8, Table 8). For each run, we evaluate the noiseless outputs and the causal prior at 10,000 Sobol configuration–fidelity pairs. The estimate ξ^\hat{\xi} is the largest absolute difference between them, divided by the standard deviation of each output. This follows Assumption 5.2 with w=0\boldsymbol{w} = \mathbf{0}, which gives an upper bound on ξ\xi. To amplify the residual, we learn the CPM from 10% of D^\hat{\mathcal{D}} and from shuffled outcomes. Both increase ξ^\hat{\xi} and the AUR on all five problems, consistent with the regret bound of Theorem 5.8.

Q2. The main theoretical result is Theorem 5.8, which holds for Algorithm 1 as used in the experiments. The sublinear rate is a secondary result for a variant (Appendix B.6), and we use Algorithm 3 only to prove it. Its forced queries incur the cost of the target fidelity independent of their expected hypervolume gain. VoCI-HVKG weighs this gain against the cost of each query (see W2). We added this to the Discussion (Section 7) in the revision.

Reviewer_wwUBOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends Causal Bayesian Optimization (CBO) to the multi-objective and multi-fidelity setting. To this end, it adapts the causal prior from CBO to incorporate multiple fidelities and objectives via tensor-product constructions, and adapts the existing hypervolume knowledge gradient (HVKG) acquisition function to this multi-fidelity setting. Theorem 5.8 provides a confidence set-based anytime guarantee; convergence is established under additional forced exploration (Alg 3). Experiments demonstrate improvement over non-causal baselines.

Critical strengths
  • This particular combination of multi-fidelity, multi-objective and causal BO is sensible and has not been considered.
Critical weaknesses
  • The paper is a rather incremental and not particularly surprising combination of existing ideas (causal BO, tensor product kernels across modalities and fidelities, hyper volume-focused acquisition functions etc.)
  • While Figure 1 suggests that experiments involve sim-to-real transfer for a robotic navigation tasks, it appears that the “real robot” experiments are simulated experiments on a surrogate fitted on real data (Sec. 6.1). While the authors motivate this by reproducibility, I would have considered including additional experiments demonstrating downstream utility on the real robotic platform much more compelling.
Questions for authors
  • I would appreciate a correction in case my assessment on the sim-to-real experiments (under weaknesses) is mistaken.
2 comments

Thank you for your review, and for noting that combining multi-fidelity, multi-objective and causal Bayesian optimization is sensible and has not been considered before.

Robot experiments. Your reading is correct. The robot navigation experiments run on a surrogate benchmark, not on the physical robot. We built it from 300 runs: 200 on the physical TurtleBot 4 and 100 in Gazebo. Figure 1 shows these three data sources, and we revised its caption. We chose a benchmark for three reasons.

  1. The evaluation requires a known Pareto front. Beyond hypervolume, we report GD+\mathrm{GD}^{+} and IGD+\mathrm{IGD}^{+} of each method's recommended Pareto set against the true front at the target fidelity (Section 6.1), and its attainment probability as a function of distance to this front (Figure 6). On the benchmark, we obtain the front from 20 NSGA-II runs on the true objectives, each with a population of 1,000 for 100 generations (Table 5). On the physical robot, the true front is unknown, so the regret-based comparison of the paper could not be computed. A front assembled from the methods' own robot runs would favor the regions those methods explored, and could not serve as a neutral reference.
  2. Scale and pairing. Our results pair all methods and ablation variants on the same seed, so differences in Table 6 come from the methods. On the physical robot, the paired t-tests of Table 6 would require identical conditions across methods, which are hard to guarantee over weeks of hardware time.
  3. The property under test comes from real hardware. The cross-fidelity bias that RESCUE must handle is not a synthetic term. It is the gap between Gazebo, the physical robot without obstacles and the physical robot with obstacles, which the benchmark reproduces (Figure 11).

Validation. We validated the benchmark against held-out runs, 20% of the 300 that were not used for fitting (Table 10, Figure 11). We also added a ranking stability test to the revision (Appendix A.7, Table 11). We recompute the regret of every recommended Pareto set under benchmark models retrained with other random states and under models with a systematic bias sized to the test error. RESCUE keeps the lowest mean AUR under both (see also our response to Reviewer at34, W3).

This follows standard practice. A benchmark validated against held-out runs, with multiple seeds and initial designs shared across methods, follows current guidance on benchmarking Bayesian optimization in robotics (Stenger et al., 2026, Section 8.1 (https://arxiv.org/abs/2609.09403)). It is also standard practice outside robotics, including YAHPO Gym, which our constrained XGBoost HPO problem uses (Pfisterer et al., AutoML Conf 2022).

Physical deployment. Deploying the recommended configurations on the physical robot would show downstream utility and would complement the benchmark, but it cannot replace the benchmark for comparing methods. Covering the recommended sets of RESCUE and HVKG over all 20 seeds, with a single run per configuration, would take ≈9,100 physical runs, or ≈850 hours of robot time. We added this estimate to Appendix A.7 in the revision. A few deployment runs would mainly test whether the benchmark predicts outcomes on the physical robot, which our validation on held-out runs already measures.

On incrementality. RESCUE adds the following beyond combining existing methods.

  1. Unlike causal BO, which assumes a known causal graph (Section 1), RESCUE learns a causal model over configurations, fidelities, objectives and constraints from data, and uses it to set the prior mean and covariance of a multi-output multi-fidelity GP.
  2. VoCI-HVKG learns how much to trust this prior within a cost-aware, feasibility-aware acquisition.
  3. Theorem 5.8 and Corollary 5.9 give a regret bound that does not depend on the magnitude of cross-fidelity bias when the causal prior error is bounded. In comparison, the regret-difference bound of robust multi-fidelity BO (Mikkola et al., AISTATS 2023) grows exponentially with TT, and its authors state that it is useful only in the first rounds. Our bound holds at every iteration (Section 5, Appendix B.7).

We clarified this in the revision. Section 6.1 now states that the robot experiments run on a benchmark derived from real-robot samples, which provides a known true Pareto front and identical conditions for all methods within each seed. It also cites Stenger et al. (2026) for this protocol. Appendix A.7 now reports the robot time that a physical deployment of the recommended sets would require.

We hope this clarifies the experimental setup and the contribution, and we would be grateful if the reviewer would consider it in the final assessment.

Thank you for the clarification. I will keep my score.

Reviewer_VJrLOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  • Describe so called “Causal performance model” that learns causal structure for prior?
  • Describes corresponding acquisition function
  • Provide theoretical analysis on RESCUE
  • evidence RESCUE in synthetic and real world examples

Note for upcoming discussion: Assuming causal sufficiency is not a reason for a rejection, but a necessary assumption in many cases, albeit hard to justify.

Critical strengths
  • Succinct literature review
  • Comprehensive clear original idea: multi-objective multi-fidelity BO has not been discussed as in this work and this paper is therefore a valuable contribution
  • Clear improvements over multiple iterations, see questions
Critical weaknesses
  • the introduction of a causal performance model is unconventional, but not fundamental a weakness
  • causal sufficiency is a strong, but unavoidable assumption in this first step of the literature
Questions for authors

Previous questions resolved by authors in this submission:

  • Cf. Discussion 7: There seems to be one reference that has considered causal calculus in the MFBO setting: https://openreview.net/forum?id=elj9C1sqp4 -> this has been added

  • The confidence intervals in Fig2 are still quite wide and RESCUE is often enclosed in the intervals of other competing strategies -> the confidence intervals are now significant

  • What’s the purpose of introduction a ‘causal performance model’ which seems to have the identical definition to a structural causal model, with tuple {U, V, F}? Both references (pear 2009, Peters 2017) do not mention it and it seems to be mention as a fact, but I haven’t read about it in 10 years of causal inference study. If this is a new, distinct, concept or definition, what makes it unique and necessary to introduce? -> the authors have decided to continue their use of the terminology, which is no problem

  • Page 3 line 147: “Computing p(Y| do(X = x) , s) often involves evaluating intractable integrals which can be approximated using observational data” Do you have a reference for this? I am unsure how intractable integrals are related to approximations using observational data. -> this has been resolved by removal

  • The healthcare example is a standard in the CBO literature, but it seems to contrived to aim to minimise Statin and PSA. Statin is a drug, i.e. an intervention, and it’s hard to make a case that this is conceptually sound to minimise? -> this has been resolved by removal

Remaining question:

  • How do you propose to extend this work into a problem space where causal sufficiency does not hold?
1 comment

Thank you for the careful review and the positive assessment.

RESCUE takes the CPM as an input (Section 3.2). Our experiments learn the CPM with PC and DirectLiNGAM, which assume causal sufficiency, and a causal discovery method that relaxes this assumption (such as FCI) can replace them. We see two directions for extending RESCUE beyond causal sufficiency.

  1. Discovery under latent confounding. With latent confounders, no DAG over the observed variables describes the causal structure. FCI produces a partial ancestral graph, and some interventional effects are then not identifiable. Jaber et al. (ICML 2019) give a complete algorithm that computes an interventional distribution from this graph whenever it is identifiable. RESCUE could use these identified effects as the prior mean. For an effect that is not identifiable, partial identification gives a lower and an upper bound on its value. The prior mean could then take the midpoint of these bounds, with a prior variance that increases with the gap between them. If the midpoint is far from the true effect, the causal trust ω^\hat{\boldsymbol{\omega}} decreases the weight of this prior, as it does for any misspecified causal prior in the current formulation.
  2. Interventions from the optimization loop. RESCUE learns the CPM once from D^\hat{\mathcal{D}}, before optimization starts. At each iteration, VoCI-HVKG selects the configuration and fidelity of the next query, and the result is added to the interventional data D\mathcal{D}. Latent confounders cannot bias the effect of the configurations and fidelities in D\mathcal{D} on the outcomes. RESCUE could pool D^\hat{\mathcal{D}} and D\mathcal{D} and add a variable that marks whether each sample is observational or interventional. FCI could relearn the causal graph from the pooled data, following joint causal inference (Mooij et al., JMLR 2020). RESCUE could then refine the CPM as D\mathcal{D} accumulates.

We added these directions to the Discussion (Section 7) in the revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision

We thank all reviewers for their feedback. We uploaded a revised PDF with the changes below. Table, figure and equation numbers in all our responses refer to the revised version.

New results

  • Table 11, Appendix A.7 (Reviewers at34, wwUB). Ranking stability of the robot navigation benchmark. We recompute the regret of every recommended Pareto set under benchmark models retrained with other random states and under models with a systematic bias sized to their test error. RESCUE has the lowest AUR on every one of the ten alternative models.
  • Table 8 and Eqs. 7 and 8, Appendix A.5 (Reviewer at34). A data-driven estimate of the causal-model error ξ\xi on every benchmark. Learning the CPM from 10% of D^\hat{\mathcal{D}} or using the misspecified causal prior increases both the estimate and the AUR on all five problems, consistent with Theorem 5.8.

Both new experiments reproduce from the code linked in our Reproducibility Statement.

Clarifications

  • Section 6.1 (Reviewer wwUB). The robot navigation experiments run on a benchmark derived from real-robot samples, which provides a known true Pareto front and identical conditions for all methods within each seed.
  • Figure 1 (Reviewer wwUB). The caption now states that the three settings are the data sources of the robot navigation benchmark.
  • Appendix A.7 (Reviewer wwUB). The robot time that a physical deployment of the recommended sets would require.
  • Section 7 (Reviewers VJrL, at34).
    • Hidden confounders, and directions for relaxing causal sufficiency under latent confounding.
    • Why Algorithm 1 does not guarantee the decay of σˉT\bar{\sigma}_T and why Algorithm 3 is used only for the rate.
  • Appendix A.3.2 (Reviewer at34). The cost GP is fit once before optimization starts.
  • Table 3. The robot row now states that the benchmark model is trained on 240 of the 100 Gazebo and 200 real-robot runs, with 60 held out.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.