acceptodds
Under review as a conference paper at ICLR 2027

On the Sufficiency of Rehearsal in Avoiding Undesired Future

Abstract

When a model predicts an undesired outcome, deciding how to avoid it naturally calls for identifying causal relations between actions and outcomes. Prior studies suggest that effective actions can often be chosen by judging their influence on desired outcomes through rehearsal in a hypothetical world, even without faithfully modeling the underlying causal mechanisms. This raises a foundational question: whether rehearsal generally suffices for decision-making without identifying those mechanisms, and if so, what information it requires beyond vanilla prediction. We address this by formalizing the influence information required for decision-making with structural equation models and showing that this information does not in general entail complete causal identification. We further characterize the conditions under which the desired influence is discernible from observational data and develop a principled procedure for determining it under partial structural knowledge. Our theoretical and empirical results support the feasibility of rehearsal in avoiding undesired outcomes with limited structural knowledge.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Avoiding Undesired Future, Rehearsal Learning, Influence

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4BjCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether decision-making aimed at avoiding undesired futures requires full identification of exact causal effects. It defines a binary influence indicator representing whether the desired outcome is at least as likely as its complement, and shows that this information is weaker than the full interventional distribution. It then characterizes when the indicator can be determined using partial knowledge of common causes rather than a complete graph, and proposes a procedure that actively queries an expert about whether variables are common causes. Synthetic tasks and BERMUDA-based simulations show that the indicator can be determined with fewer queries than are needed for precise effect estimation.

Critical strengths
  1. The paper clarifies how much information is actually needed for decision-making. Theorem 1 formalizes the distinction between the influence indicator for a fixed task and full interventional information (p. 4). It clearly demonstrates that recovering exact causal effects and identifying actions that meet a particular success criterion are different problems.
  2. The conditions for determining the indicator from partial knowledge are concrete. Theorem 2 gives a necessary and sufficient condition: all common-cause candidates compatible with the observational distribution must yield the same indicator (p. 7). In Appendix C.2, two candidates assign different success probabilities to an action, 0.55 and 0.625, yet support the same action judgment because both exceed 0.5 (pp. 17–18). This example directly illustrates a situation in which neither the full common-cause set nor the corresponding effect needs to be identified exactly.
  3. The theory is connected to a knowledge acquisition procedure. Querying additional common-cause information when candidates disagree, and stopping when they agree, provides a concrete use of the theory. In Table 1, the MI strategy has a lower mean query count than the comparison strategies, and DYAD and TETRAD show a clear difference between the query counts required to determine the indicator and to estimate the effect (pp. 7–8).
Critical weaknesses
  1. The theory concerns a restricted success criterion rather than general optimal action selection. The basic indicator in Definition 4 determines whether the success probability is at least 0.5 (p. 4). It cannot distinguish between actions with success probabilities of 0.51 and 0.95, and does not directly account for action costs or outcome-specific utilities. The authors explain that other positive odds thresholds can be accommodated, but the current experiments focus on a single criterion. The contribution is therefore best interpreted as determining whether task-specific criteria are met, rather than solving the full decision-making problem.

  2. There is a gap between the exact determinability guarantee and practical computation and estimation. Theorem 2 takes as input the set of candidates compatible with the exact observational distribution. The practical procedure relies on a Markovian setting, observable common causes, and correct expert answers, and the number of candidates may grow exponentially with the number of unresolved variables (Section 4, pp. 5–7). The current guarantees do not cover the risk of misclassifying probabilities near the threshold with finite samples or the effects of incorrect expert answers. The query scores are also heuristic and should not be read as guaranteeing the minimum number of queries.

Questions for authors

NA

1 comment
AuthorsOpenReview ↗

Response to Reviewer 4BjC

Thank you for recognizing our work and for the insightful feedback. We hope the following responses can fully address your concerns; please let us know if any points remain unclear.


The authors explain that other positive odds thresholds can be accommodated, but the current experiments focus on a single criterion. The contribution is therefore best interpreted as determining whether task-specific criteria are met, rather than solving the full decision-making problem.

Following your suggestion, we extended our empirical evaluation beyond the single odds threshold of one. Specifically, we evaluate MI on Tetrad across multiple probability thresholds τ∈0.35,0.50,0.65\tau \in \\{0.35, 0.50, 0.65\\} (the odds threshold of 1 corresponds to the probability threshold of 0.5):

ThresholdQRQ_RQIQ_I
0.350.00±0.000.00 \pm 0.004.52±0.914.52 \pm 0.91
0.502.05±0.302.05 \pm 0.304.52±0.914.52 \pm 0.91
0.654.71±1.044.71 \pm 1.044.52±0.914.52 \pm 0.91

The results show that when τ=0.35\tau = 0.35 or τ=0.50\tau = 0.50, discerning the influence indicator requires far fewer queries than causal-effect estimation. When τ=0.65\tau = 0.65, the required query counts become comparable because the threshold is very close to the true interventional probability of 0.66250.6625. The margin is only 0.01250.0125, smaller than the effect-estimation tolerance 0.020.02. A smaller margin requires more precise adjusted probabilities and thus tends to require more queries.


The practical procedure relies on a Markovian setting, observable common causes, and correct expert answers ... or the effects of incorrect expert answers.

Thank you for raising these points.

  • On the Markovian setting. Our analysis in Section 3 incorporates general recursive SCMs including semi-Markovian models. Also, we show that the gap between causal identification and rehearsal discernibility is already non-trivial even under a Markovian setting, by providing a concrete example in our response to Reviewer HqEL (Q1).
  • On incorrect expert answers. The sensitivity experiments with corrupted expert responses show that the binary influence indicator can tolerate expert errors when the threshold margin is moderate (see Reviewer HqEL, Q2).

Theorem 2 takes as input the set of candidates compatible with the exact observational distribution. ... The current guarantees do not cover the risk of misclassifying probabilities near the threshold with finite samples ...

Our theoretical results are established at the population level. With finite samples, smaller sample sizes naturally make discerning the influence indicator more difficult, requiring more queries or observational samples to avoid misclassification. This is reflected in the multi-threshold experiment above (Q1) and also in the experiments with varying sample sizes in our response to Reviewer TpjS (Q3). Nevertheless, the experimental results generally remain consistent with the implications of our theoretical findings.


... the number of candidates may grow exponentially with the number of unresolved variables (Section 4, pp. 5–7). ... The query scores are also heuristic and should not be read as guaranteeing the minimum number of queries.

As noted in Section 4, the number of candidate common-cause structures can grow rapidly with the number of variables. In practice, the candidate space can be reduced using domain-specific ordering constraints (e.g., excluding variables known to occur after the actionable variable). We also noted that the query scores are information-theoretic heuristics. Designing query strategies with theoretical guarantees on the number of required queries is an interesting direction for future work.

Thank you for the thoughtful comments. We have incorporated your suggestions into Section 4 of the updated manuscript.

Reviewer_TpjSOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper looks at the Avoiding Undesired Future (AUF) problem and provides formal results for Rehearsal as an effective process for AUF. The authors use SCM and provide sufficiency results for discernibility of influence under different forms of structural knowledge, e.g., graph structure. Finally, the authors provide a querying algorithm for influence discernibility. Experiments are conducted to show efficacy of the model.

Critical strengths
  1. The problem is interesting.
Critical weaknesses
  1. Most experiments are on synthetic datasets. While the experiments exist on one real world dataset BERMUDA, the results suggest that a single query is enough for achieving 0 error.

  2. The choices of baselines are by design restricted versions of MI. Z-MI and Y-MI show very similar performance on Triad and Tetrad datasets in Fig 3. While Y-MI seems to be doing better than or exactly the same as MI in Figure 6 across all plots. This reduces the significance of the empirical evidence provided for the developed theory.

  3. Experiments are very limited. In principle, I would perform experiments to adversarially test the underlying method, which is not done in this case.

  4. Z-MI and Y-MI are missing in real data? Am I missing something here?

  5. Writing has to be carefully checked. Section 5 says each experiment is repeated ten times, while Appendix D.1 says 1,000 times. Which one produced the reported standard deviations?

Questions for authors
  1. Can the authors provide more experiments to support and motivate the usefulness of their theoretical results?
  2. Why standard deviation in Z-MI in dyad is very high?

I think the authors should design and test their method on more adversarial setting than a simpler one which merely support their method. I put the rating as 2 but I implicitly tend to reject the paper, than accept it at the moment.

5 comments
AuthorsOpenReview ↗

Response to Reviewer TpjS (Part 1)

Thank you for the thoughtful feedback. We hope the following responses can fully address your concerns; please let us know if any points remain unclear.


While the experiments exist on one real world dataset Bermuda, the results suggest that a single query is enough for achieving 0 error.

Thanks for this comment. We would like to point out that needing only 1 query to reach 0 indicator error on Bermuda is not a limitation of the case study, but clear evidence supporting our theoretical implications.

Specifically, for actionable variables such as TATA and DICDIC in Bermuda, there are three true common causes in the underlying causal graph.

  • Before querying (t=0t=0), confounding bias makes the unadjusted estimator misclassify the influence indicator (eR>0e_R > 0).
  • After querying just 1 common cause using the proposed strategy, the remaining confounding bias from the other two unqueried common causes is already small enough. As a result, the estimated probability crosses to the correct side of the decision threshold, driving eRe_R to 0.
  • Meanwhile, as shown in Figure 5 (bottom) and Figure 7, the causal-effect deviation eIe_I is still noticeable at t=1t=1 and continues to decrease in subsequent steps as the remaining confounders are queried.

In short, this demonstrates that correct rehearsal decisions can be made with limited structural knowledge well before the causal effects are accurately estimated, providing strong empirical support for our main theoretical insight.


The choices of baselines are by design restricted versions of MI. Z-MI and Y-MI show very similar performance on Triad and Tetrad datasets in Fig 3. While Y-MI seems to be doing better than or exactly the same as MI in Figure 6 across all plots. This reduces the significance of the empirical evidence provided for the developed theory.

Thanks for the comments. Below, we clarify the purpose of the experiments, and explain the results of baselines one by one.

  1. Empirical support for our theoretical implications: Our main theoretical insight is that discerning influence indicators requires less structural information than identifying causal effects. This is reflected in our experimental results, particularly in Table 1, where QR<QIQ_R < Q_I holds generally across methods. It is the gap between QRQ_R and QIQ_I, not the performance difference between methods, that supports our theoretical implications.
  2. Why Z-MI and Y-MI show similar performance on Triad and Tetrad: Z-MI and Y-MI serve as natural one-sided ablation baselines of MI. In the synthetic tasks, their first queries are misled by the action's ancestor AA and the outcome's ancestor BB, respectively, causing both methods to require one additional query compared to MI.
  3. Why Y-MI performs similarly to MI on Bermuda: In the causal graph of Bermuda given in Appendix D.2, the true common causes lie upstream and affect the outcome through multiple paths. They have high mutual information with the outcome, which dominates the ranking, so the variables most informative about the outcome (selected by Y-MI) happen to coincide with the true common causes.
AuthorsOpenReview ↗

Response to Reviewer TpjS (Part 2)

Can the authors provide more experiments to support and motivate the usefulness of their theoretical results?

Thanks for the suggestion. Following your suggestion, we have added new experiments to further show the usefulness of our framework: (1) an additional benchmark based on the Sachs protein-signaling network (Sachs et al., 2005), and (2) a sample-efficiency study across different observational sample sizes.

  1. Additional benchmark (Sachs): We evaluate on the widely used Sachs Bayesian network (from the bnlearn repository). We generate 100,000 observational samples in each of ten runs, as in Bermuda. We take Erk=HIGH\textit{Erk}=\text{HIGH} as the desired outcome, and evaluate PKA, Raf, and Mek as actionable variables. The required query counts (QRQ_R and QIQ_I) are reported below:
MethodPKA (QRQ_R / QIQ_I)Raf (QRQ_R / QIQ_I)Mek (QRQ_R / QIQ_I)
Random4.80±2.604.80 \pm 2.60 / 7.00±2.727.00 \pm 2.720.00±0.000.00 \pm 0.00 / 6.40±1.696.40 \pm 1.694.10±2.344.10 \pm 2.34 / 5.50±3.175.50 \pm 3.17
MI2.00±0.002.00 \pm 0.00 / 7.20±1.837.20 \pm 1.830.00±0.000.00 \pm 0.00 / 3.70±0.463.70 \pm 0.462.00±0.002.00 \pm 0.00 / 3.60±3.203.60 \pm 3.20
  • We make the following observations:
    • Across all cases, discerning the influence indicator consistently requires fewer queries than estimating the causal effects.
    • For Raf, consensus is reached without any queries. For PKA and Mek, MI requires only 2 queries, while Random requires more than 4 queries.
  1. Sample efficiency: We also evaluate MI on synthetic tasks across various observational sample sizes N∈100,500,1000,5000,10000N \in \\{100, 500, 1000, 5000, 10000\\} (over 1,000 repetitions per task). Results are summarized as follows:
TaskN=100N=100 (QRQ_R / QIQ_I)N=500N=500 (QRQ_R / QIQ_I)N=1000N=1000 (QRQ_R / QIQ_I)N=5000N=5000 (QRQ_R / QIQ_I)N=10000N=10000 (QRQ_R / QIQ_I)
Dyad3.24±2.053.24 \pm 2.05 / 4.68±1.014.68 \pm 1.011.08±0.891.08 \pm 0.89 / 4.40±1.064.40 \pm 1.061.07±0.761.07 \pm 0.76 / 4.08±1.204.08 \pm 1.201.01±0.321.01 \pm 0.32 / 2.86±1.282.86 \pm 1.281.00±0.151.00 \pm 0.15 / 2.29±0.862.29 \pm 0.86
Triad5.09±1.645.09 \pm 1.64 / 5.39±1.495.39 \pm 1.494.44±1.094.44 \pm 1.09 / 5.32±1.095.32 \pm 1.094.27±0.974.27 \pm 0.97 / 5.18±1.095.18 \pm 1.093.29±0.573.29 \pm 0.57 / 3.79±1.183.79 \pm 1.183.04±0.223.04 \pm 0.22 / 3.25±0.793.25 \pm 0.79
Tetrad4.81±2.704.81 \pm 2.70 / 6.57±1.346.57 \pm 1.342.68±1.522.68 \pm 1.52 / 6.29±1.106.29 \pm 1.102.57±1.232.57 \pm 1.23 / 6.23±1.036.23 \pm 1.032.20±0.622.20 \pm 0.62 / 5.31±1.145.31 \pm 1.142.05±0.302.05 \pm 0.30 / 4.52±0.914.52 \pm 0.91
  • As shown in the table, discerning the influence indicator consistently takes fewer queries than estimating the causal effects in all cases. While a very small sample size (N=100N = 100) needs more queries due to finite-sample variance, moderate sample sizes (N=1,000N = 1,000) already allow our method to discern the influence indicator with few queries (close to the results at N=10,000N = 10,000). This further validates the insight that discerning influence indicators requires less structural information than estimating causal effects when observational data is limited.
AuthorsOpenReview ↗

Response to Reviewer TpjS (Part 3)

In principle, I would perform experiments to adversarially test the underlying method, which is not done in this case.

Following your suggestion, we designed two additional experimental settings to further evaluate our methods:

  1. Adversarial distractor setting (Adv-Distractor). We design an SCM with 3 common causes, 4 action-only distractors (which drive the actionable variable via an AND-gate but have zero effect on outcome), and 3 outcome-only distractors (which drive outcome via a threshold gate but have zero effect on action). This is specifically designed to fool one-sided heuristics so that Z-MI tends to prioritize action-only distractors, while Y-MI tends to prioritize outcome-only distractors.
  2. Adversarial near-threshold setting (Adv-Boundary). We modify the structural equation of outcome in Tetrad so that its true interventional probability (0.515) lies very close to the decision threshold 0.5. This is used to test whether Algorithm 2 can still stop early when the threshold margin is narrow.

Results over 100 repetitions are reported as follows:

MethodAdv-Distractor (QRQ_R)Adv-Distractor (QIQ_I)Adv-Boundary (QRQ_R)Adv-Boundary (QIQ_I)
Random3.20±2.403.20 \pm 2.407.99±2.017.99 \pm 2.014.66±0.994.66 \pm 0.995.34±0.835.34 \pm 0.83
Z-MI5.00±0.955.00 \pm 0.956.95±0.226.95 \pm 0.224.38±0.494.38 \pm 0.494.77±0.424.77 \pm 0.42
Y-MI4.00±0.924.00 \pm 0.925.99±0.105.99 \pm 0.105.24±0.455.24 \pm 0.455.77±0.425.77 \pm 0.42
MI1.16±0.441.16 \pm 0.442.98±0.142.98 \pm 0.144.25±0.464.25 \pm 0.464.77±0.424.77 \pm 0.42

We make the following observations:

  • In Adv-Distractor, Z-MI and Y-MI are misled by their respective distractors and require more queries than Random to discern the influence indicator. MI performs better than others as it assigns near-zero scores to the distractors.
  • In Adv-Boundary, while the narrow margin naturally requires more queries to resolve the uncertainty, discerning the influence indicator still requires fewer queries than estimating causal effects across all methods.

Z-MI and Y-MI are missing in real data?

Thanks for pointing this out. Actually, the results of all four strategies (including Z-MI and Y-MI) on the Bermuda dataset were presented in Figures 6 and 7 in Appendix E, which were deferred to the appendix due to space limitation. To avoid confusion, we have added a clearer pointer to Figures 6 and 7 in Section 5 of the updated manuscript.


Writing has to be carefully checked. Section 5 says each experiment is repeated ten times, while Appendix D.1 says 1,000 times. Which one produced the reported standard deviations?

Thanks for the suggestion. We would like to clarify that the synthetic tasks were repeated 1,000 times (as stated in Appendix D.1), while the Bermuda task was repeated 10 times (as stated in Appendix D.2). We have revised Section 5 to avoid ambiguity in the updated manuscript.


Why standard deviation in Z-MI in Dyad is very high?

This is a good question. In Dyad, the particular functional form of the structural equations makes the true common causes and the action-only distractor have exactly the same mutual information with the actionable variable. This does not hold in Triad or Tetrad, where the distractor has higher mutual information with the actionable variable. Consequently, finite-sample fluctuations make the Z-MI query order less stable in Dyad, leading to larger variations in causal-effect deviation across runs.


Reference

[1] Sachs, Karen, et al. "Causal protein-signaling networks derived from multiparameter single-cell data." Science, 308(5721): 523–529, 2005.

Reviewer_HqELOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper asks whether an action makes a desired outcome exceed a threshold, without identifying the exact intervention probability. It considers possible common cause structures and asks experts questions until the structures agree on the threshold judgment. A reasonable problem, but I am not convinced the information requirements and practical benefit are sufficiently justified.

Critical strengths
  1. Useful distinction between knowing the exact probability and knowing whether it exceeds a threshold.
  2. A procedure for checking agreement under incomplete structural knowledge.
  3. Experiments comparing different strategies for selecting expert questions.
Critical weaknesses
  1. Strong assumptions: all relevant confounders observed, and expert answers about their causal roles assumed correct.
  2. Given access to these variables and experts, why not learn the relevant causal structure and apply adjustment? Missing comparison with causal discovery followed by adjustment over the remaining possible graphs.
  3. The main idea seems closely related to partial identification. Bounds can settle a threshold without identifying the probability. Not clear what rehearsal adds beyond this.
  4. The query savings are not fully convincing. Threshold correctness is compared with effect estimation within 0.02, two different accuracy requirements. Also needs clarification whether the experimental stopping rule certifies agreement across all compatible structures.
Questions for authors
  1. What is the benefit over causal discovery and adjustment, using the same observational data and expert access?
  2. How sensitive is the procedure to incorrect expert answers? Could one wrong answer remove the true structure and produce a false conclusion?
6 comments
AuthorsOpenReview ↗

Response to Reviewer HqEL (Part 1)

Thank you for the constructive feedback. We hope the following responses can fully address your concerns; please let us know if any points remain unclear.


Strong assumptions: all relevant confounders observed, and expert answers about their causal roles assumed correct.

Thanks for the comment. We would like to clarify that these assumptions are not required in Section 3:

  • In Section 3, our analysis incorporates semi-Markovian models with unobserved confounders. Also, it does not require expert queries.
  • In Section 4, we focus on Markovian models and oracle queries (a common setting in classical back-door adjustment and active structure learning) in order to cleanly study how much structural knowledge is needed for rehearsal discernibility.

Moreover, we would like to point out that this standard setting is far from trivial. Even under causal sufficiency, there is already a large gap between causal identification and rehearsal discernibility. For example, consider the following SCM (left) and its causal structure (right):

\left\\{ \begin{aligned} Z &\leftarrow N_Z, & N_Z &\sim \operatorname{Bern}(3/5),\\\\ C &\leftarrow Z\oplus N_C, & N_C &\sim \operatorname{Bern}(1/4),\\\\ Y &\leftarrow (Z\land C)\oplus N_Y, & N_Y &\sim \operatorname{Bern}(1/4). \end{aligned} \right. &\qquad \begin{array}{ccccc} && C &&\\\\ &\nearrow&&\searrow&\\\\ Z&&\longrightarrow&&Y \end{array} \end{array}$$ - *Causal discovery from observational data is challenging*. One can only identify, in this case, a completed partially directed acyclic graph (CPDAG) representing a Markov equivalence class of six possible DAGs. With the additional knowledge that $Z$ causes $Y$, there are still 3 possible DAGs, and it remains unclear whether $C$ is a common cause of $Z$ and $Y$. - In contrast, *discerning the influence indicator is much easier*. Given the same conditions as above, the influence indicator is determined as 1 (as adjusting for either $\emptyset$ or $\\{C\\}$ shifts the odds in favor of the desired event $Y=1$ ). This does *not* need to know whether $C$ is a common cause or not. This example shows that, **even under causal sufficiency, the gap between causal identification and rehearsal discernibility is already evident**. Thus, this standard setting is particularly *insightful* and *worth studying*. We have added this discussion and example in the updated manuscript. --- > How sensitive is the procedure to incorrect expert answers? Could one wrong answer remove the true structure and produce a false conclusion? Thanks for this question. While a wrong expert answer could remove the true structure, it does *not* necessarily lead to a false conclusion in rehearsal. Specifically, a wrong expert answer, which incorrectly excludes a true confounder, typically biases the adjusted probability, but as long as the probability does not cross the decision threshold, the influence indicator can still be correctly determined. For example, even if an expert mistakenly excludes the true common cause $B$ in Example 2 (Appendix C.2), leaving only the candidate $\\{A\\}$, the adjusted probability still produces the same influence indicator. To empirically evaluate this sensitivity, we corrupt each expert response independently with an error probability $\eta \in \\{0.05, 0.10, 0.20\\}$ on synthetic tasks. The results for the full-budget indicator mistake rate ($e_R$) and causal-effect deviation ($e_I$) are reported as follows: | Task | $\eta=0.00$ ($e_R$ / $e_I$) | $\eta=0.05$ ($e_R$ / $e_I$) | $\eta=0.10$ ($e_R$ / $e_I$) | $\eta=0.20$ ($e_R$ / $e_I$) | | --- | --- | --- | --- | --- | | Dyad | 0.0000 / 0.0097 | 0.0010 / 0.0172 | 0.0100 / 0.0272 | 0.0340 / 0.0463 | | Triad | 0.0000 / 0.0087 | 0.1440 / 0.0216 | 0.2620 / 0.0339 | 0.4970 / 0.0594 | | Tetrad | 0.0000 / 0.0094 | 0.0000 / 0.0230 | 0.0040 / 0.0346 | 0.0300 / 0.0576 | We make the following observations: - The results suggest that the margin between the true interventional probability and the decision threshold (0.5) plays an important role in the sensitivity to expert errors. - On Dyad and Tetrad, the true interventional probabilities (0.7 and 0.6625) are well above 0.5, and confirming a subset of common causes is already sufficient to discern the influence indicator. Thus, $e_R$ remains very low ($\le 1\\%$ when $\eta \le 0.1$, and $\approx 3\\%$ when $\eta = 0.2$). - On Triad, the true interventional probability (0.55) is very close to the threshold, and confirming all three common causes is required to obtain the correct influence indicator. It is natural that the results are more sensitive to expert errors in this case. An uncertainty-aware querying method would mitigate this issue, which we leave for future work. We have added this sensitivity analysis and discussion in the updated manuscript.
AuthorsOpenReview ↗

Response to Reviewer HqEL (Part 2)

Given access to these variables and experts, why not learn the relevant causal structure and apply adjustment?

This is a very good question. Actually, this work also adopts adjustment. We agree that if causal discovery were powerful enough to recover a comprehensive causal model, it would certainly be highly useful for the AUF problem we consider, as we also noted in Section 1 of our submission. However, causal discovery generally relies on strong assumptions (e.g., the causal faithfulness assumption) and is often error-prone in practice. Therefore, from the perspective of analyzing influence discernibility, we prefer not to rely on intermediate discovery results that are likely to introduce errors in practice.

Below, we elaborate more on why directly acquiring the underlying causal graph is difficult and costly.

  • First, learning the causal graph from observational data is inherently difficult.
    • Under standard assumptions, causal discovery from observational data can generally recover only a Markov equivalence class (MEC), represented by a CPDAG. Thus, it is often infeasible to determine the required adjustment set from observational data alone.
      • For example, consider a simple case with three variables CC, ZZ, and YY, where the true causal graph is C→Z→YC \to Z \to Y and C→YC \to Y. Under the causal faithfulness assumption, classic causal discovery methods (such as the PC algorithm) can only recover the undirected skeleton C−Z−Y−CC - Z - Y - C. Its MEC contains six possible DAGs, with no adjustment set valid across all of them.
    • In addition, the SCMs used in the tasks in Appendix D.1 do not satisfy the causal faithfulness assumption (i.e., AA is marginally independent of YY, despite the directed path A→Z→YA \to Z \to Y). In such cases, the theoretical guarantees of faithfulness-based causal discovery methods no longer hold, and these methods may fail to recover the correct MEC.
  • Second, querying experts about the causal graph is costly.
    • While one can query experts to disambiguate the remaining graphs in an MEC, recovering the full causal graph may require up to O(∣V∣2)O(|\mathbf{V}|^2) edge queries (where ∣V∣|\mathbf{V}| is the number of observed variables). In contrast, this work bypasses learning the internal DAG structure and requires at most O(∣V∣)O(|\mathbf{V}|) common-cause queries.
AuthorsOpenReview ↗

Response to Reviewer HqEL (Part 3)

Missing comparison with causal discovery followed by adjustment over the remaining possible graphs.

Thanks for your suggestion. Below we make a comparison with causal discovery followed by adjustment over compatible DAGs. Specifically, we follow an IDA-style procedure (Maathuis et al., 2009) as follows:

  1. Use the PC algorithm to learn a CPDAG from observational data.
  2. Enumerate all compatible DAGs in the CPDAG.
  3. Compute interventional probabilities using the adjustment formula for each DAG.
  4. Output the influence indicator if all adjusted probabilities give the same indicator, and output FAIL otherwise.

For a fair comparison in the active setting, here we also equip this baseline with expert queries (denoted as PC + Active Query). Specifically, by following Algorithm 2, whenever the compatible DAGs do not agree on the influence indicator, we query the oracle expert for common-cause knowledge to narrow down the compatible DAGs, and then perform adjustment over the remaining graphs.

We evaluate the mistake rate of the influence indicator and the counts of correct / wrong / FAIL results over 1,000 runs per task (with 10,000 samples in each run). Results are reported as follows:

MethodDyadTriadTetrad
PC + Adjustment98.7% (13 / 968 / 19)99.4% (6 / 978 / 16)99.9% (1 / 984 / 15)
PC + Active Query98.7% (13 / 968 / 19)99.4% (6 / 978 / 16)99.9% (1 / 984 / 15)
Ours0.0% (1000 / 0 / 0)0.0% (1000 / 0 / 0)0.0% (1000 / 0 / 0)

As shown in the table, our method achieves zero mistake rate across all tasks, while the PC-based baselines fail in most runs. We make the following observations to explain why they fail:

  • The learned CPDAGs lead to a false consensus. This is because the underlying SCMs violate causal faithfulness. As a result, the conditional independence tests in the PC algorithm often miss the directed path from ZZ to YY, and the equivalence class collapses into only 1 or 2 misspecified DAGs (with a small fraction of runs failing to produce a valid CPDAG due to directed cycles). Out of the 3,000 runs in total, the PC algorithm recovers the true full DAG only twice (2/1,000 on Dyad, and 0/1,000 on Triad and Tetrad).
  • The misspecified initial graphs make subsequent expert queries useless. Interestingly, we find that PC + Active Query has the exact same mistake rate as PC + Adjustment. The reason is that the misspecified DAGs in the learned CPDAG already agree on a wrong influence indicator at step t=0t=0. Thus, the stopping rule is triggered at the very beginning, and the expert is never queried. In contrast, our method does not rely on learning the full graph. It directly examines candidate common-cause structures, thus avoiding this issue.

We have added this comparison in the updated manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.