- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines whether decision-making aimed at avoiding undesired futures requires full identification of exact causal effects. It defines a binary influence indicator representing whether the desired outcome is at least as likely as its complement, and shows that this information is weaker than the full interventional distribution. It then characterizes when the indicator can be determined using partial knowledge of common causes rather than a complete graph, and proposes a procedure that actively queries an expert about whether variables are common causes. Synthetic tasks and BERMUDA-based simulations show that the indicator can be determined with fewer queries than are needed for precise effect estimation.
Critical strengths
- The paper clarifies how much information is actually needed for decision-making. Theorem 1 formalizes the distinction between the influence indicator for a fixed task and full interventional information (p. 4). It clearly demonstrates that recovering exact causal effects and identifying actions that meet a particular success criterion are different problems.
- The conditions for determining the indicator from partial knowledge are concrete. Theorem 2 gives a necessary and sufficient condition: all common-cause candidates compatible with the observational distribution must yield the same indicator (p. 7). In Appendix C.2, two candidates assign different success probabilities to an action, 0.55 and 0.625, yet support the same action judgment because both exceed 0.5 (pp. 17–18). This example directly illustrates a situation in which neither the full common-cause set nor the corresponding effect needs to be identified exactly.
- The theory is connected to a knowledge acquisition procedure. Querying additional common-cause information when candidates disagree, and stopping when they agree, provides a concrete use of the theory. In Table 1, the MI strategy has a lower mean query count than the comparison strategies, and DYAD and TETRAD show a clear difference between the query counts required to determine the indicator and to estimate the effect (pp. 7–8).
Critical weaknesses
-
The theory concerns a restricted success criterion rather than general optimal action selection. The basic indicator in Definition 4 determines whether the success probability is at least 0.5 (p. 4). It cannot distinguish between actions with success probabilities of 0.51 and 0.95, and does not directly account for action costs or outcome-specific utilities. The authors explain that other positive odds thresholds can be accommodated, but the current experiments focus on a single criterion. The contribution is therefore best interpreted as determining whether task-specific criteria are met, rather than solving the full decision-making problem.
-
There is a gap between the exact determinability guarantee and practical computation and estimation. Theorem 2 takes as input the set of candidates compatible with the exact observational distribution. The practical procedure relies on a Markovian setting, observable common causes, and correct expert answers, and the number of candidates may grow exponentially with the number of unresolved variables (Section 4, pp. 5–7). The current guarantees do not cover the risk of misclassifying probabilities near the threshold with finite samples or the effects of incorrect expert answers. The query scores are also heuristic and should not be read as guaranteeing the minimum number of queries.
Questions for authors
NA
Response to Reviewer 4BjC
Thank you for recognizing our work and for the insightful feedback. We hope the following responses can fully address your concerns; please let us know if any points remain unclear.
Following your suggestion, we extended our empirical evaluation beyond the single odds threshold of one. Specifically, we evaluate MI on Tetrad across multiple probability thresholds τ∈0.35,0.50,0.65 (the odds threshold of 1 corresponds to the probability threshold of 0.5):
The results show that when τ=0.35 or τ=0.50, discerning the influence indicator requires far fewer queries than causal-effect estimation. When τ=0.65, the required query counts become comparable because the threshold is very close to the true interventional probability of 0.6625. The margin is only 0.0125, smaller than the effect-estimation tolerance 0.02. A smaller margin requires more precise adjusted probabilities and thus tends to require more queries.
Thank you for raising these points.
Our theoretical results are established at the population level. With finite samples, smaller sample sizes naturally make discerning the influence indicator more difficult, requiring more queries or observational samples to avoid misclassification. This is reflected in the multi-threshold experiment above (Q1) and also in the experiments with varying sample sizes in our response to Reviewer TpjS (Q3). Nevertheless, the experimental results generally remain consistent with the implications of our theoretical findings.
As noted in Section 4, the number of candidate common-cause structures can grow rapidly with the number of variables. In practice, the candidate space can be reduced using domain-specific ordering constraints (e.g., excluding variables known to occur after the actionable variable). We also noted that the query scores are information-theoretic heuristics. Designing query strategies with theoretical guarantees on the number of required queries is an interesting direction for future work.
Thank you for the thoughtful comments. We have incorporated your suggestions into Section 4 of the updated manuscript.