- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an Evidential Casual Discovery (EviCD) framework that reformulates pairwise casual discovery over a 4-modeled relations Fundamentally, the authors tried to solve a problem in traditional casual discovery by predicting Dempster-Shafer basic probability assignments (BPA) via three restricted-focal network heads rather than predicting scalar probabilities or single-point estimate graphs. The proposed method produces two diagnostic metrics (vacuity, and conflict), which measures support left uncommitted for selective abstention, and incompatibility among channel-level assessments respectively. Experimentally, the authors demonstrated EviCD competitive nature in structural recovery, selective prediction, while also providing explicit continuous confounding score.
Critical strengths
- Unlike traditional methods that output scalar edge probabilities or single point-estimate graphs, EviCD predicts Dempster–Shafer mass functions across 4 modeled relations. 2.EviCD achieves the highest aggregate selective prediction score on synthetic benchmarks. EviCD also maintains high selective quality as graph scales compared to AVICI.
- Evaluation of unseen external benchmarks to further prove the transfer of this without retraining.
Critical weaknesses
- The adopted a pignistic likelihood learning, which is a many-to-one mapping. As a result,the separation into vacuity and conflict could be driven by architecture parameterization, not necessarily the observed data.
- EviCD does not uniformly lead in traditional point-estimate structure metrics across all settings, where specialized continuous optimization methods excel.
Questions for authors
- Although, the model reliance on stimulation protocol is substantiated by the authors. Could the authors include experiments in which the performance changes with out-of-distribution shifts?
- Current training penalizes vacuity when observational data cannot identify the direction. Can training objectives be redesigned to prevent artificial suppression of vacuity?
- Although, the authors presented a well organized paper. They failed to mention the rationale behind the GraphSAGE Encoder?
3 comments
I like to appreciate the authors for the elaborate responses. Nonetheless, I am of the opinion that the complementary role of EviCD should be leading and emphasized.
Moreso, the argument as regards artificial suppression of vacuity is not well substantiated.
Nevertheless, the responses are largely convincing and offer a positive outlook for extension.
Thank you for the constructive comments and positive feedback. We will more clearly emphasize the complementary role of EviCD in the paper. The main contribution is not that vacuity or conflict replaces predictive confidence, but that EviCD augments competitive causal-relation prediction with explicit diagnostics of uncommitted and incompatible evidence.
We agree that this mechanism is not fully isolated by the current experiment. Nevertheless, the Gaussian/Laplace stress test provides supporting evidence that singleton supervision can limit evidential imprecision under directional ambiguity. We will clarify this interpretation and present equivalence-aware/set-valued supervision as a natural extension.
Thank you again for helping us clarify and strengthen the paper.
Thank you for recognizing the value of the structured evidential representation, selective prediction, and zero-shot external transfer. We address the remaining concerns below.
1. Pignistic likelihood and the separation of vacuity and conflict
We agree that the pignistic likelihood does not uniquely identify the underlying BPA: different mass functions can induce the same pignistic probabilities. We therefore do not interpret m(Ω) and K as uniquely data-identified uncertainty components.
Within EviCD, however, vacuity and conflict have different evidential meanings: m(Ω) measures uncommitted support, whereas K measures incompatibility among committed channel evidence. The restricted focal families and relation-specific heads constrain the admissible decompositions, while observations determine the instance-specific outputs. We therefore interpret them as model- and training-distribution-relative diagnostics. The controlled perturbations (Table 2; Appendix H, Tables 8–10) support this distinction: noise, sample size, and signal strength primarily affect vacuity, whereas increasing latent-confounder strength produces a stronger response in conflict.
2. Point-estimate structural performance
We agree that EviCD does not uniformly outperform methods specialized for conventional point-estimate structural recovery.
Its main advantage is complementary: EviCD provides a structured evidential representation of each causal-relation decision, including uncommitted support and cross-channel conflict, supporting selective prediction and inspection of the evidence state in addition to structural recovery. This advantage is also reflected in the zero-shot external evaluations (Table 3; Appendix J, Table 16), where EviCD remains competitive structurally while showing strong selective performance under heterogeneous distribution shifts.
We will therefore position EviCD as providing competitive causal-relation prediction, zero-shot transfer, and explicit evidential diagnostics, rather than uniform superiority on conventional structural metrics.
3. Out-of-distribution transfer
Our submitted paper already includes a zero-shot distribution-shift evaluation. EviCD is trained only on synthetic ER/SF data and applied without retraining to Sachs, DREAM4, and Smith fMRI. As reported in Table 3 and Appendix J (Table 16), EviCD obtains the lowest SHD and highest selective score on Sachs, the highest AUROC and lowest SHD on DREAM4, and the highest selective score with competitive SHD on fMRI. These results support useful zero-shot transfer of the learned relation representation and selective ranking under external distribution shifts. We will make this setting and the corresponding results more explicit in the revised manuscript.
4. Preventing artificial suppression of vacuity
We agree that singleton supervision can suppress evidential imprecision when observations do not identify a unique direction. A natural solution is to extend EviCD with equivalence-aware/set-valued supervision. If observations support a direct relation but cannot distinguish Dij from Dji, the target can be the set {Dij, Dji} rather than either directional singleton, allowing the model to assign mass directly to the unresolved directional set.
Directional non-identifiability also does not necessarily imply complete vacuity. Mass on {Dij, Dji} represents unresolved orientation, whereas m(Ω) represents broader lack of commitment. The Gaussian/Laplace stress test in Appendix H.3 therefore illustrates how singleton supervision can suppress evidential imprecision under directional ambiguity and motivates supervision that preserves unresolved orientation explicitly.
5. Why GraphSAGE?
We choose GraphSAGE because pairwise causal relations can depend on broader multivariate context rather than only statistics of (i,j). Message passing allows each variable representation to incorporate information from other variables before constructing ordered-pair representations. Its shared inductive aggregation also supports different graph sizes and external datasets. GraphSAGE is a standard backbone rather than a methodological contribution; other permutation-equivariant GNN encoders could replace it.
We will revise the manuscript to clarify these claim boundaries, make the zero-shot transfer setting more explicit, describe equivalence-aware supervision as a natural extension, and add the rationale for the GraphSAGE encoder.