acceptodds
Under review as a conference paper at ICLR 2027

EviCD: Structured Evidential Uncertainty for Causal Discovery

Abstract

When causal graphs support downstream decisions, knowing which predicted relations warrant commitment can be as important as the graph itself. Existing methods typically return a point estimate or an edge confidence, which does not by itself reveal whether uncertainty reflects weak commitment or incompatible relation-specific assessments. We propose EviCD, a graph neural network that predicts per-pair Dempster–Shafer mass functions over four modeled relations: , , latent confounding without a direct edge, and no modeled relation. A restricted-focal architecture yields two complementary model-level diagnostics: vacuity , measuring support left uncommitted, and conflict , measuring incompatibility among channel-level assessments. These quantities support selective prediction, allowing EviCD to abstain on high-vacuity pairs under a retained-accuracy guarantee conditional on calibration. EviCD does not resolve observational non-identifiability; its outputs are simulator-relative assessments. On the main synthetic benchmark, EviCD attains the highest aggregate selective-prediction score among methods with an available selective confidence, while maintaining competitive structural accuracy and providing an explicit continuous confounding score. Controlled perturbations further show that increasing latent-confounder strength primarily changes , whereas changes in noise, sample size, and signal strength primarily affect .

Primary area:
causal reasoning
Keywords:
causal discovery, uncertainty quantification, evidential deep learning, Dempster-Shafer theory, identifiability, selective prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VHX4OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an Evidential Casual Discovery (EviCD) framework that reformulates pairwise casual discovery over a 4-modeled relations Fundamentally, the authors tried to solve a problem in traditional casual discovery by predicting Dempster-Shafer basic probability assignments (BPA) via three restricted-focal network heads rather than predicting scalar probabilities or single-point estimate graphs. The proposed method produces two diagnostic metrics (vacuity, and conflict), which measures support left uncommitted for selective abstention, and incompatibility among channel-level assessments respectively. Experimentally, the authors demonstrated EviCD competitive nature in structural recovery, selective prediction, while also providing explicit continuous confounding score.

Critical strengths
  1. Unlike traditional methods that output scalar edge probabilities or single point-estimate graphs, EviCD predicts Dempster–Shafer mass functions across 4 modeled relations. 2.EviCD achieves the highest aggregate selective prediction score on synthetic benchmarks. EviCD also maintains high selective quality as graph scales compared to AVICI.
  2. Evaluation of unseen external benchmarks to further prove the transfer of this without retraining.
Critical weaknesses
  1. The adopted a pignistic likelihood learning, which is a many-to-one mapping. As a result,the separation into vacuity and conflict could be driven by architecture parameterization, not necessarily the observed data.
  2. EviCD does not uniformly lead in traditional point-estimate structure metrics across all settings, where specialized continuous optimization methods excel.
Questions for authors
  1. Although, the model reliance on stimulation protocol is substantiated by the authors. Could the authors include experiments in which the performance changes with out-of-distribution shifts?
  2. Current training penalizes vacuity when observational data cannot identify the direction. Can training objectives be redesigned to prevent artificial suppression of vacuity?
  3. Although, the authors presented a well organized paper. They failed to mention the rationale behind the GraphSAGE Encoder?
3 comments

Thank you for recognizing the value of the structured evidential representation, selective prediction, and zero-shot external transfer. We address the remaining concerns below.

1. Pignistic likelihood and the separation of vacuity and conflict

We agree that the pignistic likelihood does not uniquely identify the underlying BPA: different mass functions can induce the same pignistic probabilities. We therefore do not interpret m(Ω)m(Ω) and KK as uniquely data-identified uncertainty components.

Within EviCD, however, vacuity and conflict have different evidential meanings: m(Ω)m(Ω) measures uncommitted support, whereas KK measures incompatibility among committed channel evidence. The restricted focal families and relation-specific heads constrain the admissible decompositions, while observations determine the instance-specific outputs. We therefore interpret them as model- and training-distribution-relative diagnostics. The controlled perturbations (Table 2; Appendix H, Tables 8–10) support this distinction: noise, sample size, and signal strength primarily affect vacuity, whereas increasing latent-confounder strength produces a stronger response in conflict.

2. Point-estimate structural performance

We agree that EviCD does not uniformly outperform methods specialized for conventional point-estimate structural recovery.

Its main advantage is complementary: EviCD provides a structured evidential representation of each causal-relation decision, including uncommitted support and cross-channel conflict, supporting selective prediction and inspection of the evidence state in addition to structural recovery. This advantage is also reflected in the zero-shot external evaluations (Table 3; Appendix J, Table 16), where EviCD remains competitive structurally while showing strong selective performance under heterogeneous distribution shifts.

We will therefore position EviCD as providing competitive causal-relation prediction, zero-shot transfer, and explicit evidential diagnostics, rather than uniform superiority on conventional structural metrics.

3. Out-of-distribution transfer

Our submitted paper already includes a zero-shot distribution-shift evaluation. EviCD is trained only on synthetic ER/SF data and applied without retraining to Sachs, DREAM4, and Smith fMRI. As reported in Table 3 and Appendix J (Table 16), EviCD obtains the lowest SHD and highest selective score on Sachs, the highest AUROC and lowest SHD on DREAM4, and the highest selective score with competitive SHD on fMRI. These results support useful zero-shot transfer of the learned relation representation and selective ranking under external distribution shifts. We will make this setting and the corresponding results more explicit in the revised manuscript.

4. Preventing artificial suppression of vacuity

We agree that singleton supervision can suppress evidential imprecision when observations do not identify a unique direction. A natural solution is to extend EviCD with equivalence-aware/set-valued supervision. If observations support a direct relation but cannot distinguish DijD_{ij} from DjiD_{ji}, the target can be the set {DijD_{ij}, DjiD_{ji}} rather than either directional singleton, allowing the model to assign mass directly to the unresolved directional set.

Directional non-identifiability also does not necessarily imply complete vacuity. Mass on {DijD_{ij}, DjiD_{ji}} represents unresolved orientation, whereas m(Ω)m(Ω) represents broader lack of commitment. The Gaussian/Laplace stress test in Appendix H.3 therefore illustrates how singleton supervision can suppress evidential imprecision under directional ambiguity and motivates supervision that preserves unresolved orientation explicitly.

5. Why GraphSAGE?

We choose GraphSAGE because pairwise causal relations can depend on broader multivariate context rather than only statistics of (i,j)(i,j). Message passing allows each variable representation to incorporate information from other variables before constructing ordered-pair representations. Its shared inductive aggregation also supports different graph sizes and external datasets. GraphSAGE is a standard backbone rather than a methodological contribution; other permutation-equivariant GNN encoders could replace it.

We will revise the manuscript to clarify these claim boundaries, make the zero-shot transfer setting more explicit, describe equivalence-aware supervision as a natural extension, and add the rationale for the GraphSAGE encoder.

I like to appreciate the authors for the elaborate responses. Nonetheless, I am of the opinion that the complementary role of EviCD should be leading and emphasized.

Moreso, the argument as regards artificial suppression of vacuity is not well substantiated.

Nevertheless, the responses are largely convincing and offer a positive outlook for extension.

Thank you for the constructive comments and positive feedback. We will more clearly emphasize the complementary role of EviCD in the paper. The main contribution is not that vacuity or conflict replaces predictive confidence, but that EviCD augments competitive causal-relation prediction with explicit diagnostics of uncommitted and incompatible evidence.

We agree that this mechanism is not fully isolated by the current experiment. Nevertheless, the Gaussian/Laplace stress test provides supporting evidence that singleton supervision can limit evidential imprecision under directional ambiguity. We will clarify this interpretation and present equivalence-aware/set-valued supervision as a natural extension.

Thank you again for helping us clarify and strengthen the paper.

Reviewer_CaTUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a graph neural network architecture for predicting causal relationships and expressing uncertainty from observational data. For each variable pair, it considers two possible causal directions, latent confounding without a direct edge, and neither relation. Three prediction heads produce Dempster–Shafer mass functions, which can assign support to individual relations or sets. Their combination provides two diagnostics: vacuity, measuring uncommitted support, and conflict, measuring disagreement between the heads. As such, vacuity is used to withhold answers; conflict is reported to reveal internal disagreement.

The main contributions are the above-mentioned structured uncertainty representation, an algebraic characterization of conflict, and a guarantee that stricter abstention approximately preserves or improves retained accuracy under a stated calibration assumption. The model is trained on simulated datasets with known causal structures; its uncertainty scores are not claimed to establish causal identifiability.

Critical strengths
  1. The method distinguishes weak support from disagreement between prediction heads and can withhold uncertain answers. Vacuity provides a rule for withholding predictions, while conflict offers an additional diagnostic.

  2. Controlled experiments test how the uncertainty scores respond to noise, sample size, and confounding. Encouraging results on external datasets without retraining also suggest practical value.

Critical weaknesses
  1. The experiments do not isolate the benefit of the evidential representation. A key missing comparison is the same encoder, features, and training data with an ordinary classifier that withholds predictions using confidence or entropy. That is, building a simpler version of EviCD and comparing the two fairly, to see whether its special uncertainty representation actually helps. Without this, the gains cannot be clearly attributed to the proposed uncertainty model.

  2. The paper also does not show that using conflict improves decisions beyond vacuity alone. These gaps leave the practical benefit of the extra score unclear.

  3. The related-work discussion misses important predecessors: Zhao et al. (2020) distinguish insufficient evidence from conflicting evidence in GNNs; Tong et al. (2021) combine deep networks with prototype-based Dempster–Shafer classifiers; and Geifman and El-Yaniv (2017) study confidence-based abstention. These works do not contain EviCD’s exact causal architecture, but the paper should explain more clearly what its uncertainty representation adds beyond these established approaches.

Questions for authors
  1. Does vacuity identify unreliable causal predictions better than ordinary prediction confidence? This could be tested using EviCD's own predicted probabilities. A comparison with a standard classifier trained on the same data would also help establish the benefit of the proposed evidential approach.

  2. Does conflict help identify mistakes beyond vacuity and categorical confidence? What evidence supports its added practical value, given that the abstention rule uses only vacuity?

  3. Could you clarify the methodological differences from earlier uncertainty-aware GNNs (Zhao et al., 2020), deep evidential classifiers (Tong et al., 2021), and selective classification (Geifman and El-Yaniv, 2017)? Which advances go beyond adapting these ideas to causal relations?

3 comments

Thank you for recognizing the value of distinguishing weak/uncommitted evidence from disagreement between evidential channels, as well as the controlled uncertainty analyses and zero-shot external evaluations. We address below what the evidential representation adds beyond an ordinary classifier, the practical role of conflict KK, and the relation to prior evidential and selective-classification methods.

1. The benefit of the evidential representation

The main benefit is that EviCD retains the state of the evidence supporting a causal-relation decision, rather than representing uncertainty only through a four-class probability vector. The categorical prediction, uncommitted support m(Ω)m(Ω), and incompatibility KK among relation-specific channels remain separately observable.

Using the same EviCD predictions, vacuity does not outperform ordinary confidence/entropy for pure abstention or error ranking: AUCselAUC_{sel} is 0.728 ± 0.036 for vacuity, 0.814 ± 0.023 for 1−max⁡BetP1-\max BetP, and 0.853 ± 0.036 for BetP entropy; error-AUROC is 0.531 ± 0.042, 0.762 ± 0.037, and 0.734 ± 0.041, respectively. Vacuity instead measures support that remains uncommitted within the evidential representation.

To isolate the evidential formulation from the encoder/features, we trained an end-to-end four-class Softmax counterpart with the same features, GraphSAGE/pair representations, simulator distribution, training protocol, and 200k-step budget. Softmax-CE achieved AUROC/F1 0.747 ± 0.047 / 0.194 ± 0.013; class-balanced Softmax improved to 0.778 ± 0.039 / 0.361 ± 0.033; EviCD achieved 0.818 ± 0.033 / 0.446 ± 0.020. Therefore, the evidential formulation provides both structured information about the evidential state and improved causal-relation prediction beyond the shared encoder/features.

2. The practical role of conflict KK

In Dempster–Shafer theory, conflict quantifies incompatibility among committed evidence. EviCD inherits this interpretation: the forward, reverse, and symmetric channels act as relation-specific evidence sources, and KK measures incompatibility among their assessments during fusion. In short, vacuity reflects insufficient commitment, whereas conflict reflects incompatible commitment.

Our additional analysis shows that KK is not an improved abstention score: adding KK to categorical confidence leaves error-AUROC unchanged (0.762 → 0.762) and changes error-AUPRC only from 0.526 to 0.535. We therefore do not claim that KK improves the vacuity-based abstention rule; its role is diagnostic. This is also supported by the controlled perturbation experiments (Table 2; Appendix H, Tables 8–10): latent-confounder strength produces a stronger response in KK, whereas noise, sample size, and signal strength primarily affect vacuity.

3. Relation to Zhao et al. (2020), Tong et al. (2021), and selective classification

Zhao et al. (2020) distinguish uncertainty types in GNNs; EviCD instead constructs restricted-focal mass functions with distinct causal roles and separates uncommitted support from cross-channel incompatibility. Tong et al. (2021) combine deep representations with prototype-based Dempster–Shafer classification; EviCD organizes evidence into forward, reverse, and symmetric channels over DijD_{ij}, DjiD_{ji}, C, and N, with incompatibility retained during fusion. Geifman and El-Yaniv (2017) establish confidence-based selective classification; we do not claim abstention itself as new.

Our contribution is therefore not GNNs, prototypes, Dempster–Shafer theory, or selective prediction individually, but the structured decomposition of causal-relation evidence into forward, reverse, and symmetric restricted-focal channels, their evidential fusion, and explicit diagnostics of uncommitted and incompatible support. We will incorporate these controlled comparisons, clarify the diagnostic role of conflict, and expand the related-work discussion accordingly.

Thank you for the detailed response and additional experiments. The matched Softmax comparison substantially addresses my first concern and provides encouraging evidence that EviCD improves predictive performance. I also appreciate the clarification of the related work.

However, the new results raise an important question about the main contribution: vacuity performs worse than ordinary confidence and entropy for identifying unreliable predictions, and conflict provides little additional information for error detection.

While the authors clarify that conflict is intended as a diagnostic, its practical value beyond ordinary predictive uncertainty remains unclear. Moreover, the improvement over Softmax does not isolate whether the benefit comes from the evidential representation itself or the multi-head architecture and training.

Overall, the additional experiments considerably strengthen the paper, although the practical value of the proposed uncertainty decomposition is not yet fully established.

Dear reviewer, thank you for the follow-up.

Regarding vacuity vs. ordinary confidence and entropy, we agree that vacuity is less effective for ranking prediction errors, as reflected by the error-AUROC and selective-prediction results. However, we would distinguish this from the broader notion of “unreliable predictions.” Vacuity is not designed as an error-probability score; it measures how much evidential support remains uncommitted.

The controlled perturbations (Table 2; Appendix H) show that vacuity responds systematically to changes in data informativeness, including noise, sample size, and signal strength. This distinction is particularly relevant in causal discovery, where ground-truth causal relations are often unavailable and prediction error cannot always be directly assessed.

Our research goal is not to solve every aspect of uncertainty with a single score, but to provide complementary evaluation beyond predictive confidence. We thanks again for your constructive comments that help us better clarify and improve the paper.

Reviewer_1WxKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

EviCD is an amortized causal discovery model that combines a GraphSAGE encoder over summary statistics with Dempster–Shafer belief functions. For each variable pair, it predicts either edge direction, latent confounding, or no relation. It also reports vacuity for uncommitted evidence and conflict for disagreement between prediction channels. The paper studies these scores theoretically and evaluates graph recovery, calibration and abstention on synthetic graphs and the Sachs, DREAM4 and fMRI benchmarks.

Critical strengths
  1. I like the distinction between uncommitted evidence and disagreement between channels. It gives two diagnostics to inspect, and the perturbation experiments test how they respond to noise, sample size and confounding.

  2. The authors explicitly discuss what these scores cannot tell us. In particular, they explain how singleton simulator labels shape the uncertainty outputs and why neither score certifies causal identifiability. This helps keep the interpretation tied to what the model actually learns.

Critical weaknesses
  1. My main concern is that the result tables do not agree. The eight SHD values in Tables 13–14 average to 49.25 for DAG-GNN and 72.375 for AVICI. Table 1 instead reports 195 and 255. These differences are large enough to change the claimed ranking against EviCD. I cannot judge the structural recovery claims until the aggregation is explained and the tables are reconciled.

  2. The baseline scoring also makes the gains hard to assess. AVICI’s confounding proxy is thresholded before auroc, while EviCD uses a continuous score (Appendix F). AVICI also has no native confounding output, so this comparison needs qualification. It is unclear whether directed baseline scores are symmetrized for relation-presence auroc. For uncertainty, comparisons are limited to AVICI and GES-based stability selection. The benefit of the evidential head beyond the features and encoder therefore remains unclear.

Questions for authors
  1. Clarification: Do Tables 1 and 13–14 use the same runs, SHD definition and aggregation weights? Which values support the ranking stated in Section 3.2?

  2. Clarification: How are baseline outputs mapped to the four relation classes? Are scores symmetrized for relation-presence auro, and why are AVICI’s confounding scores thresholded before computing auroc? Please also clarify whether Figure 2(a) shows accuracy–coverage area or precision–recall area.

  3. Comparison affecting my assessment: Does vacuity improve abstention over softmax confidence or entropy when using the same input features, GraphSAGE encoder and training data? This would help isolate what the evidential formulation adds.

1 comment

Thank you for the careful reading and for recognizing the value of distinguishing uncommitted evidence from disagreement. We have re-audited the structural evaluation and performed additional controlled experiments to clarify what the evidential formulation adds beyond the shared encoder and ordinary confidence.

1. Structural-result inconsistency and corrected ranking

We are sorry for the mistake in Table 1. The AVICI and DAG-GNN rows in Table 1 were generated by an older evaluation script, whereas Tables 13–14 contain the per-setting results from the current evaluation pipeline. The per-setting runs and SHD definition are the same; the inconsistency arose at the aggregate reporting stage. Re-aggregating the same eight ER/SF settings with the current pipeline gives:

  • AVICI: AUROC 0.824 ± 0.027, F1 0.465 ± 0.027, SHD 72.4 ± 17.5;
  • DAG-GNN: AUROC 0.601 ± 0.043, F1 0.538 ± 0.024, SHD 49.3 ± 10.4;
  • EviCD: AUROC 0.818 ± 0.033, F1 0.446 ± 0.020, SHD 173.6 ± 45.9.

We also identified a legacy scoring path for EviCD relation-presence AUROC in the table. The intended score is sijrel=1−BetP(N;mij)s_{ij}^{rel}=1-BetP(N;m_{ij}), as stated in Appendix F. We will regenerate the affected tables using a single consistent evaluation pipeline and revise the corresponding structural-ranking statement in Sec. 3.2.

2. Baseline mapping, symmetrization, and Figure 2(a)

Baseline mapping. Following Appendix F, all hard predictions are mapped to the common frame with relations DijD_{ij}, DjiD_{ji}, C, and N. Directional predictions map to DijD_{ij} or DjiD_{ji}, absence to N, and FCI bidirected PAG marks to C. Methods without an explicit latent-confounding output cannot otherwise predict C.

For directional score matrices, relation-presence scoring is symmetrized at the pair level using sijrel=max(∣sij∣,∣sji∣)s_{ij}^{rel}=max(|s_{ij}|,|s_{ji}|). For EviCD, the corresponding continuous score is 1−BetP(N)1-BetP(N).

AVICI confounding score. AVICI does not natively represent latent confounding. Among the compared baselines, only a limited subset explicitly represents latent confounding. As defined in Appendix F and noted in the Table 1 footnote, the original evaluation used a binary AVICI proxy that equals 1 when pij>0.2p_{ij}>0.2 and pji>0.2p_{ji}>0.2, and 0 otherwise. We agree that thresholding before AUROC is not an appropriate continuous-score comparison. We will therefore remove this proxy from direct C-AUROC comparison with EviCD's continuous BetP(C) and retain it only as a clearly labelled binary heuristic reference. AVICI remains included for relation-presence and structural comparisons, where its outputs are directly applicable.

Figure 2(a). Figure 2(a) reports retained accuracy versus coverage and the corresponding AUCselAUC_{sel}. “AUPRC” is a legacy axis-label error and will be corrected.

3. Relationship between vacuity and confidence-based abstention

Clarification. Vacuity is not intended to replace ordinary predictive confidence for selective rejection, but to represent the amount of evidential support that remains uncommitted within the belief-function representation. Confidence/entropy summarizes uncertainty in the final categorical prediction, whereas vacuity m(Ω)m(Ω) describes how much support remains uncommitted to more specific hypotheses.

Direct comparison. We compared these quantities on exactly the same EviCD predictions, keeping the trained model, features, encoder, and categorical outputs fixed:

ScoreAUCselAUC_{sel}Error-AUROC
Vacuity m(Ω)m(Ω)0.728 ± 0.0360.531 ± 0.042
1−maxBetP1-max BetP0.814 ± 0.0230.762 ± 0.037
BetP entropy0.853 ± 0.0360.734 ± 0.041

Thus, ordinary confidence/entropy is more effective for pure abstention and error ranking. Vacuity provides complementary information about uncommitted evidence, rather than another estimate of prediction error.

We further trained an end-to-end four-class Softmax counterpart with the same features, GraphSAGE/pair representations, simulator distribution, training protocol, and 200k-step budget. Softmax-CE achieved AUROC/F1 0.747 ± 0.047 / 0.194 ± 0.013; class-balanced Softmax achieved 0.778 ± 0.039 / 0.361 ± 0.033; EviCD achieved 0.818 ± 0.033 / 0.446 ± 0.020.

The plain Softmax is strongly affected by the dominant N class; class-balanced CE substantially improves it, but EviCD remains stronger. Therefore, ordinary confidence/entropy is preferable for scalar abstention, while the structured evidential formulation provides complementary diagnostics and improves causal-relation prediction beyond the shared encoder/features. We will add these controlled comparisons and clarify the distinct roles of predictive confidence and vacuity in the revised manuscript.

Comments on the paper

2 comments

Thank you for fixing the tables and for comparing against a standard classifier with same features and encoder. That comparison supports the prediction model. However, the corrected results show more graph errors than AVICI and/or DAG GNN. The confidence from EviCD itself works better for choosing what predictions to keep than its proposed uncertainty score and the extra disagreement score seems to add little to spotting such mistakes. Overall, I keep my score because the benefit of these extra scores remains somewhat unclear to me.

Thank you for the follow-up. We agree that categorical confidence is better for identifying prediction errors. However, vacuity and conflict are not intended to replace confidence for this purpose.

Their role is to characterize the evidential state: vacuity reflects insufficient committed support, whereas conflict reflects incompatible relation-specific support. The controlled perturbations (Table 2; Appendix H) support this distinction. Beyond the synthetic setting, EviCD also transfers without retraining to Sachs, DREAM4, and fMRI, with competitive structural recovery and strong selective performance, whereas AVICI shows less consistent selective-reliability transfer across these external benchmarks.

Thus, confidence, vacuity, and conflict serve different purposes rather than competing as the same uncertainty score. Anyway, thank you for your comments and time. We will make this distinction clearer in the future version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.