- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper considers the PC algorithm using Fisher’s Z tests and models the absolute values of the test statistics collected during its skeleton search as i.i.d. observations from a two-component Gaussian mixture model. It estimates test-level priors and likelihoods and converts them into posterior probabilities of edge existence. The method is evaluated on synthetic data generated from linear Gaussian DAG models.
Critical strengths
None
Critical weaknesses
-
The modeling assumptions are insufficiently justified, and the interpretation of the resulting scores as posterior probabilities remains unsupported.
-
The paper assumes independence among Fisher’s Z test statistics. However, many tests reuse the same samples and involve highly overlapping conditioning sets, making complete independence an overly simplifying assumption. The authors neither analyze the effects of ignoring these dependencies on posterior estimation and calibration nor provide corresponding empirical validation.
-
The paper assumes that the Fisher’s Z test statistics are identically distributed. The authors fit a single mixture model to statistics obtained from different variable pairs and conditioning set sizes. However, the alternative effect distributions may differ across these tests. The paper does not justify the shared model or examine whether probabilities are well calibrated within each category of tests.
-
The paper assumes a linear Gaussian SEM. While this is not inherently problematic as a simplifying modeling choice, the resulting joint distribution is multivariate Gaussian. Under this setting, the experiments should compare against Bayesian methods that directly model graph structure in terms of edge probabilities, calibration, and computational cost, to establish the practical value of the proposed approximation.
-
-
The empirical benefits are limited. PROPeR is the only external baseline, and comparisons with other uncertainty estimation methods are missing. Moreover, posterior calibration is actually worse (Table 1, PROPeR (MLE)).
Questions for authors
-
The Introduction discusses several uncertainty estimation methods. Could the authors include representative methods as additional experimental baselines?
-
Could the authors include a simple bootstrap baseline? Although bootstrap frequencies are not posterior probabilities, their ranking performance, probabilistic prediction performance, and computational cost can still be compared.
-
Why is fixed? What happens when truly non-adjacent variables require a separating set larger than 3?
-
How does the method perform in non-linear and non-Gaussian settings?
-
Under the paper’s linear Gaussian setting, the variables follow a multivariate Gaussian distribution. What advantages does the proposed method offer over frequentist uncertainty assessment methods or Bayesian structure learning methods that directly exploit this distributional structure? Could the authors explain why their modeling approach is necessary?