- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SVI-DAG aims to estimate not just one causal graph, but a range of plausible graphs and their probabilities. It uses a flexible model to represent connections that tend to occur together, a way to include prior knowledge about likely connections, and a particle-based method to explore different node orderings. The authors test it on simulated data and the Sachs biological dataset.
Critical strengths
-
Estimating several plausible graphs is worthwhile when the data do not determine one clear graph. The paper gives a coherent method and tests what happens when major components are removed.
-
The evaluation looks at both graph recovery and the quality of the reported probabilities. On Sachs, SVI-DAG has the best expected F1 score when comparing fully directed graphs (Table 2), meaning it achieves the best balance of recovered and spurious connections under that measure.
Critical weaknesses
-
The extra complexity has no clear overall payoff on the real-data benchmark. On Sachs, SVI-DAG has the worst Brier score among six methods for fully directed graphs; this score measures how well its predicted edge probabilities match the reference graph. Its expected structural Hamming distance, the number of edge changes needed to match that graph, ranks 4th of six for both directed and equivalence-class comparisons. It is also the third slowest method (Table 1). Its F1 result is a real strength, but the paper needs to explain when that gain matters more than the weaker probability estimates, larger graph error, and run time.
-
Removing a proposed component sometimes helps. In Table 4, removing the particle-based SVGD step improves structural Hamming distance from 69.3 to 62.0 and F1 from 26.9% to 34.8%, although other measures become worse. Removing the prior knowledge causes a much larger decline than removing the other components. The authors should explain these trade-offs and compare methods using the same prior information before attributing the gains to the new graph model.
Questions for authors
- For the intended use, which matters most: recovering graph connections, ranking likely connections, or assigning accurate probabilities? How should readers weigh the Sachs results across these measures?
- Can the authors compare methods given the same prior information and explain why removing SVGD improves graph distance and F1 in Table 4?