- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
A Bayesian network structure learning algorithm returns a DAG that is guaranteed to be optimal, that is, there is no DAG that has higher score. But this does not guarantee that the DAG is causal.
One way to compute the score is to used a pretrained model such as TableFPN. One inputs one part of the training data into TableFPN and it outputs a histogram. Then we use probabilities derived from the histogram and the remainder of the training data to compute log-likelihood of the data. A problem with this approach is that the score is not score-equivalent, that is, DAGs belonging to the same Markov equivalence class are not guaranteed to have the same score.
The paper uses a projection to convert these score into a score-equivalent score. The projection works as follows. Let be the score of variable v given a parent set S. Then we can choose values for every such that we minimise
s.t. for all and .
The condition guarantees that the t-score is score-equivalent.
The paper uses both the original and the projected scores as learning objectives and compares the DAGs that optimal with respect to a score to the data-generating DAG.
Critical strengths
The topic is interesting if researched properly.
Critical weaknesses
The problem with the paper is that it is extremely badly written. This makes it difficult to assess other merits of the paper.
For example, the projection is a central part of the paper but there is no clear formulation. I had to spend time to try to construct the formulation that is written in the summary. It is also not clear what is the point of the weights.
Similarly, Proposition 5 talks about a label indegree vector. But it is never defined. Without a definition, the statement does not make sense.
The paper uses technical sounding terminology without clearly stating what the terms mean. For example, it has sentences like "A matched-domain control isolates how cells unusable by a graph can influence its projected score." The problem is that it is not clear what a matched-domain control is and what are cell unusable by a graph.
It is also somewhat unclear to me what are the contributions of the paper. The authors list 180 score tables as the contributions. However, how some models perform on some arbitrary synthetic data is not interesting. What is interesting is generalisable conclusions that can be drawn based on the experiments. As far as I can see, there is not much generalisable results.
I also have trouble to understand the overall goal of the evaluation. The paper starts by saying that a score can prefer a wrong causal graph. A causal graph is generally a DAG. But then the paper compares CPDAGs that represent Markov equivalence classes. However, a Markov equivalence class is a set of graphs that can represent the same probability distributions. And if we have found the correct Markov equivalence class then all but one graph are wrong causal graph.
Furthermore, if the point is to evaluate, to which degree the optimal graph is causal, then what is the point to benchmark with non-optimality guaranteeing methods? Why just not compare the highest scoring structure and the true structure? What does PPO or BOSS-style search results tell about a) scores or b) the algorithms in question?
Questions for authors
What is "a learned score"?
What is "a causal reading"?
What is "an forced-opposite query"?
What is "over-direction"?
What is "a parent-count correction"?
Page 4. Does "Conditional law" mean "Conditional distribution"?