acceptodds
Under review as a conference paper at ICLR 2027

Exact Search, Uncertain Arrows: Auditing Learned Score Tables

Abstract

A learned score can prefer the wrong causal graph even when search finds its exact optimum. To understand this mismatch, we introduce an audit framework that separates the ability to find high-scoring graphs from the ability of the score to favor accurate structures. We also examine which arrow directions the predictor favors. Our framework uses tables that store one score per variable and candidate parent set. We project these scores so that all graphs in a Markov equivalence class receive the same total score, then use exact search to find a highest-scoring graph. The component removed by this projection compares how well opposite edge directions fit the data. Better predictive fit alone does not establish the causal direction. We build 180 full ten-variable TabPFN tables and compute their exact optima, enabling repeated search comparisons without further predictor queries. On 70 tables, policy-gradient search finds a graph closer to the true structure than the highest-scoring graph, despite receiving a lower score. BOSS-style search does so on 51. These reversals occur mainly on scale-free graphs, where the highest-scoring equivalence classes often direct edges that the true class leaves undirected. This orientation error persists from 500 to 2000 observations. On Erdős-Rényi graphs, the highest-scoring graph is usually closer to the true structure than the graphs returned by heuristic search. For larger graphs, we find the highest-scoring graph within a restricted search space or using a table with estimated entries. Together, these tables provide a reusable benchmark for assessing search methods and identifying when higher scores fail to yield more accurate causal structures.

Primary area:
causal reasoning
Keywords:
causal discovery, score-based structure learning, score equivalence, exact search, TabPFN

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UTJ2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

A Bayesian network structure learning algorithm returns a DAG that is guaranteed to be optimal, that is, there is no DAG that has higher score. But this does not guarantee that the DAG is causal.

One way to compute the score is to used a pretrained model such as TableFPN. One inputs one part of the training data into TableFPN and it outputs a histogram. Then we use probabilities derived from the histogram and the remainder of the training data to compute log-likelihood of the data. A problem with this approach is that the score is not score-equivalent, that is, DAGs belonging to the same Markov equivalence class are not guaranteed to have the same score.

The paper uses a projection to convert these score into a score-equivalent score. The projection works as follows. Let l(v,S)l(v, S) be the score of variable v given a parent set S. Then we can choose values for every t(v,S)t(v, S) such that we minimise

∑v,Swi,S(l(v,S)−t(v,S))2\sum_{v, S} w_{i, S} (l(v, S) - t(v, S))^2

s.t. t(x,W)+t(y,W∪x)=t(y,W)−t(x,W∪y)t(x, W) + t(y, W \cup x) = t(y, W) - t(x, W \cup y) for all x≠yx \neq y and W⊆V∖{x,y}W \subseteq V \setminus \{x, y\}.

The condition guarantees that the t-score is score-equivalent.

The paper uses both the original and the projected scores as learning objectives and compares the DAGs that optimal with respect to a score to the data-generating DAG.

Critical strengths

The topic is interesting if researched properly.

Critical weaknesses

The problem with the paper is that it is extremely badly written. This makes it difficult to assess other merits of the paper.

For example, the projection is a central part of the paper but there is no clear formulation. I had to spend time to try to construct the formulation that is written in the summary. It is also not clear what is the point of the weights.

Similarly, Proposition 5 talks about a label indegree vector. But it is never defined. Without a definition, the statement does not make sense.

The paper uses technical sounding terminology without clearly stating what the terms mean. For example, it has sentences like "A matched-domain control isolates how cells unusable by a graph can influence its projected score." The problem is that it is not clear what a matched-domain control is and what are cell unusable by a graph.

It is also somewhat unclear to me what are the contributions of the paper. The authors list 180 score tables as the contributions. However, how some models perform on some arbitrary synthetic data is not interesting. What is interesting is generalisable conclusions that can be drawn based on the experiments. As far as I can see, there is not much generalisable results.

I also have trouble to understand the overall goal of the evaluation. The paper starts by saying that a score can prefer a wrong causal graph. A causal graph is generally a DAG. But then the paper compares CPDAGs that represent Markov equivalence classes. However, a Markov equivalence class is a set of graphs that can represent the same probability distributions. And if we have found the correct Markov equivalence class then all but one graph are wrong causal graph.

Furthermore, if the point is to evaluate, to which degree the optimal graph is causal, then what is the point to benchmark with non-optimality guaranteeing methods? Why just not compare the highest scoring structure and the true structure? What does PPO or BOSS-style search results tell about a) scores or b) the algorithms in question?

Questions for authors

What is "a learned score"?

What is "a causal reading"?

What is "an forced-opposite query"?

What is "over-direction"?

What is "a parent-count correction"?

Page 4. Does "Conditional law" mean "Conditional distribution"?

Reviewer_3jj4OpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper asks whether wrong graphs from a learned causal score come from weak search or from the score itself. It builds full score tables with TabPFN on small synthetic graphs. It then finds the exact best-scoring graph and compares it with heuristic searches.

The paper also adjusts each table so that equivalent graphs receive the same score. The part removed by this adjustment expresses a preference between edge directions. The paper shows that this preference reflects predictive fit rather than causal direction.

On many scale-free graphs, heuristic searches find lower-scoring but more accurate graphs. A higher score therefore does not always mean a more accurate graph. The paper proposes its tables as a reusable benchmark for search methods.

Critical strengths
  • The adjustment gives a clear way to separate a score for each equivalence class from a preference between edge directions. The paper shows that this preference reflects predictive fit rather than causal direction.
  • The paper is careful about the limits of its claims. Its score tables with exact optima could serve as a benchmark for search methods.
Critical weaknesses
  • Comparing the exact optimum of a fixed score table with the true graph has been used before. Earlier work did this for classical scores (Liu et al., 2012; Malone et al., 2018) and for TabPFN scores on five-node graphs (Sypniewski et al., 2025). Extending it to ten variables is a modest step, since exact search at this size is standard. The main new element appears to be the adjustment that splits each score into a class score and a direction preference.
  • The paper motivates this adjustment as giving one score per equivalence class, not as improving accuracy. On scale-free graphs, the unadjusted optimum is more often closer to the truth. It is therefore unclear what the adjustment gains in practice.
  • The audit needs the true graph and complete score tables. It therefore applies to small synthetic benchmarks rather than to the analysis of real data. The introduction asks whether to improve the search or the score, but says little about why this question matters for learned scores in particular.
Questions for authors
  • Is there a setting in your results where the adjusted score leads to more accurate graphs than the raw score? Or is there a task that needs one score per equivalence class? An answer here would show whether the paper’s main new element has a practical payoff.
  • Beyond the larger graph size, what does the audit reveal that the earlier five-node TabPFN study (Sypniewski et al., 2025) could not? For example, could the over-direction on scale-free graphs appear at five nodes at all? An answer here would clarify the novelty.
Reviewer_KcJ5OpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper audits learned causal scores by fixing local-score tables, projecting them onto a score-equivalent space, and computing certified optima.

Critical strengths
  1. Fixing the score table and certifying its optimum makes the search-versus-score distinction convincing. The reusable tables could be a useful benchmark.

  2. The controls are careful, including raw versus projected scores, fixed-skeleton comparisons, and limits of parent-count corrections. The paper is also clear that predictive-fit diagnostics do not validate causal arrows.

Critical weaknesses
  1. The main ten-variable results use fixed scoring realizations. Stability under new context splits and predictor seeds remains untested; the separate six-dataset bootstrap study does not resolve this for the main SF results.

  2. The added value beyond classical score equivalence and exact search could be stated more sharply. For a benchmark contribution, query cost, solver resources, and access to the tables and certificates also need a consolidated description.

Questions for authors
  1. Do the SF over-direction pattern and the raw/projected accuracy difference persist across new context splits and predictor seeds?

  2. How sensitive are these results to the auxiliary-noise construction for empty parent sets and to nonfinite-score replacement?

  3. Are the full tables, generating graphs, and optimality certificates available for verification, and what resources are needed to reuse them?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.