acceptodds
Under review as a conference paper at ICLR 2027

A Diagnostic Radius for Knowledge-Induced Causal Effect Estimation

Abstract

Observational causal discovery usually identifies an equivalence class of causal graphs rather than a unique graph. In applied settings, background or expert knowledge can further constrain uncertain edge orientations, after which sound orientation rules propagate their logical consequences. We give a query-specific certificate: the fewest closure orientations that must be contradicted before a fixed adjustment set can fail, conditional on a correct equivalence class, computed exactly by retractions without enumerating the space of knowledge states. On synthetic graphs whose generator spreads instances across radius levels by targeting the treatment's separation from the adjustment set, this breakdown radius tracks survival to corrupted knowledge at a rank correlation of 0.75 to 0.81 under three- and four-tier corruption, against at most 0.14 for structural Hamming distance to the true graph. When the query is instead drawn without targeting separation, the radius ranks survival at 0.29 and the structural proxy separation at 0.34, both well above SHD, with the radius still leading SHD by about 0.3. On published networks, with knowledge scenarios constructed by large language model elicitation, it ranks survival within fixed knowledge states, where knowledge counts are constant. We release code and benchmarks so that a breakdown radius can be reported alongside any causal estimate that relies on an adjustment set.

Primary area:
causal reasoning
Keywords:
background knowledge, causal discovery, sensitivity analysis, covariate adjustment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_h8pCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the causal edge orientation problem based on a Markov equivalent class. In particular, the paper defines the breakdown radius concept that measures the distance between a base MPDAG and the closest MPDAG that triggers a change of the causal effect estimation, aiming to measure a causal model’s resistance to problematic orientation knowledge. The work shows an algorithm that exactly computes the breakdown radius, the correlation between the breakdown radius and adjustment set validity, and the relationship between breakdown radius and causal effect estimation bias, by theory and experiments.

Critical strengths
  1. The work is well motivated. It aims to propose a metric for causal discovery such that the metric measures the constructed causal model’s robustness against problematic orientation knowledge. This direction is potentially important for LLM-assisted causal discovery methods.
  2. The research framework is relatively complete, covering metric computing, validity, and impact on causal effect estimation.
Critical weaknesses
  1. One of my main concerns is the high computational complexity of the breakdown radius. A shortcoming of the paper is the absence of the complexity analysis. However, I would expect significant hardness for computing the metric. This also limits its implementation.
  2. The breakdown radius mainly considers the so defined closure orientation. It measures the error tolerance of a proposed causal model to possible problematic orientations, in terms of a target causal effect estimation task. However, it does not specify the accuracy of the proposed causal model. In other words, the paper assumes that the Markov equivalent class is correct, however, this is a strong assumption. Under this assumption, the metric neglects the bias caused by the recovery of the CPDAG.
  3. The contribution is mild. Although the paper establishes nontrivial theoretical analysis for the exact computation of the breakdown radius, the framework of graph-space distances for assessing adjustment robustness has already been explored in existing work. The theoretical novelty therefore mainly lies in specialising this framework to expert/LLM knowledge oriented causal graphs.
  4. The experimental results are not fully convincing. For instance, (1) a large part of the graphs are small networks, limiting the scope of the investigation. (2) The breakdown radius should be sensitive to the selection of the treatment/outcome pair, however, the pair selection process is not clear in the experiments on published networks.
Questions for authors

See the weakness part.

4 comments
AuthorsOpenReview ↗

Comment 1 of 2.

We thank the reviewer for the thorough review. We are glad that the reviewer finds the framework complete, and we share the view that a robustness measure for orientation knowledge matters for LLM-assisted causal discovery, where such knowledge is cheap to obtain and hard to check. The concerns raised led to an addition in the revision: a complexity analysis, and the pair selection rule. We answer each point below.

W1 Complexity analysis

We agree that the analysis was missing. It is now in a new Appendix C.8, with a pointer from Section 5.

What we prove. For one treatment and one outcome, the setting of the paper, the radius is computable in polynomial time. The validity of ZZ depends on the parents of the treatment only, and the nearest failure is always one change of the treatment’s parent set away from the analyst’s state (Proposition C.5; argument in Appendix C.9, complete proofs in Appendix J). On all 679,047 instances whose treatment component has at most six vertices, this returns the same radius as the search over every parent set.

What the experiments ran. Algorithm 1, which is also exact, so every reported radius is unchanged. Its explicit search costs ∣KG0∣O(rval)⋅poly(∣V∣)|K_{G_0}|^{O(r_{\mathrm{val}})} \cdot \mathrm{poly}(|V|), where KG0K_{G_0} is the set of closure orientations of the analyst's state (Proposition C.3). This exponent bounds the algorithm, not the problem (Remark C.2), and in practice it is rarely reached (timings of Appendix D.2):

GraphsQueriesAlgorithm 1, medianListing the class, medianRatioAlgorithm 1 under 1 sListing finishes
8 to 24 nodes1,5440.37 ms2.4 ms699.8%100%
32 to 64 nodes4220.96 ms90 ms90100%100%
100 nodes1416.0 ms15 s2,500100%89%
150 nodes14415 ms2.2 h*5⋅1055 \cdot 10^{5}97.9%3%
200 nodes14340 ms4 years*3⋅1093 \cdot 10^{9}98.6%0%
300 nodes13852 ms2.5⋅1032.5 \cdot 10^{3} years*101210^{12}91.3%0%
400 nodes142115 ms5⋅1085 \cdot 10^{8} years*101710^{17}89.4%0%
link, 724 nodes8571 ms2⋅10262 \cdot 10^{26} years*103510^{35}100%0%

*Projected from the number of DAGs tested per second when listing does not finish within its limit; on 30 classes run for 1,800 s the projection was within a factor of 2 of the measured rate. The ratio is the ratio of the two medians. On the five published networks of 724 to 1,041 nodes, Algorithm 1 is exact on all 388 query–state units that admit a certificate, the slowest in 55 s.

Proposition C.5 is an analysis of the problem, and we kept Algorithm 1 for two reasons. First, its search uses a single property of failure, that it is upward-closed (Theorem 1), so the same search also computes the bias-tolerance radius of Section 7, whereas the argument of Proposition C.5 relies on the form of the adjustment criterion and is specific to validity. Second, the exponent is paid only by deep radii, and there the solver stage takes over; our answer to Reviewer qQYf (W1) discuss when the solver is preferable to the search and how it compares with other exact solvers.

What remains open. With several treatments or outcomes the argument no longer applies; we leave that case, including an analysis of NP-hardness, to future work (Remark C.2).

W2. The correct equivalence class is a strong assumption

We agree that it is an assumption, and the paper states it as a limitation (Section 8). Two remarks on its role.

It is the assumption of the estimate itself. An adjustment set read off an MPDAG is valid only if the class is correct, with or without a radius. The radius adds no assumption to those of the estimate it accompanies. It addresses the second source of error, the background knowledge, which the data cannot check. It measures robustness to that knowledge, not the accuracy of the graph (Section 1).

How much of the class it needs. A new Appendix H.4 examines this, learning the class with ten CPDAG learners on four Gaussian networks (nn = 100 to 100,000). An extra adjacency cannot break ZZ within the certified budget, while a missing adjacency or a wrong v-structure can (Remark 8). When the learned class has no missing or reversed edge among the ancestors of XX, YY and ZZ, the certificate held in all 46,838 such cases, so it rests on the class being correct around the query, not everywhere. This also improves with data: under GRaSP the share of invalid committed sets falls from 0.63 at nn = 100 to 0.06 at nn = 100,000, and the radius on the learned class equals the true-class radius on 91% of queries. We recommend reporting the radius with the discovery method and sample size.

AuthorsOpenReview ↗

Comment 2 of 2.

W3. Novelty against existing graph-space distances

The framework of graph-space distances is due to Taeb et al. (2026), and Section 2 credits it as our closest predecessor. The results of the paper do not follow from that framework; they hold because of the structure of the space we apply it to.

A graded space. The general MPDAG poset of Taeb et al. is not graded. The poset of knowledge states over a fixed CPDAG is, which gives the distance in closed form (Corollary 1).

Search by retraction only. The nearest failure is always reached by withdrawing orientations from the analyst's state (Theorem 1). This is what makes exact computation possible without enumerating the space.

What the radius means. All valid adjustment sets share one radius (Proposition C.7.1), and the radius is the distance to the nearest state that is not amenable. It is therefore a property of the query and the knowledge, not of the chosen set.

A certificate for the analyst. The output is not only a distance: it names the orientations whose retraction breaks ZZ, and within the radius the committed set carries no identification bias under a linear-Gaussian model (Theorem 2). Empirically, the radius ranks survival inside a fixed knowledge state, where counts of orientations, SHD and claim accuracy are constant (Table 1).

W4(1). Mostly small networks

The size of the graph does not limit the method, and the revision adds two checks at scale. On generated graphs of up to 400 nodes, the hybrid answers almost every query in under one second (Appendix D.2). On the five largest published networks (link and four munin variants, 724 to 1,041 nodes), the radius is computed exactly on every query that admits a certificate, the slowest in 55 s (Appendix I). The survival ranking of Table 1 stays on the 24 networks where survival was scored. The synthetic graphs are small by design: they are generated to spread instances across radius levels, and their radii reach the depths where the solver is needed.

W4(2). Selection of treatment–outcome pairs on the published networks

We agree that the rule was not stated; it is now in Appendix I.

The rule. A pair (X,Y)(X, Y) enters Table 1 when:

  • XX lies in or next to an undirected component of the CPDAG;
  • YY descends from XX in the true graph;
  • the empty set is not a valid adjustment set;
  • withdrawing one true orientation from complete knowledge invalidates the optimal set.

These conditions keep the queries where background knowledge matters for the adjustment. The rule is applied to every ordered pair on networks of at most 56 nodes and to a fixed seeded sample on larger ones. No step looks at a radius.

A check without the true graph. Because this rule reads the true graph, we repeated the ranking on the 528 released queries, which are chosen from the CPDAG alone. There the radius ranks survival within knowledge states at τb=0.63\tau_b = 0.63 [0.45,0.76][0.45, 0.76] (Section 6.3, Appendix I). The ranking therefore also holds on queries selected without the true graph.

I appreciate the authors’ answers. Most of my concerns and questions have been addressed. I am happy to raise my score to 3.

A follow-up question for the authors is: Could the authors clarify whether Appendix H.4 counts disagreements only within the true ancestral subgraph or across a full DAG completion of the learned CPDAG? If the count is local, I do not see how the guarantee follows from Remark 8(i), since matching the true orientations near the query may force additional changes elsewhere.

Thank you for your careful assessment and for raising your score. Your review improved the paper on several points, and your follow-up question helps us state Appendix H.4 more precisely.

The count is global, over a full DAG completion of the learned CPDAG. For each committed set we build one DAG D∈[C^]D \in [\hat C] in four steps:

  1. Impose the true orientations on A=AnD⋆({X,Y}∪Z)A = \mathrm{An}_{D^\star}(\{X,Y\}\cup Z).
  2. Close under Meek’s rules.
  3. Add the analyst’s closure orientations that still close.
  4. Complete to a DAG.

We then count kerr=∣KG0∖dir(D)∣k_{\mathrm{err}} = |K_{G_0} \setminus \mathrm{dir}(D)| over the whole graph. Any change elsewhere that is forced by matching the truth near the query is part of DD. If it contradicts one of the analyst’s closure orientations, it is counted. Using one completion rather than the best one can only make the condition kerr<rvalk_{\mathrm{err}} < r_{\mathrm{val}} harder to meet.

Why the guarantee follows.

  1. If kerr<rvalk_{\mathrm{err}} < r_{\mathrm{val}} (or rval=∞r_{\mathrm{val}} = \infty), then ZZ is valid in DD, by Corollary 1(c) (or 1(b)).
  2. The localisation is only in the comparison between DD and D⋆D^\star. Every edge of D⋆[A]D^\star[A] is an edge of DD with the same orientation. Deleting from DD every edge outside D⋆[A]D^\star[A] keeps ZZ valid, by the edge-deletion argument of Remark 8(i) applied to the full DAG DD.
  3. What remains is D⋆[A]D^\star[A] plus isolated vertices. AA is ancestral in D⋆D^\star and contains every causal path from XX to YY, every forbidden node in ZZ, and every path that could be open given ZZ. So validity of ZZ in D⋆D^\star is decided on D⋆[A]D^\star[A], and ZZ is valid in D⋆D^\star.

Empirically, the guarantee held in all 46,838 cases. In the camera-ready version we will define the count and give this argument explicitly in Appendix H.4, and we will reword “as Remark 8(i) implies” accordingly.

Thank you again for the rigorous discussion and for the time invested across the window.

Reviewer_C2BdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Observational causal discovery recovers a Markov equivalence class, where experts then can apply background knowledge to orient remaining undirected edges into an MPDAG. Because expert (human or LLM) assertions cannot be checked against data, the authors propose the breakdown radius, a query-specific diagnostic quantity that measures the fewest closure orientation revisions needed to invalidate a committed adjustment set. The paper proves monotone reachability, enabling exact computation via retraction search and SAT certificates. It also proves that within the certified radius under a linear-Gaussian SCM, the committed set carries zero population identification bias. Empirical benchmarks on over 4,000 synthetic instances and 33 published networks with LLM-elicited knowledge confirm the radius effectively ranks adjustment survival under corruption, outperforming Structural Hamming Distance.

Critical strengths

The introduced metric outputs an exact orientation-error budget and witness set for target-specific query robustness rather than relying on global graph metrics like SHD. It seems to be a useful method for the community.

The paper evaluates over 4,000 synthetic instances and 33 published networks with LLM-elicited knowledge across 6 language models, demonstrating strong performance under knowledge corruptions.

Critical weaknesses

The heavy reliance on formal lattice theory, Meek closure operations, and SAT solver formulations lacks sufficient high-level visual intuition, hindering readability.

The method assumes a correct skeleton and v-structures; finite-sample discovery errors or unmeasured confounders outside CPDAG are not captured. It would strengthen the paper if it included some experimentation on how sensitive the method is to these.

Lightweight graph separation performs comparably in untargeted synthetic regimes, though the introduced radius remains exact and defined where s is absent.

Tables 1, 9, 11, and 13 report the uncorrected 2,422-unit cohort despite Appendix H disclosing and fixing a 94-unit empty-set pipeline bug. While the appendix confirms all conclusions hold on the full 2,516-unit cohort, reporting known bugged numbers in the main text is poor editorial practice; the primary tables must be updated for the camera-ready version.

Questions for authors

In Section 7, the effect-scale bias profiles are evaluated on linear-Gaussian SCMs. Have the authors evaluated how standardizing variables (e.g., following Reisach et al., 2021 and Ormaniec et al., 2025) affects the radius, ensuring that bias thresholds are not influenced by variance accumulation along longer causal paths?

Could the authors provide a gentler, more intuitive visual walkthrough of the join-semilattice and retraction concepts in the main text to improve readability for a general ML audience?

Could the radius be coupled with skeleton-stability or bootstrap checks to handle finite-sample causal discovery errors? If so, how?

1 comment

We thank the reviewer C2Bd for the encouraging review. We are glad that the reviewer finds the radius as useful to the community for the reason we mostly care about: it gives an error budget where global metrics such as SHD say nothing about. The reviewer’s requests improved the paper: the revision adds a visual walkthrough, more ablations and analyses. We answer each point below.

W1 Visual intuition for the lattice, the Meek closure and the SAT part

We agree. To make the construction easier to follow, Figure 2 in Section 5 now walks through retraction and join on four nodes, and we recomputed the toy example of Figure 3: panel (b) enumerates every state above G0G_0, one row per retraction, and marks the retraction that breaks ZZ, so the radius can be read off directly. The SAT stage has no figure; it asks, as constraints, whether some failing state lies within kk retractions (Appendix D.1).

W2 Correct skeleton and v-structures

Yes: the radius is conditional on the class, and Appendix H.4 now measures what this costs on learned classes (ten CPDAG learners, four Gaussian networks, nn = 100 to 100,000, ten seeds, truthful or LLM knowledge). Three findings:

  • The certificate holds whenever the class is correct near the query: with no missing or reversed edge among the ancestors of XX, YY and ZZ, and fewer wrong claims than the radius, the committed set was valid at truth in all 46,838 such cases; with LLM knowledge, every set within the radius was valid (12,139), while 53% of those beyond it were invalid.
  • Only errors at the query matter: with no class error touching XX, YY or ZZ, only 0.24% of committed sets were invalid.
  • Such local errors become rare with data: under GRaSP the invalid share falls from 0.63 at nn = 100 to 0.06 at nn = 100,000, and the learned-class radius equals the oracle radius on 91% of queries.

Local errors persist on one network with high in-degree, missing adjacencies can still break a set with infinite radius (Remark 8), and causal sufficiency remains an assumption.

W3 Separation does as well on untargeted queries

We agree, and Section 6.2 says so: where separation is defined, reading the graph is a good heuristic. We introduced separation ourselves to check if the exact search adds anything over it. It does:

  • A guarantee, not a score: Fewer than rvalr_{\mathrm{val}} wrong orientations cannot invalidate ZZ. A small separation suggests only that a failure is near.
  • A certificate: The radius names the orientations whose retraction breaks ZZ, which tells the analyst what to verify.
  • Defined in every query: Separation is undefined when no member of ZZ lies in the treatment’s component. The radius is defined in every query.

On synthetic graphs, the radius still ranks survival at a fixed separation, where separation barely does at a fixed radius.

W4 The uncorrected 2,422-unit cohort in Tables 1, 9, 11 and 13

Agreed; all four tables now report the full cohort (Tables 1, 19, 21 and 23 of the revised PDF; Tables 1, 9, 11 and 13 of the submission).

Q1 Standardising the variables in Section 7

Yes, in a new Appendix E.6. There are two different operations to separate.

  • The radius. It is computed from the CPDAG, the knowledge and ZZ only. No distribution entered.
  • Standardising the generated variables. The bias thresholds of Section 7 are relative to the effect, so rescaling any variable leaves them unchanged. We recomputed the profiles on standardised variables for all 490 queries and draw pairs, and every one is identical.
  • Standardising inside the generator. This is a different distribution so individual profiles can change. Most queries keep the same profile, and the profiles remain as diverse as under the original generator. Therefore, the conclusions of Section 7 do not rest on variance accumulation.

Q2 A visual walkthrough of the join-semilattice and of retraction

Yes, this is the new Figure 2 in Section 5, on four nodes.

Q3 Coupling the radius with skeleton-stability or bootstrap checks

Yes, as two reports that answer two different questions:

  • a stability check asks whether the class is right;
  • the radius asks how many wrong orientations the adjustment set tolerates, given the class.

The second does not replace the first, and the pair is not a joint certificate, because the guarantee of the radius holds only if the class is correct.

How. Appendix H.4 says where the stability check should be spent. Extra adjacencies are harmless within the certified budget (Remark 8), and on learned classes every invalid set traces to a missing adjacency or a reversed arrow among the ancestors of X, Y or ZZ. A bootstrap can therefore concentrate on the adjacencies and v-structures around the query, not on the whole skeleton, and the radius is then reported on the class that passes. We leave the validation of such a combined procedure to future work.

Reviewer_qQYfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the question of how sensitive an adjustment set is against graph structure errors from human experts. Starting from an analyst’s MPDAG, it defines the breakdown radius as the minimum number of edge orientation revisions needed to invalidate the adjustment set. It further shows that this radius can be computed using only orientation retractions without enumerating all possible knowledge states.

Critical strengths
  1. In general, the paper is clearly written and the formulations of the theoretical results are clean.
  2. I think the problem on the robustness of adjustment set is interesting, though the paper could provide more practical motivation.
  3. The paper presents empirical studies on synthetic graphs and published networks to examine whether a larger radius predicts a higher survival rate of the adjustment set under random orientation perturbations. It also compares the radius’s predictive performance with baseline metrics such as SHD.
Critical weaknesses
  1. Algorithm 1 contains two parts: the upward search with depth b and the SAT certificate. The choice of SAT over other optimization approaches, when it becomes preferable to explicit search, and how to select the appropriate parameter bb is not thoroughly motivated.

Minor comments: 2. Definition 3: I don't think the choice of optimal adjustment set here (e.g., backdoor adjustment set) is unique; this needs to be clarified. Also, a brief discussion on how this may impact the radius should be added. 3. Line 299: The meaning of "a set survives wrong knowledge longer" is unclear. 4. Line 35: oriented by "a person" -> "an analyst".

Questions for authors
  1. Depending on the analyst’s knowledge state, an adjustment set that is invalid in the true DAG may have a larger radius than one that is valid. Could the authors clarify the practical implications of this possibility and whether such cases can be potentially detected when the true DAG is unknown?
  2. For a fixed graph and causal query, different valid adjustment sets may lead to different radius. How much does the radius vary across these sets, and could adjustment-set selection jointly determine the robustness?
2 comments
AuthorsOpenReview ↗

Comment 1 of 2.

We thank the reviewer qQYf for the constructive questions. We are glad that the reviewer finds the paper and the theoretical results clearly written and formulated, and considers the robustness of adjustment sets an interesting problem. The questions on the uniqueness of the adjustment set led to a result that makes the paper simpler. We answer each point below.

Practical motivation

Our setting is an effect estimate adjusted with orientations supplied by an expert or a language model, which the data cannot check and an analyst can afford to verify only in part. The breakdown radius turns this into a decision rule:

  • rval=∞r_{\mathrm{val}} = \infty: no revision of the knowledge can invalidate the adjustment set, and the estimate can be reported as is;
  • rval=1r_{\mathrm{val}} = 1: one named orientation should be verified first;
  • rval=kr_{\mathrm{val}} = k: up to k−1k-1 wrong orientations are tolerated.

In the statin example (Section 4), the analyst asserts four orientations and adjusts for {Age,Smoke}\{\text{Age}, \text{Smoke}\}; Meek’s rules add a fifth, Chol →\to Statin, which the analyst never stated. Here rval=3r_{\mathrm{val}} = 3: the set can fail only if BMI →\to Chol, Smoke →\to BMI and Chol →\to Statin are all wrong, in which case Smoke may lie downstream of the treatment. The certificate thus tells the analyst how many errors are tolerable and which orientations to examine, including one they did not know they relied on.

W1 The SAT stage, the explicit search and the depth bb

Why a constraint solver over other optimisation approaches. The query “is there a Meek-closed state within kk retractions of G0G_0 where ZZ fails?” is an exact feasibility question over Boolean orientation variables, with closure, acyclicity and v-structure constraints. An infeasible answer proves that no failure lies within kk (App. D.1). We translated the same encoding to a mixed-integer program (SCIP, HiGHS, CBC) and to weighted CNF (the MaxSAT solver RC2 and the SAT solver CaDiCaL), and answered 2,083 queries with each (new App. D.3). All return the same radius, so the result does not depend on the solver. CP-SAT (which is the solver we use in Algorithm 1) is the fastest on every query: the MILP solvers take a median 3.5 to 19 times as long and leave 8 to 94 queries unfinished in 300 s, and the SAT and MaxSAT solvers are 1.4 to 3.3 times slower overall once translation to clauses is included.

When it is preferable to explicit search. The upward (retraction) search is cheapest when the nearest failure is close, but its cost grows exponentially with the radius, so the solver takes over beyond the search depth. Listing the whole equivalence class is competitive only on small classes. The table below summarises the regimes, with details in App. D.2.

RegimePreferred methodKey figures
Nearest failure within depth bbUpward (retraction) searchAnswers most queries on its own; the solver stage is reached on only 6 of the 256 timing instances
Nearest failure beyond depth bbConstraint solverAvoids the cost of upward search, which grows exponentially with the radius, and proves that no nearer failure exists
Small equivalence class (n≤24n \le 24; up to about 10310^3 DAGs)Either; listing can winListing faster on 4.9% of queries (n=8n = 8–2424) and 51.7% of synthetic ones, saving a median of 42 ms
Medium class (n=32n = 32–100100; about 10210^2 to 10410^4 DAGs)HybridHybrid under 1 s on 100%; listing faster on at most 0.2%
Large class (n≥150n \ge 150; over 10610^6 DAGs)HybridHybrid under 1 s on 89–99%; listing finishes on 3% at n=150n = 150 and 0% from n=200n = 200

How to select the depth bb. bb affects running time only: both stages return the same radius whenever the solver completes (Remark 6), so no tuning is needed for correctness. We use b=3b = 3 throughout (App. D).

M2 Uniqueness of the optimal adjustment set in Definition 3

In Definition 3, ZZ is the optimal adjustment set of Henckel et al. (2022), as introduced in Section 3, and validity is understood under the generalized adjustment criterion (GAC), which holds exactly when ZZ is valid in every DAG of the MPDAG. The back-door criterion is only sufficient, so it can miss valid sets. Other valid sets exist, but the choice does not change the radius: all sets valid in G0G_0 under the GAC fail at exactly the same knowledge states and share one radius (Proposition C.7.1). Under the back-door criterion this no longer holds and the radius can depend on the set (App. D.1), which is why we use the GAC. We will state this explicitly in Definition 3.

AuthorsOpenReview ↗

Comment 2 of 2.

M3 Line 299, “survives wrong knowledge longer”

Agreed, the sentence is unclear. It means that we test whether a larger radius predicts that ZZ stays a valid adjustment set in the corrupted, re-closed knowledge state as the analyst’s claims are corrupted more heavily, measured by the survival AUC of Section 6.1 (the fraction of draws in which ZZ is still valid, averaged over a ten-point intensity grid). It concerns validity under the corrupted knowledge, not in the data-generating DAG. This wording now replaces the opening sentence of Section 6.

M4 Line 35: oriented by ’a person’ → ’an analyst’

Agreed; the revised version now reads “an analyst orients the remaining edges”.

Q1 A set invalid in the true DAG with a larger radius than a valid one

Can this happen? Within one knowledge state, no: all sets valid in G0G_0 share one radius (Proposition C.7.1). If the true DAG lies in the class, they are valid or invalid together, and invalid only if it contradicts at least rvalr_{\mathrm{val}} closure orientations (Corollary C.7.2). Across different knowledge states, it can: a state containing many errors can still have a large radius.

Practical implication. The radius measures robustness, not accuracy. It says how many orientation errors the conclusion can absorb before the adjustment set may fail, so a large radius certifies that the causal conclusion is less sensitive to the knowledge used, not that the knowledge or the resulting estimate is correct (Section 1). It should be reported beside the estimate for this purpose and not used to rank analysts or knowledge sources by accuracy.

Detection when the true DAG is unknown. The data cannot check these orientations, so such cases cannot be detected from the data, including by comparing estimates across valid sets (Su and Henckel, 2022; App. H.4). What the certificate adds is the orientations whose revision breaks the set, which can be verified directly. Discovery errors, such as a missing adjacency, lie outside the radius, which is conditional on the equivalence class (Remark 8).

Q2 Radius across the valid adjustment sets of one query

It does not vary: all sets valid in G0G_0 fail at the same knowledge states and share one radius (Proposition C.7.1). The measurements agree: on 2,772 published-network queries with a valid alternative set, no listed valid set had a radius different from the committed set’s (above 60 nodes, at most 20 sampled sets per query), and Proposition C.7.1(i) held on all 12,060 four-node and 381,680 sampled five-node combinations. The submitted Appendix F said that valid sets can reach different radii; that was wrong and is replaced. Since the radius is the distance to the nearest state where no adjustment set is valid (Proposition C.7.1(iv)), the choice among valid sets rests on efficiency alone (Appendix F).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion

We thank the Area Chair and the three reviewers. Reviewer C2Bd describes it as "an exact orientation-error budget and witness set for target-specific query robustness rather than relying on global graph metrics like SHD".

Status. All three ratings stand at weak acceptance. Reviewer h8pC moved from weak rejection to weak acceptance after the rebuttal, writing: "Most of my concerns and questions have been addressed." The reviewers find the work "well motivated" (h8pC) and the formulations of its theoretical results "clean" (qQYf), and write that it "seems to be a useful method for the community" (C2Bd). The weaknesses and questions of Reviewers qQYf and C2Bd each have a point-by-point answer in the thread.

Concerns and where the revised PDF answers them.

  • Novelty (h8pC W3). The poset of knowledge states over a fixed CPDAG is graded, unlike the unrestricted MPDAG poset of Taeb et al. (2026), whose distance construction we apply (Section 2). This gives the distance in closed form (Corollary 1). A nearest failing state is reached by retractions alone (Theorem 1), and the certificate names orientations whose retraction breaks the set.
  • Complexity analysis (h8pC W1). The new complexity analysis (Appendix C.8) shows that, for one treatment and one outcome, deciding rval≤kr_{\mathrm{val}} \le k is in P (Proposition C.5): it is enough to examine single moves of the treatment's parent set. On all 679,047 enumerated instances (connected chordal components of at most six vertices), the minimum over these moves equals the minimum over all parent sets. The exponential factors therefore bound Algorithm 1, not the problem (Remark C.2), and every reported radius is unchanged.
  • Scale and pair selection (h8pC W4). On the five published networks of 724 to 1,041 nodes, the radius is exact, never censored, on all 388 of the 1,540 query–state units that admit a certificate, the slowest in 55 s (Appendix I). The selection rule is stated in Table 26. On the 528 released queries, chosen from the CPDAG alone, the radius ranks survival within knowledge states at τb\tau{_b} = 0.63 [0.45, 0.76] (731 units on 18 networks; Section 6.3).
  • Equivalence class (h8pC W2, C2Bd W2). The new Appendix H.4 learns the class with ten CPDAG learners on four Gaussian networks, at sample sizes from 100 to 100,000. When the class is correct near the query and fewer of the analyst's closure orientations are wrong there than the radius (or the radius is infinite), the committed set is valid in the generating DAG in all 46,838 such cases. With the elicited claims on such classes, every committed set within the radius is valid (12,139), while 810 of the 1,514 beyond it (53%) are invalid.
  • Separation as a proxy (C2Bd W3). With separation held fixed, the radius still ranks survival, at τb\tau{_b} = 0.49 [0.40, 0.56] on tiered knowledge and 0.38 [0.31, 0.43] under flips. With the radius held fixed, separation orders survival at 0.04 [−0.12, 0.20] and 0.12 [0.05, 0.23] respectively (Appendix H). The radius is also defined on queries where separation is not, such as 1,033 of the 2,516 units of Section 6.3.
  • Solver and choice of set (qQYf W1, Q1, Q2). Nine exact methods, among them MILP, MaxSAT and SAT solvers, were run on 2,083 queries, and every method that finished returned the same radius (Appendix D.3). All sets valid in the analyst's state G0G_0 under the generalized adjustment criterion share one radius (Proposition C.7.1), so the radius belongs to the query and the knowledge, not to the chosen set. No set valid in G0G_0 and invalid in the generating DAG has a larger radius than a set valid in both (Corollary C.7.2).
  • Presentation (C2Bd W1, Q2, W4). Figure 2 now walks through retraction and join on four nodes, and Table 1 reports the full cohort (2,516 pooled units).

Thank you for a discussion that improved the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.