acceptodds
Under review as a conference paper at ICLR 2027

When Does Causal Repair Ordering Survive Federated Evidence Formation?

Abstract

Federated causal analysis can have enough data overall to answer a target question while no individual client can form the evidence needed for a particular test. The issue matters when the same target can be settled by several alternative sets of conditional-independence (CI) tests, which we call exact repairs. Federation can reverse this centralised preference: even reallocating the same pooled observations can make a shorter repair less likely to form than a longer one. The gap can be arbitrarily large: a target solvable by one query structurally and after pooled query-statistic aggregation can require arbitrarily many locally formed queries, with the gap persisting through any prescribed bounded aggregation depth. Formation ease and target-decision value can also rank repairs differently. Under random federation, we characterise how route support demands and federation scaling determine how much aggregation is needed, and give a concrete condition that preserves the centralised ordering. Experiments on public Bayesian networks show shallow recovery with limited aggregation, cases where a more expensive repair attains higher local readiness than every cheaper alternative, and strong sensitivity to how an unchanged pooled sample is split across clients.

Primary area:
causal reasoning
Keywords:
causal discovery, federated learning, conditional independence, query complexity, evidence aggregation, statistical decision theory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MhrtOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers an interesting causal question under the federated setting: When does a centralised preference for a shorter sufficient evidence route survive federated evidence formation? In this paper, the authors prove that an unbounded 1:h:1 repair-cost separation that can persist through any prescribed bounded aggregation depth; and also characterize the formation depth under random federation. The experiments report readiness frontiers over aggregation depth, sample size and client allocation.

Critical strengths
  • The paper proposes a clear, new, and interesting question.
  • Each part is well organized and demonstrated, e.g., structural cost, formability, target-decision value, with both positive and negative results.
  • The 1:h:1 construction with independently selectable depth and randomised lower bounds provides a rigorous worst-case picture.
  • The experiments consider multiple seeds, making the results reliable, overall the experiments are extensive with grounded theories.
Critical weaknesses
  • There are some assumptions in the paper, whether they can be satisfied in real world is not sufficiently demonstrated in the paper. Some of them can be idealized, such as Assumption 2.
  • In the federated setting, communication and privacy issues are important. Those topics are not sufficiently discussed.
  • Including some real world case study or examples can enhance the reliability and utilities of this paper.
Questions for authors

See above.

3 comments

Thank you for your suggestions on the practical aspects of the paper. We’ve responded in the rebuttal and will soon share a revised manuscript with an expanded practical discussion and an example using real measurements. We’d be happy to clarify anything you’d like to discuss further. If the response addresses your concerns and you find the additions improve the paper, we’d appreciate your considering a higher rating.

Thanks for the detailed and candid response. My main concerns regarding the assumptions, communication and privacy, and real-world evidence have been adequately addressed. Given the novel and well-motivated question, the rigorous theoretical analysis with both positive and negative results, and the extensive experiments across multiple seeds and settings, I therefore maintain my positive assessment (3: Weak acceptance).

I do not raise my score further, for the following reasons, which concern the overall scope of the work rather than missing experiments:

  1. Gap between support readiness and decision reliability. The theory is developed at the support level under an idealized correct-or-abstain model. As your new results show, readiness does not by itself guarantee correct causal decisions in practice.
  2. Mostly worst-case and constructed results. The strongest results rely on carefully constructed instances, and the experiments use small Bayesian networks. It remains unclear how often these phenomena arise in realistic federated settings.
  3. Accessibility. The paper is dense and introduces many interacting notions. It also relies heavily on a long appendix, which makes it hard for the broader ICLR audience to follow.

Thank you for the thoughtful follow-up and for maintaining your positive assessment. We appreciate your clarification of the remaining concerns about scope and accessibility.

We agree that support readiness alone does not guarantee reliable causal decisions. Theorem 4 (Section 4.3) makes the distinction between full-repair readiness and decision value precise for limiting target-decision experiments, within the correct-or-abstain model and under a common graph-independent joint readiness-mask law. The finite-sample CI experiments in Section 5 separately examine the effects of estimated-label errors; these results do not constitute a general finite-sample reliability guarantee.

The stochastic formation laws in Theorem 2 and the sufficient preservation condition in Corollary 3 complement the constructed separations. Together, they identify conditions for reversal and preservation, without claiming how common reversals are in realistic federated deployments.

We have uploaded a revised manuscript. To improve accessibility, it includes a concrete hospital example, an explicit CHILD target in Section 3, a concise repair-selection procedure in Section 4.1, and finite-sample decision results with their scope explained in Section 5. We have also moved the detailed adaptive witness characterisation to Appendix C while retaining a short explanation of its role in the main argument. We hope these revisions make the paper easier to follow and would be happy to clarify any remaining questions.

Reviewer_GBuTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors study federated causal analysis in which a target relation can be resolved by several alternative sets of conditional-independence tests, termed exact repairs. They establish a 1:h:1 separation between structural, client-local, and pooled repair costs, show that client-local formation can reverse the repair ordering even with pooled information fixed, and give a support condition that preserves the centralized ordering. Experiments on public Bayesian networks support the theory.

Critical strengths
  1. The authors formal results for both repair-order reversal and preservation.

  2. The experiments include complete lower-cost repair frontiers and a same-pooled-sample allocation comparison. These controls make the main claims more convincing.

Critical weaknesses
  1. The main results rely on Assumptions 1 and 2: clients agree on the labels of admissible CI queries, and a query returns the correct label once its support threshold is met. In practical federated causal discovery, local CI tests may disagree.

  2. The paper is difficult to follow. The presentation is poor and the notions are dense, which make the work difficult to follow. An example may be helpful.

Questions for authors
  1. Assumption 1 allows clients to have different distributions but requires them to agree on every admissible CI label. What happens when two clients give different population CI labels for the same query? In that case, how would an exact repair and its pooled counterpart be defined?

  2. In Theorem 4, the readiness mask is constructed to be independent of the candidate graph. How restrictive is this requirement? Does the target-decision trichotomy still hold when the support distribution depends on the candidate graph?

  3. Corollary 3 requires comparing Cov_T across repairs. How could this condition be checked without knowing the candidate causal class and its CI labels?

  4. The experiments measure whether the required conditioning cells have enough samples but do not evaluate the correctness of the resulting CI decisions. Could the authors add an experiment using actual finite-sample CI tests and report whether the repair ordering still reverses?

3 comments

Thank you for your detailed questions. We’ve replied point by point and will soon share a revised manuscript that includes the finite-sample CI results. If you have any remaining questions about the analysis or the experiment, we’d be glad to clarify. If you feel your questions have been addressed and the new results strengthen the paper, we’d be grateful if you would consider raising your score.

Thank you again for your detailed questions. The revised manuscript is now uploaded, including the requested finite-sample CI results and clarifications on shared CI semantics, graph-dependent readiness masks, and coverage checks.

We would be grateful for your feedback on these additions and clarifications. If you have any remaining questions about the analysis or experiments, we would be glad to discuss them.

AuthorsOpenReview ↗

A Sincere Follow-up Before the Discussion Deadline

Dear Reviewer,

Thank you again for your time and thoughtful review. With approximately one day remaining in the discussion period, we sincerely hope to hear your thoughts on our responses and revised manuscript. Your feedback is very important to us, and we would be grateful for any further comments or remaining concerns you may wish to share.

Thank you very much for your time and consideration.

Reviewer_Q7oaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates when the structural preference among alternative sets of conditional-independence (CI) queries, termed exact repairs, is preserved under federated evidence formation. It shows that a structurally cheaper repair can become less feasible when evidence must be formed locally, even when the pooled observations remain unchanged. The authors establish an unbounded separation between structural, client-local, and pooled query costs, characterize aggregation-depth requirements and repair-order reversals, and provide sufficient conditions for preserving the original ordering. Experiments on public Bayesian networks support these theoretical findings through controlled studies of aggregation, client allocation, and repair readiness.

I am not an expert in the field of CI testing. However, from my expertise in federated learning (FL) and years of reading that literature, I can say that this paper solves an important problem, but communicates it poorly. It looks nothing like an FL paper and would possibly not be used by that community. I recommend rejection and suggest the authors rewrite the paper, keeping in mind the ML readers, who benefit from a clear motivation, a concise problem setting, and an algorithmic methodology. While the experiments can stay as is, I suggest bringing most of Appendix K to the main paper to show rigor. Consequently, most of the theoretical results (except the main ones) can move to Appendix.

Critical strengths
  1. The paper goes beyond demonstrating isolated ordering reversals by establishing an unbounded separation between structural, client-local, and pooled adaptive repair costs, including randomized adaptive lower bounds. The bounded-aggregation and stochastic formation-depth results further characterize when this distortion persists and when aggregation can eliminate it. These results provide a rigorous characterization of an important limitation of federated causal evidence formation, making the contribution more substantial than a collection of counterexamples.

  2. The empirical analysis compares complete repair-cost frontiers rather than relying exclusively on selected repair pairs, and uses matched designs to isolate the effect of federation. The aggregation-depth experiments demonstrate that limited cross-client aggregation can recover the structurally preferred repair. These strengthen the evidence that observed reversals arise from evidence fragmentation rather than changes in the underlying causal task.

Critical weaknesses
  1. The framework assumes a fixed residual DAG class and admissible query universe, and compares inclusion-minimal exact repairs derived from these objects. However, constructing and comparing the complete repair family can be combinatorially expensive. The experimental analysis reports a large number of exact repairs, despite operating on relatively small target-specific problems. The evaluation also relies on simulated federations generated from known public Bayesian-network populations rather than realistic federated causal-discovery pipelines. Without a scalable procedure for obtaining and selecting repairs, or evidence on larger and less controlled tasks, it is difficult to assess whether the theoretical characterization can guide practical federated causal discovery.

  2. The analysis assumes that every CI query satisfying predetermined support thresholds returns the correct population-level independence label, while client distributions preserve the same admissible-query CI semantics. Consequently, the theoretical and empirical results largely characterize support availability rather than the reliability of statistical evidence. In realistic federated causal discovery, CI tests exhibit finite-sample errors, variable statistical power, and heterogeneous effect strengths, even when conditioning-cell counts exceed a threshold. Since the experiments do not evaluate actual CI-test errors or downstream causal recovery accuracy, it remains unclear whether the demonstrated readiness ordering reversals translate into comparable improvements in reliable target identification. This limits the practical significance of the central findings.

Questions for authors

Please address the critical weaknesses

6 comments

Thank you for your feedback on the paper’s presentation. We’ve responded to your comments in the rebuttal and will soon share a revised manuscript that clarifies the motivation and repair-selection procedure. We’d be happy to elaborate on any points you’d like to discuss further. If the response and revision address your concerns and improve your assessment of the paper, we’d appreciate your considering a higher rating.

Thank you again for your feedback. We have uploaded a revised manuscript with a clearer motivation and problem setting, an explicit repair-selection procedure, and finite-sample CI results in the main text. These revisions incorporate the clarifications and experiments discussed in our rebuttal.

We would appreciate your thoughts on the updated manuscript and would be happy to clarify any remaining questions.

Thank you for clarifying the repair-selection procedure. The deletion algorithm addresses my concern that finding a single inclusion-minimal ready repair requires enumerating the entire repair family. My remaining concern is end-to-end scalability. The reported selection times exclude candidate-class and coverage preparation, and the CHILD experiments construct certificates relative to a known reference DAG. The results therefore leave open whether the procedure can guide repair selection when the candidate class must be learned from data or is substantially larger.

The finite-sample CI experiments address my concern that the original evaluation measured only readiness. On CHILD, the five-query strategy increases correct local target decisions, with the gain coming from fewer abstentions. The Light Tunnel result illustrates the remaining limitation. Two-client aggregation achieves a high readiness, yet only a handful of evaluations agree with the physical reference graph. I appreciate the inclusion of this result, which shows that the relationship between readiness and reliable target recovery can depend strongly on the task.

The rebuttal strengthens the empirical evaluation and clarifies how a usable repair can be selected. For my assessment, the remaining gaps in end-to-end scalability and the reliability of target recovery under realistic federated conditions are still consequential. Moreover, I was not impressed by the submitted draft of the paper, which has serious presentation issues. In my humble opinion, the reason to reject the paper outweigh its current contribution. I will retain my score as is.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.