- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the research problem of constructing linear causal abstractions (CA) for linear structural causal models (SCM). It departs from existing work by extending to a cyclic SCMs. Specifically, the authors provide an algebraic characterisation for a subclass of cyclic models and provide a sufficient graphical condition (All-In, One-Child, acyclic cluster) for the existence of an abstraction map. Furthermore, the authors propose a two stage pipeline for learning CAs for linear SCMs. The first stage includes three graph partitioning algorithms (where one algorithm claims to guarantee sufficiency). The second stage includes a constrained optimization formulation for learning the aggregation weights of a given partition. The proposed method is evaluated on synthetic graphs and economic input-output networks and the results demonstrate superior performance to UCA (unsupervised baseline).
Critical strengths
- The cyclic extension for CA construction is well motivated and addresses an immediate gap in existing works.
- The paper not only includes a novel algebraic characterisation but also a constructive pipeline that is well connected to the developed theory.
Critical weaknesses
-
The validity guarantee claim for AOC Closure is not supported. The paper claims to "propose three partitioning algorithms including one that guarantees these conditions" in the abstract. In the caption of Figure 3, it is claimed that "AOC Closure which is guaranteed to admit a valid abstraction by Theorem 8". Theorem 8 states that All-In, One-Child, and acyclicity are sufficient conditions for the existence of a valid abstraction. I interpret this as AOC Closure's output is guaranteed to satisfy the three sufficiency conditions and hence, must admit a valid abstraction. However, in Sec 4.2 where AOC Closure is proposed, the authors only states that AOC Closure will satisfy the All-In and One-Child conditions but do not mention the acyclic condition.
-
The paper's claim on providing a principled tradeoff between exactness and informativeness is weakly supported. In Figure 3, AOC closure, the only method with a claimed guarantee to admit a valid abstraction is consistently at ReconErr ~1 (which is the value of a trivial abstraction) across all three plots. The only method that consistently achieves ReconErr below 1 is Greedy All-In which doesn't have any theoretical guarantees (its output isn't guaranteed to satisfy any of the sufficiency conditions in Theorem 8). This is further supported in Figure 2 where Greedy All-In’s lower ReconErr comes with visibly higher RAE, whereas AOC Closure has lower RAE but substantially higher ReconErr (sometimes >1). Together, this means that AOC Closure is exact but not informative while Greedy All-In is informative but not exact. This weakens the paper's claim to offer "a principled trade-off between exactness and informativeness" as the paper only proposes two extreme options.
Questions for authors
Questions:
-
Can the authors theoretically support the claim that the AOC Closure algorithm is guaranteed to satisfy the three sufficiency conditions?
-
Can the authors provide any additional support for claiming a principled trade-off between exactness and informativeness beyond a method that is exact but non-informative and a method that is informative but inexact?
-
With regard to the metric RAE which is defined as the max over all subsets of {1, ..., k}, the paper computes RAE only over the empty set and singletons due to intractability for large k. However, the experiments in Table 1 and Figure 3 all use k=10. Full enumeration would then require computing over 1,024 subsets. Could the authors report RAE computed over more than just singleton subsets at k=10 or report RAE computed over all subsets at a smaller k (k=5 would only require computing over 32 subsets)?
2 comments
Thank you again for your careful review. In response we have proved that AOC Closure’s output satisfies all three sufficiency conditions — it now initialises from strongly connected components, so every partition it returns is a coarsening of them and is acyclic (Section 4.2, Appendix A.8). We have also shown that the hyperparameter in the GradCAP algorithm acts as a dial that trades off exactness with intra-cluster edges (Section 5, Table 1). Expanding our synthetic experiments to different graph settings and considering win-rate against AOC Closure, we see that the different methods we propose are advantageous in different settings (Section 5, Figure 2; Appendix E.4) with some cross-over when the tolerance (e.g. Cyclic graphs in Figure 2 with ). We now report RAE over all subsets rather than singletons, both on synthetic graphs and across sixteen economies in out Leontief experiments (Appendix E.5, Table 4).
We would be glad to address any further questions while the discussion period allows.
We thank reviewer 38wS for taking the time to review our paper and for their considered comments.
AOC Closure guarantee
You are right that we left the acyclic condition implicit and could have spelt this out more. In the statement of Theorem 8 we claim in passing that the cluster graph will be acyclic whenever the low-level graph is acyclic and the All-In condition is satisfied. Here is the full argument for this.
Suppose we have an acyclic low-level graph G and a partition π that satisfies the All-In condition. Let Gπ denote the cluster graph and suppose we have a cycle in this graph i0→i1→⋯→il−1→i0. By the All-In condition, we can construct a low-level path beginning with r0∈π(i0), going to r1∈π(i1) and so on each time picking a low-level variable rj∈π(ij) for j∈{0,...,l} in the order of the cycle. Since we assume G is a finite graph, this path will eventually contain a cycle contradicting the acyclicity of G.
So when G is acyclic, AOC Closure is guaranteed to admit a valid abstraction. For cyclic G, we can ensure that Gπ is acyclic by beginning with the strongly connected components (SCCs) as our initial partition and running AOC Closure to find a coarsening of it that satisfies the All-In and One-Child conditions. The argument above then adapts. Suppose such a coarsening had a cycle i0→⋯→il−1→i0 in its cluster graph. The low-level walk constructed above is now closed, so it contains a low-level cycle; the variables on that cycle therefore lie in a single SCC, hence in a single block of the initial partition, and hence in a single cluster of any coarsening of it. This contradicts the distinctness of i0,…,il−1. Note that finding the initial SCC partition does not affect the asymptotic runtime complexity of the algorithm.
Exactness-Informativeness trade-off
AOC Closure and Greedy All-In do indeed give two extreme points on the RAE-ReconErr curve and are intended to act as baselines for other methods. Further experiments show that OC Closure gives a genuine trade-off between the two methods, particularly when the low-level graph is cyclic and its cycles are contained within clusters. In our latest version of the paper we present results that make this explicit. OC Closure has no guarantee, since it targets One-Child but not All-In, but it is exact on almost every cyclic instance we tested. Indeed under One-Child each cluster is constrained by at most one collinearity equation and a cycle of clusters defines an eigenvector equation which generically admits a solution. The solution is not in general marginalisation-free; this is only guaranteed if it additionally satisfies the All-In Condition.
At the end of section 4 we briefly mention a gradient-based method, GradCAP, which admits a regulariser on cluster size via a hyperparameter λintra. Our latest results show that the largest cluster shrinks in size monotonically as λintra increases and that selecting among a small grid of λintra at a fixed RAE tolerance (a selection that requires no ground truth, since both RAE and ReconErr are computable from A alone) yields an abstraction that improves on AOC Closure's ReconErr up to 77% of the time for cyclic graphs whose cycles are contained within clusters.
Computing exact RAE
Thank you for your suggestion. We have ensured that the full RAE is now computed whenever k≤12. This does not materially change any of our results.