- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper presents an identifiability result in nonlinear causal represenation learning. Both the structural causal model among the latent variables as well as the mixing functions that generate the observed variables are allowed to be fully flexible, nonlinear functions. The main focus of the paper is on recovering the latent causal graph and the mixing functions under perfect interventions, where each intervention is allowed to target multiple nodes.
Critical strengths
The problem of identification in a fully non-linear model with multi-nodes interventions is, to the best of my knowledge, not yet considered in the literature. To me, it is an interesting problem that potentially fits the scope of ICLR. While it is known that a logarithmic number of multi-node interventions are sufficient and necessary for causal discovery if all variables are observed, this result is not yet known if the variables are latent. Thus, the manuscript closes a gap provided that all claims are correct.
Critical weaknesses
Despite having written papers on identifiability in I find the manuscript very difficult to comprehend in its current form. Moreover, related literature appears not to be enough discussed. I detail my concerns in the following:
-
The paper uses a lot of new, unusual terms, which are not properly introduced and could mostly be avoided. Examples include "residual charts", "target-environment pairs", "target signatures", "minimum-code chart", "replacement laws", "fiber normalizer", "oriented foliation", "sink peeling", "terminal-response faithfulness (TRF)", "conditional-foliation peeling (CFP)". This might come from the fact that several mathematical fields are combined (e.g. measure theorey, differentiable geometry, coding theory, graph theory), but I do not expect that readers are familiar with all of them. So novel terms should be explained, and examples and intuition should be given.
-
Sentences tend to be chains of compressed noun phrases, for example "coordinatewise-normalized environment ratios", "positive paired-support densities", and "a selective finite-sample realization in an affine–probit observation model with quadratic mechanisms". Many sentences contain both multiple reasons and conclusions, an example is "Code containment and minimum weight then align the target signatures, while uniform-distribution rigidity reduces each remaining scalar map to a reflection." There is very little intuition, examples, or "in other words" restating. While this could be due to place constraints, it makes the paper very difficult to read. The only example is in Section 3.4 and has four lines. I would appreciate more examples and intuition.
-
The structure of the paper is difficult to follow. During the text it is often referred to later theorems. To illustrate a related point, the introduction says towards the end that `"across 800 numerical evaluations, every one of the 288 returned graphs is correct, and correct returns increase from 0/160 to 160/160 over the tested sample-size range". This comes before the reader knows what an evaluation is, what abstention means, or what varies across conditions.
-
The main results appear loosely stated. For example, how are the "residual variables" in Theorem 4.3 defined exactly, or how is the "minimax number" in Corollary 4.4 defined exactly?
-
For causal discovery on observed variables, Eberhardt, Glymour and Scheines (2005) showed in their paper "On the Number of Experiments Sufficient and in the Worst Case Necessary to Identify All Causal Relations Among Variables" that multi-node interventions suffice and are necessary in the worst case. This paper should be included in the discussion of related literature since, in some sense, the manuscript generalizes it to the causal representation learning setup.
Questions for authors
The identifiability results requires that we have the joint distribution of each interventional environment and the observed environment. To me this seems to be a strong assumption. In which real-world situations could we defend it to be reasonable, and where are its limitations, i.e., when is it not a reasonable assumption?
6 comments
Response to Reviewer a6kj — Weaknesses (2/4):
W2. Clearer phrasing and the example in §3.4
Following the approach in W1, we simplify compressed expressions while preserving the technical terms needed to specify the models and conditions.
1. Finite-sample procedure and abstention
The phrase “a selective finite-sample realization in an affine–probit observation model with quadratic mechanisms” combines four points: the finite-sample procedure, its option to abstain, the observation model, and the structural mechanisms.
We retain “affine–probit observation model” and “quadratic mechanisms”, which specify the setting. We replace the compressed phrase with a direct description of the procedure and its return-or-abstain rule:
We also develop a finite-sample procedure for affine–probit observations with quadratic mechanisms. Using atomic pairs, it returns a graph when simultaneous confidence intervals resolve all decisions, and abstains otherwise.
The probability guarantee and numerical results follow this passage. The meaning of “selective” is conveyed by the return-or-abstain rule, while the formal result name in §4.6 is retained.
2. “coordinatewise-normalized environment ratios”
We expand the compressed modifier into an action while preserving the original contrast between two signals that address two identification challenges.
Original:
Support invariances determine which residual coordinates can change, while coordinatewise-normalized environment ratios determine how those coordinates propagate through the causal system.
Revised:
Support invariances determine which residual coordinates can change, while normalizing environment ratios separately along each coordinate reveals how these changes propagate through the causal system.
3. “positive paired-support densities”
This phrase specifies an identification condition. We preserve its meaning while making the relationships between the modifiers explicit.
Original:
Positive paired-support densities and distinct nonzero signatures ensure that ...
Revised:
Positive densities on the paired supports, together with distinct nonzero target signatures, ensure that ...
“On the paired supports” specifies where the densities are defined, and “target signatures” identifies the object of the second condition. Section 2.2 gives the support , the corresponding measure, and the condition's role. The introductory sentence now makes these relationships explicit without repeating the formal setup.
4. “Code containment … uniform-distribution rigidity”
The original sentence compressed the intermediate step from signature inclusion to triangular structure. We now explain the sequence: signature alignment, triangularity, and elimination of the remaining mixing through independent uniform outputs. This makes the role of the distributional constraint explicit.
Original:
Code containment and minimum weight then align the target signatures, while uniform-distribution rigidity reduces each remaining scalar map to a reflection.
Revised:
Code containment and minimum weight align their target signatures. Signature inclusion then makes triangular after reordering. Requiring independent uniform outputs removes the remaining mixing, leaving coordinate reflections.
This revision appears in the proof sketch in §4.1, between the coordinate-matching argument and the conclusion covering arbitrary rank and nested signatures. The full proof is in Appendix E, which was Appendix C in the original submission.
NOTE: All appendix references in these responses follow the revised numbering.
Response to Reviewer a6kj — Weaknesses (3/4):
W2. Worked example (continued)
5. The four-line example in §3.4
Section 3.4 provides a brief overview for the chain and links to the full calculations in Appendix H. We expand the key comparisons here to show how the example illustrates sink identification and recursive recovery.
Environments target , and environment targets . For node 3, uses a uniform replacement distribution, and uses density , where . Appendix H gives
where is strictly increasing. The first equation satisfies the identity-CDF condition in an environment that does not target node 3. The other two CDFs share their level sets and their ordering. Node 3 therefore passes both comparisons.
Node 2 is not targeted in . If it were a current sink, its conditional CDF would equal . However, with the mechanism parameter from Appendix H,
By continuity, this deviation persists on a neighborhood, so node 2 fails the test at this stage. After marginalizing node 3's residual coordinate, node 2's conditional CDFs in become . In , its CDF is a strictly increasing transformation of its causal coordinate . Node 2 then passes the test, yielding the removal order .
This calculation directly corresponds to the geometric explanation in §3.3. The revised proof sketch in §4.1 also explains signature alignment, triangularity, and the independent uniform constraints. These passages clarify the geometric mechanism and the residual-identification argument, respectively.
W3. Paper structure and the numerical summary
Thank you for the time and care devoted to following the paper's theoretical development. We have made the relationships between the construction and its guarantees easier to trace.
The paper proceeds from problem setup (§2) to identification procedure (§3) to theoretical guarantees (§4). The forward references in §3 identify the guarantee for each construction:
- Theorem 4.1: recovery of residual coordinates and target signatures.
- Theorem 4.2: sink identification.
- Theorem 4.3: full causal identification.
- Corollary 4.4: the optimal environment count, combining identification with the design lower bound in Proposition K.1.
We have added a concise, hyperlinked THEORETICAL ROADMAP near the beginning of the appendix, with an entry point in the main text. It shows how the results and proofs fit together, supporting both sequential reading and direct reference.
For the numerical summary, one evaluation means running the graph-recovery procedure on one generated instance at a specified sample size. There are 160 independent replicate sequences, covering four dimensions, four graph families, two response regimes, and five replicates per combination. Each sequence is evaluated at five sample sizes, giving evaluations.
As explained in W2, the estimator returns a graph when the decisions are resolved and abstains otherwise. Of the 800 evaluations, 288 returned a graph, all correctly, and 512 resulted in abstention. In increasing order of sample size, the correct-return counts are 0,0,16,112,160, each out of 160 evaluations. Appendix Q.7 gives the definitions and complete setup.
To introduce the evaluation conditions alongside the numerical results, we write:
Across 800 evaluations spanning dimensions, graph families, response regimes, and sample sizes, all 288 returned graphs are correct. The correct-return rate rises from 0% to 100% as sample size increases.
Here, the correct-return rate is the fraction of evaluations at a given sample size that return the correct graph, with abstentions included in the denominator.
Response to Reviewer a6kj — Weaknesses (1/4):
Comment
Thank you for the time and care you have devoted to reviewing our paper. We appreciate your suggestions on how to make the presentation clearer and more accessible. We have revised the wording, expanded key explanations, and added a Glossary and a Theoretical Roadmap. In the four accompanying comments, we address the points raised in the Weaknesses section and explain the corresponding revisions. We hope these changes make the ideas and reasoning easier to follow, and we welcome further discussion.
W1. Terminology and intuition
Thank you for raising this concern. Our analysis combines tools from several mathematical fields and introduces names for the proposed identification conditions and recovery methods. We recognize that some terms needed a clearer introduction for readers outside those fields. We have revised the terminology and its presentation in two ways:
The GLOSSARY gives concise explanations of key terms, their roles, and their intuition. It is intended to help readers follow the paper and to provide useful connections to mathematical tools that may inform future work in CRL.
The following three examples illustrate these changes. They are representative selections; the revisions are incorporated throughout the paper.
1. “target–environment pairs”
This compound phrase can be simplified. We have consistently replaced it with “across targets and environments”, which expresses the intended meaning directly.
2. “replacement laws”
In probability theory, law and distribution are standard synonyms (Norris [1, §2.3]). Here, the term denotes the probability distribution used to draw replacement values for a target variable. The word replacement specifies its role in intervention sampling.
We now consistently use “replacement distributions”. In §2.2, we explicitly connect this term to the replacement distribution with density νie. This gives the terminology, sampling definition, and subsequent conditional cumulative distribution function (CDF) explanation a consistent vocabulary. We have also replaced “replacement strength” with “replacement distributions”.
3. “fiber normalizer”
A fiber here is the coordinate line obtained by fixing the other coordinates and varying uj. The fiber normalizer is the normalizing factor obtained by integrating along that line. We retain this term because its geometric meaning helps explain the construction:
We retain “fiber normalizer” in §3.3 and in the glossary. Before its formal definition, we add:
This identifies the coordinate projection's inverse image and explains the geometry of the integral.
References
[1] J. R. Norris. Probability and Measure. Lecture notes, §2.3.
[2] Peter Bürgisser and Felipe Cucker. On a Problem Posed by Steve Smale. Annals of Mathematics, 174:1785–1836, 2011. §8.3.
[3] Persi Diaconis, Susan Holmes, and Mehrdad Shahshahani. Sampling From a Manifold. arXiv:1206.6913, 2012. Proposition 2.