acceptodds
Withdrawn from ICLR 2027

Logarithmic Interventions Suffice for Nonlinear Causal Representation Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Causal representation learning aims to recover latent variables and their causal graph from nonlinear observations. However, existing identifiability results leave open full recovery under hidden multi-node intervention targets and replacement laws that vary across target-environment pairs. In this work, we show that same-unit factual-interventional pairs separate target identification from causal ordering. A minimum-code criterion over candidate residual charts and binary target codes recovers residual coordinates and intervention signatures (binary environment-target patterns) by selecting a minimum-Hamming-weight code consistent with paired invariances. After this alignment, coordinatewise normalization of environment density ratios isolates causal-order information from replacement strength. We then introduce Conditional-foliation peeling (CFP), which compares the oriented-foliation structure of the resulting conditional CDFs to identify and recursively remove sinks, variables with no remaining children. For smooth scalar invertible-noise SCMs with an invertible decoder and fixed conditional supports, the paired laws identify endogenous variables up to permutation and componentwise strictly monotone transformations and recover the entire DAG, assuming positive paired-support densities and Terminal-response faithfulness (TRF). Any distinct nonzero signatures suffice, including codes of arbitrary rank and nesting. Matching upper and lower bounds show that the minimax number of labeled intervention environments, excluding the factual environment, is exactly for .

Primary area:
causal reasoning
Keywords:
causal representation learning, identifiability, causal discovery, paired interventions, multi-node interventions, intervention design

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_a6kjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents an identifiability result in nonlinear causal represenation learning. Both the structural causal model among the latent variables as well as the mixing functions that generate the observed variables are allowed to be fully flexible, nonlinear functions. The main focus of the paper is on recovering the latent causal graph and the mixing functions under perfect interventions, where each intervention is allowed to target multiple nodes.

Critical strengths

The problem of identification in a fully non-linear model with multi-nodes interventions is, to the best of my knowledge, not yet considered in the literature. To me, it is an interesting problem that potentially fits the scope of ICLR. While it is known that a logarithmic number of multi-node interventions are sufficient and necessary for causal discovery if all variables are observed, this result is not yet known if the variables are latent. Thus, the manuscript closes a gap provided that all claims are correct.

Critical weaknesses

Despite having written papers on identifiability in I find the manuscript very difficult to comprehend in its current form. Moreover, related literature appears not to be enough discussed. I detail my concerns in the following:

  1. The paper uses a lot of new, unusual terms, which are not properly introduced and could mostly be avoided. Examples include "residual charts", "target-environment pairs", "target signatures", "minimum-code chart", "replacement laws", "fiber normalizer", "oriented foliation", "sink peeling", "terminal-response faithfulness (TRF)", "conditional-foliation peeling (CFP)". This might come from the fact that several mathematical fields are combined (e.g. measure theorey, differentiable geometry, coding theory, graph theory), but I do not expect that readers are familiar with all of them. So novel terms should be explained, and examples and intuition should be given.

  2. Sentences tend to be chains of compressed noun phrases, for example "coordinatewise-normalized environment ratios", "positive paired-support densities", and "a selective finite-sample realization in an affine–probit observation model with quadratic mechanisms". Many sentences contain both multiple reasons and conclusions, an example is "Code containment and minimum weight then align the target signatures, while uniform-distribution rigidity reduces each remaining scalar map to a reflection." There is very little intuition, examples, or "in other words" restating. While this could be due to place constraints, it makes the paper very difficult to read. The only example is in Section 3.4 and has four lines. I would appreciate more examples and intuition.

  3. The structure of the paper is difficult to follow. During the text it is often referred to later theorems. To illustrate a related point, the introduction says towards the end that `"across 800 numerical evaluations, every one of the 288 returned graphs is correct, and correct returns increase from 0/160 to 160/160 over the tested sample-size range". This comes before the reader knows what an evaluation is, what abstention means, or what varies across conditions.

  4. The main results appear loosely stated. For example, how are the "residual variables" in Theorem 4.3 defined exactly, or how is the "minimax number" in Corollary 4.4 defined exactly?

  5. For causal discovery on observed variables, Eberhardt, Glymour and Scheines (2005) showed in their paper "On the Number of Experiments Sufficient and in the Worst Case Necessary to Identify All Causal Relations Among NN Variables" that multi-node interventions suffice and are necessary in the worst case. This paper should be included in the discussion of related literature since, in some sense, the manuscript generalizes it to the causal representation learning setup.

Questions for authors

The identifiability results requires that we have the joint distribution of each interventional environment and the observed environment. To me this seems to be a strong assumption. In which real-world situations could we defend it to be reasonable, and where are its limitations, i.e., when is it not a reasonable assumption?

6 comments
AuthorsOpenReview ↗

Response to Reviewer a6kj — Weaknesses (1/4):

Comment

Thank you for the time and care you have devoted to reviewing our paper. We appreciate your suggestions on how to make the presentation clearer and more accessible. We have revised the wording, expanded key explanations, and added a Glossary and a Theoretical Roadmap. In the four accompanying comments, we address the points raised in the Weaknesses section and explain the corresponding revisions. We hope these changes make the ideas and reasoning easier to follow, and we welcome further discussion.

W1. Terminology and intuition

Thank you for raising this concern. Our analysis combines tools from several mathematical fields and introduces names for the proposed identification conditions and recovery methods. We recognize that some terms needed a clearer introduction for readers outside those fields. We have revised the terminology and its presentation in two ways:

  1. Simplify expressions where the mathematical meaning is preserved. We use terminology familiar to the causal representation learning community when it conveys the same meaning and remains precise for readers from the original mathematical field.
  2. Retain useful concepts and explain them. Terms such as terminal-response faithfulness (TRF) and conditional-foliation peeling (CFP) name central conditions and methods introduced in this paper. We retain these terms and provide a GLOSSARY near the beginning of the appendix. For important terms used repeatedly, we also add explanations at appropriate places in the main text.

The GLOSSARY gives concise explanations of key terms, their roles, and their intuition. It is intended to help readers follow the paper and to provide useful connections to mathematical tools that may inform future work in CRL.

The following three examples illustrate these changes. They are representative selections; the revisions are incorporated throughout the paper.

1. “target–environment pairs”

This compound phrase can be simplified. We have consistently replaced it with “across targets and environments”, which expresses the intended meaning directly.

2. “replacement laws”

In probability theory, law and distribution are standard synonyms (Norris [1, §2.3]). Here, the term denotes the probability distribution used to draw replacement values for a target variable. The word replacement specifies its role in intervention sampling.

We now consistently use “replacement distributions”. In §2.2, we explicitly connect this term to the replacement distribution with density νie\nu_i^e. This gives the terminology, sampling definition, and subsequent conditional cumulative distribution function (CDF) explanation a consistent vocabulary. We have also replaced “replacement strength” with “replacement distributions”.

3. “fiber normalizer”

A fiber here is the coordinate line obtained by fixing the other coordinates and varying uju_j. The fiber normalizer is the normalizing factor obtained by integrating along that line. We retain this term because its geometric meaning helps explain the construction:

  • A fiber is the inverse image of a coordinate projection. Integration over such inverse images and the construction of conditional densities appear in related mathematical work [2, §8.3]. In our setting, integrating the environment density along a coordinate fiber gives the normalizer. Dividing by it yields the conditional density, consistent with the standard construction of conditional distributions [3, Proposition 2].
  • Normalizing and integrating along a coordinate fiber gives a conditional density and a conditional CDF. Considering these CDFs over different fixed values of the other coordinates yields the level sets compared later. Thus, the fiber connects conditioning along a coordinate to the subsequent geometric comparison.

We retain “fiber normalizer” in §3.3 and in the glossary. Before its formal definition, we add:

Fixing the other active coordinates and varying uju_j traces a coordinate fiber.

This identifies the coordinate projection's inverse image and explains the geometry of the integral.

References

[1] J. R. Norris. Probability and Measure. Lecture notes, §2.3.

[2] Peter Bürgisser and Felipe Cucker. On a Problem Posed by Steve Smale. Annals of Mathematics, 174:1785–1836, 2011. §8.3.

[3] Persi Diaconis, Susan Holmes, and Mehrdad Shahshahani. Sampling From a Manifold. arXiv:1206.6913, 2012. Proposition 2.

AuthorsOpenReview ↗

Response to Reviewer a6kj — Weaknesses (2/4):

W2. Clearer phrasing and the example in §3.4

Following the approach in W1, we simplify compressed expressions while preserving the technical terms needed to specify the models and conditions.

1. Finite-sample procedure and abstention

The phrase “a selective finite-sample realization in an affine–probit observation model with quadratic mechanisms” combines four points: the finite-sample procedure, its option to abstain, the observation model, and the structural mechanisms.

We retain “affine–probit observation model” and “quadratic mechanisms”, which specify the setting. We replace the compressed phrase with a direct description of the procedure and its return-or-abstain rule:

We also develop a finite-sample procedure for affine–probit observations with quadratic mechanisms. Using atomic pairs, it returns a graph when simultaneous confidence intervals resolve all decisions, and abstains otherwise.

The probability guarantee and numerical results follow this passage. The meaning of “selective” is conveyed by the return-or-abstain rule, while the formal result name in §4.6 is retained.

2. “coordinatewise-normalized environment ratios”

We expand the compressed modifier into an action while preserving the original contrast between two signals that address two identification challenges.

Original:

Support invariances determine which residual coordinates can change, while coordinatewise-normalized environment ratios determine how those coordinates propagate through the causal system.

Revised:

Support invariances determine which residual coordinates can change, while normalizing environment ratios separately along each coordinate reveals how these changes propagate through the causal system.

3. “positive paired-support densities”

This phrase specifies an identification condition. We preserve its meaning while making the relationships between the modifiers explicit.

Original:

Positive paired-support densities and distinct nonzero signatures ensure that ...

Revised:

Positive densities on the paired supports, together with distinct nonzero target signatures, ensure that ...

“On the paired supports” specifies where the densities are defined, and “target signatures” identifies the object of the second condition. Section 2.2 gives the support Me\mathcal M_e, the corresponding measure, and the condition's role. The introductory sentence now makes these relationships explicit without repeating the formal setup.

4. “Code containment … uniform-distribution rigidity”

The original sentence compressed the intermediate step from signature inclusion to triangular structure. We now explain the sequence: signature alignment, triangularity, and elimination of the remaining mixing through independent uniform outputs. This makes the role of the distributional constraint explicit.

Original:

Code containment and minimum weight then align the target signatures, while uniform-distribution rigidity reduces each remaining scalar map to a reflection.

Revised:

Code containment and minimum weight align their target signatures. Signature inclusion then makes T=h∘q−1T=h\circ q^{-1} triangular after reordering. Requiring independent uniform outputs removes the remaining mixing, leaving coordinate reflections.

This revision appears in the proof sketch in §4.1, between the coordinate-matching argument and the conclusion covering arbitrary rank and nested signatures. The full proof is in Appendix E, which was Appendix C in the original submission.

NOTE: All appendix references in these responses follow the revised numbering.

AuthorsOpenReview ↗

Response to Reviewer a6kj — Weaknesses (3/4):

W2. Worked example (continued)

5. The four-line example in §3.4

Section 3.4 provides a brief overview for the chain 1→2→31\to2\to3 and links to the full calculations in Appendix H. We expand the key comparisons here to show how the example illustrates sink identification and recursive recovery.

Environments A,BA,B target {1,3}\lbrace1,3\rbrace, and environment CC targets {1,2}\lbrace1,2\rbrace. For node 3, AA uses a uniform replacement distribution, and BB uses density 1+δ(2z−1)1+\delta(2z-1), where 0<δ<10<\delta<1. Appendix H gives

C3,A(u)=F3(u),C3,B(u)=NB(F3(u)),C_{3,A}(u)=F_3(u),\quad C_{3,B}(u)=N_B(F_3(u)),

where NB(z)=z+δ(z2−z)N_B(z)=z+\delta(z^2-z) is strictly increasing. The first equation satisfies the identity-CDF condition in an environment that does not target node 3. The other two CDFs share their level sets and their ordering. Node 3 therefore passes both comparisons.

Node 2 is not targeted in AA. If it were a current sink, its conditional CDF would equal u2u_2. However, with the mechanism parameter 0<b<1/20<b<1/2 from Appendix H,

By continuity, this deviation persists on a neighborhood, so node 2 fails the test at this stage. After marginalizing node 3's residual coordinate, node 2's conditional CDFs in A,BA,B become u2u_2. In CC, its CDF is a strictly increasing transformation of its causal coordinate F2F_2. Node 2 then passes the test, yielding the removal order 3,2,13,2,1.

This calculation directly corresponds to the geometric explanation Cj,eR=Nje∘FjRC_{j,e}^R=N_j^e\circ F_j^R in §3.3. The revised proof sketch in §4.1 also explains signature alignment, triangularity, and the independent uniform constraints. These passages clarify the geometric mechanism and the residual-identification argument, respectively.

W3. Paper structure and the numerical summary

Thank you for the time and care devoted to following the paper's theoretical development. We have made the relationships between the construction and its guarantees easier to trace.

The paper proceeds from problem setup (§2) to identification procedure (§3) to theoretical guarantees (§4). The forward references in §3 identify the guarantee for each construction:

  • Theorem 4.1: recovery of residual coordinates and target signatures.
  • Theorem 4.2: sink identification.
  • Theorem 4.3: full causal identification.
  • Corollary 4.4: the optimal environment count, combining identification with the design lower bound in Proposition K.1.

We have added a concise, hyperlinked THEORETICAL ROADMAP near the beginning of the appendix, with an entry point in the main text. It shows how the results and proofs fit together, supporting both sequential reading and direct reference.

For the numerical summary, one evaluation means running the graph-recovery procedure on one generated instance at a specified sample size. There are 160 independent replicate sequences, covering four dimensions, four graph families, two response regimes, and five replicates per combination. Each sequence is evaluated at five sample sizes, giving 160×5=800160\times5=800 evaluations.

As explained in W2, the estimator returns a graph when the decisions are resolved and abstains otherwise. Of the 800 evaluations, 288 returned a graph, all correctly, and 512 resulted in abstention. In increasing order of sample size, the correct-return counts are 0,0,16,112,160, each out of 160 evaluations. Appendix Q.7 gives the definitions and complete setup.

To introduce the evaluation conditions alongside the numerical results, we write:

Across 800 evaluations spanning dimensions, graph families, response regimes, and sample sizes, all 288 returned graphs are correct. The correct-return rate rises from 0% to 100% as sample size increases.

Here, the correct-return rate is the fraction of evaluations at a given sample size that return the correct graph, with abstentions included in the denominator.

Reviewer_nZRUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper considers the problem of nonlinear causal representation learning with unknown intervention targets. Specifically, they consider a more general setting, where each intervention may target multiple hidden variables, and the functional relations among variables are nonlinear. Further, they assume that same-unit pairing is available, which means that each factual sample can be paired with an intervention on the same underlying unit, where the exogenous noises of the non-intervened variables are exactly the same between the pairs. Under this assumption, the authors propose a two-stage algorithm to uniquely recover the underlying causal model. The first step is to identify the intervention targets by comparing the factual-counterfactual pair, while the second step is to recursively recover the sink variable in the underlying model. The authors show that the model can be uniquely identified under the proposed faithfulness assumption. Further, they provide a tight bound on the number of interventions for unique identifiability, as well as finite-sample analysis for reliable graph recovery.

Critical strengths
  1. The presentation of the method and the results is clear and easy to follow. All the theoretical results are well-explained through texts and examples.
  2. This paper provides a complete set of theoretical results for the considered setting, including identifiability conditions, the minimal number of interventions needed, and a finite-sample guarantee.
Critical weaknesses

One major weakness of the work is the restrictiveness of the assumption. Specifically, the considered setting assumes that we have observations of the same-unit pairing, where the exogenous noises (the random seed) of the non-intervened variables are exactly the same in the factual and counterfactual environments. This is a very strong assumption and appears to be unrealistic. I would suggest that the authors provide some real-life scenarios where this assumption can be satisfied and evaluate the performance of the proposed method in these cases.

Besides, the paper lacks a comparison with existing methods, especially in the simulation results. I believe existing methods developed for unpaired samples are also applicable in the paired-sample setting, such as Varıcı et al. (2024).

Questions for authors
  1. Is uniform exogenous noise UU a necessary assumption? My understanding is that this is a without-loss-of-generality assumption; even if a noise UiU_i has a non-uniform distribution, we can always transform it into a uniform distribution using inverse CDF, and put it in fif_i. Am I right?
  2. Does Theorem 4.1 require any additional assumption on the intervention to prevent any zero-measure parameter cancellation or proportionality?
  3. Does Theorem 4.2 require that the target signatures are distinct and nonzero?
3 comments
AuthorsOpenReview ↗

Response to reviewer nZRU — Questions (1/3):

Comment

Thank you for the time and care you have devoted to reviewing our paper!

We appreciate your questions about the assumptions and proofs behind our theoretical results. Below, we clarify the role of noise normalization and the conditions used in Theorems 4.1 and 4.3, adding intuition and a simple example alongside the derivations. We hope these explanations make the reasoning easier to follow, and we welcome further discussion.

NOTE: All section, theorem, lemma, and equation numbers cited below refer to the latest revised version of our paper.

Q1. Uniform exogenous noise

Yes, your understanding is correct: within the smooth model class of Section 2.1, uniform exogenous noise can be obtained by reparameterizing the noise variables.

Let U~i\widetilde U_i denote the original independent exogenous noises with CDFs FU~iF_{\widetilde U_i} and mechanisms f~i\widetilde f_i. When each U~i\widetilde U_i takes values in an open interval and has a strictly positive C1C^1 probability density there, both FU~iF_{\widetilde U_i} and its inverse FU~i−1F_{\widetilde U_i}^{-1} are strictly increasing C2C^2 diffeomorphisms. Assume the original mechanisms satisfy the corresponding C2C^2 regularity conditions of Section 2.1, with f~i(zpa⁡(i),⋅)\widetilde f_i(z_{\operatorname{pa}(i)},\cdot) a strictly increasing diffeomorphism onto the same fixed open interval IiI_i for every parent value. Define

f_i(z_{\operatorname{pa}(i)},u_i) =\widetilde f_i\bigl(z_{\operatorname{pa}(i)},F_{\widetilde U_i}^{-1}(u_i)\bigr).$$ Intuitively, the CDF expresses a noise realization by its percentile in the original distribution. Applying this transformation componentwise preserves independence and gives $\mathbf U\sim\operatorname{Unif}((0,1)^d)$. The inverse CDF converts that percentile back to the original noise value before the original mechanism is evaluated. The composite mechanism $f_i$ therefore retains its $C^2$ regularity, monotonicity in noise, and fixed conditional support. This transformation produces the same factual and interventional endogenous variables. Untargeted nodes still reuse the same noise realization under same-unit pairing, so all observed paired joint laws of $(\mathbf X,\mathbf X^{(e)})$ remain unchanged.
AuthorsOpenReview ↗

Response to reviewer nZRU — Questions (2/3):

Q2. Additional intervention assumptions for Theorem 4.1

No. Theorem 4.1 holds for every model satisfying the stated structural, paired-support, and target-signature conditions. It requires no additional assumption ruling out parameter cancellation or proportionality.

The proof relies directly on the geometric constraints imposed on the paired support Me\mathcal M_e by same-unit pairing. Fix an environment ee, let U=q(X)\mathbf U=q(\mathbf X) and V=q(X(e))\mathbf V=q(\mathbf X^{(e)}), and write T=h∘q−1T=h\circ q^{-1} for a feasible chart and code (h,Λ)(h,\boldsymbol{\Lambda}). When λej=0\lambda_{ej}=0, the pair-invariance constraint in Eq. (8) reads

E[(Tj(U)−Tj(V))2]=0.\mathbb E\bigl[(T_j(\mathbf U)-T_j(\mathbf V))^2\bigr]=0.

Continuity of the nonnegative integrand and positive density on Me\mathcal M_e imply Tj(u)=Tj(v)T_j(u)=T_j(v) throughout Me\mathcal M_e. The key is the form of the paired support:

Me=(u,v)∈(0,1)2d:uSec=vSec.\mathcal M_e=\\{(u,v)\in(0,1)^{2d}:u_{S_e^c}=v_{S_e^c}\\}.

Only untargeted residual coordinates must agree within a pair. With vv and all other uu coordinates fixed, each targeted coordinate uiu_i (i∈Sei\in S_e) can still vary over (0,1)(0,1), while Tj(u)T_j(u) must remain equal to the fixed value Tj(v)T_j(v). Thus TjT_j cannot depend on that targeted coordinate, giving (Lemma E.1) ∂uiTj≡0(i∈Se).\partial_{u_i}T_j\equiv0\qquad(i\in S_e).

Contrapositively, whenever TjT_j depends on uiu_i (∂uiTj≢0\partial_{u_i}T_j\not\equiv0), every environment targeting node ii must have λej=1\lambda_{ej}=1, so the support of the jj-th candidate code column contains supp⁡(σi)\operatorname{supp}(\boldsymbol{\sigma}_i). Invertibility of DT\mathrm{D}T then guarantees at least one nonzero permutation monomial in its determinant at any fixed point, matching each candidate coordinate jj one-to-one with a true coordinate ii on which TjT_j depends and yielding a minimum-weight lower bound on the code (Lemma E.2).

The true pair (q,Σ)(q,\boldsymbol{\Sigma}) is feasible and attains this lower bound. An optimal candidate code therefore cannot have any extra ones in its matched columns: every support inclusion must be an equality, aligning the candidate code with the true target signatures. Since the signatures are distinct, off-diagonal dependence can only follow strict inclusion of their supports. Ordering coordinates by this partial order makes TT triangular (Lemma E.3). The uniformity constraint also requires TT to preserve the product of independent uniform distributions. This restricts the triangular C1C^1 diffeomorphism to coordinatewise identity or reflection, up to the aligned permutation (Lemma E.6).

A simple example illustrates the information in pairing. For a uniform root Zi=UiZ_i=U_i, leaving the node untargeted gives residual pairs (Ui,Vi)(U_i,V_i) on the diagonal Vi=UiV_i=U_i. Independently redrawing Zi(e)=ViZ_i^{(e)}=V_i from the same uniform law (i.e. uniform distribution) leaves both marginals uniform, but changes the paired support to the entire unit square (0,1)2(0,1)^2, with E[(Ui−Vi)2]=\mathbb E[(U_i-V_i)^2]= E[Ui2]+E[Vi2]−2E[UiVi]=\mathbb E[U_i^2]+\mathbb E[V_i^2]-2\mathbb E[U_iV_i]= 1/3+1/3−2/4=1/6>01/3+1/3-2/4 = 1/6>0 since $ U_i,V_i\overset{\mathrm{i.i.d.}}{\sim}\operatorname{Unif}(0,1)$. The intervention is therefore visible through the pairing even without a marginal distribution shift. Different environments may likewise use identical replacement distributions. Structural path cancellation is addressed by Terminal-response faithfulness (TRF) in Theorem 4.3 (TRF is not used in Theorem 4.1).

AuthorsOpenReview ↗

Response to reviewer nZRU — Questions (3/3):

Q3. Distinct, nonzero target signatures in Theorem 4.3

Yes. Section 2.2 assumes that the target-signature columns σi\boldsymbol{\sigma}_i are distinct and nonzero, and Theorem 4.3 inherits both conditions together with TRF. For clarity, we express the argument in the true residual coordinates. Theorem 4.1 shows that the recovered coordinates differ only by a permutation and coordinate reflections, which preserve both sink tests (Lemma F.1).

To see what the two tests require, consider a true current sink jj. The response factors of other active targets do not depend on uju_j, since jj has no descendants in G[R]\mathcal G[R]. These factors cancel in conditional normalization. An untargeted sink then has the uniform conditional density, giving Cj,eR(u)=ujC_{j,e}^R(u)=u_j. For a targeted sink, the remaining factor gives Cj,eR=Nje(FjR)C_{j,e}^R=N_{j}^{e}(F_j^R), where NjeN_{j}^{e} is the replacement CDF for node jj. Thus, each targeting environment expresses the same structural coordinate FjRF_j^R through its own strictly increasing replacement CDF. The CDF values may differ, but their level sets and the ordering of those sets agree. This is the structure compared by the common oriented foliation test.

Together with TRF, the two signature conditions allow these tests to distinguish non-sinks from sinks. The argument proceeds as follows (Appendix I). Fix an ancestor-closed current active set RR. For any non-sink jj of G[R]\mathcal G[R], TRF supplies a current sink descendant ℓ\ell with ∂ujFℓR≢0\partial_{u_j}F_\ell^R\not\equiv0. Using the response factors ρie\rho_{i}^{e} from Section 4.2, the environment density ratio on the active set satisfies

\partial_{u_j}\log r_e^R=\sum_{i\in S_e\cap R}\alpha_{ji}^{e},$$ where $\alpha_{ji}^{e}=\partial_{u_j}\log\rho_{i}^{e}$. By TRF and **Lemma G.5**, whenever $e$ targets $\ell$, the term $\alpha_{j\ell}^{e}$ is not identically zero and varies nontrivially with $u_\ell$. Comparing the supports of $\boldsymbol{\sigma}_{\ell}$ and $\boldsymbol{\sigma}_j$ leads to two cases, where conditional CDFs $C_{j,e}^R$ are compared in the almost-everywhere equivalence classes of **Section 4.2.1**: 1. **Case 1: Some environment targets $\ell$ without targeting $j$.** Choose such an environment $e$. Then $\alpha_{j\ell}^{e}\not\equiv0$, and linear independence of the nonzero response terms over constant coefficients within a single environment (**Lemma G.6**) implies $\partial_{u_j}\log r_e^R\not\equiv0$. If node $j$ passed the untargeted identity test in **Eq. (15)** ($C_{j,e}^R(u)=u_j$ a.e.), **Lemma G.2** would force $\partial_{u_j}\log r_e^R\equiv0$, a contradiction. 2. **Case 2: Every environment targeting $\ell$ also targets $j$.** Distinctness of the target signatures makes this support inclusion strict, $\operatorname{supp}(\boldsymbol{\sigma}_{\ell})\subsetneq\operatorname{supp}(\boldsymbol{\sigma}_j)$. Nonzeroness of $\boldsymbol{\sigma}_{\ell}$ then provides an environment $e$ that targets both $j$ and $\ell$, while strict inclusion provides a second environment $e'$ that targets $j$ but not $\ell$. In this case, targeting $\ell$ never leaves $j$ untargeted, so we use the cross-environment comparison in **Eq. (16)**: both environments target $j$, but only one targets $\ell$. Under $e'$, node $\ell$ is an untargeted current sink, so $r_{e'}^R$ contains no $\rho_{\ell}^{e'}$ factor and its remaining factors do not depend on $u_\ell$; hence $r_{e'}^R$ is independent of $u_\ell$. If node $j$ passed the targeted foliation test in **Eq. (16)**, meaning $[C_{j,e}^R]\sim_{\mathrm{fol}}[C_{j,e'}^R]$ (common oriented foliation), **Lemma G.3** would require $\partial_{u_j}\log r_e^R$ to be independent of $u_\ell$ as well. But in the sum for environment $e$, $\alpha_{j\ell}^{e}$ varies with $u_\ell$, while sinkhood makes every other term independent of $u_\ell$. The sum therefore cannot be independent of $u_\ell$, again a contradiction. Every current non-sink $j$ therefore fails at least one of the two sink tests in **Eqs. (15)–(16)**. Once a current sink is identified, nonzeroness of its target signature also ensures that **Algorithm 1** has a targeting environment to recover its endogenous coordinate. This argument allows nested target signatures and does not require the signature matrix $\boldsymbol{\Sigma}$ to have full column rank. The experiments addressing the concerns raised in your Weaknesses section are underway, and we are compiling the results; we will post them here as soon as they are available. Thank you again for your thoughtful review! :)
Reviewer_1XKAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies nonlinear causal representation learning problem under “paired” interventions. The focus is identifying the latent DAG and the latent variables using a logarithmic number of multi-node “factual-counterfactual” pairs of environments.

Critical strengths

The studied problem is ambitious and important open problem for identifiable representation learning literature for a while. For instance, even for the simpler problem of linear CRL from logarithmic number of environments is shown to be sufficient only for limited settings (e.g., do interventions, linear causal models [Lee et al. 2026]). Thus, even if the setting of the paper is quite different from the recent interventional CRL literature, in principle it could have been an useful result.

Critical weaknesses

I have substantial experience in this field, yet I found the paper extremely difficult to follow. Starting from the very first paragraph, the paper introduces undefined or vaguely defined terms, and this trend continues throughout the paper. There is little to no explanation of how the reader should interpret the assumptions and compare to those made in related work. Even the “intervention” setting is poorly described, e.g., there is almost no justification of the considered “factual-counterfactual” pairs. To be clear, similar settings have been studied before, e.g. by Brehmer et al. (2022) as cited in the paper. However, this setting is very different from the subsequent “unpaired” interventional CRL literature (which the paper also cites). The distinction is underdiscussed.

Similar concerns arise at almost every step: why uniform noise distributions, what should we make of the TRF assumption, what criticial assumptions and/or tools make it possible to go from linear to sublinear environments, etc.

Overall, I am not able to evaluate the paper's claims correctness or importance. Even if the results are fully correct, which may well be the case, the presentation makes it impossible to assess their significance, relevance, real-world practice implications, and contributions.

Questions for authors

see the weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1XKA — Weaknesses and Questions (1/3)

Comment

Thank you for the time and care you have devoted to reviewing our paper, and for recognizing the importance of the problem. Our contribution is to identify latent causal variables and their full DAG under nonlinear mixing and nonlinear mechanisms.

We achieve this from interventions on multiple unknown targets, using the optimal number of environments. We explain below how the paired observations, assumptions, and two identification steps lead to this result, and we summarize the new experimental evidence and manuscript revisions.

NOTE: All section and theorem numbers refer to the revised manuscript.

1. What the pairs provide, and how this relates to prior work

The learner observes same-unit factual–interventional pairs and environment labels; the causal variables, exogenous inputs, and intervention targets are unknown. Target variables are independently redrawn, while non-target variables retain their mechanisms and exogenous inputs. Non-target values can still change through their parents; the shared quantity is the exogenous input. These are stochastic perfect interventions, with replacement distributions that may vary across targets and environments (§2.2, Figure 1). The observation map is smooth, invertible and unknown. Each mechanism is smooth and strictly increasing in its noise, with the same output range for all parent values (§2.1).

Pairs reveal interventions that marginals can miss. For a uniform root, leaving it unchanged produces pairs on the diagonal. Independently redrawing it from the same distribution produces pairs throughout the unit square, although both marginals remain uniform.

Brehmer et al. [1] study the same shared-noise principle with unknown single-target interventions; our environments can target several unknown variables simultaneously. Unpaired CRL instead observes each environment without the correspondence between measurements. Controlled simulators can generate such pairs by retaining non-target exogenous inputs and independently replacing target values. Brehmer et al.'s simulated robotic experiments provide a precedent [1, §5.3].

2. Why uniform noise?

Uniform noise expresses an exogenous input by its percentile, using its CDF. Within §2.1's smooth model class with fixed conditional supports, independent noises with positive C1C^1 densities on open intervals can be transformed as Ui=FU~i(U~i)U_i=F_{\widetilde U_i}(\widetilde U_i). Absorbing the inverse CDF into each mechanism preserves independence, smoothness, and all factual and interventional observations, including their pairing. This supplies a common coordinate system for identification; the full construction is given in Appendix E (Noise normalization); see also our response to nZRU, Q1.

3. What does TRF require?

A sink has no remaining children. Terminal-response faithfulness (TRF) captures the following intuition:

At every recovery step, each non-sink must influence at least one current sink descendant, with a total response that is not identically zero.

The corresponding derivative with respect to the non-sink's exogenous input must be nonzero somewhere. This allows the procedure to distinguish non-sinks from sinks under simultaneous interventions.

Proposition L.1 establishes TRF for mechanisms with nonzero edge derivatives whose sign products agree along all paths between the same nodes. It also proves probability-one validity in an explicit nonlinear analytic family under absolutely continuous sampling of the edge weights. With one single-target environment per variable, Proposition 3.3 establishes identification directly without TRF.

AuthorsOpenReview ↗

Response to Reviewer 1XKA — Weaknesses and Questions (2/3)

4. What makes logarithmically many environments sufficient?

Each variable has a binary pattern recording which environments target it. With mm environments, there are 2m−12^m-1 nonzero patterns. Two steps make this capacity usable through an unknown nonlinear observation map.

  1. First, recover exogenous coordinates and hidden target patterns (Theorem 4.1). Under the paired-support condition, untargeted exogenous coordinates agree within pairs, while targeted coordinates vary over their full support. The method searches for coordinates and patterns satisfying these equalities, with independent uniform factual coordinates. The minimum-code criterion minimizes the total number of target assignments, equivalently the number of ones in the target matrix. This recovers the exogenous coordinates up to permutation and reflection, together with their patterns. The patterns need only be distinct and nonzero; full column rank is unnecessary, allowing fewer environments than variables.

  2. Second, recover causal order and variables (Theorems 3.1 & 4.3). CFP constructs conditional CDFs from the environment distributions in the recovered coordinates. For a current sink, untargeted environments give the identity CDF. Targeting environments express the same causal variable through different strictly increasing replacement CDFs. Their numerical values can differ, but they group states into the same level sets and order those sets identically. This agreement is the common oriented foliation. Under TRF, these two comparisons identify sinks recursively, recovering the causal variables up to permutation and componentwise strictly monotone transformations, together with their full DAG.

The 2nd comparison also handles nested patterns: in a two-variable chain, the parent can be targeted in environments 1 and 2, and the child only in environment 1. Neither environment targets the child while leaving the parent untargeted. Comparing the parent's ordered CDF level sets across the two environments detects the additional child intervention. Together, the two steps attain

m⋆(d)=⌈log⁡2(d+1)⌉,d≥2.m^\star(d)=\lceil\log_2(d+1)\rceil,\qquad d\geq2.

Corollary 3.2 proves a matching lower bound through indistinguishable models when fewer environments force a zero or repeated pattern. This is therefore the exact minimum needed to identify every model in the stated class.

Lee et al. [2] obtain logarithmic complexity with unpaired data and linear mechanisms and mixing. Their design separates every node pair in both directions: each node is targeted somewhere without the other. It also uses two noise scales with distinct coordinate variance ratios. Our paired result handles nonlinear mechanisms and mixing and permits nested patterns. Pair invariance and CFP explain how this combination becomes identifiable.

AuthorsOpenReview ↗

Response to Reviewer 1XKA — Weaknesses and Questions (3/3)

5. Evidence for the contribution and clearer presentation

Our posted Experiments I response directly support the environment-efficiency result and the role of foliation, which provide two complementary results:

  • Environment efficiency: On eight-variable models with nonlinear mechanisms and mixing, CFP exceeds GSCALE-I [3] by 0.237–0.364 graph F1. It uses 75% fewer environments (4 vs. 16) at the same total observation budget. On the same sixteen-environment observations, CFP uses the correspondence within each pair and improves F1 by 0.207–0.419 over GSCALE-I.
  • Foliation's role: On seven-variable models with analytic mechanisms, nested patterns, and three environments, adding the foliation criterion raises graph F1 from 0.027 to 0.929, with the same learned exogenous representation.

Results in our Experiments I response are averaged over 30 independent models per generating condition. We have reorganized the manuscript to state full identification and the exact environment count first (§3, Theorem 3.1 and Corollary 3.2), followed by their two-step construction (§4). The main text explains noise normalization (§2.1), how distinct target patterns suffice (§4.1), and the conditional-CDF interpretation and sink tests (§4.2.1–4.2.2), with an explicit three-node example in §4.3. The ROADMAP, GLOSSARY, and comparison of intervention settings remain available in Appendices B–D (Table 2).

By the way, further Experiments II on the real-world applicability of our approach are underway, and we will post the results here as soon as they are available.

Thank you again for the thoughtful feedback! We hope these explanations and results make the contribution easier to assess, and we welcome further discussion :)

References

[1] Johann Brehmer, Pim de Haan, Phillip Lippe, and Taco S. Cohen. Weakly supervised causal representation learning. NeurIPS, 2022.

[2] Inbeom Lee, Tongtong Jin, and Bryon Aragam. Beyond Identifiability: Learning Causal Representations with Few Environments and Finite Samples. 2026.

[3] Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer. General Identifiability and Achievability for Causal Representation Learning. AISTATS, 2024.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Response to reviewers nZRU, a6kj and 1XKA — Experiments I

Further Experimental Evidence I: Comparison with Unpaired Methods

Thank you for the time and care you have devoted to reviewing our paper! We have added additional experiments compare CFP with existing unpaired methods, evaluate recovery with fewer intervention environments, and clarify the role of foliation in recovering the graph.

AuthorsOpenReview ↗

Response to reviewers nZRU, a6kj and 1XKA — Experiments I (1/2)

1. CFP improves graph recovery on the same observed data.

We compare conditional-foliation peeling (CFP) with GSCALE-I [1] and UMNI-CRL [2] on eight-variable structural causal models. In the comparisons using a common intervention design, all methods receive the same observations and environment labels. The baselines use the distributions within each environment; CFP additionally uses the correspondence within each factual–interventional pair, which links the two observations through their shared non-target exogenous inputs. Intervention targets are unknown to all estimators.

For nonlinear mechanisms and nonlinear mixing, CFP improves graph F1 by 0.207–0.419 over GSCALE-I on the same 16-environment data. The last column also reports CFP with a different, four-environment design at the same total observation budget.

MechanismObservations BBGSCALE-I (16 env.)CFP (16 env.)CFP (4 env.)
Analytic589,8240.5400.8470.904
Analytic2,359,2960.6930.9000.961
Quadratic589,8240.5140.9210.763
Quadratic2,359,2960.5330.9510.770

NOTE: BB counts factual and interventional observation vectors, including both members of each pair. Entries are mean graph F1; parentheses indicate the number of intervention environments.

The linear comparison gives a consistent result. With linear Gaussian mechanisms and linear mixing, CFP and UMNI-CRL receive the same observations from eight multi-target environments. CFP improves graph F1 by 0.243–0.450, across both graph densities and both observation budgets.

Graph edge probability ppObservations BBUMNI-CRLCFP
0.25589,8240.3480.798
0.252,359,2960.5620.844
0.50589,8240.4220.733
0.502,359,2960.5220.765

Shared exogenous inputs allow CFP to distinguish direct targets from downstream changes, even when non-target variables change. Same-unit pairing also appears in Brehmer et al. [4]. Here, the multi-target experiments examine recovery with known environment labels and hidden, overlapping target sets.

2. Fewer intervention environments, with the same total data.

The four-environment CFP results in the nonlinear table remain above the 16-environment GSCALE-I results in every row, with F1 gains of 0.237–0.364. Thus, CFP achieves higher graph F1 with 75% fewer intervention environments at the same total observation budget in these experiments.

The design in §3.1 (Corollary 3.2) targets several variables at once, giving each variable a distinct, nonzero pattern across environments. Eberhardt et al. [3] showed how such interventions reduce the number of experiments needed for causal discovery with observed variables. Here, same-unit pairing makes the intervention patterns recoverable when the causal variables are latent and mixed nonlinearly.

Theorem 4.1 establishes this recovery without requiring the binary target signatures to be linearly independent. With mm environments, there are 2m−12^m-1 distinct nonzero signatures. Under the paper's identification conditions, m=⌈log⁡2(d+1)⌉m=\lceil\log_2(d+1)\rceil environments therefore suffice for d≥2d\ge2 variables, giving four environments for the eight-variable models evaluated above.

AuthorsOpenReview ↗

Response to reviewers nZRU, a6kj and 1XKA — Experiments I (2/2)

3. Why foliation matters for graph recovery.

On seven-variable models with analytic mechanisms and nested intervention targets, adding the foliation criterion raises graph F1 from 0.027 to 0.929. Both versions use the same learned residual representation, which represents the exogenous inputs. The ablation keeps the checks in environments that do not target the candidate variable and removes the foliation criterion.

MethodGraph F1 ↑Mean absolute Spearman correlation ↑
Without the foliation criterion0.0270.908
CFP0.9290.975

Three multi-target environments; 262,144 pairs per environment and 262,144 additional factual observations. Spearman correlation measures recovery of the endogenous variables.

The ablation illustrates that high correlation with the latent variables does not ensure accurate graph recovery. Consider a parent targeted in environments 1 and 2, with its child targeted only in environment 1. No environment perturbs the child while leaving the parent untouched.

To resolve this ambiguity, CFP compares conditional CDFs across environments that target the candidate variable. For a sink, which has no remaining children, these CDFs group the same states together and order those groups identically, even when their numerical values differ. CFP uses this agreement of ordered level sets to identify sinks and recover causal order, as described in §4.2–4.4.

At each removal step, TRF requires every non-sink to have at least one remaining sink whose total response to that non-sink's exogenous input is not identically zero. Together with the other stated assumptions, this ensures that non-sinks fail at least one of the two checks (Theorem 4.2).

NOTE: Results are averages over 30 independent models per generating condition, with the same models used at both baseline budgets. A bootstrap over models gives 95% confidence intervals above zero for the F1 gains reported here. UMNI-CRL runs with no estimate contribute F1 = 0; CFP's advantage also remains when both methods are compared on models for which UMNI-CRL returns an estimate.

Thank you again for the thoughtful feedback. We hope these comparisons and the accompanying explanation make the contribution of CFP easier to assess, and we welcome further discussion.

References

[1] Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer. General Identifiability and Achievability for Causal Representation Learning. AISTATS, PMLR 238:2314–2322, 2024.

[2] Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer. Linear Causal Representation Learning from Unknown Multi-node Interventions. NeurIPS, 37, 2024.

[3] Frederick Eberhardt, Clark Glymour, and Richard Scheines. On the Number of Experiments Sufficient and in the Worst Case Necessary to Identify All Causal Relations Among N Variables. UAI, pp. 178–184, 2005.

[4] Johann Brehmer, Pim de Haan, Phillip Lippe, and Taco Cohen. Weakly supervised causal representation learning. NeurIPS, 35:38319–38331, 2022.

AuthorsOpenReview ↗

Response to reviewers nZRU, a6kj and 1XKA — Experiments II

Further Experimental Evidence II: Practical Feasibility of Same-Unit Pairing

Thank you for asking how same-unit pairs are obtained in practice, where this assumption holds or is limited, and how CFP performs on realistic data. To address this directly, we evaluated the complete CFP pipeline on rendered images from CausalCircuit, an established 3D robotic simulation benchmark [1, 2].


View the visual evidence and ablations: https://anonymous.4open.science/r/cfp_supplementary-6E43/


1. The pairing assumption corresponds to a concrete acquisition procedure.

Rather than requiring an analytical joint density as input, CFP operates directly on paired observations and anonymous environment identifiers; the joint distribution in our theory simply describes these paired observations at the population level. In CausalCircuit—which simulates a robot arm interacting with red, green, and blue lights—the simulator records a factual image, intervenes on a selected variable, and recomputes the scene while reusing the exogenous inputs (background randomness) of the remaining mechanisms.

Crucially, exogenous inputs are shared, while downstream causal variables remain free to change. Intervening on the green light, for example, can still alter the red light through their causal link even though the red light retains its original exogenous input.

2. CFP recovers causal structure from images.

Using the official CausalCircuit splits (100,000 training, 10,000 validation, and 10,000 test image pairs), we first train a four-dimensional autoencoder purely via image reconstruction. We then freeze the encoder and apply the full CFP procedure using only the image pairings and anonymous environment identifiers—leaving target identities, true causal variables, noise values, and the ground-truth graph strictly for evaluation.

The table below reports all three encoder seeds under balanced environment sampling (full comparisons against shuffled pairings and initial optimization schedules are provided at the anonymous link above):

Encoder seedDirected-edge F1SHDTarget-signature accuracyCausal-variable correlation
10.800293.75%0.611
21.0000100.00%0.780
30.000681.25%0.647
Mean0.6002.6791.67%0.679

Causal-variable correlation improves over the frozen encoder in every seed, from a mean of 0.450 to 0.679. Seed 2 recovers all five directed edges with no extra edges and identifies every target signature correctly. The table also shows the variation in graph recovery across seeds.

NOTE: Correlations are mean absolute Spearman correlations on 10,000 test units (with coordinate matching determined on a separate validation subset). Target-signature accuracy measures the proportion of correctly recovered target indicators across environments, and SHD denotes structural Hamming distance.

3. When is this information available, and what does it add?

Controlled simulators that preserve and replay exogenous inputs provide a direct setting for this acquisition procedure. In physical systems, repeated measurements of the same object alone do not guarantee shared inputs: additional disturbances and measurement error need to be accounted for. Appendix P provides a separate selective estimator for known error budgets. It returns a graph when the compatible models agree and abstains otherwise, with guarantees in Theorems P.2 and P.4 and error budgets in Proposition P.5.

Same-unit pairing is also an established regime in CRL. Brehmer et al.’s identification result [1] relies on an unlabeled mixture covering all atomic intervention targets, whereas our framework leverages anonymous environment identifiers while keeping each environment’s target set hidden and allowing multi-node interventions (while CausalCircuit uses atomic interventions, our logarithmical multi-target scaling is established theoretically and validated in our dedicated multi-target experiments). Ultimately, unit-level pairing captures cross-environment correspondence that separate environment marginals miss—a practical advantage we evaluate directly against marginal-based methods in Experiments I.

Together, the acquisition example and image results show where the required information can be obtained and what CFP can recover from it. Thank you again for encouraging us to make this practical connection explicit!

References

[1] Brehmer et al. Weakly supervised causal representation learning. NeurIPS, 2022.

[2] Qualcomm AI Research. CausalCircuit dataset: acquisition protocol and data format.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.