acceptodds
Under review as a conference paper at ICLR 2027

Singularity Vanishing Theorem for Neurovarieties and Causality

Abstract

We analyze the identifiability problem in general machine learning and establish the singularity-vanishing theorem. We first extend the classic Alexander theorem to a foundational double-point theorem for generic polynomial networks by characterizing the singular locus of their associated ideal sheaves. Building upon these algebro-geometric insights, we revisit and reformulate causal representation learning (CRL) to provide the first formal proof of identifiability under strictly non-injective transformations. We introduce algebraic extension modules paired with models or a hypothetical functional space, and show that valid interventions yield a sufficient number of independent conditions to obstruct the double-point scheme, thereby establishing a general singularity vanishing theorem and resolving a long-standing conjecture in identifiable representation learning. Furthermore, by anchoring representation learning in rigid algebraic geometry, our work implicitly echoes the recent call by Fluri et al. for the 'Hilbert Problems of AI' from geometry. We envision that our results provide a novel, grounded theoretical perspective on causality and world models—the ultimate objectives of sciences such as biology and physics—while potentially opening new avenues for theoretical guarantees across broader machine learning domains.

Primary area:
learning theory
Keywords:
algebraic geometry, causality, neuromanifolds, identifiability theory, theory of AI for Science

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bdFqOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies identifiability i.e to which degree a model's outputs determine its parameters up to obvious symmetries. It treats two models:

  • Deep polynomial networks with monomial activations x ↦ xʳ and no biases.
  • Linear causal disentanglement, X=FZX = FZ with Z=(I−Λ)−1ϵZ = (I − \Lambda)^{-1}\epsilon, where F need not be injective. The d-th cumulant of XX is ∑iκi ai⊗d\sum_i \kappa_i\, a_i^{\otimes d}, with aia_i the columns of A=F(I−Λ)−1A = F(I − \Lambda)^{-1}.

The classical tool for such decompositions is the Alexander–Hirschowitz theorem: general double points impose independent conditions on degree-d forms, H1(IX2(d))=0H^1(I^2_X(d)) = 0, outside a short list of exceptions. The paper calls this a vanishing theorem and makes three groups of claims.

  1. Section 3: a layer-wise "vanishing singularity theorem" that gives global identifiability of deep polynomial networks (Theorem 3.1, Proposition 3.2, Theorem 3.3).
  2. Section 4.1: a transfer to causal models through "module extensions" by cumulants, and identifiability from many interventions (Definition 4.1, Corollary 4.2, Proposition 4.3, Theorem 4.4, Proposition 4.5).
  3. Section 4.2 and Appendix E: a linear criterion for identifying the baseline pair (F, Λ) under multi-target interventions (Theorem E.2, Corollaries 4.6, 4.8, E.3).

The abstract presents these as the first formal proof of identifiability under strictly non-injective transformations, and as resolving a long-standing conjecture. The paper is purely theoretical and contains no experiments or applications.

The paper does not deliver on these promises. In my opinion, its singular novel contribution is Theorem E.2 and Corollary E.3 (see strenghs below). These are elementary linear algebra calculation on top of established techniques. It does not support the strong claims about identifiability that the paper makes. All other other claims in this paper are standard background that can be tracked back to published papers or have inconsistent and incomplete proofs (see weaknesses below).

Critical strengths

Theorem E.2 and Corollary E.3 extend the linear-system analysis of Carreno et al. (Section 3.3) from single-node to multi-target interventions, for soft and hard interventions respectively. The new step is eliminating the unknown post-intervention weights by a projection, which turns identifiability of the baseline parameters into one rank condition per latent node. The result assumes the mixing matrices are already aligned across contexts and that the graph and intervention targets are known. The proofs are elementary and I believe correct.

Critical weaknesses

All other claims in this purely theoretical paper are either standard results (e.g. Alexander–Hirschowitz, Lemmas D.1–D.2) or lack a valid proof. Several are not well-formed. For example, Appendix D.1 is titled "Proof of Theorem 3.3". It contains Lemma D.1, Lemma D.2 and Corollary D.3, and then stops. Theorem 3.3 does not follow from Corollary D.3. Passing from that diagram of sheaves to the diagram of H0H^0 and H1H^1 groups in Theorem 3.3 would need the vertical maps to be morphisms of sheaves of modules, which they are not because raising a section to a power is not additive. Even granting it, a commutative diagram of evaluation maps says nothing about the fibres of the map from weights to outputs. That step needs a dedicated argument. I believe Usevich et al. (Theorem 11) proves its finite version only under an extra hypothesis, and the uniqueness version is false in general.

There are many instances of this kind (e.g. Proposition 4.3, Theorem 4.4, Proposition 4.5), and I do not think they can be fixed by revision, because the irreducible mathematical content is not present. The, as far as I can tell, only novel and correct result, Theorem E.2, is linear algebra and does not use the sheaf-theoretic framework of Sections 3 and 4.1. Studying polynomial networks as toy models is interesting, but their benefit is theoretical tractability, which has to be exploited through rigorous and careful mathematics. The paper does not provide that.

The "long-standing conjecture" the abstract claims to resolve could not be identified by me. The conjecture quoted in Appendix A (Usevich et al., Conjecture 40) dates from October 2025 and concerns uniqueness, which the paper's finite notion of identifiability does not address.

Questions for authors

I want to apologize for this harsh review. I don't intent to be mean but believe that rigorous and objective evaluation is necessary in particular for theoretical contributions.

If you substantially disagree with my assessments, I encourage you to challenge them during the rebuttal phase. To facilitate a focussed discussion for example point out if and how my assessment of the claimed proof of Theorem 3.3 is wrong.

Reviewer_H7jdOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes an algebraic-geometric framework for studying the identifiability of polynomial neural networks and causal representations. The authors use double-point schemes and cohomology vanishing to characterize identifiability, and extend the framework to linear causal models with non-injective transformations and multiple interventions. The paper presents several theoretical results on the identifiability of polynomial networks and causal parameters. In particular, it derives a linear rank condition for uniquely recovering baseline causal parameters under multi-target interventions.

Critical strengths
  • The paper connects neural network identifiability with causal representation learning.
  • The proposed geometric perspective is interesting and offers a different way to study identifiability beyond tensor-based approaches.
  • The linear rank criterion for multi-target interventions provides a concrete characterization of when baseline causal parameters can be identified.
Critical weaknesses

My background is not in theoretical machine learning, and I am not familiar enough with the algebraic-geometric tools used in this paper to fully verify the proofs. My evaluation therefore focuses more on whether the presented results support the paper's claims.

  • The abstract and conclusion suggest general identifiability guarantees for neural networks and causal representation learning, but the actual results focus on polynomial networks and specific linear causal models. The causal results establish identifiability of the baseline parameters rather than the full intervention-specific model.
  • The contribution of the proposed framework is not entirely clear. In particular, the final causal identifiability criterion in Theorem E.2 is derived using linear algebra, and it is difficult to see which parts of this result rely on the earlier geometric analysis. It would be nice to provide a clearer explanation of what additional insights or guarantees the proposed framework provides compared with a more direct analysis.
  • The paper cites numerous works investigating the identifiability of polynomial neural networks (e.g., Usevich et al., 2025; Massarenti and Mella, 2025) and non-injective linear causal models (e.g., Carreno et al., 2024). However, the paper does not clearly compare their assumptions and theoretical guarantees with the proposed results. Given the strong novelty claims, a more explicit discussion of what remains unresolved in prior work would help establish the significance of the contribution.
Questions for authors
  • Could the authors provide a small numerical example illustrating how different intervention target sets affect the rank conditions in Theorem E.2?
Reviewer_F22tOpenReview ↗
Rating
3
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies identifiability in polynomial neural networks and causal representation learning. The big idea is to connect identifiability to a geometric problem: use double points and sheaf cohomology to understand when a model has enough independent constraints to recover its parameters or latent representation.

The paper first studies polynomial neural networks using neurovarieties and the Alexander-Hirschowitz theorem. It then tries to extend this view to causal models, including non-injective mixing functions and multiple interventions. One of the more concrete results is a linear rank criterion for identifying the baseline parameters (F, Lambda) when the reduced matrices A(k) and intervention structure are known.

The validation is theoretical, with proofs and examples rather than experiments. I think the overall direction is interesting. The connection between algebraic geometry and identifiability is worth exploring, although some of the arguments need to be explained more carefully, especially when moving from the polynomial network setting to causality.

Critical strengths
  1. The paper asks a good question and tries a different mathematical approach. Non-injective transformations are important in causal representation learning, and I like the idea of studying identifiability through the geometry of model representations. This could provide a useful language for understanding when identifiability fails, beyond the usual decomposition-based arguments.

  2. The results on interventions, especially the rank condition in Appendix E, are promising. The paper gives a relatively concrete way to check identifiability of the baseline parameters once the reduced matrices and intervention information are fixed. I also appreciate that it distinguishes identifying (F, Lambda) from identifying all the intervention-specific parameters. That distinction is important.

Critical weaknesses
  1. I think the main mathematical statements need tighter assumptions and a clearer link to identifiability.
  • Theorem 3.1 should state explicitly how the number of double points relates to the degree and ambient dimension. For example, three double points on P^1 cannot impose independent conditions on cubic polynomials in the claimed surjective sense, simply by dimension counting. I think this can be fixed by stating the appropriate range of parameters.

  • Vanishing of H^1 gives a statement about the evaluation map, but the next step to uniqueness of network weights is not completely clear to me. There is also a difference between a finite fiber and a unique fiber in Definition 2.3. A small complete example would help.

  1. The bridge to causal identifiability needs more explanation.
  • In Proposition 4.3, the use of non-Artinian rings to obtain infinite transcendence degree needs a more careful justification. Non-Artinian by itself does not seem sufficient.

  • Theorem 4.4 argues that adding interventions removes ambiguity. I think the authors should explicitly show how all interventions act on the same unknown parameters. A direct sum of maps alone does not guarantee that the kernel disappears. A stacked system with a clear rank argument would make the idea more convincing.

  • Appendix E starts with the reduced matrices A(k) already known and aligned. Since the paper is motivated by observations under non-injective mixing, I would like the authors to make clear whether recovering A(k) is assumed, established by earlier work, or proved in this paper.

I do not see these as reasons to dismiss the overall direction. But it would improve the work quite a bit if the authors narrowed the strongest statements to exactly what their proofs establish.

Questions for authors
  1. Can the authors state the exact conditions on the degree, dimension, and number of double points in Theorem 3.1? It would also help to explain what is new here compared with the classical Alexander-Hirschowitz result.

  2. Can you give one small polynomial network example where the cohomology condition is checked and the fiber of the network parameters is computed? This would make the relation between vanishing, finite identifiability, and unique identifiability much clearer.

  3. In Theorem 4.4, how exactly are the constraints from different interventions combined on a shared parameter space? Can the authors show the resulting matrix and the rank condition in a simple example?

  4. In the causal part, are A(k) assumed to be identifiable from the observed distributions? If yes, under what assumptions? I think this distinction is necessary to understand the generality of the result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.