acceptodds
Under review as a conference paper at ICLR 2027

The Geometry of Polynomial Group Convolutional Neural Networks

Abstract

We prove that polynomial group convolutional neural networks (PGCNNs), for an arbitrary finite group , are identifiable. In order to achieve this, we introduce a new mathematical framework for PGCNNs using the language of graded group algebras. This framework yields two natural parametrizations of the architecture, based on Hadamard and Kronecker products, related by a linear map. We compute the dimension of the associated neuromanifold, verifying that it depends only on the number of layers and the size of the group. The identifiability of PGCNNs follows from showing that the general fiber of both parametrizations is trivial up to the regular group action and rescaling.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
equivariance, convolution, identifiability, algebraic geometry, neuromanifolds, neuroalgebraic geometry

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JL4hOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the geometric properties of polynomial group convolutional neural networks (PGCNNs) for arbitrary finite groups. The authors introduce a mathematical framework based on graded group algebras and establish two related parametrizations using Hadamard and Kronecker products.

The main theoretical contributions are a characterization of the dimension of the associated neuromanifold and a proof of generic identifiability. In particular, the authors show that the neuromanifold dimension depends only on the network depth and the order of the group, while the network parameters are generically identifiable up to group actions and rescaling symmetries.

The theoretical results are supported by numerical verification using Sage and Macaulay2 for small finite groups and shallow networks. The paper extends previous results on polynomial convolutional networks and provides an algebraic-geometric framework for understanding the structure of group-equivariant neural networks.

Critical strengths
  1. Rigorous theoretical contributions.

The paper establishes a characterization of the neuromanifold dimension and proves generic identifiability of polynomial group convolutional neural networks for arbitrary finite groups. These results extend previous work on polynomial convolutional networks and provide a mathematical foundation for understanding the geometry of group-equivariant architectures.

  1. Novel algebraic framework.

The authors introduce a framework based on graded group algebras and establish a connection between Hadamard- and Kronecker-product parametrizations. This approach provides a systematic way to analyze the geometric properties of polynomial group convolutional networks and may be useful for further theoretical investigations.

Critical weaknesses
  1. Limited empirical validation.

Although the paper provides numerical verification of the theoretical results, the experiments are restricted to small finite groups and shallow networks. Moreover, no experiments on concrete machine learning tasks are presented. Evaluating PGCNNs on practical tasks would help demonstrate the relevance of the theoretical findings and strengthen the empirical contribution of the paper.

  1. Unclear implications for practical machine learning.

While the theoretical results on neuromanifold dimension and generic identifiability are mathematically interesting, their implications for learning performance, optimization, and generalization remain unclear. In particular, the paper does not establish how these geometric properties translate into practical advantages for group-equivariant neural networks. This limits the impact of the results for the broader ICLR audience.

Questions for authors
  1. Practical applications to machine learning tasks.

Could the authors clarify how the theoretical results on neuromanifold dimension and generic identifiability can contribute to practical machine learning tasks, such as image classification? In particular, what insights do these results provide for designing or training group-equivariant convolutional neural networks?

  1. Connection between theoretical results and learning performance.

The paper establishes that the neuromanifold dimension depends only on the network depth and the group size. Could the authors explain how this result relates to learning performance, optimization, or generalization? It would be helpful to understand whether the theoretical findings lead to concrete advantages over conventional convolutional neural networks.

  1. Experimental validation.

Could the authors provide experimental evidence on a concrete machine learning task, such as image classification, to demonstrate the practical relevance of their theoretical results? Such experiments would help assess whether the proposed geometric framework offers insights beyond the algebraic characterization of the architecture.

3 comments
AuthorsOpenReview ↗

Response to reviewer JL4h

Thank you for recognizing the rigor of our results and the novelty of the algebraic framework. We welcome the opportunity to clarify their practical relevance.

1. Practical machine learning.

Finite-group equivariant networks are used in molecular property prediction, where atom permutations are relevant symmetries (Gilmer et al., 2017), and medical image analysis, where rotation and reflection equivariance reflect tissue's lack of a preferred orientation (Veeling et al., 2018). Training data are expensive in both settings, motivating architectures that may require fewer samples than symmetry-agnostic models.

We study how respecting data symmetries affects function-space geometry. Understanding this relation can inform architecture design and training. Any loss is constant along parametrization fibers (Trager et al., 2020), creating flat directions. Theorem 4.5 shows that, for general filters, the parameter ambiguities are exactly group shifts between consecutive layers and filter rescalings. The relation to generalization is addressed below.

2. Dimension, learning performance, and generalization.

Please see our reply to DBMR Q2: the dimension result connects to metric entropy, which gives an upper bound on sample complexity.

3. Experimental validation.

Our contribution concerns the mathematical foundations of machine learning, in the same line as Kileel et al. (2019), Trager et al. (2020), Shahverdi et al. (2025; 2026b), and Usevich et al. (2025). These results characterize an architecture's function space. The numerical computations in Appendix I check the theorems. An image-classification benchmark would instead measure networks' accuracy with monomial activations, a different question from the geometric statements we prove. Polynomial activations provide a model in which dimension and fibers can be determined exactly; we are not proposing a new architecture.

If any proof step was unclear, we would be glad to explain it. If the soundness score reflects a specific concern, please let us know which step it concerns so we can address it in the revision. Thank you again for your time and thoughtful questions.

Extra references

J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, G. E. Dahl. Neural Message Passing for Quantum Chemistry. In Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1263-1272, 2017.

B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, M. Welling. Rotation Equivariant CNNs for Digital Pathology. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2018, Springer, Cham, September 2018, DOI 10.1007/978-3-030-00934-2_24.

Thank you very much for your explanation. However, I remain unconvinced that the contribution is sufficiently novel relative to prior work. I will therefore maintain my original score.

Thank you for the follow-up. We would appreciate a more specific explanation of the novelty concern. The original review identified a “Novel algebraic framework” as a critical strength and raised concerns about empirical validation and practical implications, which we addressed in our rebuttal. Insufficient novelty relative to prior work was not identified as a weakness in that review. Our response to reviewer DBMR, Q3, compares both our results and proof techniques with Shahverdi et al. (2025; 2026a). We establish generic identifiability for polynomial group convolutions on arbitrary finite groups. Shahverdi et al. (2025) study a different convolution architecture, while Shahverdi et al. (2026a) assume identifiability. Could you clarify which contributions you consider insufficiently novel, relative to which prior results, and which aspects of our comparison you find unconvincing?

We would also appreciate clarification of any remaining concerns underlying the soundness score of 2, following our earlier request in the first comment.

Reviewer_gL2AOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the neuromanifold of polynomial group convolutional neural networks (PGCNNs) using tools from algebraic geometry. The authors formally analyze the case in which the input space is signals on a finite group GG, by identifying filters and activations with elements of the group algebra K[G]K[G] for a field KK (of characteristic zero), and considering activation functions that take an iterated "Hadamard product" (i.e. pointwise product) of an input with itself. By proving the PGCNN can be represented as the convolution of an input-dependent tensor product and parameter-dependent tensor product φθ\varphi_\theta, several new results are obtained, which leads the authors to argue for the usefulness of this tensor representation. In particular, they obtain:

  • a proof that the PGCNN and a PNN of width ∣G∣|G| have the same parameter-to-dimension ratio;
  • a characterization of the fiber of θ↦φθ\theta\mapsto\varphi_\theta, and a subsequent one for the fiber of the PGCNN itself;
  • a proof that the PGCNN is identifiable, up to inter-layer symmetry transformations.
Critical strengths

This paper provides a rigorous exploration of GCNNs by specializing to certain polynomial activations. This is a useful demonstration of using algebraic techniques to understand the geometric properties of networks, and the authors argue convincingly that Proposition 2.7 (and its subsequent application) show the utility of their representation of the network through φθ\varphi_\theta.

The calculation of the dimension of the neuromanifold, and the PGCNN's identifiability, are in themselves results of potential interest to readers.

Critical weaknesses

None of the paper's weaknesses appear strong enough to support rejection. One question, nonetheless, it detailed for the authors below.

Questions for authors

An outstanding question is to clarify, in my opinion, the weakest point of the paper: the claim that the parameter-to-dimension result means degree or "curvature" must explain differences in performance.

A first point is that it's not clear why the parameter-to-dimension ratio should control ease of learning, or generalization error, or similar: is there more to this than a possible intuition the authors wish to dispel?

Secondly, it's not clear why the authors' result would mean the neuromanifold still explains differences in performance (e.g. through its degree or curvature). Could this not instead arise — at least in principle — through different parameter-to-function maps, with models that have the same neuromanifold?

It should also be noted that equivariance imposes constraints that are known to make the learning dynamics more difficult [1,2].

[1] Pertigkiozoglou, Chatzipantazis, Trivedi, Daniilidis. Improving Equivariant Model Training via Constraint Relaxation. NeurIPS 2024. [2] Xie, Smidt. A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models. NeurIPS 2025.

2 comments
AuthorsOpenReview ↗

Response to reviewer gl2A

Thanks for your encouraging assessment and thoughtful questions! We agree that equal parameter-to-dimension ratios do not imply that performance differences must be explained by degree or curvature. We will remove this unsupported claim.

Theorem 4.1 and generalization error.

Please see our reply to DBMR Q2, which explains the connection through metric entropy and an upper bound on sample complexity.

Could performance differences arise from the parameter-to-function map?

Yes; our results address this only partially. By Theorem 4.1, the Jacobian of Φ\Phi has maximal rank at general filters. In the terminology of Trager et al. (2020), every critical point of the loss at general filters is therefore pure, arising from the neuromanifold's geometry and the loss. Spurious critical points can only occur on a proper Zariski-closed subset of parameter space. Theorem 4.5 identifies the general fibers as group shifts between layers and rescalings, with no additional fiber degeneracy. Parametrization can still affect training through the non-general locus and gradient-descent dynamics, which our results do not address.

Degree and training. We do not determine the neuromanifold's degree or claim that it explains training differences. For square loss, training minimizes distance from a data-dependent point to the neuromanifold; for generic data, the number of complex critical points is its Euclidean distance degree (Draisma et al., 2016). The works you suggest (Pertigkiozoglou et al., 2024; Xie & Smidt, 2025), which we will cite, show that equivariance constraints can make training harder and that relaxing them can help. Whether this is reflected in the neuromanifold's geometry, for example its ED degree, or in the parametrization remains open. We will state this in the conclusion.

Extra reference

J. Draisma, E. Horobeț, G. Ottaviani, B. Sturmfels, R. R. Thomas. The Euclidean distance degree of an algebraic variety. Foundations of Computational Mathematics, 2016.

I have read the authors' comments and thank them for the clarification. I believe the updates to the submission will make its contribution more clearly positioned.

Reviewer_DBMROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the identifiability of polynomial group convolutional neural networks (PGCNNs) for an arbitrary finite group GG. Here, identifiability means that the hidden-layer filters can be recovered from the input–output map alone, up to trivial symmetries: group shifts between consecutive layers and rescaling of filters. The activation functions are monomials of degree rr. This makes the space of realizable functions (the neuromanifold) a semi-algebraic set that can be studied with algebraic geometry.

The main technical tool is the graded group algebra. The polynomial activation is written as a Hadamard power. The Hadamard product does not distribute over the group convolution, but Proposition 2.7 relates it to the Kronecker product. This gives two parametrizations of the architecture:

  • ΦΦ maps the filters to the PGCNN they implement. Its image MΦM_Φ is the neuromanifold.
  • φφ is obtained by replacing each Hadamard product with a Kronecker product. Its image is MφM_φ.

The two are related by a projective linear map ΛΛ, so that Φ=Λ∘φΦ = Λ ∘ φ (Figure 2). This lets much of the analysis take place in the larger Kronecker space, where it is technically easier.

The main results are:

  • Dimension (Theorem 4.1). For activation degree r≥2r ≥ 2, the projectivized Jacobians of both maps have full rank at a generic point. Both neuromanifolds therefore have the maximal dimension L(n−1)+1L(n−1)+1, where n=∣G∣n = |G|. As a consequence, the authors note that a PGCNN has the same ratio of parameter count to dimension as an equi-width polynomial MLP of width ∣G∣|G|. They argue that any efficiency advantage of equivariant models must therefore show up in the curvature or degree of the neuromanifold, not its dimension.
  • General fiber of φφ (Theorem 4.3). This is proved by induction on depth, using invertibility of the last-layer filter and a symmetric tensor rank result.
  • General fiber of ΦΦ (Proposition 4.4 and Theorem 4.5). Proposition 4.4 is the most technical step. It controls the intersection of a linear span of r-th powers with the variety of r-th powers, using an incidence scheme and semicontinuity of fiber dimension and fiber length. Theorem 4.5 builds on it to establish identifiability of PGCNNs for every r≥2r ≥ 2.
Critical strengths
  • The paper addresses a meaningful question, identifiability of equivariant architectures, with rigorous mathematical tools. It brings deep mathematical machinery to a machine learning problem.
  • The results hold for arbitrary finite groups and for every activation degree r≥2r ≥ 2, not only for sufficiently large degrees.
  • The factorization Φ=Λ∘φΦ = Λ ∘ φ, built from the Kronecker representation of the activation, is a very useful insight. It allows to work with a different parametrization instead of directly on the original neuromanifold.
  • The paper appears mathematically rigorous.
Critical weaknesses
  • Unclear novelty relative to prior work. The paper builds closely on Shahverdi et al. (2025) and Shahverdi et al. (2026a). Both the new results and the proof techniques need to be positioned more precisely against these works, so that the specific technical novelty is clear.
  • Restrictive input domain. The analysis assumes the input domain ΩΩ equals GG. This is mathematically convenient, since it preserves the group algebra structure at the first layer, but it is restrictive. It is unclear how the results transfer to more general domains such as sequences.
  • The efficiency claim is not shown. The claim in the contributions, Section 4.1, and the conclusion that efficiency must lie in curvature or degree lacks a supporting analysis.
  • Accessibility. The paper is very dense, and the proofs are highly technical and require familiarity with algebraic geometry.
    • Section 2 is very dense. It introduces graded group algebras without examples, which makes it hard to follow.
    • The paper does not separate the high-level ideas, which a general audience could grasp, from the harder technical arguments very well.
    • Examples appear only in the appendix.
    • As a result, readers outside the area may have difficulty understanding the paper.
  • Presentation.
    • The several parametrization maps and neuromanifolds can be hard to keep track of.
    • Projective varieties and incidence schemes are introduced in the main text without enough explanation, so the proof sketch of Proposition 4.4 is not very accessible.
    • The commutative diagram in Figure 2 appears central to the argument but receives little discussion. In particular, its implications for the dimensions of the two neuromanifolds are not spelled out.
    • Figure 1 could be improved.
    • The paper contains several typos (e.g., “circulent”, "2 ≥ i ≤ n", "the sober is a , up") and needs careful proofreading.
Questions for authors
  • The input domain is assumed to equal GG. Can you explain how this assumption could be relaxed, and how the results apply to more general domains such as sequences?
  • How does the efficiency comparison with an equi-width polynomial network depend on the assumption Ω=GΩ = G? Would the conclusion change for other domains?
  • Can you clarify the technical novelty relative to Shahverdi et al. (2025) and Shahverdi et al. (2026a), both in the results and in the proof techniques?
  • Can you discuss the commutative diagram in Figure 2 in more detail, including its implications for the dimensions of MφM_φ and MΦM_Φ? It would also help to label the diagram explicitly (e.g., θθ, φθφ_θ, the PGCNN) to link it to the problem.
3 comments
AuthorsOpenReview ↗

Response to reviewer DBMR

Thank you for your careful reading and positive assessment of the factorization Φ=Λ∘φ\Phi = \Lambda \circ \varphi. Your questions identify where further explanation is needed. We address them below and specify the changes we will make.

1. Input domain and extensions.

The assumption Ω=G\Omega=G makes the analysis tractable. For Ω=G/H\Omega=G/H, with HH a subgroup of GG (Cohen & Welling, 2016), only the first layer changes: it becomes a lifting layer K[G/H]→K[G]K[G/H]\to K[G]. We expect our analysis to extend at least in part, with two caveats. If ∣G/H∣|G/H| is much smaller than ∣G∣|G|, the activation degree must be high enough, possibly above 2, for the activated first-layer output polynomials to be linearly independent, as our proofs require. If HH has a normal subgroup NN, this inflates the general fibers. We will discuss these limitations in a remark.

2. Efficiency, metric entropy, and other domains.

We will correct the manuscript's imprecise efficiency claim and instead discuss metric entropy, which controls expressivity and gives an upper bound on sample complexity. Here s(ε,δ)s(\varepsilon,\delta) is the least number of training samples needed to guarantee generalization error below ε\varepsilon with probability greater than 1−δ1-\delta.

For an algebraic model such as a PGCNN, sample complexity is bounded above in terms of metric entropy log⁡N(M,ε)\log\mathcal{N}(\mathcal{M},\varepsilon), where N\mathcal{N} counts the ε\varepsilon-balls needed to cover a compact piece M\mathcal{M} of the neuromanifold (Marchetti et al., 2025, Thm. B.1). Zhang & Kileel (2025) bound metric entropy above using the number of model parameters; Yomdin & Comte (2004, Chapter 2) bound it below using dimension. Theorem 4.1 makes these bounds coincide for PGCNNs. Results of Finkel et al. (2025) and Usevich et al. (2025) give the corresponding conclusion for equal-width PNNs.

The comparison gives metric entropy nn times smaller for PGCNNs than for equal-width PNNs, showing how weight sharing reduces function-space complexity. We will add a short remark and a detailed appendix.

For Ω=G/H\Omega=G/H, avoiding the two obstructions in Q1 and taking sufficiently high activation degree, we expect the dimension result and the factor-nn metric-entropy comparison to persist. This extension remains an expectation.

3. Novelty relative to Shahverdi et al.

Shahverdi et al. (2025) show that the projectivized PCNN parametrization is a morphism factoring through the Segre-Veronese variety V\mathcal{V} via a birational map: ϕ~=Λ∘ν\widetilde{\phi}=\Lambda\circ\nu, where ν\nu is the Segre-Veronese embedding. This transfers known invariants, including dimension, degree, and Euclidean degree, to the neuromanifold.

For group convolutions on arbitrary finite groups, our projectivized parametrization is only rational, requiring different machinery. We analyze the Jacobian symbolically for Theorem 4.1 and use semicontinuity of fiber dimension and length for Theorem 4.5.

Shahverdi et al. (2026a) assume identifiability to study equivariant models. We prove identifiability for PGCNNs. Our results and techniques are therefore complementary to theirs.

4. Figure 2 and the two neuromanifolds.

The diagram expresses the factorization above, from the Kronecker parametrization φ\varphi to the Hadamard parametrization Φ\Phi through Λ\Lambda. Generic finiteness of Λ\Lambda means the two images have equal dimension (Theorem 4.1); birationality means the general fibers agree (Theorems 4.3 and 4.5). We will label the objects explicitly and explain these links to the theorems.

Accessibility and presentation. We will emphasize high-level ideas, move technical remarks to the appendix, bring examples into Section 2, improve explanations, and correct typos.

Extra references

Y. Zhang and J. Kileel. Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applications, 2025.

Y. Yomdin and G. Comte. Tame Geometry with Application in Smooth Analysis. Springer, 2004, Chapter 2.

Thank you very much for the responses and clarifications. The metric-entropy perspective and the extension of ΩΩ to homogeneous spaces G/HG/H both seem promising. However, the answers are very brief, and the arguments remain conjectures or sketches. The proposed clarifications and accessibility improvements cannot be properly evaluated.

Thanks for your reply! We have submitted a revised version of the paper with more details on the metric-entropy perspective. See also the official summary comment above to all reviewers. We hope that these additional details will be satisfactory.

Comments on the paper

1 comment

Thanks again to all the reviewers for the careful reading and very useful feedback. We have now uploaded a new version of the article with changes based on the discussions below. The changes are summarized as follows:

  1. We retract our claim about the significance of the equal ratio of the numer of parameters and the dimension between PGCNN and equi-width PNN.
  2. We explain how Theorem 4.1 controls the metric entropy of the neuromanifold, which bounds the sample complexity from above. For this we add a paragrapha after Theorem 4.1 and Appendix I which provides a rigorous proof of these claims.
  3. To improve accessabililty, we have added a Running Example that consists of three parts. The goal is to introduce the mathematical machinery and compute dimensions and fibers explicitly for a small example.
  4. To improve presentation, we have added a paragraph at the beginning of section 2 which motivates the mathematical machinery we introduce there. We also added few sentences to the caption of Figure 2 to connect it directly to the main theorems.
  5. We have fixed typos.

All changes, except for typos and change in the online version of the abstract, are highlighted with blue color.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.