- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper studies whether categorical equivariant networks, whose layers must respect prescribed feature transfers, can approximate every continuous target satisfying those same constraints. It shows that increasing depth and width cannot overcome certain restrictions imposed by the permitted layers. Its main contributions are:
- It proves that increasing network size may not improve expressivity. For the specified layer class, the paper identifies restrictions that survive arbitrary depth and hidden width. In certain graph settings, every network remains affine despite containing nonlinear operations, so it cannot approximate even simple compatible nonlinear targets arbitrarily well (Theorems 3.2 and 4.2).
- It explains the capabilities and limitations of more flexible scalar gates. For the graph systems studied, the paper characterizes exactly which targets become approximable when gates can depend on the input. These gates restore nonlinear behavior, but they may still fail to produce the output directions required by the target (Theorem 6.1).
- It establishes when the bundle-based construction supports universal approximation. The paper characterizes when a linear readout can recover output features from their collected views. When such a readout exists, ordinary neural networks operating inside the bundle can approximate every continuous target satisfying the consistency requirements (Proposition 7.1).
I find the paper’s broader framework and its insights into architectural expressivity interesting, and I lean toward acceptance. The theoretical results appear sound based on my understanding, although I cannot confidently verify the correctness of every proof. My confidence in this recommendation is therefore limited.
Critical strengths
The categorical perspective provides a broader analysis of equivariant architectures. By considering relations between different feature spaces, including noninvertible restrictions, the paper examines architectural constraints that are not fully described by a symmetry group alone. It distinguishes the functions allowed by the consistency requirements from those realizable by the permitted layers, and proves that this gap can persist despite arbitrary depth and hidden width. These results may provide useful guidance on when improving expressivity requires changes to the feature representation or layer construction.
Critical weaknesses
-
The presentation is difficult to follow. The main text assumes substantial familiarity with categorical thoery and does not sufficiently explain its connection to familiar equivariant architectures. A concise terminology guide and a complete worked example of the network computation would make the motivation and scope of the theoretical results clearer.
-
The practical relevance of the analyzed architecture needs stronger motivation. The complete characterizations concern particular graph feature systems, and the scalar-gate result additionally requires finite holonomy. Although these assumptions are acknowledged, their relevance to realistic learning problems remains insufficiently established. A concrete application motivating the restriction would help readers understand when the proposed criteria provide useful architectural guidance.
Questions for authors
-
Connection to equivariant approximation theory. Could the authors explicitly distinguish systematic degeneration that is unavoidable for all equivariant maps between the chosen feature spaces from additional degeneration caused by a specific model parameterization? Prior work has identified limitations in preserving orientation information with certain equivariant features [1-3]. How do the restrictions on output directions in this paper relate to these phenomena? Explaining how these phenomena can be overcome by changing the equivariant parameterization.
-
Guidelines for model design. Could the authors try to translate their theoretical findings into clear guidelines for designing expressive equivariant models? In particular, how should feature representations, nonlinear operations, and readout maps be chosen to avoid the identified expressivity barriers?
[1]: Joshi et al. (2023). "On the Expressive Power of Geometric Graph Neural Networks", ICML 2023.
[2]: Kaba et al. (2023). "Improving Equivariant Networks with Probabilistic Symmetry Breaking", NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations.
[3]: Lin et al. (2026). "Reducing Symmetry Increase in Equivariant Neural Networks", ICLR 2026.
2 comments
Thank you for your detailed response. It’s been a pleasure discussing this paper with you. Your clarification offers useful insight into the expressivity limitations of equivariant models that rely on scalarization. I have increased my confidence in my assessment accordingly and will maintain my score, which already supports acceptance.
Thank you for the careful reading and for identifying where the architectural consequences need a clearer explanation. We have added explanations of the basic terms where they first appear and a worked edge computation in Section 2, and expanded the design discussion in Sections 6–7. The additions and replacement passages are marked in red.
The holonomy group H is generated by transporting features around graph cycles. For an input x, the stabilizer H_x consists of the symmetries in H that fix x. Equivariance requires each of these symmetries to fix f(x) as well. Thus every deterministic equivariant map satisfies f(x) ∈ W^{H_x}, where W^{H_x} is the set of output vectors fixed by those symmetries. The chosen output feature space can force additional output symmetries, as studied by Lin et al. (2026).
Our layers impose a further restriction. Even after allowing multiplication by any continuous invariant scalar function, their outputs at x must lie in E(x), the set of values Lx + c where L is an equivariant linear map and c is fixed by all symmetries in H. Here an invariant scalar function is one whose value does not change under these symmetries. Theorem 6.1 proves that, for the stated graph feature diagrams with finite H, this enlarged class can approximate every continuous equivariant map uniformly on compact sets if and only if E(x) = W^{H_x} at every input. This separates the output values allowed by symmetry from those available to our layers.
For example, consider the planar action of D4, the eight symmetries of a square. A nonzero input x off the reflection axes is fixed only by the identity, so symmetry allows any output vector. Nevertheless, our enlarged class can only output a multiple of x. The identity map f(x) = x belongs to this class: its output is unchanged, so it has exactly the same symmetries as the input. Preserving those symmetries therefore does not guarantee that the class can approximate every equivariant target. In particular, it cannot approximate a target whose value at x lies outside the line through x.
For planar vectors written as complex numbers z, the added vector feature p(z) = conjugate(z)^3 supplies the missing directions. Combinations of z and p(z) with suitably chosen invariant scalar coefficients can approximate every continuous D4-equivariant map uniformly on compact sets. Section 6 gives this construction, which preserves deterministic equivariance.
We have added the suggested references to Section 9 (Discussion) and clarified their relation to our results. Joshi et al. (2023) analyze how tensor features and the invariant scalars computed from them affect geometric GNN expressivity. Lin et al. (2026) study when feature representations force outputs to have more symmetries than their inputs. We cite the suggested probabilistic symmetry-breaking paper as Lawrence et al. (2025), following the published author list and year. Their method allows individual samples to break input symmetries while the output distribution conditional on the input remains equivariant. Our construction instead adds vector features to approximate deterministic equivariant targets.
The revised Section 6 explains the separate roles of scalar and vector features: invariant scalar features distinguish inputs not related by symmetry, while equivariant vector features span the output vectors allowed by symmetry at each input. For the input distributions analyzed in the paper, supplying several independently varying input vectors can reduce average approximation error because their span is less often deficient. Duplicating a single vector in more hidden channels does not enlarge its span. Section 7 gives a test based on linear equations for whether a linear readout that commutes with the prescribed transfers can recover each output feature from the collection of its restricted copies. A successful test supplies the reconstruction used in the approximation theorem. A failed test rules out this reconstruction, but does not by itself rule out approximation.
Section 9 describes a possible application: learning response functions F_a and F_b in two local coordinate frames, with a fixed map T transferring features between them. Processing a feature and then transferring the response must agree with transferring the feature and then processing it: F_b(Tz) = T F_a(z). Our experiments use simple feature diagrams with fixed transfer maps and target responses that satisfy this equation. They test whether the permitted layers can approximate these responses and whether the proposed changes to the layers or features remove the approximation barrier. They do not establish performance on a real sensor or graph benchmark, and the revised text states this limitation explicitly.