- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors propose a range of extensions that can be applied to a given group in a geometric algebra network, and explain how to construct corresponding networks that are equivariant under these extended groups. They find that these networks can have slightly increased performance, while reducing parameter count and training time. The method and experiments largely follows previous work (CGENN, GLGENN), with the only new contribution being the systematic extension to larger groups, which was however also partly done for GLGENN. I found the paper poorly structured and formatted, making it hard to extract the contributions and results. I do not see how these concerns can be overcome during the rebuttal, and therefore recommend to reject the paper.
Critical strengths
- The authors find that networks with an extended geometric algebra equivariance group have higher performance at lower parameter count and training time, see e.g. Figure 11. However, the gains are often smaller than the uncertainties from seed variations.
Critical weaknesses
- I think the paper adds little novelty compared to previous work. It reads like a thin wrapper around the GLGENN paper (2025), which itself is a thin wrapper around the CGENN paper (2023). These papers differ in which groups are considered to build equivariant networks using GA, and then apply small modifications to the architecture and evaluate on essentially the same benchmarks. The benchmarks are not particularly relevant: volume estimation, O(8)-regression, N-body dynamics simulation are toy examples; for top tagging the authors train only on up to 1k samples whereas the dataset has 1M samples and modern datasets contain 100M or more samples as simulation is cheap; for atom position prediction the authors do not directly compare with literature results.
- Overall the authors do a poor job in presenting their results. The main results in Figure 3 and Table 2 are poorly structured and in my opinion not well explained in the text. The proposed methods show improvements in these plots, but the performance gain often has similar size as the uncertainties. The paper body extensively explains the theory behind the approach, but I found it hard to follow and to get an understanding for the meaning of the extended groups - in particular for the extended groups which are the main contribution of the paper. The authors formatted the paper to remove white space between paragraphs and sections, probably to make it fit into the page limit, making the paper look overcrowded. The appendices contain additional results and answer most questions that came up when reading the paper (compute cost, regularization when training on small datasets) - but the authors should do a better job in already addressing these points in the main paper.
Questions for authors
- I find the core claim "increasing the equivariance group beyond what is realized in the problem improves performance and reduces cost" not intuitive. The optimal equivariance group should be the one realized in the problem, not more and not less. Do the authors have an explanation for what effects in the larger group can help in the learning process?
- I do not understand how the proposed P, A, Q, B relate to actual groups. Can you explain, for the O(3) group, what operations these extensions add beyond the standard setup? It would also be good to give more intuition for what the individual extensions do, to discuss when these extensions are expected to help, and to check this in experiments.
- Experiments focus on the low-data regime (~1k training points, ~1k steps to avoid overfitting with ~100k parameter networks). The authors argue that their architectures are more parameter efficient, but from my understanding they only compare them at fixed architecture layout, whereas they could probably also significantly reduce parameter count and still achieve strong performance. Also, it would be good to discuss issues related to overfitting in the main text, as this seems to be central to the claims in the paper.
- A central problem of GA-based networks is the significantly increased compute cost. The authors discuss this in appendix H, but compare only across their own architectures and not with baselines. The authors cite the flash clifford package for efficient implementations, although there are several other papers in this direction. Did the authors explore how such optimized implementations can help in their approach?
- For top tagging, several other Lorentz-equivariant architectures have been proposed since 2023 which seems to be the papers citation cutoff. In particular, https://arxiv.org/abs/2405.14806 extends the geometric algebra transformer architecture to the Lorentz group and is now widely used in high-energy physics.
6 comments
Response to Reviewer NYGv [2/3]
3) Q3: Focus on the low-data regime?
The reviewer highlights a potential weakness regarding the focus of the experiments on the low-data regime. We would like to clarify that we do not claim our wider-group models perform best in all settings, particularly with large training sets. On the contrary, we posit that the proposed framework offers a tunable trade-off: with abundant data, more expressive models utilizing smaller groups (such as Γ) become preferable, whereas in low-data regimes, wider-group models (such as P, A, and B) are preferable. This is discussed in particular in the Configurations section (lines 348-356 of the main text). That is the reason why we concentrate on different settings with low train set size, this is the focus of our work, since we propose P, A, B, Triple. At the same time the six benchmarks that we consider (convex hull, regression, O(3)- and O(6)-N-Body, top tagging, and atomic dynamics) are standard in the equivariant literature, and many papers use the same set or a subset of experiments, in particular, CGENN, CSMPN, GLGENN, and EqMotion.
We would also like to address the reviewer's hypothesis that simply reducing the parameter count by cutting parameters in CGENN may improve performance. We conducted a new experiment on the O(6)-Convex Hull volume estimation task, where we compare the full A model (which shares weights across grades congruent to 0 or 1 modulo 4, and across grades congruent to 2 or 3 modulo 4) and (CGENN) with reduced-parameter versions of : narrow , shallow , and narrow & shallow . The depth and width of the reduced models were chosen to match the parameter count of A as closely as possible. For this task, we observe the clearest advantage of A over the reduced CGENN, especially with 256 training samples:
| Model | Width | Depth | Parameters | Test MSE, mean ± sd (256 train s.) | Test MSE, mean ± sd (4096 train s.) |
|---|---|---|---|---|---|
| A | 32 | 4 | 13697 | 11.63 ± 1.16 | 4.78 ± 0.63 |
| , full | 32 | 4 | 70753 | 11.73 ± 0.72 | 3.58 ± 0.63 |
| , narrow | 12 | 4 | 12853 | 13.76 ± 0.43 | 7.93 ± 0.31 |
| , shallow | 32 | 1 | 18913 | 13.65 ± 0.14 | 4.82 ± 0.28 |
| , narrow & shallow | 27 | 1 | 13933 | 13.77 ± 0.21 | 5.13 ± 0.12 |
4) Q4: Efficient implementation of GA layers?
We would like to clarify that we compare our models in terms of computational cost not only with our models, but also with the two closest GA baselines, CGENN () and GLGENN (Q). Table 3 shows that our larger-group networks are more parameter- and computationally efficient than these direct competitors in the small-data regime. We agree that GA-based methods require more computation time than non-GA methods due to their implementation. To the best of our knowledge, the main methods for accelerating GA-based networks focus primarily on accelerating the implementation of the geometric product, and thus these methods can be applied directly to our models. As an example, we mention the Flash Clifford package; if needed, Flash Kingdon (https://github.com/tBuLi/flash-kingdon) can be applied similarly. Flash Clifford allowed us to accelerate geometric product used in the P, A, B, Triple, Q, geometric-product layers and to reduce FLOPs by a factor of across all configurations. For example, applying Flash Clifford to the configuration for with one linear and four geometric-product layers (with normalization and residual connections) reduces the forward-pass time from 2.875 ms to 1.721 ms and the forward-backward time from 8.267 ms to 7.377 ms.
We have added this discussion and the corresponding citations to Appendix I.
Response to Reviewer NYGv [3/3]
5) Q5: Other baselines for O(1,3)-Top Tagging?
We thank the reviewer for pointing out the L-GATr work, and we have now added it and several other models to the comparison. We find that we were able to achieve an accuracy of 0.945 for the Q- or A-config and 0.941 for the Γ-config when training on 1024 samples, whereas models trained on the full dataset achieve similar or lower accuracy: 0.942 CGENN, 0.936 ResNeXt, 0.930 P-CNN, 0.932 PFN, 0.94 ParticleNet, 0.922 EGNN, 0.929 LGN, 0.942 LorentzNet, including the newly added: 0.9423 L-GATr (2024), 0.9426 PELICAN (2022), 0.940 ParT (2024), 0.933 TreeNiN (2019). We have added this comparison to Section 6.
To address the reviewer's interest in the performance of our configurations on a larger training set, we conducted a new experiment and increased the train set from 1024 to 10000 jets at the same step budget. The mean changes were +0.24 in accuracy for A and +1.18 in accuracy for Γ compared with the results for 1024 train samples.
We also tested whether applying our local constraints can preserve the performance of the L-GATr model suggested by the reviewer, which was already trained on all 1.2M training jets (a certain type of mixing architecture). Projecting one hidden map to the A-config layer removed 4640 of 1077474 parameters of L-GATr. Before any updates, the validation AUC fell from 0.986363 to 0.984992 on 403k test jets. After matched fine-tuning, the projection recovered 81.3% of the initial AUC deficit on the same test jets. The selected A-config and the original L-GATr AUCs were 0.986070 and 0.986329. Thus, for large training sets, our wider configurations can be applied to fully trained models such as L-GATr to reduce parameters and, after matched fine-tuning, recover the lost performance.
6) Novelty of ideas.
We believe our work contains enough novel and non-trivial ideas to avoid being considered a thin wrapper around CGENN and GLGENN. The main novel ideas are:
(i) Using the groups P, A, B, and Triple in equivariant networks. Changing the groups was not straightforward: for each group, we identified its action, parameterized new operations equivariant with respect to it, and proved their equivariance. We also used other techniques not employed in CGENN or GLGENN. In particular, proving that equivariance with respect to P, A, B, and Triple implies O(p,q)-equivariance is non-trivial.
(ii) Showing how to mix different equivariance groups within a single architecture (in our heterogeneous configurations). To the best of our knowledge, the idea of varying the size of the equivariance group depending on the task has not been considered before, even outside the domain of GA-based networks.
Note that the same results cannot be obtained by simple regularization or by straightforwardly cutting parameters in CGENN or GLGENN.
7) Recommendations on presentation.
We have followed the reviewer's suggestions to elaborate on the computational cost and regularization effects of wider-group equivariant models in the small-data regime in the main text. We have added Section 7, devoted to these comments. For the convenience of readers not familiar with the GA formalism, we have added Appendix G, which presents the main theorems, groups, and layers in detail for the particular case of of Minkowski space-time, as applied to the Top Tagging task.
Response to Reviewer NYGv [1/3]
We thank Reviewer NYGv for valuable feedback and for highlighting the strengths of our work, including higher performance at a lower parameter count and shorter training time.
We are happy to address the questions:
1) Q2: Relation between the proposed groups P, A, B, Q and O(p,q)?
We thank the reviewer for this insightful question. We would like to clarify that P, A, B, and Q are not extensions of O(p,q); they are extensions of the corresponding Clifford group Γp,q. The pseudo-orthogonal groups O(p,q) act on an n=p+q-dimensional vector space, whereas our groups act on a 2n-dimensional space (reflecting the dimension of the Clifford algebra). For example, in the R1,3 setup (e.g., the O(1,3) Top Tagging experiment), the transformations induced by P, A, B, and Q do not act directly on R1,3. Instead, they transform a 24-dimensional space (with basis elements such as e,e1,e2,e3,e4,…,e1234). For example, these transformations can map vectors to scalars, bivectors, trivectors, or quadvectors. Therefore, making the group wider (and getting P, A, B, Q) does not correspond to a direct extension of O(p,q), nor are we adding simple geometric operations on Rp,q (e.g. R3 or R1,3). Rather, we are changing the algebraic representation space in which the network operates. Consequently, we cannot directly discuss the geometric meaning of these transformations in the original vector space. The geometric interpretation of the groups P, A, B, and Q is therefore nontrivial; developing this underlying theory is an interesting direction for future work. While we cannot yet provide a formal theorem explaining why one configuration is better than another, we emphasize that this approach works empirically: there are cases in which these wider groups improve performance.
2) Q1: Relation between wider groups and the task symmetries? Effects of wider groups on the learning process?
We agree that in an ideal infinite-data setting, the optimal equivariance group should exactly match the symmetries realized in the problem. However, in practical finite-data regimes, enforcing a wider group equivariance (and thus a more constrained network) offers two major benefits. First, enforcing wider equivariance acts as a powerful regularizer. Second, the true underlying symmetry of a problem is often not perfectly observable or easily identifiable by human inspection, especially in 2n-dimensional underlying spaces. If a network equivariant to a wider group empirically outperforms a less constrained one (with a smaller group), it strongly suggests that the data possesses hidden higher-order structural constraints that the wider group successfully captures. Thus, widening the group is a way of understanding the deeper nature of the task.