- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies linear permutation equivariant maps between antisymmetric tensors of arbitrary order. It gives a diagrammatic construction of a complete basis for these maps and proves that the dimension of this map space is at most two. In addition, it introduces an IO pattern, so that applying these maps does not require storing large weight matrices. Experiments on the Wiki-RfA dataset show that the method can achieve performance comparable or better than existing general permutation equivariant models while using fewer parameters. The paper is well-written and clearly organized.
However, I think the novelty of this work is relatively weak compared with existing mathematical results. The dimension formula and the conclusion that the dimension is at most 2 can both be derived directly from classical representation theory results [1]. In addition, reference [2] has already given a closely related diagrammatic basis for the case where the input and output have the same tensor order. The authors should clearly compare the main results of this paper with these existing results, and clarify which parts of the proposed characterization are new contributions.
[1] William Fulton , Joe Harris, Representation Theory: A First Course, 1991
[2] Alexander Wilson. The Multiset Partition Algebra: Diagram-Like Bases and Representations, 2023.
Critical strengths
The authors provide a complete basis for equivariant linear maps with arbitrary input and output orders, and prove that the dimension of the corresponding mapping space is at most 2. This result lays a clear mathematical foundation for building equivariant networks that exploit antisymmetric structures.
Critical weaknesses
I think the main weakness of this work is the lack of clear motivation. Although this paper gives a theoretical characterization of permutation equivariant linear maps between antisymmetric tensors of arbitrary order, the practical motivation for considering higher order tensors is still unclear. The experimental section focuses on a standard directed graph dataset, where pairwise relations can already be represented by second order antisymmetric tensors (i.e., adjacency matrices). The architecture in the experiments only involve 1-order and 2-order tensors. So it is not clear what additional advantages this higher order antisymmetric representation actually offer in this setting. The authors should clarify what meaningful machine learning tasks actually benefit from such higher order representations, a theoretical justification of their benefits would also be helpful.
Questions for authors
Can the authors identify representative machine-learning tasks in which higher-order antisymmetric tensor representations offer advantages that are difficult to achieve with lower-order representations? Could the practical value of the proposed framework be further demonstrated on a task that genuinely requires higher-order antisymmetric tensors?
3 comments
Thank you for the detailed response. The additional explanations help clarify the mathematical contributions and their relationship to existing work. However, my main concern regarding the motivation and practical significance of higher-order antisymmetric tensor representations remains insufficiently addressed.
Overall, the response does not substantially change my assessment of the paper. I therefore maintain my original score.
We thank the reviewer for their reply. We are glad that our response clarified our mathematical contributions and their relationship to existing work. We are grateful for their time and feedback.
We thank the reviewer for their review, and we are glad that they found the paper to be well written and that our results provide “a clear mathematical foundation for building equivariant networks that exploit antisymmetric structures.” We answer their points below.
Wilson, 2023. Wilson’s work concerns symmetric rather than antisymmetric tensors, so we will cite it in our discussion of symmetric tensors in the Related Work section. The multiset partition algebra that was introduced in their paper is the algebra of Sn-equivariant maps on a space of degree-r polynomials in commuting variables, which decomposes into tensor products of symmetric powers of Rn, so its diagram-like basis describes equivariant maps on symmetric tensors. Our work considers equivariant maps on antisymmetric tensors, and our diagrams, which come from antispherical bipartitions and are constructed differently from Wilson’s, must account for the signs that arise from reordering indices. This is the key technical difficulty in our setting: we resolve it with our sign rules (Figure 2), which also explain why certain diagrams vanish (Lemma 4.13). We also characterise maps between different orders k=l for all n, whereas Wilson only considers maps from a space to itself and needs n≥2r.
Fulton & Harris. We agree that the dimension in Corollary 4.16 can also be recovered classically: writing Rn≅1⊕V, where V={x:∑ixi=0} is the standard representation, gives Λk(Rn)≅ΛkV⊕Λk−1V (cf. Eq. (85)). The ΛjV are irreducible (Fulton & Harris, Prop. 3.12) and are pairwise non-isomorphic, since they correspond to distinct hook partitions (n−j,1j), and Schur’s lemma then gives the dimension count. We will add this remark and the citation.
The dimension, however, is a by-product rather than our main contribution. Schur’s lemma gives the number of maps but not the maps themselves. By contrast, our construction gives an explicit basis of weight matrices in the coordinates in which antisymmetric tensors are stored, uniformly in n, and directly leads to the IO patterns that make these layers practical at scale (see Appendix J). This mirrors the characterisation of permutation equivariant layers on general tensors by Maron et al. (2019), whose Bell-number dimension likewise follows from the classical theory of the partition algebra (Jones, 1994): in both cases the value for equivariant learning lies in turning the abstract count into layers that can be implemented.
Novelty. In summary, our new contributions are 1) the explicit basis of weight matrices for every pair of orders (k,l) and every n, given by antispherical bipartitions and their sign rules; 2) the IO patterns that implement these maps with O(1) storage of each map in n; and 3) the first explicit characterisation of these layers for equivariant learning.
What the experiment tests. Our experiment is not designed to compare higher-order with lower-order representations. It compares our layers with general permutation equivariant layers (Maron et al., 2019) at the same tensor orders, with everything else held fixed. Even at orders one and two these layers had not been characterised before, and in our updated results (see our response to Reviewer drUC) they achieve a higher AUC than the general layers at every width, with fewer parameters.
Where higher orders help. Higher orders help when what we want to model involves three or more entities together. For example, consider pairwise comparison data, such as match results between teams or human preferences between language models. Let Tij be the log-odds that i beats j. The 2→3 map in Table 2 then gives Tij+Tjk+Tki for each triple. If the three comparisons fit together, then this is zero, but if they go round in a circle (i.e. i beats j, j beats k and k beats i), then it is large. Such cycles matter for both tasks, because no single ranking can explain them. Whether there is such a cycle is a question about the triple, and is not about any one pair, so an order-three tensor is the natural place to store the answer. Order-two layers, which can only combine values along rows and columns, cannot compute it directly.
A higher-order experiment. Such a task would be an interesting application in its own right, but it would not change the main contributions of the paper. The characterisation, the explicit basis, and the IO patterns are established for every order, so they do not depend on an experiment at any particular order. The role of our experiment is to show that layers that are built from the characterisation work well in practice, and our updated results show exactly that.