- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The goal of this paper is to construct and efficient rotation-equivariant network design that is 1) basis-free, 2) acts on the native image grid, and 3) does not rely on discretization of the rotation group, or (truncated) analytic filter bases. An additional goal is 4) a design that combines directly with existing CNN pipelines for classification and segmentation of 2-dimensional images. The proposed design, named CRE, obtains the goals 1-4.
At the core of the CRE design is a convolutional block which is translation equivariant, like a CNN, and whose local response is rotation equivariant. The latter is accomplished through a rotation-invariant learnable kernel which decomposes into central-pixel interaction and neighbourhood attention, whose translational and rotational equivariance is guaranteed by parameter-sharing across pixel positions and dependence only on rotation-invariant distances. The CRE model is compared via (limited) classification and segmentation experiments with several established rotational-equivariant designs (and a standard CNN as a non-equivariant baseline). In classification the CRE model is found to improve average performance, and in segmentation it is subleading, but still competitive, with the comparison models. Limitations in efficiency are stated and ideas for improvement are indicated.
Critical strengths
Competitive novel rotation-equivariant design with easily implementable convolutional blocks.
Critical weaknesses
-
The survey of related rotation-equivariant work is very short and excludes papers with partial overlap with the proposed architecture (e.g. using rotation invariant kernels), or that do not fall into the classes listed (e.g. learned equivariance). Examples include Zhdanov, Hoffmann & Cesa (2023), Implicit Convolutional Kernels for Steerable CNNs, NeurIPS 2023. Khetan et al. (2021), Implicit Equivariance in Convolutional Networks, arXiv 2111.14157
-
The experimental tests are limited in scope and the gained accuracy may be limited to certain tasks.
Questions for authors
-
The implementation of the convolutional blocks includes architectural choices, e.g. when modeling and as small multilayered perceptrons. How are these choices motivated? Have hyper parameter optimization studies been made, and if not, how informative are the results of your tests?
-
Why do you emphasize the non-equivariant networks in your review of related work? Wouldn’t it be more relevant to cover more of the proposals for rotationally equivariant networks?
2 comments
Thanks for the clarifications. I am happy with the authors' rebuttal and revisions. Consequently, I will raise the soundness score.
Response to Reviewer fJTx
We thank the reviewer for the positive assessment and the helpful references. Each point ends with where to find the change in the revised paper.
Related work misses kernel-based and learned-equivariance work (Zhdanov et al. 2023; Khetan et al. 2021).
We added both, and placed CRE relative to them.
We also added a paragraph on invariant set and message-passing models, to compare with ours. In the paper: Section 2.
Why emphasize non-equivariant networks in the related work?
Agreed. We shortened the general CNN paragraph and used the space for rotation-equivariant methods (implicit kernels, learned equivariance) and for related invariant message-passing models. In the paper: Section 2.
The experiments are limited in scope, and the gains may be task-specific.
We extended the evaluation in four directions:
On task specificity, we agree that the gains depend on the task, and now say so: they are largest for orientation-free structures such as vessels and smaller for the closed membranes of ISBI. In the paper: Section 4.2, Conclusion.
How were f_θ and g_ψ chosen; was there hyper-parameter optimization, and how informative are the results?
f_θ and g_ψ are the smallest MLPs that keep the operator non-linear before and after the averaging step, following set and message-passing networks (DeepSets, PointNet, MPNN). We did not tune hyper-parameters for CRE or for the baselines: the baselines use their published configurations, and all models share the same training protocol, so the comparison is not tilted by tuning. To show that the results do not hinge on one configuration, we added a width study and repeated seeds. CRE is already competitive at a small size, improves with width, and the result of the paper model is consistent over three seeds. In the paper: Table 9, Figure 9 (Appendix B.4).
Efficiency (mentioned in the summary).
The efficiency limitation stated in the original submission is now addressed. We rewrote the operator exactly and implemented it with fused Triton GPU kernels; it now trains faster than Steerable and Platonic at large batch sizes. In the paper: Table 6, Appendix B.9.