acceptodds
Under review as a conference paper at ICLR 2027

CRE: Convolutional basis-free rotation-equivariant operator on grids

Abstract

Scientific and medical imaging often suffers from limited labeled data, making it important to exploit structure already present in the data. These images often contain geometric symmetries. In particular, rotation is a common nuisance factor: tissues, cells, and objects can appear at arbitrary orientations. Standard CNNs exploit translation symmetry but lack built-in rotation handling, while existing equivariant methods often rely on discrete rotation groups, predefined steerable bases, lifted orientation spaces, or non-standard image representations. We propose CRE, a local, basis-free convolutional block that models rotations directly on the native image grid without explicit orientation channels. CRE combines center- relative pixel interactions with invariant neighborhood attention, replacing fixed Cartesian kernel weights with geometry-conditioned local relations. We evaluate CRE on six benchmarks: three classification datasets and three segmentation datasets. In classification, CRE achieves the best mean accuracy across continuous rotations on all three datasets, while maintaining competitive variance over different angles. In segmentation, CRE remains competitive with baselines showing that it can be used as a practical simile for CNN-based pipelines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Equivariance, classification, segmentations, cnns, continous rotations, science datasets

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_fJTxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The goal of this paper is to construct and efficient rotation-equivariant network design that is 1) basis-free, 2) acts on the native image grid, and 3) does not rely on discretization of the rotation group, or (truncated) analytic filter bases. An additional goal is 4) a design that combines directly with existing CNN pipelines for classification and segmentation of 2-dimensional images. The proposed design, named CRE, obtains the goals 1-4.

At the core of the CRE design is a convolutional block which is translation equivariant, like a CNN, and whose local response is rotation equivariant. The latter is accomplished through a rotation-invariant learnable kernel which decomposes into central-pixel interaction and neighbourhood attention, whose translational and rotational equivariance is guaranteed by parameter-sharing across pixel positions and dependence only on rotation-invariant distances. The CRE model is compared via (limited) classification and segmentation experiments with several established rotational-equivariant designs (and a standard CNN as a non-equivariant baseline). In classification the CRE model is found to improve average performance, and in segmentation it is subleading, but still competitive, with the comparison models. Limitations in efficiency are stated and ideas for improvement are indicated.

Critical strengths

Competitive novel rotation-equivariant design with easily implementable convolutional blocks.

Critical weaknesses
  • The survey of related rotation-equivariant work is very short and excludes papers with partial overlap with the proposed architecture (e.g. using rotation invariant kernels), or that do not fall into the classes listed (e.g. learned equivariance). Examples include Zhdanov, Hoffmann & Cesa (2023), Implicit Convolutional Kernels for Steerable CNNs, NeurIPS 2023. Khetan et al. (2021), Implicit Equivariance in Convolutional Networks, arXiv 2111.14157

  • The experimental tests are limited in scope and the gained accuracy may be limited to certain tasks.

Questions for authors
  • The implementation of the convolutional blocks includes architectural choices, e.g. when modeling fθf_\theta and gψg_\psi as small multilayered perceptrons. How are these choices motivated? Have hyper parameter optimization studies been made, and if not, how informative are the results of your tests?

  • Why do you emphasize the non-equivariant networks in your review of related work? Wouldn’t it be more relevant to cover more of the proposals for rotationally equivariant networks?

2 comments
AuthorsOpenReview ↗

Response to Reviewer fJTx

We thank the reviewer for the positive assessment and the helpful references. Each point ends with where to find the change in the revised paper.

Related work misses kernel-based and learned-equivariance work (Zhdanov et al. 2023; Khetan et al. 2021).

We added both, and placed CRE relative to them.

  • Implicit steerable kernels (Zhdanov, Hoffmann & Cesa, 2023) also learn kernels with MLPs, but inside the steerable framework: with group representations, steerability constraints and orientation channels. CRE conditions only on rotation-invariant distances and outputs plain feature maps, so it replaces a convolution directly.
  • Implicitly equivariant networks (Khetan et al., 2021) learn equivariance through an extra training loss, so it holds only approximately and depends on training. CRE builds it into the operator: exactly for rotations by multiples of 90° and approximately for other angles.

We also added a paragraph on invariant set and message-passing models, to compare with ours. In the paper: Section 2.

Why emphasize non-equivariant networks in the related work?

Agreed. We shortened the general CNN paragraph and used the space for rotation-equivariant methods (implicit kernels, learned equivariance) and for related invariant message-passing models. In the paper: Section 2.

The experiments are limited in scope, and the gains may be task-specific.

We extended the evaluation in four directions:

  • Segmentation under rotation. The segmentation table now evaluates every model on test images rotated from 0° to 345°. CRE is best on DRIVE and competitive on ISBI and neuronal cells, and it is the only model whose predictions do not change at 90°, 180° and 270°. The training images already show membranes, vessels and cells at many orientations, which gives every model some robustness to rotation even without augmentation; even so, CRE has the lowest variance across angles in five of the six columns. In the paper: Table 2, Section 4.2; Appendix B.7.
  • Two new medical datasets without a canonical orientation. On BloodMNIST, CRE has the best mean accuracy over angles. On TissueMNIST it has the lowest variance and is second in mean accuracy, behind Steerable, which we report. In the paper: Tables 10–11, Figure 10 (Appendix B.5).
  • CRE inside a deep standard architecture. Replacing the convolutions of a ResNet-18 with CRE layers makes it far more robust to rotation, with a five times lower variance across angles. Its upright accuracy is lower; we show this is not under-training and attribute it to isotropic kernels, which cannot use edge orientation, a strong cue on upright CIFAR-10 images. In the paper: Table 12, Figure 11 (Appendix B.6).
  • Direct equivariance measurements and an expressivity test, comparing CRE with Steerable, G-CNN and a standard CNN on feature maps, and on synthetic tasks that depend only on angles. In the paper: Appendices B.2–B.3.

On task specificity, we agree that the gains depend on the task, and now say so: they are largest for orientation-free structures such as vessels and smaller for the closed membranes of ISBI. In the paper: Section 4.2, Conclusion.

How were f_θ and g_ψ chosen; was there hyper-parameter optimization, and how informative are the results?

f_θ and g_ψ are the smallest MLPs that keep the operator non-linear before and after the averaging step, following set and message-passing networks (DeepSets, PointNet, MPNN). We did not tune hyper-parameters for CRE or for the baselines: the baselines use their published configurations, and all models share the same training protocol, so the comparison is not tilted by tuning. To show that the results do not hinge on one configuration, we added a width study and repeated seeds. CRE is already competitive at a small size, improves with width, and the result of the paper model is consistent over three seeds. In the paper: Table 9, Figure 9 (Appendix B.4).

Efficiency (mentioned in the summary).

The efficiency limitation stated in the original submission is now addressed. We rewrote the operator exactly and implemented it with fused Triton GPU kernels; it now trains faster than Steerable and Platonic at large batch sizes. In the paper: Table 6, Appendix B.9.

Thanks for the clarifications. I am happy with the authors' rebuttal and revisions. Consequently, I will raise the soundness score.

Reviewer_BUBHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CRE, a basis-free rotation-consistent local operator that operates directly on regular image grids. CRE consists of two branches: center-relative pixel interaction and local attention that depends only on radial distance. It avoids discrete rotation groups, predefined steerable bases, and explicit orientation channels. The authors evaluate the method on three classification benchmarks and three medical/scientific segmentation benchmarks. In classification under continuous test-time rotations, CRE achieves higher mean accuracy and lower accuracy variance across angles than prior rotation-equivariant baselines. In segmentation, it is competitive with equivariant U-Net variants. The main claimed contribution is a rotation-stable module that can directly replace standard convolution in existing CNN pipelines.

Critical strengths

1.The operator design is conceptually simple and practical. It works directly on the native pixel grid and can be plugged into common CNN architectures such as ResNet and U-Net without modifying the feature dimension for orientation. This is a clear advantage over steerable CNNs and SE(2) networks that require orientation lifting.

2.The continuous-rotation evaluation protocol for classification is valuable. Testing over angles from 0° to 360° reveals the performance oscillation issue of discrete group CNNs. CRE achieves the best mean accuracy on all three classification datasets and generally provides a better mean-accuracy–angle-stability trade-off than most baselines. However, this should be stated more precisely: CRE does not have the lowest variance on every dataset. On CIFAR-10, SRE has lower variance (2.82) than CRE (34.16); on PathMNIST, Harmonic has lower variance (1.38) than CRE (1.83). Thus, the more accurate claim is that CRE offers the best mean-variance trade-off, not that it uniformly achieves the lowest variance.

Critical weaknesses

1.The paper needs to distinguish strict equivariance from empirical rotation stability more clearly, and the formal justification is insufficient. The paper repeatedly describes CRE as “rotation equivariant” or “rotation-consistent by construction.” However, Appendix A.1 only proves that a standard continuous convolution with a rotationally invariant kernel is rotation equivariant. It does not directly prove that CRE’s nonlinear center-interaction branch and local self-attention branch are equivariant in the continuous domain. On a discrete pixel grid, rotations also involve bilinear resampling, changes in the sampled square window, and changes in neighborhood correspondence, so exact SO(2)equivariance is not guaranteed. The authors should clarify whether CRE is strictly discretely equivariant, approximately equivariant in the continuous domain, or empirically rotation-stable. It would be helpful to provide a discrete rotation operator definition, an error bound, or a quantitative experiment measuring feature-map alignment error as a function of rotation angle. If exact equivariance cannot be proved, the corresponding claims should be weakened.

2.The segmentation evaluation omits the controlled rotation-consistency test that is most directly related to the paper’s core claim. Section 4.2 explicitly states that segmentation does not use a rotation protocol and instead relies on mixed orientations already present in the datasets. Since the central selling point of CRE is rotation consistency, the absence of a controlled rotation test for dense prediction means it remains unverified whether this property transfers to pixel-level prediction. I recommend adding a rotation-angle sweep and reporting the mean and variance of BCE/ROC, as well as ISBI’s VrandVrand and VinfoVinfo, using the same metrics as in the paper; if applicable, mIoU could also be added.

3.A rotation-augmentation baseline is missing. All models are trained without rotation augmentation. It would be useful to add a standard CNN/U-Net trained with rotation augmentation as a baseline under a comparable training budget, and ideally also report CRE with augmentation. This would help separate the benefit of the architectural design from the common data-augmentation strategy and would indicate whether CRE and augmentation are complementary.

4.Segmentation performance is not uniformly competitive and should be discussed more carefully. On ISBI 2012, CRE is worse than the standard U-Net on BCE (0.3265 vs. 0.3053), ROC (0.9363 vs. 0.9484), VrandVrand (0.8697 vs. 0.9048), and VinfoVinfo (0.5824 vs. 0.6827). On the fluorescent neuronal cells dataset, CRE’s ROC (0.9936) is slightly lower than Steerable’s (0.9944). The paper should avoid overgeneralizing and should analyze when and on which structures CRE provides benefits, and when it does not outperform baselines.

5.The efficiency comparison mixes algorithmic cost with implementation cost. Table 6 shows that CRE has a moderate parameter count but higher peak memory and training time. Appendix A.5 acknowledges that the current implementation is not optimized. I recommend reporting theoretical FLOPs or a decomposition of memory cost, or providing an optimized/fused-kernel implementation, to distinguish intrinsic operator complexity from current implementation overhead.

Questions for authors

1.How do you define rotation equivariance on a discrete grid? After rotating the input with bilinear interpolation, is the requirement that the inverse-rotated output match pixel by pixel, or only that performance be statistically stable under rotation? Do the two CRE branches have a formal equivariance proof in the continuous domain? How does the discrete error vary with rotation angle and kernel-window size?

2.Why is the controlled rotation test omitted for segmentation? Can you add a rotation-angle sweep and report the mean and variance of BCE/ROC and, for ISBI, VrandVrand and VinfoVinfo (and mIoU if applicable)?

3.Can you add standard CNN/U-Net with rotation augmentation as a baseline and clarify the benefit of CRE relative to augmentation and whether the two are complementary?

4.In Table 1, CRE does not have the lowest variance on CIFAR-10 or PathMNIST. How do you reconcile this with the claim of “minimal accuracy variance”? Please discuss the relationship between CRE and methods such as SRE and Harmonic that have lower variance but much lower mean accuracy.

5.How much of the current runtime/memory overhead comes from the theoretical operator complexity, and how much comes from the unoptimized implementation? Can you provide FLOPs or a comparison after implementation optimization?

1 comment
AuthorsOpenReview ↗

Response to Reviewer BUBH

We thank the reviewer for the constructive review. Each point ends with where to find the change in the revised paper.

Strength 2 / Q4: CRE does not have the lowest variance everywhere.

Agreed. We now claim the best trade-off between mean accuracy and variance, not the lowest variance. SRE on CIFAR-10 and Harmonic on PathMNIST reach a lower variance only at a clearly lower mean accuracy. In the paper: abstract, Section 4.1, caption of Table 1, Conclusion.

W1 / Q1: Strict equivariance vs. empirical stability; definition, proof, and error vs. angle and kernel size.

  • Definition. We use the pixel-wise definition: rotating the input and rotating the output back must give the original output, not just a stable score.
  • Proof. Both branches only see a window as a set of (center, neighbor, radius) triples, aggregated by a mean and by attention pooling, which depend on the input but not on the order of the triples. In the continuous domain with a circular window, any rotation only reorders this set, so the layer is exactly rotation-equivariant. On the grid, the square window is only preserved by rotations by multiples of 90° and reflections, so each layer, and the full network with its pooling, is exactly equivariant to these; other angles require interpolation and are only approximately equivariant. We replaced "by construction" with this statement.
  • Error vs. angle and kernel size. Instead of a bound, we measure the error directly at every angle, block by block, before and after training. It is at float precision at multiples of 90°. At other angles a single CRE layer stays at the interpolation error for kernel sizes 3, 5 and 7, whereas a standard convolution is far above it; in trained networks the error grows with depth.

In the paper: Proposition 2 (Appendix B.1); Table 7, Figures 5–6 and kernel sizes (Appendix B.2); Section 3.

W2 / Q2: Controlled rotation sweep for segmentation.

Added, and it is now the main segmentation table. Each test patch is rotated with its real surrounding image from 0° to 345°, and every angle is scored on the same central disc. We report the mean and variance over angles of ROC-AUC and Dice, V_rand for ISBI, and the pixels whose label changes under rotation. CRE has the lowest variance in almost all cases and is the only model that changes no pixel at 90°, 180° and 270°. We also computed BCE and V_info. CRE has the lowest BCE variance on all three datasets and the best BCE on DRIVE. On ISBI, its V_info, like its V_rand, is below U-Net and Steerable. In the paper: Table 2 and Section 4.2; protocol and per-angle curves in Appendix B.7 (Figure 12).

W3 / Q3: Rotation-augmentation baseline; are the two complementary?

We trained the standard CNN with random rotations. It reaches a higher mean accuracy over angles than CRE without augmentation, and we report this, including with reduced training data. We did not train a U-Net or CRE with augmentation, because our study isolates what the operator gives without augmentation. The two are complementary, not competing. Augmentation is learned, holds only on average at the output, and requires simulating the transformation. CRE is exactly equivariant in every layer by construction, which dense prediction benefits from (its segmentations do not change at multiples of 90°), and it keeps a higher upright accuracy on CIFAR-10 and PathMNIST. Augmentation can be added on top of CRE. In the paper: Table 13, Figure 13 and discussion (Appendix B.8); pointer in Section 4.1.

W4: Segmentation is not uniformly competitive.

Agreed. We now state that CRE is clearly best on DRIVE and competitive on ISBI and neuronal cells, where U-Net, Steerable and SE2CNN are ahead on some metrics. CRE helps most on thin, orientation-free structures such as vessels; ISBI membranes form closed contours, which makes the metrics sensitive to small gaps (Figure 4). The training images also show these structures at many orientations, which gives every model some robustness to rotation even without augmentation; even so, CRE has the lowest variance in five of six columns (W2). Some of the original neuronal-cell models had been trained with flips and 90° rotations, so we retrained all of them without augmentation. In the paper: Table 2, Section 4.2, Conclusion.

W5 / Q5: Intrinsic vs. implementation cost.

Most of the cost came from the implementation. Both branches can be rewritten efficiently. Our original plain PyTorch code instead stored every k×k window explicitly. We implemented the rewritten operator with fused Triton GPU kernels that compute the same function and load the same checkpoints. Compared with the original, it uses an order of magnitude less memory, allows batches more than ten times larger and trains almost 20× faster at batch size 256, now faster than Steerable and Platonic. In the paper: Table 6 (both implementations, Appendix A.5); Appendix B.9.

Reviewer_Z8MkOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a long-standing problem of models with rotational equivariance in 2D images. The paper presents CRE, a mixture of ideas from convolutional graph neural nets and equivariant transformers. I see somewhat outdated equivariant references in this domain, and this is probably explained by the community accent shift into 3D vision and particle simulations. I strongly advise the authors to look into the 3D domain and adapt their architecture to 3D medical images (medical MNIST, for example) and also particle simulations.

Critical strengths

-- The paper presents a somewhat easy to understand and implement approach and can be adapted to various data domains including particle simulations.

Critical weaknesses

-- The paper overlooks important developments in the 3D domain and does not try to improve somewhat limited expressivity of the central cell convolution (see below), as it's built around representing radial distributions of pixels.

-- The benchmark comparison is not very rigorous, as many equivariant baselines have significantly few number of parameters. Also, on the segmentation benchmarks there is no clear advantage of the proposed architecture over all tasks. Finally, the training splits shall be explained better.

Questions for authors

-- My main criticism concerns representation learning, as you are using an oversimplified encoding of distance-dependent pair interactions between the pixels with a mean pool aggregation that can loses very important higher-order information. For example, a single CRE local encoder of the central pixel would not distinguish configurations where other pixels are randomly swapped / rotated around the central one by 90/180/-90 degrees. Including additional angular information, angular basis functions or higher-order geometric moments would naturally improve the expressivity but you don't discuss it neither present the SOTA models from the 3D field.

-- I advise you to take a look and discuss models that incorporate additional angular encoding, this can be PPFNet, DimeNet/DimeNet+, GemNet, PaiNN etc. Also look closer to NequIP and MACE and consider adapting the method for 3D particle/image data.

-- It is quite straightforward to improve the expressivity of the central convolution by replacing/extenfing it with harmonic Nets and adapting the transformer for the new equivariant output. Why you didn't consider it?

-- The paper trains on patches, which increases the number of training examples, but patches from the same original image are not independent samples. The split should therefore be explicitly confirmed in all the experiments.

-- The appendix states that the checkpoint with the lowest training loss is selected. Why so? Why not a validation loss?

-- The proposed CRE architecture has a substantially larger parameter count than most of the compact equivariant baselines, including Harmonic Networks, G-CNN, SRE, and SE2CNN. Consequently, the observed improvements in classification accuracy cannot be attributed unambiguously to the proposed radial interaction and neighborhood attention mechanisms. Parameter-matched experiments and performance-versus-capacity curves would improve the understanding the CRE capacity. Can CRE offer a better accuracy–rotation-consistency trade-off at equal parameter count and equal computational budget? Can you please provide additional experiments by reducing the number os layers or the hidden dimension to match the FLOP / parameter count of simpler baselines, most importantly, Harmonic Networks?

-- You also say "rotation-consistent by construction" - do you mean that the operator is rotation invariant? What "consistency" mean here?

-- Maybe you can draw some explicit analogies with graph convolutions? For example, how the proposed central pixel interaction is different from a graph convolutional nets (with aggregation and activation functions provided by eq 5)?

-- The paper measures the variance of classification accuracy over rotation angles as the main stability metric. It will be very useful to look at the stability of feature maps under the rotation more directly.

-- The test-time rotations are controlled transformations, not realistic distribution shifts. The 5 deg rotation is still finite. The transformations do not capture other sources of variation encountered in actual imaging, such as scale, translation, blur, noise, acquisition differences, staining variation, non-rigid deformations, etc. Thus more realistic test sets , e.g. from ISBI / DRIVE / Neuronal cells will be very useful to evaluate the method. A more convincing evaluation would combine controlled rotation-equivariance measurements with source-image-level splits and broader imaging distribution shifts.

--You may also include a synthetic benchmark specifically designed to test angular expressivity. You do not establish whether the radial-only representation preserves all the local geometric information needed by those tasks.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Z8Mk

We thank the reviewer for the detailed questions and the pointers to the 3D literature. Each point ends with where to find the change in the revised paper.

Expressivity of the radial encoding and a synthetic benchmark.

Correct for a single layer, and intended: the local kernel depends only on distances. Across layers, distances from different centers determine the angles between them, so a deeper network can recover angular information. We created a new experiment in Appendix B.3 ("Synthetic benchmark for angular expressivity") with images whose class depends only on the angle between blobs: CRE learns these angles and keeps them under rotation as well as Steerable. It cannot represent handedness, since a pattern and its mirror image have identical distances; we now list this as a limitation. In the paper: Figures 7–8, Table 8 (Appendix B.3); Proposition 2; limitations (Section 5).

3D models with angular encodings; why not harmonic extensions?

We added these works to Section 2. They gain expressivity by encoding directions or orientation-dependent features. CRE avoids this on purpose, so that its features stay plain image tensors and it can directly replace a convolution. Adding circular harmonics would turn CRE into a steerable or harmonic network, with orientation channels and a truncated basis. Orientation-free angular features, such as angles between neighbors, and a 3D version could be future work. In the paper: Section 2; Section 5.

Parameter- and compute-matched comparison (vs. Harmonic Networks).

We added a CRE with about as many parameters as Harmonic, and a capacity curve. On CIFAR-10, the small CRE still outperforms Harmonic and every baseline, including the much larger Steerable and Platonic. On MNIST and PathMNIST it is below Harmonic. Accuracy grows with width, and the paper model is consistent over three seeds. It also needs fewer multiply-accumulates per pixel than a 5×5 convolution (Table 6). In the paper: Table 9, Figure 9 (Appendix B.4); Table 6 and Appendix B.9.

No clear advantage on all segmentation tasks.

Agreed. Table 2 now evaluates every model on rotated test images. CRE is best on DRIVE and competitive on ISBI and neuronal cells, with the largest gains for orientation-free structures such as vessels. The training images already show these structures at many orientations, which makes every model somewhat robust to rotation; even so, CRE has the lowest variance in five of six columns and is the only model whose predictions do not change at multiples of 90°. In the paper: Table 2, Section 4.2, Conclusion.

Splits with patches; checkpoint selection.

All segmentation splits are made on whole images before patching (official ISBI volumes; 80/20 of images for DRIVE and neuronal cells), so no test patch is seen in training. Classification uses the official splits, now stated correctly in the appendix. We select checkpoints by training loss as there is no validation split; the test set is never used, and every model follows the same rule. In the paper: Table 3 and Appendices A.2–A.3.

"Rotation-consistent by construction": is it invariance? Measure stability directly.

Both, at two levels. The local operator is invariant: its response depends only on the center, the neighbors and their distances, so rotating the window about its center does not change it. Applying it at every pixel, like an isotropic convolution kernel, makes each layer equivariant: feature maps rotate with the input. Global pooling then makes the classifier invariant. On the grid, this is exact for multiples of 90° and reflections, and approximate otherwise. We replaced "by construction" with these statements and now measure the error directly, block by block: it is close to Steerable and far below G-CNN and a CNN. In the paper: Proposition 2 (Appendix B.1); Section 3; Table 7, Figures 5–6 (Appendix B.2).

Relation to graph convolutions.

The central branch (Eqs. 4–5) is similar to message passing on a graph linking each pixel to its k×k window, but differs from a graph convolutional network (GCN) in three ways. (i) A GCN transforms neighbors linearly and applies the activation after summing; CRE applies an MLP to each (center, neighbor, distance) triple before averaging, so it models pairwise interactions, and a second MLP after it. (ii) A GCN only knows which pixels are connected; CRE also knows their distance, but not their direction, which keeps it locally rotation-invariant. (iii) It runs densely on the grid as a drop-in convolution. In the paper: Section 2.

Realistic distribution shifts.

Rotation is deliberately the only controlled factor, since it is the property CRE claims; we now state this scope. We added two medical datasets without a canonical orientation: CRE has the best mean accuracy on BloodMNIST and the lowest variance on TissueMNIST. In the paper: Section 5; Tables 10–11 (Appendix B.5).

I thank the authors for addressing my points. The time is very short, but maybe you have a PDF version of the updated manuscript with difference highlighted?

A quick follow-up question : in steerable networks (Steerable/Harmonic) the order of the filter must normally correspond to the resolution of the data. In principle, you can match the rotation resolution of 5 deg to the filters' maximum order (the expansion order). Did you check this? (if the max. expansion order matches the rotation resolution in your experiments)?

Reviewer_5vT6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CRE, a local image-grid operator intended to improve rotation consistency without explicit orientation channels or predefined steerable bases. It combines shared center–neighbor interactions, conditioned on radial distance and aggregated by averaging, with a neighborhood attention branch using radial positional embeddings. CRE is evaluated on classification and segmentation tasks. The reported classification results are promising. Segmentation results are mixed, with a particularly strong result on DRIVE.

However, the current theoretical analysis and evaluation do not sufficiently establish continuous rotation equivariance of the proposed discrete operator. I have addittional concerns regarding the attention formulation and the interpretation of rotation-consistency measurements.

Critical strengths
  • The proposed method combining shared local interactions with radial geometry provides an understandable inductive bias, while retaining spatially aligned feature maps. Hence the motivation of proposed method is clear and intuitive

  • The network is evaluated across two tasks - classification and segmentation. The segmentation experiments demonstrate that the operator can produce dense predictions. The DRIVE results are encouraging, although the advantage does not extend consistently across all segmentation datasets.

Critical weaknesses
  • The theoretical results in the Appendix don't fully establish that the actual implementation of CRE is equivariant. While the math shows that a continuous linear convolution with a rotationally invariant kernel is rotation equivariant, the actual operator is nonlinear, input-dependent, and evaluated over discrete square neighborhoods. The paper doesn't really explain how the theoretical proposition applies to either branch or their combination.
  • Measuring the variance of aggregate classification accuracy across rotation angles (in Section 4.1) only proves robustness. Just because the overall accuracy stays the same doesn't mean individual predictions or intermediate features aren't changing different examples could be succeeding and failing while the average holds steady. Section 4.2 also skips controlled rotations for segmentation. Relying on naturally occurring orientation diversity isn't enough to prove that predictions transform consistently when you rotate the exact same image. Showing the direct discrepancies between F(Tθx)F(T_\theta x) and TθF(x)T_\theta F(x), along with the specific interpolation and boundary handling, would really help address this gap.
  • Section 3.2 suggests that attention captures interactions between different neighbors, Equation 9 only shows a single attention operation that extracts the class-token output. Since this output only depends on the class-token query and the input keys and values, the updated neighbor-token outputs are discarded and don't actually contribute within the same layer. Softmax normalization does couple the aggregation weights, but this isn't the same as using contextualized neighbor representations.
  • The segmentation results suggest that performance really depends on the task. While CRE performs strongly on DRIVE, it actually falls short of U-Net on all four reported ISBI metrics. Similarly, the claim that attention provides lower rotational variance doesn't hold up across all datasets; on MNIST, the attention-only variant actually shows substantially higher variance than either the central branch or the full CRE model.
Questions for authors
  • What exact equivariance guarantee is being claimed here? Could you also provide an argument for the nonlinear CRE operator that specifically accounts for things like square support, sampling, interpolation, and pooling?
  • Could you provide the direct equivariance errors for both individual CRE blocks and the complete networks? It would be helpful to see this both before and after training, and across both lattice-preserving and intermediate rotation angles.
1 comment
AuthorsOpenReview ↗

Response to Reviewer 5vT6

We thank the reviewer for the careful reading. Each point ends with where to find the change in the revised paper.

W1 / Q1: What exact equivariance guarantee is claimed, for the nonlinear operator on a discrete grid?

Both branches see a window only through the set of (center, neighbor, radius) triples: the central branch averages a shared MLP over this set, and the attention branch pools it with softmax weights. The guarantee, point by point:

  • Square support and sampling. A rotation by a multiple of 90° (or a reflection) maps the square window onto itself, only reorders the triples and keeps every radius. Each CRE layer, including its non-linearities and reflect padding, is therefore exactly equivariant to the eight symmetries of the grid.
  • Pooling. Layer normalization, 2×2 pooling on even-sized maps and global pooling commute with these symmetries, so the full network is exactly equivariant (segmentation) or invariant (classification) to them.
  • Interpolation. For other angles the rotated image must be interpolated on the grid, so CRE is only approximately equivariant. The error comes from this resampling, not from the learned weights, and we measure it directly (W2).

We replaced "equivariant by construction" with this statement and keep the original proposition only as the continuous-kernel motivation. In the paper: Proposition 2 with proof (Appendix B.1); end of Section 3.

W2 / Q2: Direct equivariance errors instead of accuracy variance, and a controlled segmentation test.

We agree that a stable average accuracy does not imply stable individual predictions. We added two direct measurements.

  • Feature maps, block by block, before and after training. We rotate the input, rotate each feature map back, and measure its relative difference to the original map, for the full CIFAR-10 networks and for single layers. Images are mean-padded so that rotation never cuts content, and the error of interpolating the same map twice serves as a reference floor. At multiples of 90° the error is at float precision in every block, before and after training. At intermediate angles it equals the interpolation floor at initialization and grows with depth after training. It is comparable to Steerable and far below G-CNN and a standard CNN, and CRE keeps its prediction under rotation more often than all three. In the paper: Table 7, Figures 5–6 (Appendix B.2).
  • Segmentation under controlled rotation. Each test patch is rotated with its real surrounding image (bilinear interpolation, no padded pixels) from 0° to 345°, and every angle is scored on the same central disc. Besides ROC-AUC, Dice and their variance, we count the pixels whose label changes once the prediction is rotated back (Flipped). Averages can stay constant while individual pixels change; Flipped measures exactly that, without ground truth. CRE flips no pixel at 90°, 180° and 270° on all three datasets, while every baseline does. In the paper: Table 2 and Flipped definition (Section 4.2); protocol, formula and per-angle curves (Appendix B.7, Figure 12).

W3: The attention only uses the CLS output.

The reviewer is right. The branch is attention pooling: neighbors interact only through the jointly normalized softmax weights, not through contextualized neighbor tokens. Only the query is constant (the class token is a learned parameter); the keys and values come from the window, so the output depends on the input. It is, however, invariant to permutations of the window's pixels, and rotating the window about its center only permutes pixels of equal radius. The output at each pixel is therefore invariant to rotations of its window, which makes the layer equivariant (W1). The constant query also allows the efficient implementation. In the paper: Section 3.2, caption of Figure 2 (corrected); Appendix B.9.

W4: Segmentation results depend on the task; the attention-variance claim does not hold on MNIST.

We agree with both points.

  • Segmentation. On ISBI, CRE is still below U-Net and Steerable in accuracy: ISBI membranes form closed contours, and metrics such as V_rand penalize the small gaps CRE sometimes opens (Figure 4). CRE is clearly best on DRIVE and competitive on neuronal cells. The training images already show membranes, vessels and cells at many orientations, which gives every model some robustness to rotation without augmentation; even so, CRE has the lowest variance across angles in five of six columns. We now state that the gains are largest for thin, orientation-free structures such as vessels. In the paper: Table 2, Section 4.2, Conclusion.
  • Ablation. We removed the claim. Attention alone has the lowest variance on CIFAR-10 but a high one on MNIST, so neither branch alone is consistently stable; only their combination is both accurate and stable on all three datasets. In the paper: ablation paragraph, Section 4.1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.