acceptodds
Under review as a conference paper at ICLR 2027

Equivariance and augmentation for Bayesian neural networks

Abstract

Symmetries are important for many deep learning tasks, ranging from applications in the sciences to medical imaging. However, there is an ongoing debate about whether to impose symmetry constraints on the neural network architecture (yielding equivariant neural networks) or learn them from augmented training data. Although equivariant networks are well-studied theoretically, much less is known about data augmentation, since analyzing augmentation requires control over the training dynamics. Inspired by recent results showing that augmented infinite deep ensembles are exactly equivariant, we study data augmentation for Bayesian neural networks (BNNs) trained with variational inference. We focus on variational distributions in the exponential family and derive conditions under which exact equivariance is reached. We furthermore obtain bounds on the equivariance error and introduce three novel symmetrization techniques which boost the effect of data augmentation in this setting. We conduct extensive numerical experiments which show that one of our symmetrization methods (orbit expansion) outperforms the baseline in both equivariance and overall performance. Our code is available at https://anonymous.4open.science/r/augment-BNNs-6D84this anonymous repository.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
equivariance, data augmentation, Bayesian neural networks, variational inference, exponential family

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_My9DOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work studies data augmentation for Bayesian Neural Networks trained with variational inference, as an alternative to equivariant architectures. Working within exponential family variational distributions, the authors derive conditions under which augmented training keeps an invariant subspace of natural parameters corresponding to equivariant predictive maps. They also derive bounds on the equivariance error under finite sampling. They introduce three symmetrization strategies (geometric averaging, projection, and orbit expansion) designed to place or keep the model within that invariant subspace. Experiments on FashionMNIST under C4C_4 augmentation show that orbit expansion outperforms the baseline in both equivariance and classification accuracy.

Critical strengths
  1. The theoretical framework is rigorous and well constructed, deriving conditions under which augmented training preserves the invariant subspace of equivariant parameters.
  2. The connection to deep ensembles being exactly equivariant is well motivated, framing Bayesian neural networks as a cheaper alternative is compelling.
  3. Orbit expansion outperforms the baseline in both classification accuracy and equivariance metrics across all trigger timings, demonstrating that the framework can yield practical improvements when the symmetrization strategy is well chosen.
Critical weaknesses
  1. While the paper is primarily theoretical, the experiments should validate that the theoretical framework translates to practice. This validation is only partially convincing. Orbit expansion consistently outperforms the baseline, supporting the theory, but geometric averaging and projection do not improve accuracy over the baseline. For a paper arguing that symmetrization strategies grounded in the theoretical framework are beneficial, this is a concern that deserves explanation. Are these methods failing due to theoretical reasons, or implementation choices?

  2. The experimental validation is limited to FashionMNIST with C4C_4 symmetry, which is the simplest possible setting. For a theory paper, this is understandable, but the theoretical contributions are general enough that at least one additional setting would significantly strengthen the empirical validation. It currently remains unclear whether the theoretical guarantees translate to more complex symmetries or domains.

  3. The restriction to finite groups is a significant theoretical limitation that the paper acknowledges but leaves for future work. Given that many practically relevant symmetries are continuous, such as arbitrary rotations in medical imaging or 3D symmetries in molecular property prediction, this limits the scope of the current contribution. Some discussion of the conceptual obstacles, or lack thereof, to extending the framework would strengthen the paper.

Questions for authors
  1. The best results are obtained when orbit expansion is triggered at epoch 100. Why was a higher trigger epoch not explored, and would further delaying the intervention improve results?
  2. Are there other symmetrization methods that could fall within this theoretical framework and potentially outperform orbit expansion?
  3. The experiments are limited to FashionMNIST. How difficult would it be to apply this framework to other domains, and do the authors expect similar gains in, for example, medical imaging or molecular property prediction?
  4. Some forms of augmentation, such as adding noise, do not correspond to invertible group transformations and therefore fall outside this framework. How significant is this restriction in practice, and are there plans to extend the framework to cover such cases?
  5. The closure condition required by Theorem 1 restricts the usable variational families and transformations. How difficult would it be to verify or satisfy this condition in new settings, and are there practical guidelines for doing so?
  6. Geometric averaging and projection do not outperform the baseline in accuracy. Is this expected from the theory, or is it a limitation of the experimental setup? Are there settings where these methods would be expected to perform better?
1 comment
  • Projection and geometric averaging are struggling. Why?

Prompted by the reviewers question, we have revisited this issue and have identified two likely factors.

First, when evaluating the strategies, one needs to take into account the nature of the equivariant space HGH_G. They differ between the approaches, since they depend on the representation used — geometric averaging uses ρch\rho^{\mathrm{ch}}, whereas projection and orbit expansion uses ρgcnn\rho^{\mathrm{gcnn}} (see Appendix H). We have added a remark about this. Now, the spaces have distinct natures — the filters in HGchH_G^{\mathrm{ch}} must be individually rotation invariant, reducing their expressivity compared to the filters in HGgcnnH_G^{\mathrm{gcnn}} which are of arbitrary shapes but related to each other. This makes it plausible that approaches based on HGchH_G^{\mathrm{ch}} fare worse.

Secondly, we realized that projecting / averaging takes away energy from the weights, and hence change their scaling of the weights. In the updated experiments, we therefore rescale the weights after projecting. With this modification, they fare much better - they then achieve comparable or better accuracy compared to the baseline. For more details on this, see the global comment about the updated experiments. We do however not reach the performance of orbit expansion. It hence seems that the projection operation alters the weights more drastically than the orbit insertion, and the training hence starts in a region of the parameter space further from optimal, from which it cannot recover. See also the discussion in Appendix K.

  • More challenging datasets

In the updated experiments section, we also test our approach on histological data. See the global comment for more information.

  • Restriction to finite groups

While we agree that we leave the full details of the infinite group case to future work, we have outlined concrete steps in this direction in Appendix A.

  • Timing of expansion trigger

We have in fact explored later triggers in Appendix K, and triggering at epoch 150 gave no gain over epoch 100. With a fixed budget of 500 epochs, a later expansion trades off Stage-2 training of the full-width network against longer Stage-1 training of the smaller network, and an interior optimum is therefore to be expected.

  • Other symmetrization techniques

The theoretical framework allows for any symmetrization strategy to be applied — the central insight is that as soon that we have made sure that η∈HG\eta \in H_G, the training dynamics that we consider will not move away from HGH_G. We do not rule out that there are symmetrization techniques that lead to better results, in particular for other groups, and we do not claim that the methods we have proposed are optimal.

We actually tried one more strategy in (unpublished) experiments — simply initializing with η=0\eta=0. This lead however to very bad results, so we did not include them in the manuscript.

We want to emphasize that the symmetrization strategy of Nabarro et. al. that we consider in the updated experimental section is not an example of a strategy of enforcing η∈HG\eta \in H_G. That strategy rather applies symmetrization directly to the function x↦Fη(x)x\mapsto F_\eta(x). Through the compatibility assumption, this can be reinterpreted to be a symmetrization of the distribution q↦1/∣G∣∑g∈GTg#qq\mapsto 1/|G| \sum_{g\in G} \mathcal{T}_g \# q, but this transformation in general yields a distribution that is not in the exponential family any more — a Gaussian will typically be converted to a multimodal distribution, for instance.

  • Augmentation not based on groups

When the augmentations considered do not form a group, the mathematical formalization will become very different. We do not expect our results to easily generalize.

  • The closedness conditions

For any given exponential family, the applicability can be checked explicitly by verifying that (11) and (12) hold for the transformation considered.

We note that Gaussian families, that are very popular in practice, are typically included. More generally, closure holds for permutation actions (e.g., in G-CNNs and under rotations, reflections, and channel permutations) when using product exponential families, such as Gaussian or Laplace. For general orthogonal actions, suitable choices include isotropic or full-covariance Gaussian families. In contrast, such as families with coordinates-dependent structure, such as fixed sign patterns or positivity constraints, that need to be avoided. We have discussed this in Appendix J, where we have also tested training an exponential family that does not fulfill the closedness condition. As expected, it does not have the same equivariance behavior as when the closedness condition is satisfied.

Reviewer_cdFYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how data augmentation can induce equivariance in Bayesian Neural Networks. It takes a variational approach based on the exponential familyand show conditions under which the training dynamics preserve group symmetries and achieve equivariance. The manuscript also derives bounds for the effect of non-invariant priors and finite-sample approximation on the equivariance defect. It explores three symmetrization strategies with empirical validation on Fashion-MNIST image classification.

Critical strengths

The paper is well-motivated and provides a solid theoretical framework. It establishes conditions under which invariances are preserved during optimization along with additional results concerning the influence of non-invariant priors and finite-sample bounds on equivariance defects. Essentially, the theoretical analysis shows that, under suitable conditions, the invariant parameter subspace is preserved under gradient descent optimization when the variational parameters are initialized within this subspace. Building on this result, the paper presents practical methods of constructing invariant variational parameters. This provides a clear connection between the theoretical results and the proposed methods.

Critical weaknesses

The main theoretical results of the paper appear to be direct consequences of [Nordenfors et al., 2025], once Theorem 1 is established (particularly Theorems 2--3). In fact, under the volume-preserving hypothesis of Assumption 2, most finite dimensional results from [Nordenfors et al., 2025] appear to be directly applicable to the setting of the manuscript. A clear discussion of this close relation (similarities and differences) is missing from the manuscript. No discussion is also provided on the constants CdataC_\text{data} and CweightC_\text{weight} from Theorem 4 (from the proofs it would appear to depend on the magnitude of ff, which makes sense given the concentration inequalities used). This is crucial given that they affect training complexity, a substantial issue for equivariant models.

This leads to a major issue with the manuscript: connection to existing literature. In particular, the methods put forward in Section 3.4 are well-known in the literature on equivariant models. In fact, they are the key means by which such models are constructed. Consider [Cohen & Welling, 2016; Kondor & Trivedi, 2018], but also [Chen et al., A group-theoretic framework for data augmentation. JMLR, 2020; Satorras et al., E(n) equivariant graph neural networks, 2021; Brandstetter et al., Geometric and physical quantities improve E(3) equivariant message passing, ICLR, 2022; Deng et al., Vector neurons: A general framework for SO(3)-equivariant networks, ICCV, 2021] among others. A discussion of this close relation (similarities and differences) is missing from the manuscript. The manuscript also fails to compare to other data augmentation studies of Bayesian neural networks, such as [Nabarro et al., Data augmentation in Bayesian neural networks and the cold posterior effect, UAI, 2022].

This leads to a key weakness of the paper: its experimental validation. The results are not compared to any alternative data augmentation framework, including [Nabarro et al.], or Bayesian group-equivariant architectures, those mentioned above or others. This is crucial given that a key challenge of equivariant models is that they are extremely sensitive to training conditions. In fact, it is well-known that equivariant models are hard to train [Pertigkiozoglou et al., Improving equivariant model training via constraint relaxation, NeurIPS, 2024; Manolache et al., Learning (Approximately) Equivariant Networks via Constrained Optimization, NeurIPS, 2025]. Providing alternative baselines is therefore crucial, otherwise it is impossible to know how good or effective the proposed solution is.

What is more, the main experiments focus on a relatively small convolutional architecture and only on Fashion-MNIST. Although this is a suitable controlled setting for validating the theory, it is not sufficient to establish the scalability or applicability of the proposed techniques. Experiments involving more challenging datasets, deeper architectures, and more complex group actions (such as those provided in the aforementioned works) would help determine whether the observed improvements generalize well.

Questions for authors
  • What is the exact relation of the proof techniques used to establish Theorems 2-3 to those from [Nordenfors et al., 2025]?
  • Is there a specific reason why do the experimental evaluations do not comparisons to existing group-equivariant architectures? Or alternative Bayesian data augmentation methods? Or non-Bayesian NNs? Or larger, more complex architectures and datasets?
3 comments

We thank the reviewer for their careful reading of the paper and constructive comments. We address each of the questions and/or weaknesses below.

  • Relation to [Nordenfors et. al, 2025]

We agree with the reviewer that the proof technique has many similarities to the techniques in Nordenfors et. al. The main technical difference in the proof of Theorem 2 compared to their approach is indeed encapsulated in Theorem 1. In the setting of Nordenfors et al, the closedness condition of the parameters is easy to check — here, one needs to show the invariance of a family of probability measures instead. We have added a comment.

Theorems 3 and 4, and the strategies to ensure an initialization in HGH_G from sections 3.4 are however unique to the Bayesian setting, and do not even have an analogue in the non-Bayesian setting of Nordenfors et. al.. Those contributions are hence completely independent of Nordenfors et. al.

  • The values of the constants

As the reviewer points out, the bound on the constants given by the proof only depends on the magnitude of ff. We have added a subclause to the statement of the theorem.

We would like to rectify a potential misunderstanding here: Theorem 4 has little to do with training complexity. It is rather about how many Monte Carlo samples of θ\theta are needed to realize a low equivariance defect of the true mean FηF_\eta by an empirical estimate F^η\widehat{F}_\eta. Also note that Theorem 4 does not assume that FηF_\eta is equivariant, it holds regardless of whether the ‘true mean equivariance defect’ ΔFeq(η)\Delta_F^{\mathrm{eq}}(\eta) is zero or bigger than zero.

  • Suggested references on geometric deep learning

While our constructions of course take inspiration from the extensive literature on geometric deep learning, our contributions go beyond the listed references in important ways. None of them are treating Bayesian neural networks, which is the motivation of our work, and hence in particular are not addressing the problem of projecting/initializing a probability distribution to be GG-invariant.

In particular, we would like to point out that our approach is only very tangentially related to the Vector Neuron approach, that only applies to SO(3)\mathrm{SO}(3)-equivariance. The other references are more relevant, but in our view not more than as examples of seminal works in geometric deep learning. While we do not discuss the relation to previous deep learning approaches at length, we do explicitly point out that our orbit expansion method is inspired by the by now classical GCNN-framework of Cohen and Welling.

  • The relation to Nabarro et al

We thank the reviewer for pointing out this reference. It proposes an alternative way of defining the likelihood to take into account that augmented training examples are not independent. In short, the proposed method in that paper is to symmetrize the predictive maps p(y∣x)p(y|x) before computing the logarithm. This clearly yields an equivariant map but at a high computational overhead since several inference steps (one for each augmentation) of the network are necessary to calculate a single Fη(x)F_\eta(x).

In the updated manuscript, we added a remark where we point out that we are analyzing the usual augmentation strategy that does not take dependencies between the augmented samples into account. We also add the method proposed in Nabarro et al. as a baseline in the experiments — see the global comment.

  • Challenges of training equivariant models

Note that the only step we perform to induce equivariance is that we enforce η∈HG\eta \in H_G once. Our main result is that the equivariance then comes ‘for free’ through data augmentation! This is in stark contrast to traditional exactly equivariant approaches, where η∈HG\eta\in H_G is explicitly enforced throughout training.

In the updated version of the manuscript, we have included a comparison to a Bayesian version of the p4cnn-architecture. See the global comment for further details.

I thank the authors for their careful response. Still, my main concerns about the positioning of the paper and novelty of its results remain. As such, I cannot recommend the acceptance of the manuscript in its current form.

Relation to [Nordenfors et. al, 2025]

From the response, I am taking then that the theoretical contributions of the paper are focused on Theorem 3 and 4 and on the initialization techniques. In that context, I believe that the matter of limited contribution remains. Theorem 3 is a straightforward variational result akin to typical results from variational Bayes. Theorem 4, on the other hand, is certainly a novel statistical concentration result. However, the matter of its effective value remains seen as the dependence of the "constants" are not made explicit in the derivations (see below).

The novel version of the manuscript also appears to suggest that (21) is novel. This smoothing structure is typical in the equivariance literature, e.g., [Manay et al., "Integral invariants for shape matching", TPAMI'06; Kondor, "A novel set of rotationally and translationally invariant features for images based on the non-commutative bispectrum," 2007; Bietti et al., "On the Sample Complexity of Learning under Invariance and Geometric Stability", NeurIPS'21]. As such, I am wondering what exactly is meant by the comment in line 361.

The values of the constants

The added hypothesis in the theorem statement is appreciated, but it does not determine how the constants are affected by the magnitude of ff. Explicit values of the constants would remove all ambiguity. If this is not possible, then the manuscript should explicitly state why and give at least indications as to how they depend on different parameters.

Finally, the authors are correct to point out my poor choice of words with respect to "training complexity." Perhaps a more precise statement would have been that Theorem 4 is crucial as it establishes the number of MCMC samples needed and, therefore, the complexity of the Bayesian NN. As such, the constants do matter.

Suggested references on geometric deep learning

While I agree that this related works are not about Bayesian NNs per se, some of them use constructions that would directly apply. As such, it is important to comment why they would (or would not) lead to equivariant models.

The contribution of the paper We think that the reviewer is misunderstanding us when they dismiss theoretical contributions other than Theorem 3 and 4. To generalize the results of Nordenfors et. al is not trivial. Theorem 1, which is crucial to determine which parameters yield invariant distributions, is novel.

The constant Explicit constants can directly be retrieved from the proof. Cdata=22M2C_{\mathrm{data}}=2\sqrt{2}M^2, Cweight=(82+81/log⁡(2)M2≈20.9227M2C_{\mathrm{weight}}=(8\sqrt{2}+8\sqrt{1/\log(2)}M^2\approx 20.9227M^2.

The novelty of orbit averaging We do not claim that orbit averaging per se is new; what is new is the realization that it can be applied to the natural parameters of the exponential family to yield invariant distributions.

Other approaches In the updated manuscript, we have added experiments where we compare to the augmentation strategy of Nabarro and a 'Bayesianized' GCNN, applying the constructions mentioned in the review directly. They indeed yield equivariant models, but are more expensive at training and/or inference. Please see also the global comment on the new experiment section.

Reviewer_t5v4OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper analyzes Bayesian learning of equivariant models through variational inference, assuming the model p(y∣fθ(x))p(y|f_\theta(x)) satisfies a set of "compatibility" assumptions relating group transformations of data space to those of parameter space. The authors provide some theoretical framing for the problem, motivating methods for obtaining equivariant predictive distributions they then test empirically.

On the theoretical side, four results are proven:

  • regular minimal exponential families of distributions on θ\theta have pushforwards under group transformation that are given by a group action on the natural parameter of the family if and only if they are closed under the group;
  • starting from an invariant prior, gradient updates in variational inference give an invariant variational posterior;
  • the augmented log likelihood of the negative-ELBO-loss minimizer converges to the optimal augmented log likelihood;
  • the non-equivariance obtained by sampling from the variational posterior is controlled at standard parametric rates.

Methodologically, two methods of symmetrization (making the variational posterior invariant) are proposed: orbit averaging and "orbit expansion" (constructive weight sharing from pre-trained weights). These are tested empirically, compared against a non-equivariant Bayesian baseline model. Orbit expansion gives slightly improved accuracy, and both methods improve equivariance.

Critical strengths

The paper shows effectively how, under data augmentation, equivariance can be induced in Bayesian NNs through the variational distribution. The experimental demonstration that equivariance defect decreases throughout training is a strong point in telling this story, and the symmetrization experiments show improved equivariance as well.

Additionally, the self-contained characterization of exponential families closed under the group is of potentially of independent interest.

Critical weaknesses

There is an issue in the interpretation of Theorem 3: convergence of the errors does not by itself imply the equivariance defect goes to zero. The authors support that it does go to zero in experiment, so the question here is to clarify the strength of their claim, and what is meant to be proved versus merely suggested. It's also worth noting that the focus on the structure of fθ(x)f_\theta(x), and Fη(x)=Eq[fθ(x)]F_\eta(x)=E_q[f_\theta(x)], seems unnatural in the theory section. In particular, while the authors call FηF_\eta the predictive, that is a misnomer: the variational predictive distribution (which is the natural object to study here) is Eq[p(y∣fθ(x))]E_q[p(y|f_\theta(x))], but in general this is not the same as p(y∣Fη(x))p(y|F_\eta(x)).

Additionally, the experimental section doesn't offer a strong comparison of the proposed method to other equivariant methods. For example, what is the benefit of enforcing equivariance through averaging, as opposed to enforcing equivariance in a BNN?

Questions for authors

What current methods is the proposed symmetrization meant to improve over, and how do the experiments demonstrate that?

2 comments

We thank the reviewer for their careful reading of the paper and constructive comments. We address each of the questions and/or weaknesses below.

  • Convergence of risk vs. convergence towards equivariant model.

We agree with the reviewer that this section was unclear. It is indeed the case that a convergence of the risks does not imply that the maps FηF_\eta converges towards an equivariant map. We do however note that we have empirical evidence of this happening in Section 4.1. We have added a comment and changed the wording in Section 3.3 accordingly.

  • FηF_\eta vs. Eq[p(y∣fθ(x)]E_{q}[p(y|f_\theta(x)]

We understand the reviewer’s point. Our reasoning behind naming FηF_\eta the predictive map is that we imagine using the mean E[fη(x)]E[f_\eta(x)] as a means of making predictions using the BNN. To study the probability distribution pη:=Eq[p(y∣fθ(x)]p_\eta := E_q[p(y|f_\theta(x)] of course gives more detailed information.

However, our technique can be used to prove an equivariance statement for pηp_\eta as well. Indeed, the assumption (8) on the forward model reads p(y∣fθ(x))=p(gy∣gfθ(x))p(y|f_\theta(x))=p(gy|gf_\theta(x)), and the compatibility condition (2) on ff that p(gy∣gfθ(x))=p(gy∣fgθ(gx))p(gy|gf_\theta(x))= p(gy|f_{g\theta}(gx)). Now, for η∈HG\eta\in H_G, we have that if θ∼qη\theta \sim q_\eta, gθ∼qηg\theta \sim q_\eta. Consequently, p(y∣fθ(x))∼p(gy∣fθ(gx))p(y|f_\theta(x)) \sim p(gy|f_{\theta}(gx)). Taking the expectation yields pη(y∣x)=pη(gy,gx)p_\eta(y|x)=p_{\eta}(gy,gx). We have included the definition of pηp_\eta and the revised proof in the updated pdf.

  • Experimental section

We address the shortcomings of the experimental section more thoroughly in a global comment. Let us here only clarify that we are not enforcing equivariance through averaging the maps x↦Fη(x)x\mapsto F_\eta(x) after training (symmetrization). When we are speaking of averaging, we are rather applying the average operator to the parameter η\eta, and we only do it once during training.

Thank you for the clarifications and work to update the submission.

My main concerns have been addressed. I believe theoretical contribution and ideas in the submission are of interest to the community. On the experimental side, I am slightly less convinced that the proposed method is superior to existing ones, but the added experiments provide some clarity into the tradeoffs.

I will increase my score to reflect these improvements.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global comment about update of experimental result

All of the reviewers raised concerns about our experimental sections. The critique was justified, and we have made several changes to the section. We list here the concerns of the reviewers and how we have addressed them. For the full results of the added experiments, see the updated Section 4.

  • Modification of projection strategy

When applying the averaging strategy, we are effectively changing the scaling of the neural network weights. We have therefore rerun the projection experiment, and now also rescale the weights to match the Frobenius norm before the projection. This makes the projection strategy more effective, and it can now match the (naive) baseline.

  • Additional baselines

We have added two baselines to our experiments. The first one is a Bayesian version of an equivariant architecture, more specifically the p4cnn architecture. That is, we learn a Gaussian distribution over the weights of the p4cnn models through variational inference. We then make predictions in the same manner as for the other Bayesian networks. By construction, the p4cnn achieves excellent equivariance behaviour — it is not perfectly equivariant only because the discretized value F^η(x)\widehat{F}_\eta(x) of the true average varies in between evaluations. It also achieves a slightly higher accuracy (80.6% vs. 80.1%) on the FashionMNIST dataset compared to orbit expansion. However, it takes 13% longer to train. Hence, the orbit expansion strategy achieves almost the same performance as p4cnn, but it is cheaper.

We have also added the alternative method of data augmentation of Nabarro et. al suggested by reviewer cdFY - we call it orbit likelihood. In essence, this amounts to symmetrizing the function Fη→1/∣G∣∑gFη(g⋅)F_\eta \to 1/|G| \sum_g F_\eta(g\cdot) during both training and inference. This also yields an (almost, for the same reasons as above) equivariant map, and furthermore excellent accuracy (82.4) on the FashionMNIST dataset. This one is however very expensive, in particular at inference, since the neural network needs to be evaluated on 4 transformed versions of the input in order to calculate one prediction.

To conclude, the orbit expansion method does not quite outperform the new baseline on the FashionMNIST-example, but is cheaper and simpler.

  • More realistic datasets

We have performed another run of experiments on the NCT-CRC-HE dataset of histological slices. The classification problem in this dataset is invariant to rotations in the plane. On this more realistic dataset, the orbit expansion strategy performs best, on par with the Bayesian p4cnn, but again faster. Interestingly, the orbit likelihood approach performs worse compared to the baseline (86% accuracy for orbit expansion, 80.6% Nabarro, 80.4% naive baseline). We are not entirely sure why this is, but speculate that it depends on the fact that the net we use in this case is deeper. The p4cnn and orbit expansion strategies are directly affected by the depth — it causes a higher drop in effective parameter count — whereas the orbit likelihood strategy is independent of depth. We emphasize, however, that conclusions cannot really be drawn from this one experiment, and future work is warranted here.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.