acceptodds
Under review as a conference paper at ICLR 2027

Signal-noise factorization isolates nuisance variation into removable subspaces

Abstract

Recent theoretical work identified fundamental properties of representational geometry that shape inference ability of deep neural networks. These include signal-noise factorization (SNF), the ability to segregate signal from noise, and signal-signal factorization (SSF), the ability to segregate task-specific and task-irrelevant signals. Here, we built new regularizers that explicitly reinforce these two properties during training. We compared networks trained with these regularizers to -regularized baseline networks on the CIFAR-100 image classification task to understand how our new regularizers shape representation geometry and impact performance on a well-known computer vision baseline. Enhancing SNF via regularization improved model performance but enhancing SSF did not. Motivated by biomedical diagnostics applications, we next investigated how our new regularizers affected performance on the BloodMNIST dataset treated with MedMNIST-C corruptions at five severity levels, and found even larger performance gains using the SNF regularizer. To understand the mechanism by which signal-noise factorization produces improved performance, we analyzed the nuisance subspaces across regularization regimes, finding that the SNF-regularized models represent noise in distinct subspaces, separate from class-relevant signal. Because this geometry is explicit, the dominant corruption-induced directions can be estimated on held-out data and projected out of the representations. This manipulation led to a substantial gain in categorization accuracy. These results demonstrate that regularizers that enforce signal-noise factorization can produce substantial improvements on computer vision tasks that contain out-of-distribution image distortions at inference time. They also highlight how explicitly shaping representations affects model performance: isolating nuisance variables from categorical ones is more important than maintaining factorized representations of categorical variables.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation geometry, factorized representations, signal-noise factorization, regularization, out-of-distribution robustness, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ajEMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes to adapt signal-noise factorization (SNF) and signal-signal factorization (SSF) into regularizers for ANN training. These terms are borrowed from a recent Neuroscience paper by Wakhloo et al. By penalizing the representation at the penultimate layer of a ResNet18 model in such a way as to increase these two properties of the class-conditional geometry in the representation the hope is to improve model performance in an interpretable way so the effect of representational structure and its effect on performance can be more directly studied.

Experiments are performed on the CIFAR100 and BloodMNIST datasets. The results suggest that these regularizers (specifically, SNF) might pose a slight improvement in performance but more importantly, through a mechanistic subspace analysis on (on BloodMNIST), SNF regularization is shown to have potential for handling OOD noise well by modeling "nuisance variances" in different subspaces than true signal variance.

Critical strengths
  • The goal is timely and useful if achieved. Having a principled regularizer that also supports a level of interpretation and robustness to definable noise should have good potential in fields like XAI and medical imaging.
  • The proposed regularization terms are simple even though they place some restrictions on how batches can be sampled.
  • CIFAR100 and BloodMNIST and ResNet18 are decent choices of datasets and architectures to investigate this approach.
  • The mechanistic subspace analysis is interesting and the findings are intriguing, but generality is not supported.
Critical weaknesses

The following points are, what I believe to be, the key weaknesses of the paper. See the questions for the authors below for more detail.

  • Missing literature - The related work is missing existing works that explicitly manipulate the class-conditional geometry of ANN representations. Ideally it would also be useful to compare against these approaches in terms of performance and interpretability.
  • Questionable HP tuning protocols and baselines - As far as I can tell the unregularized baselines are not tuned and then the regularized alternatives use a kind of nested sweep where L2 is added (and sweeped) and then SNF or SSF regularization is added to that (and sweeped again). We can regard the additional sweeps as approximately including the preceeding ones (at lambda=0) so that improvements (or at least very little chance of reduction in performance) is expected. This along with weak initial baselines and conservative differences in accuracy makes the impact on performance questionable.
  • Overstated results - The interpretations of Fig 2 are overly flattering towards the proposed regularizers, leaving out key observations.
  • Inconsistent handling of the two datasets - On CIFAR100 the effect on the target variables (s and f) with and without L2 regularization is demonstrated, but it is lacking the mechanistic subspace analysis performed on BloodMNIST. Secondarily, the BloodMNIST setting is missing the unregularized performance comparison. This makes the claim of "...offer some initial evidence of generality." questionable. A core argument seems to be that applying SNF regularization, increases f and s, and this means that noise is moved into a separate subspace of the representation. However, this is never demonstrated as a chain of events on one dataset.

I understand that, as highlighted in the title, the main contribution might not the improvement of performance (even though it is claimed as a contribution in the abstract) but the isolation of "nuisance variation". However, it is hard to justify analysing "how our new regularizers shape representation geometry and impact performance..." in the absence of convincing impacts on performance.

Questions for authors

The following are more details on the key weaknesses. Addressing or refuting these concerns would be required for me to change my assessment.

  • Section 1.1 seems to suggest that the novel part of the work is that it actively manipulates class-conditional geometry in the representational space in a way that is specific and interpretable such that the link between representational structure and performance can be studied. It is missing existing works that explicitly manipulates the class-conditional structure of the representation, e.g. Center loss (Wen et al., ECCV 2016), DeepLDA (Dorfer et al., ICLR 2016), Induced Neural Collapse (Yang et al., NeurIPS 2022; Liu et al., AISTATS 2023), Orthogonal projection loss (Ranasinghe et al., ICCV 2021), and Orthogonal Low-rank Embedding (Lezama et al., CVPR 2018)). If the authors cannot justify omitting these works then comparing against them experimentally might be warranted.
  • I feel like potentially weak baselines, questionable HP tuning, and very conservative gains over them are cause to question the validity of the gains.
    • According to the text the unregularized baselines were untuned (or at least never disclosed to have been). The L2 penalty was tuned for the L2 regularized models. The SSF and SNF penalties were then tuned at a constant L2 penalty for the SSF and SNF regularized models. This nested search means L2+SNF effectively "contains" the L2 model (as the SNF strength goes to zero), so it amounts to searching over an additional dimension. It would have been more principled to give L2 and L2+SNF the same tuning budget, without the latter reusing the L2 value selected for the former.
    • The Fig. 2d models were not re-tuned either.
    • We also note that it sounds like the baselines are using the required but unconventional batch size of 1350 using a stratified sampler.
    • The CIFAR100 models all look like they are underperforming when compared to similar models in the literature: a standard ResNet-18 trained from scratch on CIFAR-100 should reach to around 77% (with standard mirror+crop augmentation; DeVries & Taylor, 2017, Table 1) or 75% (weiaicunzai/pytorch-cifar100 reference implementation).
  • Section 5.1 gives a favourable reading of Figure 2; together with the previous point, this makes it unclear whether the effects warrant a mechanistic explanation.
    • The text compares conditions only within the No-L2/With-L2 groups. It omits that L2 alone yields higher s than SNF alone, and doesn't remark that SSF raises s more than SNF, which targets s directly.
    • The text says SNF "improves" accuracy and SSF "consistently" lowers it, but both differences fall within the error bars (n = 3); L2 alone gives a larger gain than either regularizer alone.
    • SSF has the largest effect on f and s yet lowers accuracy. The paper attributes this to neural collapse/over-constraint rather than treating it as evidence against the premise that higher f and s reduce error (Eq. 1).
    • The interpretation of Fig 2D also ignores that L2 alone degrades less than SSF or SNF alone.
  • No subspace analysis on CIFAR-100 is provided. The mechanistic analysis is done only on BloodMNIST, and no reason is given. CIFAR-100-C, which the authors downloaded (A.2.7), provides paired clean/corrupted images, so the Fig. 4 analyses could be computed for CIFAR-100 in exactly the same way, on a 100-class problem rather than one with an at most 8-dim signal subspace. In general, the two datasets are handled inconsistently: f and s are reported only for CIFAR-100 (Fig. 2a–b); No-L2 conditions only for CIFAR-100; the OOD tests differ (weak/strong colour jitter vs MedMNIST-C corruptions); and the subspace analysis is only for BloodMNIST. Secgtion 6.2 nevertheless argues that "consistent trends across both offer some initial evidence of generality", but few trends are actually measured on both datasets, so this claim is not supported. In particular, the mechanism in the title and abstract rests on a single dataset.

Some minor points that, if fixed, would improve the paper:

  • The line above Eq. 1 says E_g is a bound, but it looks like Wakhloo et al. does not present it as a bound.
  • More clarity on bridging the setting in Wakhloo and the current one would be useful.
  • Clarifying how the regularizers precisely target s and f would be good.
  • Undefined terminology: "nuisance variance", "population responses", "the task's latent factors", "effective dimensionality of the population".
  • The main text says the HP sweeps are "joint" but A.2.6 says the HPs were searched individually.
  • The Fig. 3B caption mentions AUC that isn't shown.
  • The PCA truncation to 50 components in Fig. 4A leaves open the possibility that the proportional explained variance in those 50 components might be different across models, making comparison shaky.
  • How big was the validation set?
  • It is not always clear if accuracy refers to validation or test set results.

References:

  • Wen, Y., Zhang, K., Li, Z., & Qiao, Y. A Discriminative Feature Learning Approach for Deep Face Recognition. ECCV 2016.
  • Dorfer, M., Kelz, R., & Widmer, G. Deep Linear Discriminant Analysis. ICLR 2016. arXiv:1511.04707.
  • Yang, Y., Chen, S., Li, X., Xie, L., Lin, Z., & Tao, D. Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network? NeurIPS 2022. arXiv:2203.09081.
  • Liu, X., Zhang, J., Hu, T., Cao, H., Yao, Y., & Pan, L. Inducing Neural Collapse in Deep Long-tailed Learning. AISTATS 2023 (PMLR 206).
  • Ranasinghe, K., Naseer, M., Hayat, M., Khan, S., & Khan, F. S. Orthogonal Projection Loss. ICCV 2021. arXiv:2103.14021.
  • Lezama, J., Qiu, Q., Musé, P., & Sapiro, G. OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning. CVPR 2018. arXiv:1712.01727.
  • DeVries, T., & Taylor, G. W. Improved Regularization of Convolutional Neural Networks with Cutout. arXiv:1708.04552, 2017.
  • weiaicunzai. pytorch-cifar100 (GitHub repository). https://github.com/weiaicunzai/pytorch-cifar100
3 comments
AuthorsOpenReview ↗

Response to Reviewer ajEM (Part 1/2)

We thank the reviewer for this thorough critique. The revision addresses most of these points directly, and several were valid errors in the submitted version that we have corrected. We group our response by the reviewer’s four concerns.

  • The reviewer is right that several related works belong in Sec. 1.1, and we have added them: Center loss, Deep LDA, Induced Neural Collapse, OPL, and OLÉ, along with AugMix and debiasing regularizers. We also now state how SSF and SNF regularization might be similar or different. While SSF is similar to the class-mean orthogonalization methods, the remaining few either compress within-class variance in all directions or remove a specified nuisance attribute. SNF, instead, uses only class labels and constrains noise solely within the signal-coding directions, leaving the orthogonal complement free. On an experimental comparison: our aim is not to win a robustness benchmark but to isolate what enforcing each geometric property does. To this end, we include an analysis of how the SSF regularization term impacts the geometry of the hidden layer it is applied to.
  • We have corrected the tuning description and addressed each sub-point.
    • The search was separate, and the submitted text wrongly called it joint. We corrected this (Sec. 3, A.2.5): L2 was selected first on clean validation accuracy, then held fixed while sweeping the geometric penalty. All selection used clean validation accuracy, so the corruption results were never selected for.
    • On the nested-search concern: the point is well taken in principle, but it does not produce the gains here. On BloodMNIST the selected L2+SNF model has lower clean validation accuracy than L2 alone (0.981 vs. 0.984, Table A.2), so the extra search dimension bought nothing on the selection criterion and the corruption benefit appears despite this, not because of it. On the untuned baselines: all conditions share the same training recipe (App. A.2.3–A.2.4, following the original MedMNIST protocol for the medical datasets), and only regularization strengths were swept. We did not rerun an equal-budget joint sweep, which we acknowledge as a limitation.
    • The Fig. 2d color-jitter experiment has been removed. The revision evaluates the main models directly on MedMNIST-C (Sec. 5) and CIFAR-100-C (Sec. A.1.5), so no separately tuned models are involved.
    • On the batch size: the stratified batch of 1350 is required by the SNF penalty, which inverts the per-batch label covariance and so needs every class present in each batch. It is applied identically to all conditions including the baselines, so it cannot advantage the regularized models. We scale the learning rate accordingly (A.2.3).
    • On CIFAR-100 absolute accuracy: our setup differs from the cited references in two ways. We use the stock torchvision ResNet-18 (7×7 stride-2 conv, max-pool), which downsamples 32×32 inputs immediately, whereas the ~75–77% references use a CIFAR-adapted stem (3×3 conv, no max-pool), and we train with the large stratified batch the SNF penalty requires. All conditions share this setup, so between-condition comparisons are controlled, though we agree absolute accuracy is lower. We now state this in App. A.2.1.
  • We agree that the original reading of Sec. 5.1 and Fig. 2 was too favorable. On CIFAR-100, SSF raises f and s the most (0.65 and 0.71, versus 0.22 and 0.31 unregularized) yet has the lowest clean and corrupted accuracy (69.6% and 45.7%, versus 71.3% and 47.1%; Sec. A.1.5). We take this, as the reviewer suggests, as evidence that higher f and s do not by themselves reduce error in our setting. We no longer present Eq. 2 as a predictor of our models’ accuracy: it describes an in-distribution Hebbian readout, and our setting differs in readout, label structure, and test distribution (Sec. 2). We also found that s is ambiguous for a second reason. Under SNF it can rise through shrinkage of the signal subspace rather than removal of noise from it. We therefore added a scale-invariant eviction metric, R, which normalizes in-subspace corruption energy by in-subspace signal energy (Sec. 5.2, Fig. 3c). On BloodMNIST and OrganAMNIST, SNF lowers R relative to L2 (0.58 vs. 1.22; 0.56 vs. 0.85), while SSF does not (1.29; 0.81). SNF also gives the largest accuracy gain under corruption on BloodMNIST, with a smaller gain on OrganAMNIST that falls within seed variability. On CIFAR-100, however, R does not separate the conditions the same way (unregularized 1.28, SSF 1.14, SNF 2.02): removability of SNF's leading corruption directions still holds there (Fig. A.6f), but genuine eviction does not. We therefore claim the eviction mechanism only for the medical datasets, and we do not claim that any single geometric statistic, s or R, predicts accuracy across settings.
AuthorsOpenReview ↗

Response to Reviewer ajEM (Part 2/2)

  • CIFAR-100 subspace analysis and dataset consistency. We agree the two datasets were handled inconsistently, and the revision makes the analyses parallel.
    • We added the mechanistic subspace analysis on CIFAR-100 using CIFAR-100-C (Sec. A.1.5, Fig. A.6), as the reviewer suggested, including the eviction metric and projection recovery.
    • We added a second medical dataset, OrganAMNIST (Sec. 5.3, Fig. 5), so the central mechanism is now shown on two datasets rather than one, which directly addresses the “mechanism rests on a single dataset” concern.
    • f and s are now reported across datasets, and the OOD evaluation is MedMNIST-C / CIFAR-100-C throughout rather than color jitter.

We hope these revisions resolve the reviewer’s concerns, and we would welcome guidance on whether any specific experimental comparison would most strengthen the paper in their view.

Reviewer_ajEMOpenReview ↗

What is the utility of these findings?

I thank the authors for the substantial revision. Making the analyses more consistent across datasets largely resolves my fourth point, and the reframing largely resolves my concern about overstated performance claims. However, the revision also makes it clear that the regularizers bring little performance benefit on either clean or corrupted data. I acknowledge that we see slight improvement on corrupted BloodMNIST.

The authors general point on reframing away from performance gains is noted, but that leaves a concern: what is the utility of these regularizers? Showing how a technique works is only impactful if it is linked to some utility increase (e.g. performance, robustness, efficiency, interpretability, scalability, fairness, understanding, etc.). The current version seems to suggest that the utility is: (a) in "understanding" the geometric structure of the representation or (b) the idea of projecting "out" nuisance factors.

The understanding is limited to the knowledge that when SSF is regularized latent mean class vectors tend to be more orthogonal and when SNF is applied noise is isolated in subspaces (at least for the two medical datasets) that can potentially be projected out. These are properties that have not been shown to be missing when employing similar existing regularizers from the literature.

As far as I can see, the "projecting out" requires that we have representative clean and corrupt versions of samples with the specific type of corruption that we want to project out (it does not generally transfer between corruption types). With this type of ground truth you could fine tune a linear head to be robust to the corrupted samples (this is not compared against), potentially not needing the SNF regularizer. The conclusion suggests that the regularizer might help with cross-clinic generalization, but this is speculation since it is untested.

Without the performance gains I struggle to see the impactfulness of the technique even if we assume that all findings are valid and sound. My concerns about baselines and tuning remain but I think this question on utility needs to be answered first.

Reviewer_do9hOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents two regularizers based on theoretical work on representational geometry. The signal-signal factorization (SSF) regularizer pushes class representations to be orthogonal, and the signal-noise factorization (SNF) regularizer pushes noise out of the class-signal directions. Both regularizers are applied to the penultimate layer of a ResNet-18 and are compared to unregularized and L2-regularized baselines. The authors report that SNF regularization slightly improves clean accuracy on CIFAR-100, as well as the accuracy on corrupted BloodMNIST images. SSF regularization does not help. Furthermore, they show that SNF-regularized models move the corruption-induced variance out of the signal subspace.

Critical strengths
  1. The paper reports accuracy numbers, but does not stop there, and also tries to explain where the gain comes from. The subspace analysis is one of the most interesting parts in the paper. Under the new SNF regularization only 16% of the corruption-induced variance falls into the signal subspace, compared to 61% for L2. After removing the corruption directions found on the validation data, accuracy on corrupted test images reises bs over 8 points. For the L2 and SSF models, this step makes the accuracy worse. This result shows a link between how the representation is organized and how robust the model is.
  2. The SNF regularizer is simple, grounded in theory, and improves corrupted accuracy on BloodMNIST without a clear degradation on clean data. The authors also report an hones result for SSF, which is useful because it shows that not every geometry property from the theory helps.
Critical weaknesses
  1. The empirical evidence is too limited for the given claims. The paper does not compare against standard robustness methods like AugMix or other representation regularizers. Furthermore, CIFAR-100-C is only used to measure geometry, and accuracy on this dataset is not reported. On CIFAR-100, the gains are quite low and within the noise, and on clean BloodMNIST, SNF is even worse. All results are based on three seeds without statistical tests, even though the paper calls some results "statstically indistinguishable".
  2. The geometry measurements in the paper do not fully support its main claim. On CIFAR-100, the SSF model reaches a higher signal-noise factorization score than the SNF model, yet it performs worse. Thus, the metric the paper is built on does not explain the results, and it stays not clear what SNF regularization changes. The SNF loss depends strongly on how large the activations are, and a learnable BatchNorm is right before the penalty. Therefore, the model could reduce its loss by shrinking its activations, which would not be picked up by the signal-noise factorization score. In addition, the projection experiment needs corrupted validation data with the same corrupted types as in the test set. This is closer to adapting to known corruptions than to the out-of-distribution setting the abstract describes.
Questions for authors

Clarification:

  1. Why does SSF regularization reach a higher signal-noise factorization score than SNF regularization on CIFAR-100 while performing worse? And how does this fit the claim of the paper that improving signal-noise factorization drives the gains?
  2. Does SNF regularization reduce the activation scale before the penalty? Could you report results with a scale-invariant version of the loss?
  3. Could you report accuracy on CIFAR-100-C and compare against other standard robustness methods and representation regularizers?
  4. Was the hyperparameter search joint or separate?
  5. How well does the projection work when the corruption subspace is estimated on corruption types that do not appear in the test set?

Suggestions for future work:

  1. Evaluation on other architectures such as vision transformers.
  2. Experiments on datasets with many more classes, as the method requires every class to be present in every batch.
4 comments
AuthorsOpenReview ↗

Response to Revierwe do9h (Part 1/2)

We thank the reviewer for their constructive feedback and comments. We address all weaknesses and questions below:

Critical weaknesses

  • Reviewer GbTD raised the same concern; we repeat our response here for convenience: We recognize the interest in comparing performance to alternate regularization schemes, however, we note that this is out of scope for the current project. We have, however, added these methods, including debiasing regularizers and AugMix, to the related work (Sec. 1.1), where we note a key distinction: debiasing regularizers target the removal of specified nuisance variables and AugMix acts on the training distribution, whereas SNF regularization requires only class labels and constrains noise only within the signal-coding directions. Our focus is on how SSF and SNF regularization change representation geometry and note in the future directions that representation geometry of other regularization schemes should be further studied to understand what structural properties of hidden representations produce ideal performance.
  • We agree that datasets were handled inconsistently. To this point, we add our results from performing the BloodMNIST analysis to all datasets, and utilize the CIFAR-100-C dataset to compute performance rather than the corruption-strength experiment presented previously. We have additionally softened our claims about accuracy/performance, focusing instead more deeply on the mechanistic findings for each regularization scheme, which we intended to be the major findings of the paper. To keep the main text focused on the medical datasets, CIFAR-100 results are now in Sec. A.1.5, Fig. A.6, including CIFAR-100-C accuracy across 19 corruption types and five severities (Fig. A.6d). We have also removed inferential phrasing such as “statistically indistinguishable,” consistent with our policy of reporting per-seed comparisons without formal tests at n=3 (App. A.2.2), and now report the small clean-accuracy cost of SNF regularization on BloodMNIST directly (Fig. 2c).
  • The reviewer is correct that on CIFAR-100 SSF reaches a higher s (0.71) than SNF (0.38) while generalizing worse. We agree this shows s is not a predictor of accuracy, and we have revised the text accordingly. Eq. 2 is an expression for the error of a supervised Hebbian readout evaluated in-distribution (Wakhloo et al., 2026); our setting differs in the readout, the discrete labels, and the shifted test distribution (Sec. 2). We therefore use s to identify which property to target, not as a quantitative predictor, and Sec. 4.1 now notes explicitly that a higher s can arise either from reorganizing noise out of the signal subspace or from shrinking variance globally, so s alone cannot establish the mechanism.
  • The reviewer correctly identifies that the BatchNorm before the penalty lets the model lower L_s by reducing activation scale. Because s is scale-invariant while L_s is not, this would not register in s. To separate shrinkage from genuine eviction, we added a scale-invariant metric R, the in-subspace corruption energy normalized by in-subspace signal energy (Sec. 5.2, Fig. 3c), and we report activation norm, BatchNorm , and in-subspace corruption energy directly (Sec. A.1.2, Fig. A.2). We distinguish two quantities the reviewer’s concern bears on: R measures the ratio of in-subspace corruption to in-subspace signal, while the projection test (Figs. 3d, 5g, A.6f) measures whether the leading corruption directions overlap the directions the classifier relies on. These need not move together, and their relationship is dataset-dependent. On BloodMNIST and OrganAMNIST, R falls below the L2 baseline (0.58 and 0.56 vs. 1.22 and 0.85), meaning in-subspace corruption contracts faster than the signal, marking genuine eviction. On CIFAR-100, R rises to 2.02, and we now state plainly that the reduced in-subspace fraction reflects shrinkage of the signal subspace rather than eviction of corruption from it (Sec. A.1.2, A.1.5). Crucially, though, the leading corruption directions SNF produces remain removable even there: projecting out same-corruption directions raises SNF accuracy from 46.9% to 51.7% (+4.8% at k=15) and keeps it above baseline out to k=45, exceeding both SSF and the unregularized model (Fig. A.6f). The datasets therefore differ in degree rather than in kind. The core mechanism, SNF producing corruption directions the readout can ignore, operates on all three; what varies is whether it is accompanied by genuine eviction from the signal subspace (medical datasets) or by shrinkage (CIFAR-100). Our central claims foreground the medical datasets because that is where eviction is cleanly demonstrated, and we now present CIFAR-100 explicitly as the shrinkage-dominated case.
AuthorsOpenReview ↗

Response to Reviewer do9h (Part 2/2)

  • The reviewer is right that estimating the corruption subspace requires examples of those corruption types. We have reframed this result accordingly: Sec. 5.2 now describes it as a corruption-aware adaptation procedure rather than an out-of-distribution result, and we separate it from SNF’s accuracy gain under corruption, which requires no knowledge of the corruptions. Consistent with the reviewer's intuition, subspaces estimated from other corruption types do not improve average accuracy on any dataset (cross-corruption curves in Figs. 3d, 5g, and A.6f), although on BloodMNIST contrast, blur, JPEG, and pixelation corruptions partly share a subspace (Fig. A.4). The procedure therefore adapts to known corruptions rather than generalizing to unseen ones.

Questions

We hope the above comments address most of the reviewer’s questions. We respond to the outstanding questions below:

  • A scale-invariant formulation (for example, penalizing 1/s directly) is a sensible design improvement and a natural next step. We note, however, that the shrinkage confound this would address is isolated by our R metric, which normalizes in-subspace corruption energy by in-subspace signal energy and so is invariant to activation scale. A full scale-invariant variant would require re-tuning and retraining across all three datasets and all regularization schemes, which is beyond the discussion window, so we leave it to future work.
  • We note that the main goal of the paper is to study the representational geometry induced by these two regularization regimes and compare them to one commonly used regularizer (L2). As such, performance comparisons with other regularization regimes is out of scope. We note, in the future directions, that comparisons to other regularization schemes and their effects on representation geometry would be an interesting study of which structural properties induce the best performance. We have adjusted the abstract and introduction to make the scope of the project more clear and expand on the mechanism induced by SSF regularization (Fig. 4, Sec. 5.2), which demonstrates that SSF regularization induces representation geometry similar to that of class-mean orthogonalization methods such as Induced Neural Collapse (Yang et al., 2022).
  • The search was separate, not joint, and we have corrected the main text, which previously described it as joint (Sec. 3, A.2.5). We first swept L2 strength alone and selected it by clean validation accuracy, then held that value fixed while sweeping the SNF or SSF strength, so that each geometric penalty is evaluated as an addition on top of the same selected L2 baseline. All selection used clean validation accuracy, with the test set held out. Reviewer ajEM pointed out that a joint search may have been more principled compared to our nested approach as a nested approach may gain additional performance by just including a better model. We note that this was not the case in our study: on BloodMNIST, the selected L2+SNF model has slightly lower clean validation accuracy, the selection criterion, than L2 alone (0.981 vs. 0.984, Table A.2).

We hope these additions address the concerns. The reviewer’s critique directly motivated the R metric and the shrinkage analysis, which we think materially strengthened the paper.

Dear Reviewer,

As the author–reviewer discussion deadline approaches, I would like to remind you to review the authors’ rebuttal and indicate whether your concerns have been adequately addressed.

If you have any remaining concerns or questions, please share them with the authors before the discussion period ends.

Thank you again for your time and valuable feedback.

Reviewer_do9hOpenReview ↗

Reply to Response to Reviewer do9h

Thank you for the detailed response and the revisions. Most of my points were addressed well, especially, I appreciate the eviction metric R.

At the same time, the revisions make the empirical case narrower, and the following concerns remain:

  1. Even though the R metric is a good way to separate shrinking from eviction, it shows that a large part of what SNF does is shrinking the representation. As the loss is unchanged, this limitation should be stated more prominently than in the conclusion.
  2. The CIFAR-100-C results show no robustness gain, and the gains on BloodMNIST are small. Without an experimental comparison to a standard robustness method, it is hard to judge how relevant the observed geometry is in practice.

Overall, I appreciate the more carefully formulated claims.

Reviewer_GbTDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Bulding upon a recent theoretical formulation which identifies two property of representation learning (signal-noise factorization SNF and signal-signal factorization SFF), authors propose two regularizers which enforce this property during neural network training. The resulting representations are better aligned with the classes in the data and yield improved classification accuracy across two datasets (CIFAR-100 and MedMNIST-C).

Critical strengths
  • The proposed approach in theory is quite interesting: being able to distinguish between real signal and nuisance signal, and minimize the amount of noise that the network captures is a solid contribution
  • The story of the paper is clear, simple and well motivated. I appreciate the proposition of the regularizers based on the theoretical work in neuroscience
Critical weaknesses

I still however have some concerns.

Major :

  • Identifying nuisance direction in real data is not easy at all. I have the feeling that this theoretical frameworks works only in very controlled settings and easy datasets. The experiments do not answer my doubt as they are limited to easy datasets such as CIFAR-100 and blood-mnist. CIFAR, in my opinion, has hardly much value by now: doing well on it does not guarantee the method will scale to more realistic datasets. BloodMNIST looks very simple too, and the distortions applied are so glaring that it seems trivial to me that some latent direction would capture them. In summary, the paper needs some experiment which shows the method works also on real data beyond toy experiments.

  • There is no comparison with other methods. Perhaps the authors are not familiar with the very close field of neural network debiasing (with regularization terms) or but I would expect to see how a debasing regularizer compares to the proposed regularizers.

Minor:

  • Equations 3 (and 4) could benefit from a more in-depth explanation on how they work
  • some notation is used before being introduced (e.g. Ψ\Psi in eq. 1)
Questions for authors
  • Authors say that the SNF-regularized models represent noise in distinct subspaces. It would be interesting to see an interpretation analysis of these subspaces, e.g. is it possible to determine which nuisance factor each subspace corresponds and/or interpret it ? I am thinking about cases in which you do not know in advance what nuisance variables are in the data

  • Similarly, can SSF be used to find and interpret the main "components" of signal in the input data ? Similarly to what sparse-autoencder are supposed to do

2 comments
AuthorsOpenReview ↗

Response to Reviewer GbTD

We thank the reviewer for the concerns and constructive comments. Below, we have responded to each critical weakness and question:

Critical Weaknesses

  • We concur with the reviewer that identifying and removing nuisance directions in real data is not easy. In response, we have pushed CIFAR-100 results to the appendix and included an analysis of a second medical imaging dataset from the MedMNIST database: OrganAMNIST. This dataset is an organ identification task from CT scans and is notably harder due to lack of color and 2D image slices taken from 3D scans, which can make adjacent organs look structurally similar depending on where the slice is taken. We perform the same set of analyses on this new dataset and find that the central mechanism replicates: SNF regularization lowers the eviction metric R relative to its matched baselines (0.54 vs. 0.87 without L2; 0.56 vs. 0.85 with L2), and removing a small number of within-corruption directions improves corrupted accuracy by roughly 4% (Sec. 5.3, Fig. 5). We note that OrganAMNIST, like BloodMNIST, is a low-resolution benchmark, and validation on larger-scale data with natural distribution shift remains a limitation of this work (Sec. 6).
  • We agree that the corruptions applied at inference time are easily identifiable, but these corruptions further support our main finding that SNF regularization produces a representation structure where corruption-induced variance is not entangled with the true signal. All models see the same corruptions, yet on BloodMNIST 61% and 58% of corruption-induced energy falls inside the signal subspace under L2 and SSF regularization, versus 16% under SNF regularization (Fig. 3b), and only SNF lowers the shrinkage-invariant eviction metric R (Fig. 3c). Under L2 or SSF regularization, projecting out the nuisance subspace also projects out the true signal. Within-corruption removability holds on all three datasets, while genuine eviction holds on the two medical datasets but not on CIFAR-100 (Sec. A.1.2, A.1.5).
  • We recognize the interest in comparing performance to alternate regularization schemes, however, we note that this is out of scope for the current project. We have, however, added these methods, including debiasing regularizers and AugMix, to the related work (Sec. 1.1), where we note a key distinction: debiasing regularizers target the removal of specified nuisance variables and AugMix acts on the training distribution, whereas SNF regularization requires only class labels and constrains noise only within the signal-coding directions. Our focus is on how SSF and SNF regularization change representation geometry and note in the future directions that representation geometry of other regularization schemes should be further studied to understand what structural properties of hidden representations produce ideal performance.
  • Regarding the minor points: Sec. 2 now defines Φ\Phi, Ψ\Psi, Ω\Omega (Eq. 1) before the generalization-error expression (Eq. 2) and defines each of its terms in network terms, and Sec. 2.1 explains what the numerator and denominator of s and f (Eqs. 3-4) measure.

Questions

  • SNF regularization itself requires no knowledge of the nuisance factors: training uses only class labels, and the models never see the corruption types. When examples of a corruption are available, its subspace is identifiable and partly interpretable. Sec. A.1.3 (Fig. A.4) shows that on BloodMNIST, the SNF model's subspaces for contrast changes, blur, JPEG compression, and pixelation overlap, so that removing any one improves accuracy on the others, while the remaining corruption types occupy more distinct subspaces. On OrganAMNIST and CIFAR-100, the leading directions are largely corruption-specific (Fig. 5g, Fig. A.6f). Identifying and interpreting these subspaces without examples of the nuisance, for instance from within-class variance outside the signal subspace, is an important direction for future work.
  • We agree that analyzing the geometry of the SSF-regularized models is interesting. We include an analysis of the SSF-induced model geometry (Fig. 4, Sec. 5.2, second-to-last paragraph), demonstrating that SSF regularization pushes class means toward the most orthogonal (simplex-ETF) arrangement their norms permit, similar to class-mean orthogonalization methods such as Induced Neural Collapse (Yang et al., 2022), rather than inducing the interpretable nuisance structure produced by SNF regularization. Unlike sparse autoencoders, however, SSF regularization does not discover new components of the signal: its class-coding directions are defined by the labels, so it organizes known task variables rather than revealing unlabeled ones. Using factorization objectives to discover unlabeled structure is an interesting direction for future work.

We hope these responses address the reviewer’s concerns. We appreciate the time and effort the reviewer put into their comments.

I thank the authors for their response. I suggest authors mark the changes in the revision with colored text.

However, I still have concerns:

  • Thanks for the new experiment. I still do not consider this a realistic dataset (the signal and noise are still simple), but it works as further evidence.

  • "We agree that the corruptions applied at inference time are easily identifiable, but these corruptions further support our main finding that SNF regularization produces a representation structure where corruption-induced variance is not entangled with the true signal." I don't see how the two sentences are in opposition. To me, the second sentence is a natural consequence of having easily identifiable corruptions. My point is that results on easier dataset may not be conclusive and indicative of the scaling on real data where noise is more complex.

  • "We recognize the interest in comparing performance to alternate regularization schemes, however, we note that this is out of scope for the current project" I don't fully agree with this. One of the main contribution is that SNF improves robustness and performance of the model, which is exactly the same goal as debiasing methods. One of the advantages of SNF/SSF could be providing an interpretation of the noise subspaces, which debiasing usually aim to discard as noted by the authors, however the current work is limited in this sense

  • Related to both points above the figures A.4, 5g, A.6f indeed shows that the leading directions and results are largely corruption-specific, and in A.4 seem to focus on the strongest corruptions (which is aligned to the point strongest=easier). On real data, I would expect to see corruptions which are less distinct and more complex

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to all Reviewers

We thank the reviewers for their time, careful review of the work, and constructive feedback. Several concerns were shared. As such, we clarify the project scope and summarize the new experiments presented here and respond to each reviewer individually below. Additionally, an updated PDF of our paper has been attached.

Project scope/framing Multiple reviewers expressed that the framing of the project implied development of a regularizer that improved performance. While SNF regularization does improve accuracy under corruption on BloodMNIST (with smaller or no gains on OrganAMNIST and CIFAR-100), this project is meant to be a scientific exploration of how theory-derived regularization schemes impact representation structure. As such, we have updated the abstract and introduction to make this distinction more explicit and softened claims about accuracy/performance to avoid reading as a performance-first account. We have also included an analysis of the geometric changes induced by SSF regularization.

SSF regularization-induced geometry We agree that a mechanistic study of SSF regularization would help the paper. As such, we have included analysis which demonstrates that SSF regularization induces more orthogonal class means and pushes the representation towards the ideal Simplex-ETF, compared to SNF or L2 regularization.

Shrinkage-invariant eviction metric One reviewer noted that the SNF loss could be lowered by shrinking activations, which s would not detect. We added a scale-invariant eviction metric R, the in-subspace corruption energy normalized by in-subspace signal energy (Sec. 5.2, Fig. 3c), and report the signal-subspace norm, BatchNorm scale, and in-subspace corruption fraction directly (Sec. A.1.2, Fig. A.2). SNF regularization lowers R on both medical datasets (BloodMNIST: 0.58 vs. 1.22 for L2; OrganAMNIST: 0.56 vs. 0.85 with L2), indicating genuine eviction, but raises it on CIFAR-100 (2.02 vs. 1.28 unregularized), where the reduced in-subspace fraction instead reflects shrinkage.

Comparison to other regularizers Multiple reviewers expressed concerns about the lack of comparison to alternate regularization schemes. While we recognize the interest in these schemes, our goal in this project was not to develop a new SOTA regularization scheme, but instead to explore what adding theory-derived regularizers might change about representation geometry in neural networks. We understand that this is likely due to framing of the project. We retain L2 regularization, one of the most commonly used regularizers, as the baseline, with SSF regularization as a contrasting geometric regularizer. We have also added the suggested methods (Center loss, Deep LDA, Induced Neural Collapse, OPL, OLÉ, AugMix, and debiasing regularizers) to the related work (Sec. 1.1), including how each relates to SSF and SNF regularization.

Dataset inconsistency Several reviewers commented on CIFAR-100 and BloodMNIST datasets being used differently and that this reduced the generality of the claims. As such, we have aligned the analyses across all three datasets, showing Wakhloo’s theoretical s and f, clean test accuracy, corrupted accuracy, and within vs cross-corruption accuracy, and a mechanistic analysis of SSF-regularization. Within-corruption removability of SNF's leading corruption directions holds on all three datasets, while genuine eviction from the signal subspace holds on the two medical datasets but not on CIFAR-100, where the effect is dominated by shrinkage (see the eviction metric R below). In addition, we added a second medical imaging dataset, OrganAMNIST (grayscale CT), a different modality from BloodMNIST, and moved CIFAR-100 results to the appendix (Sec. A.1.5) so that the main text focuses on the medical datasets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.