- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes to adapt signal-noise factorization (SNF) and signal-signal factorization (SSF) into regularizers for ANN training. These terms are borrowed from a recent Neuroscience paper by Wakhloo et al. By penalizing the representation at the penultimate layer of a ResNet18 model in such a way as to increase these two properties of the class-conditional geometry in the representation the hope is to improve model performance in an interpretable way so the effect of representational structure and its effect on performance can be more directly studied.
Experiments are performed on the CIFAR100 and BloodMNIST datasets. The results suggest that these regularizers (specifically, SNF) might pose a slight improvement in performance but more importantly, through a mechanistic subspace analysis on (on BloodMNIST), SNF regularization is shown to have potential for handling OOD noise well by modeling "nuisance variances" in different subspaces than true signal variance.
Critical strengths
- The goal is timely and useful if achieved. Having a principled regularizer that also supports a level of interpretation and robustness to definable noise should have good potential in fields like XAI and medical imaging.
- The proposed regularization terms are simple even though they place some restrictions on how batches can be sampled.
- CIFAR100 and BloodMNIST and ResNet18 are decent choices of datasets and architectures to investigate this approach.
- The mechanistic subspace analysis is interesting and the findings are intriguing, but generality is not supported.
Critical weaknesses
The following points are, what I believe to be, the key weaknesses of the paper. See the questions for the authors below for more detail.
- Missing literature - The related work is missing existing works that explicitly manipulate the class-conditional geometry of ANN representations. Ideally it would also be useful to compare against these approaches in terms of performance and interpretability.
- Questionable HP tuning protocols and baselines - As far as I can tell the unregularized baselines are not tuned and then the regularized alternatives use a kind of nested sweep where L2 is added (and sweeped) and then SNF or SSF regularization is added to that (and sweeped again). We can regard the additional sweeps as approximately including the preceeding ones (at lambda=0) so that improvements (or at least very little chance of reduction in performance) is expected. This along with weak initial baselines and conservative differences in accuracy makes the impact on performance questionable.
- Overstated results - The interpretations of Fig 2 are overly flattering towards the proposed regularizers, leaving out key observations.
- Inconsistent handling of the two datasets - On CIFAR100 the effect on the target variables (s and f) with and without L2 regularization is demonstrated, but it is lacking the mechanistic subspace analysis performed on BloodMNIST. Secondarily, the BloodMNIST setting is missing the unregularized performance comparison. This makes the claim of "...offer some initial evidence of generality." questionable. A core argument seems to be that applying SNF regularization, increases f and s, and this means that noise is moved into a separate subspace of the representation. However, this is never demonstrated as a chain of events on one dataset.
I understand that, as highlighted in the title, the main contribution might not the improvement of performance (even though it is claimed as a contribution in the abstract) but the isolation of "nuisance variation". However, it is hard to justify analysing "how our new regularizers shape representation geometry and impact performance..." in the absence of convincing impacts on performance.
Questions for authors
The following are more details on the key weaknesses. Addressing or refuting these concerns would be required for me to change my assessment.
- Section 1.1 seems to suggest that the novel part of the work is that it actively manipulates class-conditional geometry in the representational space in a way that is specific and interpretable such that the link between representational structure and performance can be studied. It is missing existing works that explicitly manipulates the class-conditional structure of the representation, e.g. Center loss (Wen et al., ECCV 2016), DeepLDA (Dorfer et al., ICLR 2016), Induced Neural Collapse (Yang et al., NeurIPS 2022; Liu et al., AISTATS 2023), Orthogonal projection loss (Ranasinghe et al., ICCV 2021), and Orthogonal Low-rank Embedding (Lezama et al., CVPR 2018)). If the authors cannot justify omitting these works then comparing against them experimentally might be warranted.
- I feel like potentially weak baselines, questionable HP tuning, and very conservative gains over them are cause to question the validity of the gains.
- According to the text the unregularized baselines were untuned (or at least never disclosed to have been). The L2 penalty was tuned for the L2 regularized models. The SSF and SNF penalties were then tuned at a constant L2 penalty for the SSF and SNF regularized models. This nested search means L2+SNF effectively "contains" the L2 model (as the SNF strength goes to zero), so it amounts to searching over an additional dimension. It would have been more principled to give L2 and L2+SNF the same tuning budget, without the latter reusing the L2 value selected for the former.
- The Fig. 2d models were not re-tuned either.
- We also note that it sounds like the baselines are using the required but unconventional batch size of 1350 using a stratified sampler.
- The CIFAR100 models all look like they are underperforming when compared to similar models in the literature: a standard ResNet-18 trained from scratch on CIFAR-100 should reach to around 77% (with standard mirror+crop augmentation; DeVries & Taylor, 2017, Table 1) or 75% (weiaicunzai/pytorch-cifar100 reference implementation).
- Section 5.1 gives a favourable reading of Figure 2; together with the previous point, this makes it unclear whether the effects warrant a mechanistic explanation.
- The text compares conditions only within the No-L2/With-L2 groups. It omits that L2 alone yields higher s than SNF alone, and doesn't remark that SSF raises s more than SNF, which targets s directly.
- The text says SNF "improves" accuracy and SSF "consistently" lowers it, but both differences fall within the error bars (n = 3); L2 alone gives a larger gain than either regularizer alone.
- SSF has the largest effect on f and s yet lowers accuracy. The paper attributes this to neural collapse/over-constraint rather than treating it as evidence against the premise that higher f and s reduce error (Eq. 1).
- The interpretation of Fig 2D also ignores that L2 alone degrades less than SSF or SNF alone.
- No subspace analysis on CIFAR-100 is provided. The mechanistic analysis is done only on BloodMNIST, and no reason is given. CIFAR-100-C, which the authors downloaded (A.2.7), provides paired clean/corrupted images, so the Fig. 4 analyses could be computed for CIFAR-100 in exactly the same way, on a 100-class problem rather than one with an at most 8-dim signal subspace. In general, the two datasets are handled inconsistently: f and s are reported only for CIFAR-100 (Fig. 2a–b); No-L2 conditions only for CIFAR-100; the OOD tests differ (weak/strong colour jitter vs MedMNIST-C corruptions); and the subspace analysis is only for BloodMNIST. Secgtion 6.2 nevertheless argues that "consistent trends across both offer some initial evidence of generality", but few trends are actually measured on both datasets, so this claim is not supported. In particular, the mechanism in the title and abstract rests on a single dataset.
Some minor points that, if fixed, would improve the paper:
- The line above Eq. 1 says E_g is a bound, but it looks like Wakhloo et al. does not present it as a bound.
- More clarity on bridging the setting in Wakhloo and the current one would be useful.
- Clarifying how the regularizers precisely target s and f would be good.
- Undefined terminology: "nuisance variance", "population responses", "the task's latent factors", "effective dimensionality of the population".
- The main text says the HP sweeps are "joint" but A.2.6 says the HPs were searched individually.
- The Fig. 3B caption mentions AUC that isn't shown.
- The PCA truncation to 50 components in Fig. 4A leaves open the possibility that the proportional explained variance in those 50 components might be different across models, making comparison shaky.
- How big was the validation set?
- It is not always clear if accuracy refers to validation or test set results.
References:
- Wen, Y., Zhang, K., Li, Z., & Qiao, Y. A Discriminative Feature Learning Approach for Deep Face Recognition. ECCV 2016.
- Dorfer, M., Kelz, R., & Widmer, G. Deep Linear Discriminant Analysis. ICLR 2016. arXiv:1511.04707.
- Yang, Y., Chen, S., Li, X., Xie, L., Lin, Z., & Tao, D. Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network? NeurIPS 2022. arXiv:2203.09081.
- Liu, X., Zhang, J., Hu, T., Cao, H., Yao, Y., & Pan, L. Inducing Neural Collapse in Deep Long-tailed Learning. AISTATS 2023 (PMLR 206).
- Ranasinghe, K., Naseer, M., Hayat, M., Khan, S., & Khan, F. S. Orthogonal Projection Loss. ICCV 2021. arXiv:2103.14021.
- Lezama, J., Qiu, Q., Musé, P., & Sapiro, G. OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning. CVPR 2018. arXiv:1712.01727.
- DeVries, T., & Taylor, G. W. Improved Regularization of Convolutional Neural Networks with Cutout. arXiv:1708.04552, 2017.
- weiaicunzai. pytorch-cifar100 (GitHub repository). https://github.com/weiaicunzai/pytorch-cifar100
3 comments
Response to Reviewer ajEM (Part 2/2)
- CIFAR-100 subspace analysis and dataset consistency. We agree the two datasets were handled inconsistently, and the revision makes the analyses parallel.
- We added the mechanistic subspace analysis on CIFAR-100 using CIFAR-100-C (Sec. A.1.5, Fig. A.6), as the reviewer suggested, including the eviction metric and projection recovery.
- We added a second medical dataset, OrganAMNIST (Sec. 5.3, Fig. 5), so the central mechanism is now shown on two datasets rather than one, which directly addresses the “mechanism rests on a single dataset” concern.
- f and s are now reported across datasets, and the OOD evaluation is MedMNIST-C / CIFAR-100-C throughout rather than color jitter.
We hope these revisions resolve the reviewer’s concerns, and we would welcome guidance on whether any specific experimental comparison would most strengthen the paper in their view.
What is the utility of these findings?
I thank the authors for the substantial revision. Making the analyses more consistent across datasets largely resolves my fourth point, and the reframing largely resolves my concern about overstated performance claims. However, the revision also makes it clear that the regularizers bring little performance benefit on either clean or corrupted data. I acknowledge that we see slight improvement on corrupted BloodMNIST.
The authors general point on reframing away from performance gains is noted, but that leaves a concern: what is the utility of these regularizers? Showing how a technique works is only impactful if it is linked to some utility increase (e.g. performance, robustness, efficiency, interpretability, scalability, fairness, understanding, etc.). The current version seems to suggest that the utility is: (a) in "understanding" the geometric structure of the representation or (b) the idea of projecting "out" nuisance factors.
The understanding is limited to the knowledge that when SSF is regularized latent mean class vectors tend to be more orthogonal and when SNF is applied noise is isolated in subspaces (at least for the two medical datasets) that can potentially be projected out. These are properties that have not been shown to be missing when employing similar existing regularizers from the literature.
As far as I can see, the "projecting out" requires that we have representative clean and corrupt versions of samples with the specific type of corruption that we want to project out (it does not generally transfer between corruption types). With this type of ground truth you could fine tune a linear head to be robust to the corrupted samples (this is not compared against), potentially not needing the SNF regularizer. The conclusion suggests that the regularizer might help with cross-clinic generalization, but this is speculation since it is untested.
Without the performance gains I struggle to see the impactfulness of the technique even if we assume that all findings are valid and sound. My concerns about baselines and tuning remain but I think this question on utility needs to be answered first.
Response to Reviewer ajEM (Part 1/2)
We thank the reviewer for this thorough critique. The revision addresses most of these points directly, and several were valid errors in the submitted version that we have corrected. We group our response by the reviewer’s four concerns.