- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper extends frequency analysis to texture-driven domains and identifies low-frequency shortcuts. The main claim is that neural network predictions are dominated by low-frequency components even though higher-frequency components have higher predictive power. The paper introduces a pruning strategy that eliminates low-frequency components from training and test sets, and empirically validates its effectiveness on both in-domain and out-of-domain data. The paper further studies the impact of large-scale training and image resolutions on low-frequency shortcuts.
Critical strengths
- The paper extends frequency analysis to texture-driven domains and identifies low-frequency shortcuts in neural networks trained on these domains.
- The main claim is supported by extensive empirical validation across various datasets and model architectures under well-controlled validation settings.
Critical weaknesses
- W1: The paper focuses on identifying low-frequency shortcuts in texture-driven domains but provides limited explanation of why these arise, particularly in texture-driven domains. As a result, it remains unclear whether low-frequency shortcuts arise from texture-driven domains or from the biases of neural networks. Although the paper claims that texture/shape bias of neural network architectures is orthogonal to the goal of this work, further discussion or analysis of the underlying causes would help clarify the phenomenon.
- W2: The paper lacks of both conceptual and technical novelty. Prior works have investigated frequency shortcuts and effectiveness of high-frequency components, leaving domain extension as the main novelty. Also, frequency analysis relies on commonly used methods with an additional pruning strategy, which is closer to an engineering approach than a novel algorithm.
That said, I am willing to revise my rating based on the discussion.
Questions for authors
- Q1: The finding that pruning low-frequency components leads to better generalization seems task-specific, as texture-driven tasks require texture-specific information, and results with non-texture-domain dataset (e.g., CIFAR10) do not exhibit a similar phenomenon. Is this observation generalizable across different domains?
- Q2: What is the purpose of the paragraph `Prior Theoretical Work.' in Section 5? Current version of the paper does not provide why this discussion is relevant.
- Q3: In Section 6, the paper states that `generalization performance decreases if the corruption corrupts the HFCs that the model has started to rely on'. However, in some datasets performance drop on OOD data occurs after pruning far fewer components than ID data. Isn't it possible to say that the model also relied on some part of the HFCs rather than dominant LFCs?
- Q4: In Section 7, frequency analysis on GalaxyMNIST and EuroSAT is said to have a skewed spectral behavior towards LFCs, but pruning LFCs results in moderate drops in in-domain accuracy unlike other texture-driven datasets. This raises the question of whether low-frequency shortcuts and improvements from pruning are consistently observed across texture-driven datasets.
- Q5: Section 4 states that validation accuracies are reported in the main paper. Is there a reason why test accuracies are not used for the analysis?
- Q6: How exactly is `accuracy contribution' measured?
11 comments
Response to Reviewer CDzK – Part 2
Q1: The finding that pruning low-frequency components leads to better generalization seems task-specific, as texture-driven tasks require texture-specific information, and results with a non-texture-domain dataset (e.g., CIFAR10) do not exhibit a similar phenomenon. Is this observation generalizable across different domains?
Response: We are not sure what you mean by “different domains”. As we show, out of 12 different combinations of four domains and four models, we achieve improved ID and/or OOD accuracy for 10 cases. This shows that, yes, our results do appear in multiple domains again and again.
If you mean whether it generalizes to all the domains, the answer is no. Low-frequency shortcuts do not appear in all domains, all the time. This is almost impossible for any real-life phenomenon, unfortunately. We study a large number of domains and use cases and report successful results on a large number of domains while also reporting the failures across different use cases.
It has been repeatedly shown that neural networks are biased towards low-frequency components (our [22; 31; 51; 64; 67; 70; 72; 81; 98; 99; 105]). We are showing that this LFC bias is a significant problem for numerous real-world domains that require information at higher frequencies.
[1] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report No. CNS-TR-2011-001, 2011. https://www.vision.caltech.edu/datasets/cub_200_2011.
Q2: What is the purpose of the paragraph `Prior Theoretical Work.' in Section 5? Current version of the paper does not provide why this discussion is relevant.
Response: Simplicity bias is fundamentally a theoretical concept. We touch on the theoretical aspect and cite the relevant work for the interested reader.
Q3: In Section 6, the paper states that `generalization performance decreases if the corruption corrupts the HFCs that the model has started to rely on'. However, in some datasets performance drop on OOD data occurs after pruning far fewer components than ID data. Isn't it possible to say that the model also relied on some part of the HFCs rather than dominant LFCs?
Response: It is certainly possible that the model relies on some part of the HFCs. LFCs are dominant, yes. That doesn’t mean they are the only source of information.
Q4: In Section 7, frequency analysis on GalaxyMNIST and EuroSAT is said to have a skewed spectral behavior towards LFCs, but pruning LFCs results in moderate drops in in-domain accuracy unlike other texture-driven datasets. This raises the question of whether low-frequency shortcuts and improvements from pruning are consistently observed across texture-driven datasets.
Response: These two datasets, particularly GalaxyMNIST, look for different levels of shape cues. Nevertheless, we agree that low-frequency shortcuts might not surface in all use cases. It is an important vulnerability of neural networks trained with gradient descent that can surface in numerous important use cases, as we have extensively shown.
Q5: Section 4 states that validation accuracies are reported in the main paper. Is there a reason why test accuracies are not used for the analysis?
Response: Yes, having both validation and test set results shows that a system designer or deep learning engineer can look for low-frequency shortcuts by using his/her dataset and deploy it in real time and expect similar behavior. Having both validation and test-set results makes our work practically more relevant, showing that, by pruning LFCs, we are not overfitting to the validation set, but safely generalizing to any unseen images.
Q6: How exactly is `accuracy contribution' measured?
Response: We fix the training set to a specific number of LFCs pruned. We then measure the validation accuracy by using validation images having the exact same number of pruned LFCs as the training set. Let’s call this value A. We then modify the validation set images by pruning the same number of LFCs and also one more specific diagonal, say, the 123rd diagonal. We then re-measure the validation accuracy by using these modified validation images. Let’s call this value B. The difference between A and B, i.e., A-B, gives the accuracy contribution of the 123rd diagonal. We repeat this for all diagonals from 1 to 447, and obtain a spectral graph, as one of those in Figure 5. One diagonal is one frequency component. We usually report diagonals from 1 to 224, as after the 224th diagonal, the contributions are usually near zero.
Thank you for your detailed response to my review. I summarize my follow-up comments below.
Revised W1
My original concern was that the paper did not sufficiently explain why low-frequency shortcuts arise in new texture-driven domains.
From the authors’ response, I understand that (1) the shortcuts are due to both the low-frequency bias of neural networks and the high-frequency dependency of texture-driven domains and (2) they distinguish their research question from those addressed by prior work on texture/shape bias.
The wording “texture-driven domains suffer from low-frequency shortcuts” could be read as attributing the phenomenon to the domains themselves. I suggest that the authors clarify in the paper that the observed shortcuts arise from the interaction described in their response.
Revised W2
The authors’ response helped me understand that investigating the value of high-frequency components (HFCs) and showing that reliance on low-frequency components (LFCs) can limit performance are distinct research questions. However, I still have concerns about novelty, and I would appreciate clarification of the claim about data reduction.
(1) Conceptual novelty of the explanation for low-frequency shortcuts
The authors cite prior work for the premise that neural networks favor LFCs:
“Neural networks are biased towards learning from low-frequency components (LFCs) (our [22; 31; 51; 64; 67; 70; 72; 81; 98; 99; 105]).”
They then explain the observed shortcuts as follows:
“So, the shortcuts are due to both the low-frequency bias of neural networks and the high-frequency dependency of texture-driven domains.”
I agree that extending the analysis to new domains is one of the paper’s strengths. The experiments also show that the models rely on LFCs in texture-driven domains and that pruning LFCs can improve performance. However, it remains unclear what new insights the paper offers into low-frequency shortcuts in texture-driven domains beyond showing how the known preference for LFCs manifests in texture-driven tasks.
For example, how does the texture-driven nature of these domains contribute to shortcuts? What does the paper add beyond applying existing theory to explain why the model’s preference for LFCs gives rise to shortcuts in these domains? Based on the evidence presented so far, the main contribution is limited to an empirical extension of prior work to new domains, and I remain uncertain about the conceptual novelty. Please clarify what new insight the paper offers into how the shortcuts arise.
(2) Distinction from prior work that uses frequency analysis to improve performance
The authors highlight that “We show that you can achieve similar and even better results by simply studying the data.” Prior work has also used frequency analysis to understand differences in model performance and to guide improvements. Section 2 describes DCT as a commonly used method, and Appendix N cites prior work that prunes frequency coefficients by diagonal. Bai et al. [1], for example, use DFT to separate the low- and high-frequency components of images, analyze the performance of ViTs and CNNs on those components, and further investigate how existing data augmentation methods affect ViTs’ use of HFCs. In their own words:
… we make a hypothesis that ViT models are less effective in capturing the high-frequency components of images than CNN models, and verify it by a frequency analysis. Inspired by this finding, we first investigate the effects of existing techniques for improving ViT models from a new frequency perspective, and find that the success of some techniques (e.g., RandAugment) can be attributed to the better usage of the high-frequency components.
Given this prior work, the claim that studying the frequency content of the data can improve performance leaves the distinction unclear. Please explain how LFC pruning in texture-driven tasks differs from prior approaches and what new empirical findings this analysis adds.
(3) Meaning of “50% less data”
The authors state, “Particularly, for Tiny-Gen-Images, we can remove 50% of the data and achieve 10% higher accuracy.” The response leaves it unclear which experiment supports these findings. Figure 5 labels the setting for TinyGenImgs as “224 LFCs pruned,” which appears to correspond to setting approximately half of the DCT coefficients in each image to zero. Section 2 (p. 3, line 123) states that the images are then transformed back into RGB for training and inference. Please identify the experiment behind this claim and specify whether “50% of the data” refers to the fraction of frequency coefficients set to zero, a reduction in the number of training images, or a measured reduction in the amount of data stored, transferred, or processed.
Response to Reviewer CDzK – Part 1
W1: The paper focuses on identifying low-frequency shortcuts in texture-driven domains but provides limited explanation of why these arise, particularly in texture-driven domains. As a result, it remains unclear whether low-frequency shortcuts arise from texture-driven domains or from the biases of neural networks. Although the paper claims that texture/shape bias of neural network architectures is orthogonal to the goal of this work, further discussion or analysis of the underlying causes would help clarify the phenomenon.
Response Part 1: Neural Network vs. Texture-Driven Domains. Neural networks are biased towards learning from low-frequency components (LFCs) (our [22; 31; 51; 64; 67; 70; 72; 81; 98; 99; 105]). Texture-driven domains perform classification using fine-grained higher-frequency details. Hence, neural networks’ bias towards LFCs forms an easy shortcut for texture-driven classification tasks, resulting in suboptimal accuracy. Removing the shortcuts from the training and test sets improves the accuracy, sometimes as high as by up to 10%.
So, the shortcuts are due to both the low-frequency bias of neural networks and the high-frequency dependency of texture-driven domains.
Response Part 2: Texture/Shape Bias vs. Low-Frequency Shortcuts. Yes, these two dimensions are completely independent. Looking for high-frequency information to differentiate different types of textiles or fake-vs-real images is different than looking for low-frequency information when images are corrupted with random textures. Geirhos et al. (our [35]) change the texture of images in shape-driven tasks. This is not what we are doing. We analyze classification tasks that look for high-frequency information. None of the existing work has looked at such datasets. Furthermore, what level of texture corruption changes what part of the frequency spectrum and how much are open research questions, as we also say in our related work.
W2: The paper lacks both conceptual and technical novelty. Prior works have investigated frequency shortcuts and the effectiveness of high-frequency components, leaving domain extension as the main novelty. Also, frequency analysis relies on commonly used methods with an additional pruning strategy, which is closer to an engineering approach than a novel algorithm.
Response: Our work does not investigate the effectiveness of high-frequency components. It investigates the harm of low-frequency components, which none of the existing work has done. Furthermore, none of the prior work has performed an interventional analysis, where they change the training set. They only perform observational analysis, where they modify only the test set. Almost every single experiment in our work requires re-training the model from scratch, identifying a new shortcut based on how neural networks work. It has implications for interpretability and predictability, as we are investigating spectral behavior in connection with ID/OOD accuracies.
Yes, new domains are a great strength of our work. It is not the only strength, however. We also show groundbreaking results. These domains are already being studied by numerous groups using sophisticated network architectures or training regimes [1, 2]. We show that you can achieve similar and even better results by simply studying the data. This opens a whole new line of research. Now that the datasets are free of low-frequency shortcuts, what kind of network architectures, data augmentation algorithms, and training regimes should we use to achieve an even higher accuracy? How can we create a pre-training set for a new foundation model while studying shortcuts side-by-side? These are exciting and groundbreaking research questions. As we have AI in our daily lives more and more, identifying shortcuts and transparently improving model quality by studying its learning behavior, e.g., spectral behavior, is key to making AI models more reliable and predictable.
Our results are even more interesting, as we show that we can achieve a higher accuracy with less data. Particularly, for Tiny-Gen-Images, we can remove 50% of the data and achieve 10% higher accuracy. This is unique. The conventional approach to neural networks and data compression is that reducing data typically causes loss of quality, or at best, preserves the quality. For the first time, we are showing that you can actually achieve a higher quality with less data. This is groundbreaking, as the cost of storing, moving, and processing data is ever increasing.
[1] Jia Xue, Hang Zhang, Kristin Dana. Deep Texture Manifold for Ground Terrain Recognition. CVPR 2018. https://arxiv.org/abs/1803.10896.
[2] Mingjian Zhu, Hanting Chen, Mouxiao Huang, Wei Li, Hailin Hu, Jie Hu, Yunhe Wang. GenDet: Towards Good Generalizations for AI-Generated Image Detection. https://arxiv.org/abs/2312.08880.