- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper aims at improving our understanding of recognition strategies in humans and neural networks. In particular, it compares humans and neural networks (e.g., CNNs, ViTs) on a specific case of partial occlusion, i.e., camouflage occlusion. The occlusion is carried out by vertical, horizontal or diagonal occluding stripes that are either of the dominant color of the background, the dominant color of the object or for control in black. The main results show that performance of humans as well as neural networks (to differing extends) is decreased when images are occluded with camouflage stripes. Additionally, the paper proposes three metrics based on the attribution technique GradCAM++ to further analyze the bahavior of neural networks when exposed to camouflage occlusion.
Critical strengths
-
Relevant and interesting research question: Investigating how contextually colored occlusions affect object recognition in humans and neural networks is a genuinely interesting research question.
-
Exploration of training augmentation with colored-occlusion is a reasonable approach to investigate whether models can learn increased robustness to camouflage occlusions.
-
I also value the reproducibility efforts of the authors by providing the code for the experiments.
Critical weaknesses
-
Human–model comparison uses different tasks: without making the task better comparable, it is hard draw conclusions. Generally accuracies can be a supplement, but models would have to be evaluated on the exact same tasks as the humans were evaluated to compare them in a fair way (potentially with agreement of high-level predictions between the camouflaged image and the original)
-
CNN–ViT training recipes are not controlled: CNN recipes used are very old, the comparison to ViTs may be confounded by weaker training recipes. The ConvNeXt paper [1] experimentally showed that the majority of differences in performance between CNNs and ViT stem from different training recipes used. Moreover, Burgert et al. [2] showed that state-of-the-art training makes CNNs very similarly robust to feature suppression as ViTs, most differences between the architectures are attributable to different training recipes, too.
-
GradCAM-based metrics offer limited additional insights: GradCAM is architecture and layer-dependent. It is hard to attribute GradCAM visualization difficulties for corrupted images with camouflage occlusion to the actual difficulties of the model to recognize the object and properties of the attribution method itself. The observed artifacts may reflect sensitivity to out-of-distribution corruptions or limitations of GradCAM, rather than meaningful differences in the models' internal representations or recognition strategies.
-
Colored-CutOut augmentation experiment evaluation is limited: only one small dataset is used and the difference between CutOut black (normal CutOut) to colored-Cutout with the dominant colors of the image is too small to judge the positive effect on simply one dataset. For reasonable ViT-training and results the dataset size is too small and/or the used standard augmentation pipeline is too weak (shown by the performance which is only about 2/3 of the ResNet). Further, a control via random colors chosen for colored-CutOut is missing. In the current shape the contribution and potential conclusion that can be drawn from that experiment are limited.
-
Limited interpretation and connection to existing literature: While the paper cites some relevant prior work, the discussion and interpretation of results is largely descriptive. Given that the aim of the paper is to improve our understanding of recognition strategies in humans and neural networks, a deeper discussion of potential mechanisms, alternative explanations and connection to the broader literature on comparing humans and models would be beneficial. As an example: Burgert et al. [2] or Subramanian et al. [3].
-
Clarity in writing and formatting concerns: In the current form, the paper does not meet ICLRs standards for clarity in writing, presentation and formatting. To help the authors to improve their manuscript, I list several issues that stood out during the review. This list is not exhaustive, and the manuscript would benefit from a thorough revision.
Miscellaneous
- I would not list the code as a contribution, in final version code should be accessible, but I do not see it as a major contribution; though a relevant thing to provide
Clarity, Presentation and Formatting
- Caption of Figure 1 with typo, this needs to be better explained and be more precise, assume the reader sees the Figure before anything else
- In Introduction, the credit to the original occlusion-based data augmentation should be given to Cutout (you do cite it later, so I would cite it in the introduction too)
- L067/068: Newer perspectives on feature reliance of DNN should be accounted, too [2]
- L077-081: Results cannot „aim“, and saying this work recreates partial occlusion is underspecified; please revisit those sentence and improve clarity
- Contributions do not align well how the abstract is written
- If the generated views (scene, object, control, original) are going to be separate datasets, please make it explicit, I felt like 3.1 and 3.3 did no use the same and precise wording
- L223: is it 320 images or 240 or 480? Until now, I thought 80 augmented (each of them has 4 version) + 160 random images.
- L233: Typo „select models“
- Equations 1,2,3: You first mention NOP, NOR and then AIO and define them in a reversed way, this is confusing
- The Equations are introduced in a confusing way, not every variable is fully explained
- L264/463: If I understood the paper well, I would not write „finetune“ here, better write "manually selected", finetuning is too much associated with training a model
- Limitations and future work should be presented in the conclusion, at the very end
- Format: Text is horizontally too close to some Figures
- Format: not right way of citing the paper in the text, in latex, use e.g., \citep{he_2016} to cite properly
- Typo: e.g., L087 and L102 missing full dots and in the conclusion, too, L399 classical written capitalized
References
[1] Liu et al., "A ConvNet for the 2020s", CVPR 2022.
[2] Burgert et al., "ImageNet-tained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression", NeurIPS 2025.
[3] Subramanian et al., "Spatial-frequency channels, shape bias, and adversarial robustness", NeurIPS 2023.
Questions for authors
Please see critical weaknesses.