- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work investigates how infants' early degraded vision shapes visual representation learning. To simulate degraded vision, the authors generate line drawings from photographs. Then, they train vision models while shifting the training data from line drawings to photographs. Controlled experiments show that the curriculum improves the learning efficiency and the shape bias of the models.
Critical strengths
Evaluations are extensive and appropriate.
Critical weaknesses
This work is not mature enough for presentation in this conference.
- The paper does not cite major studies that directly relate to the present research questions [1,2].
- The paper presents methodological issues. First, measuring the cosine similarity between CLS tokens does not reflect changes in the representational geometry. If all representations of layer l are rotated by the same angle at layer l+1, the cosine similarity between representations at layers l and l+1 would be small. Yet the relationships between l+1 representations are identical. To address this limitation, I recommend to perform a representational similarity analysis (RSA). Second, the accuracy on Imagenet-100 is relatively low, compared to results obtained with classical SSL methods (e.g. [3]). Third, controlled experiments are required to verify that the performance gains do not stem from increasing the number of training images: Photo -> Line and Photo + Line.
- In cognitive science, the link between the line drawings and the early degraded vision is unclear. One major property of early degraded vision is low contrast sensitivity. Thus, low spatial frequencies (such as thin edges) should be difficult to detect, which is not the case there. In machine learning, the experiments do not meet the standards of the field, which is Imagenet-1k training.
- Some paragraphs are hard to follow, such as "Concentration of channel responses".
[1] Cai, Y., Lin, Q., Nunna, B. S., & Zhang, M. (2026). Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13766-13780). [2] Lu, Z., Thorat, S., Cichy, R. M., & Kietzmann, T. C. (2026). Adopting a human developmental visual diet yields robust and shape-based AI vision. Nature Machine Intelligence, 1-14. [3] Aubret, A., Teulière, C., & Triesch, J. (2026). Seeing the whole in the parts with self-supervised representation learning. Neurocomputing, 132973. desvribe the method in the related works
Questions for authors
I will increase my score to acceptance if the authors:
- clarify how the present results relate to [1,2], especially [2] regarding the shape bias.
- clarify that the improvements come from the proposed curriculum, and not from using more images.
- use RSA to compare representations across layers, or remove the claims related how representations change through the neural hierarchy.
- explain the precise relationship between the photographs and early degraded vision.
- test the model on Imagenet-1k.
If that is not possible, I encourage the authors to pursue their research direction, re-frame the story, and resubmit the paper if the preliminary results are confirmed.
3 comments
I thank the authors for the response, which addressed some of my concerns.
Self-supervised accuracy: that is true, however different studies have also tested a Resnet18 and obtained a better performance than what is obtained in the present paper [1]. The exact SSL method do not exactly match, but they tested methods that are expected worse than DINO, which is presented in the present paper. Could the authors provide the results of their hyper-parameter analysis (especially, learning rates) during pre-training ?
ImageNet-1K and scaling: the authors report only the training loss of the ViT. Performance on a validation/transfer dataset must be reported. Focusing on a YOLOv8 backbone is also unusual; standard practice is to train a RN50 or ViT-S/B on Imagenet-1K. Comparison against baselines (Table 5), and most of the experiments, should be performed on Imagenet-1k. STL10 contains only 10 classes, there is a plethora of visual shortcuts (e.g. water = boat, sky=airplane) than can be learned to perform the classification task. Thus, performance on this dataset is not informative of the quality of a learning model, in general. If computational resources are a limitation, I encourage the authors to run the most important experiments on Imagenet-1k, and eventually do analyses on Imagenet-100, or at least Tiny-Imagenet/CIFAR100.
[1] Ozsoy, S., Hamdan, S., Arik, S., Yuret, D., & Erdogan, A. (2022). Self-supervised learning with an information maximization criterion. Advances in Neural Information Processing Systems, 35, 35240-35253.
Thank you for your follow-up and for recognizing the concerns addressed by our revisions.
1. Self-supervised baselines
We did not conduct an independent hyperparameter sweep. We used solo-learn and consistently reproduced its published ImageNet-100 results for ResNet-18 with 400 pretraining epochs: 74.92% for DINO and 74.28% for SwAV under offline linear evaluation.
Table 1 in Ozsoy et al. (2022) also draws its ImageNet-100 ResNet-18 baseline results from solo-learn. Its BYOL (80.32%), MoCo-V2 (79.28%), and VICReg (79.40%) results match those in solo-learn’s evaluation table, which also reports the DINO and SwAV baselines reproduced in our study:
https://github.com/vturrisi/solo-learn#imagenet-100
Although reproducing these results does not establish that the configurations are optimal, we are confident that our Photo-only runs match the published baselines. Under matched training budgets, Line→Photo improves DINO from 74.92% to 77.22% and SwAV from 74.28% to 77.62%.
2. ImageNet-1K and evaluation scale
We agree that repeating the principal curriculum comparisons on ImageNet-1K with ResNet-50 or ViT would strengthen the paper. Unfortunately, we lack the computational resources and time to complete these experiments within the rebuttal period. We also agree that the ImageNet-21K training-loss result alone does not establish improved validation or transfer performance.
Our evaluations beyond STL10 include supervised recognition on ImageNet-1K with a YOLOv8 backbone, where accuracy improves from 67.87% to 68.82%. On ImageNet-100, we evaluate supervised recognition, blur controls, and DINO and SwAV pretraining. We also evaluate object detection on PASCAL VOC and transfer to depth estimation on NYU Depth V2 and semantic segmentation on ADE20K.
These experiments provide evidence across datasets and tasks, but we recognize that the lack of standard-backbone ImageNet-1K comparisons is a limitation of the current study.
We hope the revisions and additional experiments have addressed your other concerns about the study’s framing, controls, and representational analyses. Thank you for your continued consideration. We would be happy to clarify any remaining questions.
We thank the reviewer for the detailed feedback and for recognizing the breadth of our evaluations.
1. Related work and shape bias
We now discuss Cai et al. and Lu et al. in “Developmentally Motivated Visual Learning” and compare adaptations of their sensory transformations within our supervised STL10 setup. Both sensory and structural curricula increase shape bias. Line→Photo achieves the highest recognition accuracy among the tested conditions, while blurred grayscale yields the highest shape bias; see “Developmental sensory curricula.” Our contribution examines the broader representational and transfer consequences of early structural learning.
2. Training budget and controls
The recognition and mixing controls use matched total training budgets, counting all stages. Drawings come from the same underlying photographs.
Photo+Line achieves the highest recognition accuracy (75.19%), while Line→Photo produces the strongest shape bias (61.45%). Both outperform Photo→Mixed (69.09% recognition, 38.18% shape bias) and have higher mean neuronal structure sensitivity at all four stages. The overarching response provides the full comparison. Photo→Line is reported in “Recognition across training and evaluation modalities.”
3. Representational geometry
We replaced cross-layer CLS cosine alignment with RSA in the ViT. Using the same evaluation images, we construct cosine-distance dissimilarity matrices and compute Spearman correlations between each layer and its model’s own final layer. These correlations are higher for Line→Photo at every intermediate layer; see “Representational geometry across network depth.”
4. Developmental interpretation
Line drawings serve as a computational surrogate for early structural representations, rather than infant retinal images. The pretrained generator supplies the structural prior; we do not assume it is innate in infants. Our hypothesis concerns how early structural experience guides later learning. Blurred-line controls retain recognition gains at all three blur levels, showing that the benefits persist when contour precision is reduced; see “Contour precision.”
5. ImageNet-1K and scaling
On ImageNet-1K, Line→Photo improves recognition accuracy with a YOLO-v8 backbone from 67.87% to 68.82%. Our ViT-B/16 experiments on ImageNet-21K provide further evidence under matched optimizer-step budgets. Together, these results show that the benefits of early structural learning extend to larger datasets and models.
6. Self-supervised accuracy
Our DINO and SwAV runs use ResNet-18 and standard solo-learn implementations. Aubret et al. use CO-SSL objectives and ResNet-50/RF-ResNet-50 backbones, so their absolute accuracies are not a matched comparison.
7. Writing clarity
We rewrote “Concentration of channel responses” to explain the averaging, ranking, and normalization explicitly, and distinguish this measurement from PCA dimensionality and RSA.
We hope these revisions address your concerns about the interpretation and soundness of the study.