- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This submission studies the concept of inheritance, i.e. the idea that techniques can be transferred between different families of architecture within a proper formal framework. Under the formalization of neural network provided in the manuscript, the authors show that generalized convolutional networks (GCNN) can be regarded as a more general model class than generalized feedforward networks (GFFN). The authors use this formalization also to motivate a fine-tuning technique, that they call projection. This technique freezes convolutional filters but combines them with learnable weights.
While the submission’s overall premise of aiding the transfer of insights between architectural families is interesting, I found major parts of the paper rather disconnected from this motivation. I did not find any mistakes in the paper’s proofs. The experimental evaluation uses a large variety of datasets and is well documented.
Overall, this paper is a potentially interesting contribution to CNN finetuning but in the current form overstates its contributions and could benefit from more detailed experimental evaluation.
I comment on critical strengths, weaknesses, and questions below. I am open to increasing my score if weaknesses are addressed during the rebuttal phase.
Critical strengths
The experimental results suggest that the proposed projection method has promising performance in CNN finetuning on many image classification datasets.
Critical weaknesses
I believe that motivation and experimental evaluation are somewhat disconnected. The authors motivate their work with the transfer of techniques or practices between different model families, but their evaluation focuses only on fine tuning of pretrained CNNs. It is not clear to me how this connects to the idea of inheritance. The paper appears to me more convincing as the proposal of a CNN finetuning technique than as a general study of inheritance.
It is known that an FNN with an imposed weight sharing pattern is equivalent to a CNN. I believe it would be good to discuss how the authors’ formalization relates to this perspective and potentially soften the strong claim that “There is currently no inheritance framework between common deep learning models.” [l.71]
Questions for authors
The authors report standard deviations for their accuracy results. Across how many samples is this standard deviation calculated? What constitutes a sample (e.g., is it an independently chosen seed)?
How does the authors’ approach relate to the one developed in Mo, Cheng, & Fan (2019) arXiv:1906.04950? (I acknowledge that this work is not peer reviewed, but think discussing it might still be interesting as it seems closely related.)