- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies toy-sized ReLU networks and their training artefacts when tasked with learning simple piecewise functions. It is hard to follow along and to locate the research question it tries to tackle. The empirical data fails to add value to the story, the theorem comes without a proof and formal definitions of the vocabulary used, and the appendix remains largely unrelated.
Critical strengths
None
Critical weaknesses
The goal of the paper is not clear, and the execution is unclean. The learnability of piecewise linear functions with ReLU networks is unsurprising. The empirical examples and data do not follow a clear, research-question-guided setup and remain meaningless to me.
Questions for authors
What are the main goals of the work and what are you trying to show with the experiments? What does Theorem 3.1 add to our knowledge about universal approximation/convergence results for ReLU networks and gradient descent? How does it link to stochastic gradient descent?
Thank you for your comments and questions. We appreciate your feedback on the research question and on what the experiments are intended to show. We will address these points in future revisions and improve the presentation of the definitions and theoretical arguments.
Your suggestions will also inform our future work, where we plan to study more general settings with more training samples and deeper networks.