- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies generalization bounds for neural networks with saturating activations such as tanh. The authors use the pushforward density to avoid the infinite bounds caused by inverse Jacobians in prior Koopman-based approaches. They derive data-dependent Rademacher complexity bounds and support the analysis with both theoretical and experimental results.
Critical strengths
None
Critical weaknesses
My main concern is the paper’s limited scope and significance. It mainly addresses a technical limitation of an existing Koopman-based framework, but does not really show what new insight into neural networks or generalization follows from this. The central idea is essentially a standard change-of-variables argument, and the more involved technical results do not add much conceptual understanding beyond the specific problem considered. I also have doubts about the theoretical correctness.
Questions for authors
-
Could the authors clarify the significance of studying this question?
-
The proof of Theorem 4.2 seems to have gaps in both the uniform parameter Lipschitz argument and the use of Dudley’s entropy integral. Could you clarify these issues?
-
Specifically, in lines 1125–1132, the uniform parameter Lipschitz bound relies on bounded layer inputs. However, the first-layer input can be unbounded (e.g., Gaussian). How does the claimed bound hold in this setting?
-
In lines 1139–1155, how is the application of McDiamid's inequality and Dudley's entropy integral justified? The increments are not generally centered, and the pointwise upper bound does not imply the claimed bounded-difference estimate.