- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies energy proxies in three-dimensional ReLU networks with uniformly contractive original updates. The central finding of the paper is that the least rotational defect can select an arbitrarily inaccurate energy proxy, even when the original recurrence is uniformly contractive. The experiments provide a comparison of input-excited rotation ranks proxy errors and rotational defect, using ReLU classifiers and a recurrent transformer.
Critical strengths
The paper presents an analysis framework for energy proxies for a specific family of ReLU networks. In the defined scope, it contributes with a failure theorem for energy proxy selection, input and curvature dependent reliability conditions, and controlled and trained network evidence. The paper also presents strong mathematical theory and derivations supporting the claims.
Critical weaknesses
-
Presentation and clarity could be substantially improved. In general, the paper is difficult to follow. The abstract appears confusing because the sentences are not sufficiently coherent, and the terms are not clearly defined. The paper would benefit greatly from overview illustration.
-
The paper is not sufficiently self-contained. The introduction should do a better job introducing central terms, such as energy proxies, asymmetric interactions, rotational defects, contractive updates, and equilibria.
-
The contributions of the paper, albeit valid in the defined scope, appear quite niche, and the authors provide little insight regarding their utility in broader context.
-
The mathematical formulations appear interesting but are difficult to follow as the paper does not provide sufficient context as to their motivation or method.
Questions for authors
-
Could the authors provide clear definitions of energy proxies, proxy errors, field approximation, answer fidelity, and rotational defect?
-
What is “a three-dimensional ReLU family” and how is it defined?
-
How do the proposed analysis framework and results generalize to dimensions greater than 3?