- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the representational limitations of bounded parameterizations with convex or weakly convex squared losses. The main result shows that sufficiently small negative curvature forces the prediction sign matrix to have sign-rank at most (q+1), where (q) is the parameter dimension. Combining this with a classical Hadamard sign-rank lower bound, the authors obtain an exponential dimension separation from compact ReLU representations. They also provide a construction showing that the leading constant in the curvature threshold is asymptotically optimal.
Critical strengths
The paper establishes a quantitative relationship between weak convexity and sign-rank, with a simple and self-contained proof. The construction showing that the leading constant in the curvature threshold is asymptotically tight is also a nice addition.
Critical weaknesses
My main concern is the limited novelty and significance of the results. The exact convexity phenomenon is closely related to existing results on convex representations and sign-rank, particularly Chornomaz et al. (2025), and the geometric argument is quite elementary in this specialized setting. The main weak-convexity theorem provides a quantitative extension, but the extension itself seems technically straightforward, relying on a weak-convexity inequality followed by the same convex-hull separation argument. The exponential ReLU separation is then largely a direct consequence of classical sign-rank lower bounds. Overall, I find it difficult to identify a substantial new conceptual or technical contribution.
I also find the writing rather formulaic and repetitive, at times reading like AI-generated text. In particular, the manuscript spends considerable space on qualifications about what the results do not establish, while offering relatively little deeper intuition or motivation for why the results are important. This further weakens the presentation of an already fairly narrow contribution.
Questions for authors
Could the authors clarify the novelty of Theorem 2 relative to Chornomaz et al. (2025)? The underlying convex-hull separation argument seems essentially the same, with the main additional step being a straightforward application of the weak-convexity inequality. Is there a more substantive technical distinction?
It seems that Theorem 2 can be derived fairly directly from Theorem 3 of Chornomaz et al. (2025) by constructing a suitable convex surrogate from the weakly convex squared loss. Is there any obstacle to this reduction? If not, what is the substantive novelty of Theorem 2 beyond the existing result?