- Rating
- 2
- Confidence
- 1(−1)
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper analyzes SGD convergence rates for linear regression and examines how curvature, noise, and the choice of output rule (such as full averaging and tail averaging) affect prediction error. To show this, it does not require the gradient-noise covariance to be uniformly bounded relative to the curvature matrix. Instead, it assumes that this covariance has finite trace. It also provides experiments on synthetic data to validate its theoretical results.
Critical strengths
The paper extends prior convergence guarantees under a weaker noise covariance assumption, requiring a finite trace rather than uniform bounds relative to the curvature matrix.
Critical weaknesses
-
The core contribution feels a bit incremental. Since linear regression is already well studied, the main novelty here is proving a sharper characterization under a relaxed noise assumption. It would help to clarify what's new in the proof technique and why this broader assumption matters in practice. If the theory is somewhat incremental, demonstrating these ideas in more realistic settings would make a stronger case for their importance.
-
The presentation could be a bit clearer and easier to navigate. With the dense notation and shifts between definitions and assumptions, it takes some extra effort to follow the main thread. Adding more intuitive commentary and walking through how the results connect would really help improve the flow. Additionally, making the convergence rates more explicit would strengthen the takeaway. Because the current bounds rely heavily on auxiliary kernels and spectral functionals, readers may not immediately see how the rates improve as iterations progress.
Questions for authors
see weaknesses above