- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies stationary last-iterate distributions of constant-step least-squares SGD under temporally dependent observations. Its central contribution is an explicit separation between distributional approximation and moment stability: a persistent-refresh design with standard Gaussian marginals produces a stationary error with every positive moment infinite, although the normalized stationary error converges to a Gaussian as the stepsize decreases.
Critical strengths
1.The one-coefficient anti-concentration argument in Appendix G convincingly explains how sufficiently long runs of an extreme covariate overwhelm their probability of occurrence. The coexistence with uniform geometric ergodicity makes the example particularly useful. 2. Lemma A.2 handles noncommuting positive-semidefinite matrices, and the block argument produces a common bound over sufficiently small stepsizes without assuming integrability of the exponential prefactor. 3. Weak convergence, conditional TV convergence, unconditional first-order approximation, and feasible coverage have separate assumptions. The authors correctly emphasize that the limiting Gaussian covariance need not be the limit of stationary covariance matrices.
Critical weaknesses
-
The constants and admissible stepsize threshold depend on the design law, while the corrected approximation has an unspecified remainder. The schedule in Corollary 5.6 is asymptotically sufficient for each fixed admissible law, but does not provide a usable finite-stepsize guarantee. This limitation is visible in Table 7: for the log-Weibull model with and , baseline coverage is , whereas corrected coverage is . The correction remains less accurate despite the small stepsize.
-
Corollary 5.4 uses population and . The feasible variance-estimation argument establishes asymptotic coverage, but consistency alone does not preserve an coverage expansion. The manuscript does not provide an implementable estimation scheme with conditions ensuring that calibration error is negligible at the required scale.
-
The general projective-score theory is broad, but the sharp TV expansion relies on an exact Gaussian mixture representation. For non-Gaussian errors, the established result is a weak limit.
-
Minor issues:
- Theorem 4.1 should explicitly reference the standing causal Bernoulli-shift framework. A finite design second moment and alone would not imply the result for an arbitrary stationary design; the block-ergodicity argument is essential.
- A compact assumptions table would help readers distinguish pathwise existence, the projective-score CLT, the quenched CLT, conditional TV convergence, and unconditional first-order TV approximation.
- The distinction between a correction to the approximating Gaussian law and an expansion of stationary covariance is crucial and should remain prominent wherever “covariance correction” is used.
- Appendix N.5 should explicitly identify the design used for the non-Gaussian-error and two-time OU experiments. Specifying does not uniquely distinguish the refresh and AR constructions.
- The small legends and uncertainty bars in the main figures make some comparisons difficult to inspect. Enlarged panels would improve readability.
Questions for authors
-
Can you provide an explicit finite-stepsize remainder bound or a practical diagnostic for when the correction improves approximation, at least for the Gaussian-refresh family? How does the onset depend on persistence ?
-
What conditions on estimators of and , and on their calibration sample sizes, preserve the coverage improvement? Can this be established when calibration observations overlap the SGD history?
-
Under independent Gaussian errors, when should a practitioner use the deterministic first-order correction instead of the exact conditional procedure in Appendix K? Please clarify the computational and precision tradeoffs.
-
Can you add an asymmetric non-Gaussian error distribution and a multidimensional independent-Gaussian-error experiment that directly tests the matrix correction in Theorem 5.2?