- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper studies distributed Hessian sketching, where each worker uses a small local sketch to approximate a shared Newton direction and the server averages the resulting directions. Below a sketch-size threshold, stable shifted-inverse methods retain a directional bias that cannot be removed by adding workers. The paper instead replacing the local inverse with a negatively shifted gradient flow stopped at finite time. Under specific head–tail spectral and gradient assumptions, it proves an improvement in limiting mean-direction loss from to as . It also provides finite-worker averaging guarantees. Finally, the paper gives an experimental evaluation that demonstrates the advantage of the proposed algorithms.
Critical strengths
- The paper identifies a clear limitation of distributed averaging: additional workers reduce variance but cannot remove the systematic bias of small local sketches. The proposed method addresses this limitation without increasing the local sketch size. The main theoretical separation is also interesting.
- The paper gives a numerical evaluation that supports its theoretical analysis and demonstrates the advantage of the proposed methods.
Critical weaknesses
-
The writing of this paper is not easy to follow, especially in the abstract and introduction. For example, techinical terms such as “fold threshold” and “stable endpoints” are introduced without sufficient explanation. The meaning of the error measure and the regime in which the improved rate is beneficial are also unclear at first reading. I suggest the authors revise these sections to explain the main results and their assumptions more clearly.
-
(Minor) The negative-shifted finite-time filter is adopted from Zhao (2026), so the main novelty lies in its Hessian-sketch analysis and distributed guarantees.
Questions for authors
See the above question.
3 comments
Thanks to the authors for the detailed response. I think my previous questions have been addressed, and I have raised my score to 3. One remaining concern, as also acknowledged in the rebuttal to Reviewer Em8f, is the lack of theoretical guarantees when the sketch size remains fixed as the dimension grows.
However, I still believe the presentation could be further improved, and I encourage the authors to polish the writing in the next revision.
Thank you for reading our responses, including those to the other reviewers. We are very grateful for your interest in our work, and we promise to polish the writing in the next revision to make the presentation clearer.
Response to Reviewer ju6x
Thank you for the careful reading and for pointing to the terms that need definitions. In the revised PDF (changes highlighted in teal), the introduction now intuitively explains the fold, ζ, the error measure, the regime the paper studies and the main result, and the abstract defines the fold and ζ.
Here is a summary of the clarified definitions:
W2 (novelty relative to Zhao, 2026). Yes: the worker filter is the one Zhao (2026) introduced for linear regression, and the revised introduction now says so. However, discovering and overcoming the fold barrier in distributed second-order optimization requires fundamentally new analysis. Our problem setting, impossibility barrier, loss geometry, and theoretical guarantees are distinct:
We would be glad to clarify any remaining questions. Thank you again for your constructive feedback.