- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper aims to propose a refined second-order method for federated learning. The main contribution seems to be tackling the local computation burden of Hessian, which is done through a sampling process. Further, a method is used for computing the Hessian inverse to avoid numerical instability.
Critical strengths
-
Studying of second-order methods for federated learning can be of particular interest if they actually can outperform first-order methods under the same communication/computation budget.
-
The paper aims to solve legitimate limitations of second-order methods: high computation complexity and numerical instability.
Critical weaknesses
The paper suffers from a variety of limitations which makes it unsuitable for publication in its current form:
-
The text of the paper is extremely hard to follow as the text lacks a cohesive story. This can be caused by an excessive use of AI or lack of proofreading by the authors.
-
The contributions of the paper are not crystal clear: (i) The computation of the inverse of Hessian matrix seems to be borrowed from existing techniques in the literature, and (ii) Hessian approximation using column sampling is also borrowed from a more generic techniques in Algebra. As a result, it is not clear what is the ground-breaking contribution in the paper.
-
The simulation results of the paper are extremely limited. The paper compares with two methods: FedAvg (McMahan et al., 2017) and SCAFFOLD (Karimireddy et al., 2020). These are highly outdated methods that are not even second-order methods in nature. If the paper wants to show its advantage over the first-order methods that use the same communication/computation, it should move toward papers published in 2025/2026 and compare its method against state-of-the-art methods. Without such comparisons, it cannot be judged whether the proposed method in the paper can actually beat the best first-order methods.
Questions for authors
The paper suffers from major limitations that makes it unsuitable for publication in its current shape. Please refer to the comments above (I do not have a specific question and mostly there are a series of concerns that lead to my evaluation of the paper).