- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies how initialization affects BFGS and L-BFGS with a fixed objective and optimizer configuration. For a structured convex pinball-loss covex objective, It constructs infinitely many recurring line-search patterns with distinct convergence rates per trial, relates contraction to initialization sensitivity, and proves finite-horizon persistence under perturbations. The theory is interesting but effectively one-dimensional. Experiments investigate these patterns and initialization-dependent optimization costs in some toy examples.
Critical strengths
Theoretical contribution - explicit infinite hierarchy of convergence regimes. The paper strengthens the known initialization-dependent behavior by constructing infinitely many recurring line-search patterns with exact per-trial convergence rates and a contraction–sensitivity relation. The construction and phenomenon appear interesting, but its scope and experiments seem limited.
Critical weaknesses
-
The scope of the contribution is narrower than the framing. “Initialization shapes the computational regime of adaptive optimization” suggests a broad mechanism. The theorem establishes specially chosen trajectories of one nonsmooth scalar model under a specified line-search rule. In addition, there is an limited connection between theory and experiments. The exact regimes are effectively one-dimensional, and robustness holds only over finite horizons. The real-data experiments show cost variation without demonstrating the recurring regimes or asymptotic rate differences established theoretically. This leaves the broader practical significance insufficiently supported.
-
Unclear presentation and scope. I found the setup and presentation of the results difficult to follow. The main text should more clearly distinguish the scalar construction, multidimensional embedding, and finite-horizon guarantees, and summarize the key proof ideas.
Questions for authors
-
Which theoretical conclusions extend beyond the one-dimensional invariant subspace to genuinely coupled high-dimensional objectives or other losses?
-
Do the existing experiments demonstrate recurring regimes or persistent convergence-rate differences beyond the analytic construction, or only differences in finite-time optimization cost?