- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies nonlinear least-squares fits of two-component models initialised by splitting a converged one-component fit into two coincident halves. It shows that this point is critical, that it is a strict saddle iff (for Gaussian peaks, whenever ), that the Gauss–Newton matrix has zero curvature along the escape direction, and it bounds finite-difference curvature estimates under evaluation noise (Prop. 1, Cor. 1, Lemma 1, Prop. 2). It then proposes SG-LM: Levenberg–Marquardt with the Jacobian and the residual curvature taken from a neural surrogate of the single-component shape, trained on function values, followed by a negative-curvature line search on the true model. Experiments cover biexponential, Gaussian and K-Voigt families (–), a transmission variant, grown tanh networks, and a synthetic binary-star problem with the Korg code (). Costs are counted in true evaluations, with surrogate derivatives and surrogate training excluded.
Critical strengths
- The computations of Prop. 1 and Remark 3 are explicit and checked numerically. The identity behind Cor. 1 is a clean observation.
- Table 1 separates the contribution of the surrogate Jacobian from that of the surrogate curvature, and the limitations section reports the wall-clock break-even. Code and result logs are provided.
Critical weaknesses
- The theory is elementary and largely known. The criticality of the embedded point and its classification by the sign of the residual-weighted matrix are Fukumizu & Amari (2000, Thm. 3) and the splitting matrix of Liu et al. (2019), as the paper acknowledges (l. 80–84). The "blindness" claim reduces to and for , which holds by definition. Prop. 1 is a Hessian computation, Lemma 1 expands a Rayleigh quotient in an eigenbasis, and Prop. 2 is the standard bias–variance trade-off of a central difference.
- The phenomenon is not specific to Gauss–Newton. At a critical point , so the LM step is zero whatever the curvature model. The same holds for GD, Adam, natural gradient and K-FAC (Table 5). Footnote 1 concedes this. The obstruction comes from initialising exactly at a stationary point (median , l. 320); from a loosely converged fit, the escape is "accidental" (l. 357–358, Table 18).
- Obvious baselines are missing. These include: a non-coincident split with plain LM/TRF; zero-amplitude insertion, which by Remark 2 has no such saddle; greedy insertion (sliding Frank–Wolfe); exact Hessian–vector products with a negative-curvature step on the true model, where autodiff is available; and Korg's native autodiff (l. 461).
- Scope and presentation. The theory covers two components of a single shape. Experiments are synthetic, with . The abstract and contributions consist largely of numerical ranges with parenthetical qualifications (l. 29–38, 98–117), which makes the main claim hard to extract.
Questions for authors
- Which statement of Prop. 1 about Gauss–Newton is not equally true of any method whose step vanishes with the gradient at a critical point?