acceptodds
Withdrawn from ICLR 2027

Initialization Transfer for Nonconvex Statistical Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Nonconvex estimation can give different solutions from different starting values. With limited target data, an estimate from a related source population can provide a useful start. Population differences can make this start inaccurate and can bias a fit that pools both datasets. In this article, we study initialization transfer, which uses the source estimate only to start fitting with target data. We show how source accuracy and population differences determine entry into a region where target fitting converges. For a fixed number of parameters, reliable entry and enough updates give the same large-sample uncertainty as the target-only local solution. In noiseless sparse signal recovery, a sufficiently close source estimate permits recovery from a target sample nearly linear in the number of nonzero coefficients. For noisy recovery from squared Gaussian measurements in fixed dimension, pooling can lose target coverage even when its source fraction tends to zero. Checking the completed fit and restarting when needed preserves nominal large-sample coverage without source accuracy conditions. Simulations and a handwritten-digit analysis show improved target fitting when target data or iterations are limited, and smaller errors than pooling at larger population differences.

Primary area:
learning theory
Keywords:
Initialization transfer, Transfer learning, Nonconvex estimation, Statistical inference, Distribution shift, Phase retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_awGPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focusses on nonconvex estimation that related sources can be used to give good initialization. They provided view for sparse phase retrieval analysis with ‘initialization transfer’ that fit model with sources data and use this only for starting point, or ‘pooling’ that fit combined source and target data. They claim local strong convexity theory to show the convergence to target local solutions. They also connect source accuracy to the sample size needed for sparse target refinement. Experiments were conducted on phase retrieval, matrix completion, and NMF for handwritten-digits data.

The reviewer belieeves the paper should better clarify their contribution against existing works. Currently it looks standard and missing comparison.

Critical strengths
  1. The formulation is clear and analysis is complete. For example, the theorem 3 gives exact limiting target coverage.

  2. The authors are honest on limitations like “This is a comparison of sufficient conditions, not a lower bound for all target-only methods. In dense phase retrieval, target spectral initialization already succeeds under the sample condition of Corollary 2.”.

  3. The check and restart is simple to compute.

Critical weaknesses
  1. The main technique and conceptual contribution look quite standard. The author does not tell the audience what’s the novelty and significance. For example, any start in the strongly convex basin of attractions convergences to the same local solution is well-known fact. Uniform gradient and hessian concentration carries the population curvature to the empirical loss. The reviewer would love the author to clarify what goes beyond existing theory and compare their contribution and novelty more detailed with referenced paper. Severaly closely related works are cited but not compared, Cloud the author also discuss their work against:

Obst et al. (2022); Jones-McCormick, Jagannath and Sen, 2025; Ganesh et al.,2023; Chen, 2023

  1. The NMF has no global-optimality guarantee for coordinate descent solver. Meanwhile, the factors are only identifiable up to scaling and permutation ambiguity. It means for any factors you can scale and counter scale the two factors, but still give the same matrix product, and hence the same loss. The loss is therefore constant along a continuous family of solution. The Hessian has zero eigenvalues in these rescaling directions so the local strong convexity cannot hold.

  2. The experiments are limited. When one experiment is limited to 30 source iterations. Meanwhile, the NNDSVDa, single random, and pooled fits reach the 200-iteration cap on every split. The reported metric should have ablation with iteration budget. Otherwise, we don’t know if it’s because sources are in better starting point or not converging.

Questions for authors

Cloud the author answer my questions in the 'Weaknesses' section?

Severaly closely related works are cited but not compared, Cloud the author also discuss their work against at least:

Obst et al. (2022)

Jones-McCormick, Jagannath and Sen, 2025

Ganesh et al.,2023

Chen, 2023

Reviewer_n1hEOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies using an estimate from a related source population to initialize nonconvex estimation on target data, characterizing when this improves recovery while preserving target-only asymptotic uncertainty. It also shows that pooling source and target data can distort confidence coverage, whereas checking the source-initialized fit and restarting when necessary preserves valid inference.

Critical strengths

The paper gives statistical comparison between using source information for initialization and including it in target estimation, under some assumptions.

Critical weaknesses

The contribution appears incremental, and the main inference results rely on a specialized assumptions with limited practical significance. The empirical evidence is limited to simulations and a small handwritten-digit reconstruction experiment, where the gain over random initialization is modest.

Questions for authors

NA

Reviewer_pnjAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

Paper summary: In "Initialization Transfer for Nonconvex Statistical Estimation," the authors study the initialization of gradient descent for nonconvex statistical estimation problems. They consider a setting in which both a source problem and a target problem are available. An estimate θ^S\hat{\theta}_S obtained from the source problem is used to initialize gradient descent on the target problem.

The authors establish conditions under which θ^S\hat{\theta}_S provides a suitable initialization for gradient descent on the target problem, leading to a statistically meaningful estimate. These conditions are first developed in an abstract setting under standard regularity assumptions, namely local strong convexity and smoothness of the target population objective.

To illustrate settings in which their theoretical results apply, the authors focus on phase retrieval and sparse phase retrieval. For these problems, they describe how to construct θ^S\hat{\theta}_S using source data and establish sample size requirements for successful estimation. Finally, the authors present numerical experiments to support their theoretical findings.

Critical strengths

None

Critical weaknesses
  1. Limited novelty of the general theoretical results. The theoretical results in Section 3 appear to follow from standard arguments in nonconvex statistical estimation. In particular, Theorem 1 establishes a decomposition of the estimation error into optimization and statistical components. Under the assumed local strong convexity and smoothness conditions, together with sufficient concentration of the empirical Hessian, gradient descent converges linearly to the local empirical minimizer. The resulting statistical error can then be bounded using standard arguments. Similar analyses, often under substantially more challenging assumptions, are well established in the literature. See, for example, "Instability, Computational Efficiency and Statistical Accuracy" and the references therein.

The subsequent initialization results essentially establish that, when the source estimator lies sufficiently close to the target population parameter, it belongs to the region in which the local convergence guarantees apply. The conditions ensuring this proximity appear to follow from elementary geometric arguments.

Consequently, it is unclear what new theoretical insight these results provide beyond existing local convergence analyses for nonconvex statistical estimation.

  1. Limited novelty of the phase retrieval results. It is unclear under what conditions the proposed source-based initialization strategy offers a theoretical advantage over applying spectral initialization directly to the target problem, followed by gradient descent. In particular, given the requirements on the accuracy of θ^S\hat{\theta}_S and the sample sizes needed to obtain such an estimate, it is not evident that the proposed approach improves upon existing target-only methods.

For dense phase retrieval, the authors acknowledge that target-only spectral initialization reaches the desired convergence region under the same sample size requirements. For sparse phase retrieval, the improved target sample complexity relies on assuming that a sufficiently accurate source estimator is already available. However, the cost of obtaining this estimator is not fully accounted for in the comparison.

Consequently, the theoretical results do not clearly establish when or why the proposed approach should be preferred over existing initialization strategies.

Questions for authors

I would appreciate it if the authors could address the weaknesses outlined above, particularly regarding the novelty of the theoretical results and the advantages of the proposed approach over existing initialization strategies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.