- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies recovery of realized latent memberships in a two-component Gaussian random-design regression model with balanced, fixed but unknown labels and unknown regression coefficients, covariance, and noise level. It establishes a minimax Hamming-error scale of , up to an exponentially small upper-bound remainder, and the exact-recovery criterion when is at most a sufficiently small multiple of . It also proposes bounded-residual spectral initialization followed by hard least-squares refinement. A uniform contraction analysis for margin-weighted Hamming loss establishes efficient attainment of the recovery scales. Simulations examine recovery, computation, initialization, and several departures from the main model.
Critical strengths
- The statistical question and resulting distinction are valuable. Recovering the realized labels is different from estimating shared regression components. The random margin explains why an order-r error remains even with known coefficients, and why exact recovery depends on the sample minimum rather than a fixed separation. The oracle formula makes this mechanism particularly transparent.
- The paper goes beyond an oracle calculation. The profile-RSS upper bound and balanced oracle subfamily connect the information-theoretic lower and upper bounds while accounting for label switching. The efficient procedure then addresses the computational limitation of exhaustive partition search.
- The local analysis is a substantive contribution. The use of margin-weighted loss, uniform control over local coefficient pairs, and the separate treatment of in-sample oracle errors address the lack of a uniform positive margin and the dependence created by reusing observations. This is more informative than assuming independently generated iterates or imposing a deterministic separation gap.
- The initialization and experiments support the theoretical story. Bounded residual weights preserve the population contrast direction and avoid the unfavorable concentration behavior of unbounded squared residuals. Appendix F includes the weighting ablation, unequal group sizes, intercepts, covariance and common-center changes, and non-Gaussian/dependent noise. The paper appropriately distinguishes the improvement in spectral direction from the smaller improvement after refinement in many settings.
Critical weaknesses
- The practical stopping rule is not fully connected to the stated efficient guarantee. Theorem 1.2 requires even though depends on unknown quantities. Remark 3.1 suggests stopping when assignments stabilize, but the paper does not give a data-independent polynomial iteration cap with the same guarantees, or fully specify the stopping/capping rule used for the proposed method in the numerical study. Stabilization makes further exact iterations redundant; the remaining question is how long one must wait and how the implemented procedure is covered by the theorem. This appears addressable within the existing contraction argument.
- The scope of “sharp limits” should remain explicit. The results are for two components, Gaussian design and independent Gaussian noise, balanced groups, and for an unspecified sufficiently small κ. The Hamming upper bound contains an exponentially small remainder, so it is not a uniform constant-factor equivalence to r for arbitrarily tiny positive r; the exact-recovery result is an asymptotic criterion rather than a characterization of the full transition curve. The experiments outside the model are useful, but they do not extend the theorem. I view these as scope limitations, not flaws in the main statistical claim as precisely stated in Theorem 1.1.
- Some comparative empirical conclusions depend on adapted baseline choices. Appendix F fixes a search radius and angular grid for SVD-grid and sets the unspecified regularization multiplier of the convex initializer to one. Such choices are documented, but a limited sensitivity check would make the broad empirical superiority claim more convincing. Figure 7 already shows that the bounded and squared-residual initializers often perform similarly after refinement; the strongest algorithmic justification is therefore the sample-complexity analysis and the more difficult dimensional regimes, rather than a uniform empirical advantage from bounded weighting.
Questions for authors
- Can the contraction analysis yield an r-independent polynomial iteration cap attaining the stated risk bound, with early stopping on stable assignments? Please also state the exact stopping rule and cap used in the reported runtime and recovery results.
- How sensitive are the SVD-grid and convex-initializer comparisons to a small, prespecified range of grid resolutions/search radii and regularization multipliers? Existing simulation settings should suffice for this clarification.
- Could the introduction distinguish order-optimal Hamming risk with an exponential remainder, the exact-recovery criterion, and oracle transition curves more explicitly? Also, which part of the efficient-recovery proof is the main obstacle to extending beyond Gaussian design or two components?
3 comments
Response to Reviewer hyUn (Part 2/2)
W3 & Q2: Sensitivity of baseline comparisons
W3: Some comparative empirical conclusions depend on adapted baseline choices. Appendix F fixes a search radius and angular grid for SVD-grid and sets the unspecified regularization multiplier of the convex initializer to one. [...] a limited sensitivity check would make the broad empirical superiority claim more convincing.
Q2: How sensitive are the SVD-grid and convex-initializer comparisons to a small, prespecified range of grid resolutions/search radii and regularization multipliers? Existing simulation settings should suffice for this clarification.
Thank you for the suggestion. We evaluated the sensitivity of both initializers to their tuning parameters across the designs of Figure 4, with each method followed by hard least-squares refinement.
To provide a concise summary, Tables 1 and 2 report the performance averaged across the tested range of each column parameter, based on 100 Monte Carlo replications. Columns , , and report the mean mis-clustering error (%), while column reports the mean exact-recovery probability (%).
Table 1. SVD-grid sensitivity to angular spacing and radius multiplier , with default and search radius . Entries are percentages averaged across the tested values of each column parameter, reporting mis-clustering error for and exact-recovery probability for .
| Initialization | ||||
|---|---|---|---|---|
| Proposed | 9.84 | 52.86 | 11.28 | 6.44 |
| 10.72 | 52.57 | 19.84 | 7.80 | |
| 10.57 | 52.29 | 19.50 | 8.48 | |
| 10.69 | 52.57 | 19.44 | 7.77 | |
| 10.69 | 52.29 | 20.00 | 7.91 | |
| 10.79 | 52.00 | 19.41 | 8.64 | |
| 10.71 | 52.29 | 19.41 | 8.00 | |
| 10.77 | 52.71 | 20.00 | 7.96 | |
| 10.76 | 52.00 | 20.08 | 8.59 | |
| 10.59 | 52.29 | 19.19 | 7.59 |
Table 2. Convex initializer sensitivity to the regularization multiplier , with default and original value . Entries use the same averaging and percentage metrics as Table 1.
| Initialization | ||||
|---|---|---|---|---|
| Proposed | 9.84 | 52.86 | 11.28 | 6.44 |
| 9.87 | 53.00 | 23.48 | 6.41 | |
| 9.87 | 53.00 | 23.48 | 6.40 | |
| 9.86 | 52.86 | 23.48 | 6.40 | |
| 9.88 | 52.86 | 23.48 | 6.45 | |
| 9.92 | 53.00 | 23.48 | 6.66 | |
| 10.49 | 52.57 | 23.53 | 6.81 | |
| 11.76 | 51.86 | 23.60 | 7.07 |
SVD-grid.
Changing the angular spacing and search radius yields similar overall performance for SVD-grid. The proposed method achieves better average performance than all tested parameter choices across the four settings, with particularly clear gains at small and large .
Convex initializer.
For the convex initializer, we find that using a smaller regularization multiplier improves performance. When varying , however, its average mis-clustering error remains higher than that of the proposed method, especially at larger dimensions. The convex method is also more computationally demanding in the experiments.
We have updated all comparisons throughout the paper to adopt the optimized multiplier for the convex formulation initialization. For the sensitivity experiment, full details are included in Appendix G.11.
We once again thank you for your constructive review.
I will maintain my current score
Thank you for the detailed responses and additional experiments. These largely address my main concerns, and I will maintain my current score.
Response to Reviewer hyUn (Part 1/2)
We thank you for your careful reading and helpful comments. We are happy to engage in further discussion, and our point-by-point responses are detailed below.
W1 & Q1: Practical stopping rule and iteration cap
We agree. In the revised manuscript, we now require K≥min(⌈5log(1/r)⌉,n) in Theorem 1.2, so capping at K=n iterations achieves the guarantees without knowing r.
Moreover, Algorithm 3.1 is an alternating minimization procedure. Once assignments stabilize (z(k+1)=z(k)), the updates reach a fixed point. All subsequent iterations stay the same. Stopping upon stabilization is therefore exact and covered by the theorem. We have clarified this explicitly in Remark 3.1.
For the experiments, the proposed method stops when the assignments stabilize or after 100 refinement iterations following the initial refit.
W2 & Q3: Scope and model extensions
Hamming risk and exact recovery. In finite-sample analysis, the exponentially small term Ce−cn reflects the standard design-failure probability. When nr≳1, recovery is governed by the Hamming risk of order min(1,r) as the exponential term is negligible. When nr≪1, the criterion shifts to exact recovery with non-asymptotic failure probability O(nr)+Ce−cn, where the exponential remainder is likewise negligible for exact recovery and does not affect the nr→0 threshold. We have made this distinction explicit in the introduction.
Oracle transition curve. In our paper, q(r)=arctan(2r)/π is the single-membership Bayes error curve for the symmetric oracle problem. The curve q(r) is plotted primarily as an intuitive visualization. Providing an exact characterization of the finite-sample transition curve is an interesting problem, and we leave it for future work.
Extensions beyond Gaussian designs.
Extending our results to general covariates involves two main issues.
Algorithmic guarantees. The main challenge is on the algorithmic side, which relies on two properties of Gaussian designs. The first is identifying the contrast direction through a Gaussian Stein-type identity, and the second is using rotational symmetry to control design covariances on wedge-shaped misclassification sets. For non-Gaussian designs, extending the initialization would require either suitable moment identities or another construction, such as truncation, to recover the contrast direction.
Statistical limits. The main statistical difficulty is controlling the covariate mass near the decision boundary, which determines how the recovery error depends on r. For Gaussian covariates, the projected density near zero is bounded and positive, leading to the order-r risk. Beyond the Gaussian case, heavier mass near zero makes the two components harder to separate even when the noise is small. A natural way to extend the minimax theory is to impose margin conditions on the projected covariates, which quantify the boundary separation and characterize the resulting decay rate in r.
Extensions to multiple components.
For multiple components, the primary obstacle is initialization. In the two-component setting, a single contrast vector provides a sign rule that immediately yields an initial partition. With K0≥3 components, the second-moment matrix only identifies the at most (K0−1)-dimensional span of pairwise contrasts. Turning this subspace into an accurate initial partition requires jointly resolving the individual regression vectors and their memberships. Once a sufficiently good initial clustering is available, we believe the local contraction of hard least-squares refinement can achieve optimal mis-clustering rates.