acceptodds
Withdrawn from ICLR 2027

Tangent-Aligned Quadratic Regression for Tangent Space Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Local quadratic fitting estimates tangent spaces by learning an orientation and polynomial coefficients. We study a tangent-aligned normal graph with an intercept and no normal linear term, so the optimized frame is the fitted tangent. Profiling the coefficients separates coefficient conditioning, residualized linear separation, and full local orientation curvature. We construct an exact quadratic saddle with nonsingular coefficient and linear designs but a singular profiled Hessian: predictor rotation cancels linear tilt information. A deformation gives curvature proportional to \(\delta^2\), and a perturbation bound preserves weak curvature without exact symmetry. Conversely, symmetric isotropic graphs receive a nonnegative curvature contribution. We also establish conditional noisy-anchor recovery and local optimization bounds. Experiments with observed-data safeguards cover synthetic and image neighborhoods. Matched ablations show improved noiseless quadratic reproduction from full optimization, but no uniform accuracy benefit under noise or cubic model mismatch. The results explain which orientation information survives coefficient refitting and distinguish local objective geometry from practical recovery guarantees.

Primary area:
learning theory
Keywords:
manifold learning, tangent space estimation, local polynomial regression, quadratic regression, Riemannian optimization, orthogonal group, curvature estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kWhAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies tangent-space estimation through local quadratic regression with a learned orientation, a free intercept, and no normal linear term. By profiling out the regression coefficients, it distinguishes coefficient conditioning, residualized linear separation, and full orientation curvature. Its main theoretical contribution is an explicit example showing that well-conditioned regression designs can coexist with a singular profiled Hessian. The paper also provides conditional recovery bounds, local optimization analysis, and numerical comparisons.

Critical strengths

The distinction between coefficient stability and orientation curvature is conceptually useful. The explicit cancellation construction, together with its extension to small asymmetric perturbations, provides a concrete illustration of this distinction. The experiments demonstrate improved recovery on noiseless quadratic data, and the authors are transparent about unfavorable results and the limitations of their guarantees.

Critical weaknesses

My main concern is the limited methodological and practical significance of the contribution. Joint tangent and polynomial estimation is well established, and the proposed formulation does not improve the known statistical rate. Although the local curvature analysis is interesting, the paper does not establish that this insight leads to a reliable diagnostic based on observed data, an effective algorithmic remedy, or improvements in a downstream learning task. The connection between theory and implementation also remains incomplete: the convergence results concern gradient descent within a suitably localized smooth basin, rather than the implemented L-BFGS procedure from PCA initialization (Appendix C.12). Empirically, the clearest benefit occurs on noiseless exact quadratic data. The matched comparisons do not establish an advantage over two-fit Jet estimation in the noisy quadratic settings, while full optimization has higher error in all six tested cubic-mismatch conditions (Section 4.4). External MMLS also achieves lower mean error in three of the four study categories in Table 1. Consequently, the additional optimization cost and complexity are not supported by a sufficiently compelling practical benefit. These concerns leave me unconvinced that the contribution warrants acceptance.

Questions for authors

What substantive capability does the proposed method provide beyond established local polynomial estimators and MMLS? Can the full residualized orientation Jacobian be estimated from observed data and used to improve initialization, neighborhood selection, or estimation accuracy? Can the convergence analysis be connected to the actual L-BFGS implementation and its practical parameter choices? Finally, which realistic data regime or downstream task demonstrates a clear benefit sufficient to justify full optimization over simpler one- or two-fit alternatives?

Reviewer_TQP1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors study local polynomial regression. The contribution is very blurry and unclear, and the only experimental evidence is a simplified simulation study and results on MNIST.

Critical strengths

N/A

Critical weaknesses

I found it to be impossible to identify what the contribution of this paper is, let alone what problem is being addressed and what gap the authors are trying to fill. The language is obscure and convoluted; the paper talks more about what it is not doing vs. what it is doing, and it is not even clear in the results which one the proposed method is and what is being evaluated.

Questions for authors

It is not clear 1) what problem is being solved and what the gap is w.r.t. the state of the art, 2) what the proposed contribution is, and 3) how experimental evidence validates the contribution. A couple of examples are given below. This is a non-exhaustive list and is meant as illustrative examples. The full paper requires major rewriting.

  • The abstract does not identify which problem is being addressed.
  • Line 33: “A well-conditioned coefficient design therefore need not imply [...]” the logic here is unclear. What is the argument? Why does that imply anything?
  • Line 43: “Our focus is the orientation information exposed by profiling the coefficients, rather than [...]”, again it is unclear why that should be the focus, what that buys us.
  • Line 45: “The analysis separates three objects”. What analysis?
  • Line 57: local-chart and ambient Grassmann solvers are mentioned for the first time in the list of contributions. They are not put into context and contrasted w.r.t. state of the art.
  • Line 61: What is observed-radius selection?
  • Line 65: What are “Observed-setting factorial comparisons” and “matched frame-correction ablations”? It is unclear what these experiments are supposed to show. The authors only say what they *do not prove& instead (Line 66).
  • Section 2 does not state the problem being solved. Section 3 states an optimization problem without highlighting in what practical application this problem is relevant and why it is in the shown form. The concept of a “fitted graph” is given without actually defining what we are trying to fit here.
  • Line 134: “Unconstrained least squares applies only when its solution satisfies the curvature bound.” Where does this statement come from?
  • Line 142: What are these eigenvectors of?
  • Line 153: Grassmann exponential steps are mentioned here without ever being introduced or defined.
  • Theorem 3.1: When getting to this theorem it is still not clear what is being solved. The theorem asserts “Every feasible output satisfies” but the problem is not stated.
  • The reader also looks in vain for illustrative examples in the experimental section. Lines 332 onwards an “Image transfer and localization” experiment is discussed but the problem is not properly defined and that whole paragraph is impossible to decipher.
  • An illustrative example of how blurry the contributions are is the first paragraph of the Discussion and Limitations. At no point in this paragraph do the authors state what they contribute, how whatever they did is helpful, or how it differs from state of the art. The description is a vague comparison of properties of different methods.
Reviewer_eXRbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper provides a careful analysis of tangent-space estimation through profiled quadratic regression. Its strongest contribution is an explicit cancellation example showing that well-conditioned regression designs do not guarantee positive orientation curvature, together with a perturbation analysis extending the insight beyond exact symmetry. However, the connection between this geometry and practical estimation remains incomplete: the implemented solver is not covered by the convergence guarantees, and the experiments establish only narrow accuracy benefits.

Critical strengths
  1. The paper draws a sharp and, to my knowledge, original distinction between the ability to fit a polynomial and the ability to identify the orientation it implies. This is not a rhetorical observation but is substantiated by an exact quadratic saddle on twelve points, in which the coefficient and residualized linear designs are both nonsingular while the profiled Hessian is singular.
  2. The empirical study is thorough for a paper of this length: matched ablations that share data, initialization and scale; a factorial design that crosses observed versus supplied curvature bounds with the Gram safeguard on and off; and dedicated diagnostics for the saddle mechanism, threshold behaviour, and fitting cost. The paper is also careful about the scope of its claims, devoting explicit space to where the method does not help.
Critical weaknesses
  1. The contribution is narrow relative to what the title promises. Although the analysis is solid, the accuracy advantage appears essentially only in the noiseless quadratic setting: under noise the matched intervals include zero, and under cubic mismatch a single frame correction outperforms a second fit, which in turn outperforms full optimization. Beyond the synthetic quadratic families and the two image families, the paper does not establish a regime — in terms of noise level or manifold geometry — in which the method has a stable advantage.
  2. Cost and accuracy are poorly balanced. In Table 6 (Appendix B.5), the proposed method's initialization-included fitting time is about 2.05–2.24 ms at the reported sample sizes, versus roughly 0.131–0.162 ms for the internal Jet fit and 0.66–0.86 ms for the four-iteration Iterative QF — approximately fifteen times and three times more expensive — while its accuracy in the factorial study is on par with Jet and below the external moving-least-squares baseline. For a method whose advantage appears mainly in the noiseless quadratic case, this cost profile warrants explicit discussion.
Questions for authors
  1. The paper constructs an exact quadratic saddle in which both the coefficient design and the residualized linear design are nonsingular, yet the profiled Hessian in the orientation directions is still singular. Could the authors say more about how likely this degeneracy of the orientation information is on a general local manifold, or on real point-cloud data? Specifically, how frequently do configurations of this kind arise — and if other counterexamples with the same effect exist, how frequent are they?
  2. The behaviour of the model differs markedly between the noiseless and the noisy settings. Could the authors add experiments over multiple noise levels and noise forms, in order to analyse how noise affects the estimator?
  3. The main contribution of this work is a theoretical insight rather than a practical new method. What concrete impact or application do you expect this insight to have on the design of future tangent-space estimators, on manifold learning, or on local polynomial fitting schemes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.