acceptodds
Under review as a conference paper at ICLR 2027

Regularized Regression for Hadamard Manifold-Valued Data: Finite-Sample Bounds and Tangent-Space Sparsity

Abstract

Regression for Hadamard manifold-valued data requires predictions that account for intrinsic geometry while controlling model complexity. We represent regression curves as Riemannian exponential images of bounded tangent fields and combine an intrinsic loss with penalties on their coefficients or groups. For convex tangent models containing the ambient risk minimizer, we derive finite-sample oracle inequalities under geodesic strong convexity, Lipschitz continuity, a small-ball condition and localized complexity control. The bounds connect tangent-space error, integrated geodesic error and excess risk through an explicit curvature factor. Decomposable penalties express the regularization contribution in terms of subspace compatibility and approximation error, with a sparsity factor for exactly sparse representations. Simulations on the Poincar\'e ball and symmetric positive-definite matrices show a sparsity–prediction trade-off whose benefit diminishes for dense targets. An exploratory EEG covariance-interpolation study on subjects gives lower mean held-out error than unpenalized fits, while constant and intrinsic regression baselines attain similar errors. These results identify the geometric and statistical conditions supporting tangent regularization and distinguish them from its observed numerical benefits.

Primary area:
learning theory
Keywords:
Hadamard Manifolds, Empirical Risk Minimization, Oracle Inequalities, Tangent-Space Regularization, Small-Ball Method, Manifold-Valued Regression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xqywOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies regularized regression with Hadamard manifold-valued responses. Regression curves are represented by exponentiating bounded tangent vector fields at a fixed reference point, and estimation combines an intrinsic loss with coefficient or group penalties. Under realizability, geometric regularity, small-ball, and complexity assumptions, the paper establishes finite-sample oracle inequalities and translates tangent-space error into geodesic prediction error. Experiments investigate sparsity–prediction trade-offs on hyperbolic and SPD manifolds, together with an exploratory EEG covariance-interpolation application.

Critical strengths
  • The analysis carefully distinguishes convexity of the tangent-space model from geodesic closure of its exponential image. The use of an ambient population oracle and a geodesic contraction hull addresses a substantive geometric issue.
  • The paper provides a detailed treatment of geometric distortion, localized complexity, and decomposable penalties.
  • The experiments include relevant tangent-space and intrinsic baselines, and the manuscript transparently reports limitations and unfavorable comparisons.
Critical weaknesses

My main reservation concerns the originality and significance of the contribution. Representing manifold-valued regression functions through tangent vector fields appears closely related to kernel-regression approaches based on tangent-field estimation and exponential-map reconstruction. The paper should explain more precisely what is gained beyond this existing modeling strategy and the application of established regularized-learning tools. The present contribution is concentrated on finite-sample error guarantees; it remains unclear whether it yields a qualitatively new statistical insight or a practically consequential advantage over kernel-based alternatives.

Three additional concerns limit the force of these guarantees.

First, the sparsity-dependent term does not improve the full localization term. Corollary 4 gives a bound of the form max⁡\max{r∗,λ√s/κr*, λ√s/κ}, while r* is controlled by the full model. As acknowledged in Remark F.1, the resulting bound is not an overall sparsity-adaptive rate. This weakens the theoretical explanation for the principal motivation of sparse regularization.

Second, the explicit dictionary rate requires geodesic closure, whereas the main Fourier implementation need not satisfy that condition. The general theorem may still apply through its contraction hull, but the explicit rate does not automatically transfer. The dependence on small-ball and curvature constants also makes the quantitative usefulness of the bound difficult to assess.

Third, the guarantees concern global minimizers, while the numerical procedure uses a stopping residual without a quantified global optimization gap. The empirical evidence does not clearly compensate for this disconnect: tangent LASSO has lower mean error in all 22 simulation settings, and the EEG fits are frequently essentially constant.

Questions for authors
  1. Could the authors provide a precise comparison with kernel-regression methods for manifold-valued responses that estimate tangent vector fields and reconstruct predictions through the exponential map? Which methodological or theoretical contribution cannot be obtained by combining those formulations with existing regularized-learning arguments? In particular, since Corollary 4 retains the full-model localization radius, what substantive sparse-learning advantage is established beyond a sparsity-dependent regularization term?

  2. Given the Fourier non-closure example in Remark F.2, can the authors control the actual geodesic contraction hull used by their main implementation? How do the small-ball and curvature factors behave in that example?

  3. Can the authors quantify the optimization error of the reported fits and explain how it enters the statistical guarantee? In light of the stronger tangent-LASSO results and nearly constant EEG predictions, what evidence establishes an additional benefit of intrinsic regularized regression over appropriately tuned kernel/tangent-space methods and constant predictors?

5 comments
AuthorsOpenReview ↗

Response to Reviewer xqyw

We thank the reviewer for his careful reading. Below, we address each concern and question point-by-point.

1. Originality and sparse learning (Q1)

We add a sparse intrinsic-risk guarantee for fixed tangent dictionaries on Hadamard manifolds. Theorem 5 (Section 3.6; full version: Theorem L.3) separates approximation risk, coefficient tails and global optimization error for a fixed feasible reference predictor and coordinate set. It allows misspecification without geodesic closure of the fitted dictionary image. It assumes a fixed base point and bounded population-orthonormal dictionary, i.i.d. data and the stated loss and tuning conditions.

For exactly sparse realizable targets, Corollary L.4 gives a leading term proportional to sλ2s\lambda^2, without a maximum involving the full-model localization radius. The original Corollary 4 retains that maximum; the improvement uses a new coefficient-ℓ1\ell_1 argument.

Appendix A compares Banerjee et al. (2015), who use kernel-weighted Fréchet means and exponential-map approximation, Schötz (2022), who jointly fits a base point and tangent series, and Matteo et al. (2026), who establish spherical intrinsic RKHS guarantees under cap restrictions. The proof combines established learning tools with the ambient margin, distance comparison and loss-increment control, allowing a nonconvex coefficient objective. Joint estimation of the base point remains outside our guarantee.

2. Fourier nonclosure and quantitative scope (Q2)

Under realizability, Proposition M.1 controls all geodesic contractions toward the ambient oracle, with one scalar θ\theta shared across time, including curves outside the Fourier image. Distance and the first-order multiplier scale exactly by θ\theta. A coefficient-shell argument gives

\le\max\big(r_0,2L\kappa_B\sqrt KC_q/(\zeta\sqrt n)\big),$$ where $$r_0=\frac{L}{\sqrt2\zeta\sqrt n},\quad J_0=1+\max\{0,\lceil\log_2(2B/r_0)\rceil\},\quad C_q=2\sqrt2+M\sqrt{2\log(J_0/q)}.$$ Here $\zeta>0$ and $q\in(0,1)$ is the failure-probability parameter. The dictionary, sampling and gradient assumptions remain; geodesic closure is unnecessary. Corollary M.2 separately controls design conditioning. For independent uniform times and a shared complete normalized Fourier basis, $$\Pr(\lambda_{\min}(\widehat G)<1/2) \le q_Fe^{-n/(8q_F)},\qquad K=dq_F.$$ Here $q_F$ is the scalar basis size per tangent coordinate. On the complementary event, every dictionary field's empirical squared norm is at least half its population value. The hull bound still depends on total $K$ and geometric distortion $\kappa_B$. For the original corollary retaining closure, the generic small-ball certificate can yield sufficient $K^5$ sample-size and $K^3/n$ squared tangent-radius terms, with other constants fixed (Appendix O.1). Its curvature factors are $\kappa_B^6$ for tangent error and $\kappa_B^8$ after geodesic conversion. A complete informative risk guarantee at the reported sample sizes remains unestablished. **3. Optimization error and empirical benefit (Q3)** Let $F_g$ be the half-squared intrinsic empirical objective and $F_t$ its convex tangent counterpart, with the same weighted $\ell_1$ penalty. Logarithm nonexpansion gives $F_t\le F_g$; boundedness and orthonormality give $C\subseteq\{\|\beta\|_2\le B\}$. For $z=(\log_pY_i)_i$, $D\beta=(v_\beta(T_i))_i$ and every dual vector $U$, Appendix N gives $$\mathcal D_B(U) = -\frac{n}{2}\||U\||_2^2-\langle U,z\rangle -B\||\operatorname{ST}_{\lambda w}(-D^\top U)\||_2,$$ where $\operatorname{ST}$ is coordinatewise soft thresholding and $w\ge0$ matches the fitted penalty. This lower-bounds the tangent objective on the coefficient ball. In exact arithmetic, every feasible fit therefore satisfies $$0\le F_g(\widehat\beta)-\inf_{\beta\in C}F_g(\beta) \le F_g(\widehat\beta)-\mathcal D_B(U).$$ No exact dual solution is required. Under the respective assumptions, a valid gap bound $\epsilon_{\rm opt}$ contributes $\epsilon_{\rm opt}/\varkappa$ to the original squared tangent bound and $4\epsilon_{opt}/\mu$ to Corollary L.4's squared-distance bound. Appendix O reports mean RERM upper bounds from $1.59\times10^{-4}$ to $2.29\times10^{-2}$, with continuous feasibility checked. They include relaxation error and need not vanish at an intrinsic optimum. Float64 evaluations have no interval validation. No EEG optimization-gap bound was computed. The sparse tuning threshold is at least 96 times each simulation grid's maximum, so the theorem does not certify the cross-validated fits. Mean squared prediction errors favor intrinsic RERM over ERM in all 22 settings, and tangent LASSO over RERM in all 22. In 306 of 308 EEG recordings, Fourier fits have no active nonconstant coefficients. These comparisons do not establish an intrinsic-loss advantage.

Thank you again for your careful review and constructive feedback. We hope that our responses and revisions have addressed your concerns and questions. If you find that they resolve your main concerns, we would be grateful if you would reconsider your assessment and score in light of these changes. We would be happy to clarify any outstanding points before the discussion period ends on 18 November.

Thanks for the response. The authors made substantial improvements to the technical analysis. I will take those into consideration and raise my rates. Meanwhile, I still have some questions.

  1. Can you provide an illustrative example where their sparse bound is quantitatively informative at realistic sample sizes and regularization strengths, or a sharper result that covers the fitted estimators?

2.Can you identify a setting where intrinsic geodesic RERM materially outperforms tangent-space LASSO under matched representation, tuning and computational budgets? Your current experiments generally favor tangent LASSO.

Reviewer_ZKntOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The study expresses bounded Hadamard-valued prediction curves as exponential images of fixed-reference tangent fields and fits them by using an intrinsic loss together with convex penalties. If the geometric and realizability conditions are satisfied under the small-ball, localized-complexity, and exact-minimization assumptions, then bounds are obtained for the finite-sample tangent-error, geodesic-error, and excess-risk. Decomposable penalties produce regularization terms that depend on sparsity. Moreover, in order to achieve geodesic closure, an explicit fixed-model dictionary rate is required. The simulations assess both intrinsic and tangent losses and compare the different representation options. The exploratory EEG interpolation study involves 109 subjects, 22 of whom are set aside for evaluation. Regularization yields better results than unpenalized fits; the tangent LASSO consistently has a lower mean error across the main simulations, and the EEG fits remain nearly constant.

Critical strengths
  1. The paper provides a clear distinction between geometric and statistical requirements. The ambient oracle and contraction hull address the issue of geodesic closure, whereas explicit distance comparison connects tangent regularization to intrinsic prediction error.
  2. The reporting demonstrates transparency by including simple intrinsic baselines, tangent-LASSO comparisons, structural-prior reversal, and subject-level EEG aggregation. These elements substantially strengthen the credibility of the limited empirical conclusions.
Critical weaknesses
  1. The analysis does not demonstrate the advantage of sparse learning, which is intended to motivate the estimator. Assumption 6 requires that the minimizer of the ambient bounded-class population be exactly represented within the selected tangent model. The representation-approximation term in the decomposable bound does not eliminate this realizability requirement. Furthermore, the complete localization radius remains that of the full model. A sparsity-dependent penalty term does not constitute a fully sparsity-adaptive estimation rate.
  2. Fourier's counterexample in Remark F. 2 shows therier's counterexample in Remark F.2 demonnecessity of additional geometric control; proving this condition is not automatic (p. 29). Therefore, the explicit rate cannot be applied to the main Fourier experiments solely because of the finite dictionary; it remains necessary to control the required contraction-hull complexity.
  3. The result's dependence on dimension requires further examination. When all other parameters are held constant, and only the paper's general small-ball certificate is used, the sample-size scaling is proportional to (K^5), and the primary contribution to the certified error is proportional to (K^3/n). This reflects the worst-case certificate available, not the required sample size, and shows why the fixed-model (K/n) expression cannot be interpreted as a dimension-independent sparse guarantee. Additionally, the curvature factor further diminishes the numerical significance: for a hyperbolic radius (B=6), its sixth power is approximately (1.44\times10^9) before accounting for other constants.
  4. The connection between the theorem and the computational implementation is not established, and the experiments do not provide a compelling alternative justification. Equation (2) assumes that a measurable global minimum has been achieved. However, the implementation's coefficient objective may be nonconvex, and the manuscript explicitly states that its stopping criteria do not guarantee a global objective gap (page 46). Consequently, the distance guarantee does not necessarily apply to the computed estimator.
  5. Across all 22 primary simulation settings, tangent-space LASSO yields a lower mean geodesic error based on empirical measurements. For EEG interpolation, the Fourier estimator remains constant in 306 of 308 test recordings, and the simple intrinsic baselines report nearly identical mean errors. Although these results support shrinkage, they provide limited evidence for the added value of the intrinsic-loss methodology. Furthermore, the final model specifications remain invariant to inspection of the test results, further diminishing the confirmatory value of the EEG study.
Questions for authors
  1. Is it possible to determine the complete multiplier radius for the actual nonflat Fourier model and to establish a nontrivial guarantee at a realistic sample size, accounting for all geometry and design constants?
  2. What is the quantified optimization gap that connects the fitted values to the global minimizer in equation (2)? A local stopping residual does not address this issue.
  3. Which final model specifications were determined prior to examining the test results, and what evidence demonstrates that the intrinsic method differs from tangent LASSO and constant intrinsic prediction?
5 comments
AuthorsOpenReview ↗

Response to Reviewer ZKnt

We thank the reviewer for his careful reading. Below, we address each concern and question point-by-point.

1. Realizability and sparse guarantees (W1)

The original theorem assumes realizability. Theorem 5 (Section 3.6; full version: Theorem L.3) now allows misspecification. For a fixed feasible reference predictor and coordinate set SS, it separates Δ=R(γβ∘)−R(γ⋆)\Delta=R(\gamma_{\beta^\circ})-R(\gamma^\star) from 2λ∥βSc∘∥12\lambda\|\beta^\circ_{S^c}\|_1, where γ⋆\gamma^\star is the ambient oracle. A population best-in-model reference predictor makes Δ\Delta the model-approximation error.

The general bound requires neither geodesic closure of the fitted class nor a small-ball condition. It assumes a fixed base point and bounded population-orthonormal dictionary, i.i.d. data and the stated geometry/loss conditions. For an exactly ss-sparse realizable target and λ≥4c\lambda\ge4c, Corollary L.4 gives, with probability at least 1−δ1-\delta,

\le9\lambda^2s/\mu^2+4(cB/n^2+\varepsilon)/\mu.$$ Here $\varepsilon$ bounds the feasible fit's global penalized empirical objective gap. There is no maximum with a full-model localization radius. The sufficient tuning threshold is at least 96 times each simulation grid's maximum, so this theorem does not certify the cross-validated fits (Appendix O.5). **2. Fourier hull, dimension and curvature (W2–3; Q1)** Proposition M.1 controls multiplier complexity over the contractions \$ \gamma_\theta(t)=\operatorname{Exp}_{\gamma^\star(t)} \big(\theta\operatorname{Log}\_{\gamma^\star(t)}\gamma(t)\big), \quad\gamma\in\Gamma\_{\mathcal V},\quad\theta\in[0,1], \$ toward the realizable oracle. One scalar $\theta$ is shared across time, and curves outside the Fourier image are included. Integrated distance and the first-order multiplier both scale by $\theta$, enabling this control without model closure. For a shared complete normalized Fourier basis with $q_F$ functions per tangent coordinate, $$\widehat G=I_{d_p}\otimes\widehat G_F,\quad \widehat G_F=n^{-1}\sum_i\psi(T_i)\psi(T_i)^\top,\quad K=dq_F.$$ For independent uniform times, Corollary M.2 gives $$\Pr(\lambda_{\min}(\widehat G)<1/2)\le q_Fe^{-n/(8q_F)}.$$ On the complementary event, each dictionary field's empirical squared norm is at least half its population value. This controls design conditioning; hull complexity still depends on total $K$ and geometry. The geometric distortion factor $\kappa_B$ is distinct from $K$. In the original corollary, which retains geodesic closure, the squared multiplier radius scales as $\kappa_B^2/\zeta^2$, with $\zeta\propto\kappa_B^{-2}$. This produces $\kappa_B^6$ for tangent error and $\kappa_B^8$ after geodesic conversion. With other constants fixed, the generic small-ball certificate can yield sufficient $K^5$ sample-size and $K^3/n$ radius terms (Appendix O.1). A complete informative risk guarantee at the reported sample sizes remains unestablished. **3. Global optimization gap (W4; Q2)** For half-squared loss, let $F_g$ denote the intrinsic empirical objective and $F_t$ its tangent least-squares counterpart based on $\log_pY_i$, with the same weighted $\ell_1$ penalty. Logarithm nonexpansion gives $F_t\le F_g$; boundedness and orthonormality give $C\subseteq\{\|\beta\|_2\le B\}$ for the theoretical feasible class. In exact arithmetic, Appendix N gives, for every dual vector $U$, $$\mathcal D_B(U) \le \inf_{\||\beta\||_2\le B} F_t(\beta) \le\inf_{\beta\in C}F_g(\beta).$$ Thus, for every feasible fit, $$0\le F_g(\widehat\beta)-\inf_{\beta\in C}F_g(\beta) \le F_g(\widehat\beta)-\mathcal D_B(U).$$ Neither convexity of $F_g$ nor exact dual optimization is required. Appendix O reports mean RERM upper bounds from $1.59\times10^{-4}$ to $2.29\times10^{-2}$, with continuous feasibility checked. A valid $\varepsilon$ contributes $\varepsilon/\varkappa$ to the original squared tangent bound and $4\varepsilon/\mu$ in Corollary L.4. The bounds include relaxation error and need not vanish at an intrinsic optimum. Float64 evaluations were not interval-validated. No EEG gap was computed because the required covariance inputs and coefficient paths are unavailable. **4. Empirical comparisons and EEG chronology (W5; Q3)** The records do not establish which specifications preceded test inspection. Subject-disjoint validation does not resolve this chronology; the within-recording EEG study remains exploratory. Intrinsic RERM has lower mean squared error than ERM in all 22 simulation settings; tangent LASSO improves on RERM in all 22. Across 22 EEG test subjects, subject-averaged squared geodesic errors are 2.563994 for intrinsic Fourier RERM, 2.563992 for tangent Fourier RERM and 2.563403 for constant Fréchet prediction. In 306 of 308 recordings, Fourier fits have no nonconstant coefficients. These comparisons do not establish an intrinsic-loss advantage. Estimated base points and dependent windows also prevent direct application of the fixed-model i.i.d. theorem.

Thank you again for your careful review and constructive feedback. We hope that our responses and revisions have addressed your concerns and questions. If you find that they resolve your main concerns, we would be grateful if you would reconsider your assessment and score in light of these changes. We would be happy to clarify any outstanding points before the discussion period ends on 18 November.

Thank you for the detailed response and explanation. The adjustments made address my concerns, so I will improve my rating.

Reviewer_WFeGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies regularized empirical risk minimization for curves on a Hadamard manifold. Curves are parameterized as γ=Exp⁡p∘v\gamma = \operatorname{Exp}_p \circ v, where vv is a dictionary-expanded tangent field at a fixed base point pp. Regularization is applied in TpMT_pM, while prediction is evaluated using an intrinsic loss.

The main results include a two-sided comparison, ∣v−v⋆∣L2≤Dg(γ,γ⋆)≤κB∣v−v⋆∣L2\\|v-v^\star\\|_{L^2} \le D_g(\gamma,\gamma^\star) \le \kappa_B\\|v-v^\star\\|_{L^2}, where κB=sinh⁡(ΛBB)/(ΛBB)\kappa_B = \sinh(\Lambda_B B)/(\Lambda_B B), and estimation bounds under strong convexity, Lipschitz loss, small-ball, and realizability assumptions. The paper also studies decomposable penalties and gives an explicit O(K/n)O(K/n) radius under an additional geometric closure condition.

Experiments include manifold-valued simulations and an exploratory EEG study. Tangent LASSO outperforms geodesic RERM in the reported simulation comparison, while the EEG fits are essentially constant.

Critical strengths
  • A useful geometric construction. The contraction hull gives Log⁡γ⋆(t)γθ(t)=θ⋅Log⁡γ⋆(t)γ(t)\operatorname{Log}_{\gamma^\star(t)}\gamma_\theta(t) = \theta \cdot \operatorname{Log}_{\gamma^\star(t)}\gamma(t). This enables star-shaped localization even when the original curve class is not geodesically convex.

  • A clear connection between tangent-space and intrinsic error. The two-sided comparison isolates geometric distortion through κB\kappa_B. This provides a useful way to understand how tangent-space estimation guarantees transfer to manifold-valued curves.

Critical weaknesses
  • The guarantee is restricted to a correctly specified model. Assumption 6 requires the ambient population minimizer, mapped to the tangent space, to belong to VV. The resulting bounds contain no approximation or misspecification term. The quantity Ω(vM⊥⋆)\Omega(v^\star_{\mathcal{M}^\perp}) measures sparsity within that correctly specified model; it does not address model misspecification. Calling the result an "oracle inequality" therefore suggests a broader guarantee than is established.

  • The theory does not establish a benefit from regularization. The stated bounds are minimized at λ=0\lambda = 0. The sparsity refinements reduce the regularization cost but do not show that regularization improves the estimation rate. In particular, the proposed choice λn=2LMφ2log⁡(2K)/n\lambda_n = 2LM_\varphi\sqrt{2\log(2K)/n} is not accompanied by a result showing why it is preferable to unregularized ERM. This weakens the abstract's claim of conditions supporting tangent regularization.

Questions for authors
  1. Can Theorem 3 handle vamb∉Vv_{\mathrm{amb}} \notin V through an approximation term such as R(vV)−R(vamb)R(v_V)-R(v_{\mathrm{amb}})? If not, would "estimation bound under realizability" describe the result more accurately than "oracle inequality"?

  2. Is there a regime in which the theory shows that λ>0\lambda > 0 improves over λ=0\lambda = 0? Can a suitable lower bound on λ\lambda, together with a cone-localization argument, yield a sparsity-adaptive rate such as slog⁡K/ns\log K/n?

  3. What is the full dependence of the explicit rate on KK, τ\tau, δ\delta, and κB\kappa_B? In particular, how do the small-ball constants change as the dictionary grows?

  4. How should readers interpret the EEG result when the fitted curves are essentially constant and the errors are close to those of the constant predictor?

4 comments
AuthorsOpenReview ↗

Response to Reviewer WFeG

We thank the reviewer again for his careful reading. Below, we address each concern and question point-by-point.

1. Model misspecification and oracle terminology (W1; Q1)

We agree that the original Theorem 3 assumes that the ambient population minimizer belongs to the selected model. We have renamed it an estimation bound under realizability and revised the abstract and contributions accordingly. An in-model risk minimizer belongs to the model by definition; ambient-oracle membership is stronger.

Theorem 5 (Section 3.6; full statement and proof in Theorem L.3) now allows misspecification. Let β∘\beta^\circ be fixed feasible coefficients defining a reference predictor, and fix SS with s=∣S∣s=|S|. Write Δ=R(γβ∘)−R(γ⋆)\Delta=R(\gamma_{\beta^\circ})-R(\gamma^\star), where γ⋆\gamma^\star is the ambient risk minimizer. Under Appendix L's assumptions, for λ≥4c\lambda\ge4c, with probability at least 1−δ1-\delta, every feasible measurable global ϵopt\epsilon_{\rm opt}-minimizer satisfies R(γβ^)−R(γ⋆)≤3Δ+272μλ2s+3(2λ∥∣βSc∘∥∣1+cB/n2+ϵopt).R(\gamma_{\widehat\beta})-R(\gamma^\star) \leq 3\Delta +\frac{27}{2\mu}\lambda^2 s + 3(2\lambda\||\beta^\circ_{S^c}\||_1+cB/n^2+\epsilon_{\rm opt}). Here ϵopt\epsilon_{\rm opt} bounds the global penalized empirical objective gap, and cc is the explicit stochastic threshold in Appendix L. Choosing a population best-in-model reference predictor makes Δ\Delta the model-approximation gap. The coefficient tail measures approximate sparsity.

The result assumes a fixed base point and bounded population-orthonormal dictionary, i.i.d. data, and the stated geometry and loss conditions. It requires neither geodesic closure of the fitted dictionary class nor a small-ball condition. The probability event concerns the fixed reference predictor and coordinate set.

For the original small-ball approach, restricting the model remains necessary: on a nonatomic design, indicator fields supported on sets of vanishing probability make the ambient small-ball constant zero, even in Euclidean space (Appendix D.4).

2. Regularization and sparse rates (W2; Q2)

For an exactly ss-sparse realizable target and λ≥4c\lambda\ge4c, Corollary L.4 gives

\le\frac{9\lambda^2s}{\mu^2} +\frac4\mu(cB/n^2+\varepsilon).$$ The proof combines uniform loss-increment control, penalty decomposability and the ambient population margin. The bound has no maximum with a full-model localization radius. At $\lambda=4c$, its leading term has order $$\frac{L^2\kappa_B^2M^2 \{\log(2K)+\log(J/\delta)\}}{\mu^2} \frac{s}{n}, \qquad J=1+\lceil\log_2(2\sqrt K\,n^2)\rceil.$$ The gain concerns the sparsity dependence of the upper bound. Risk domination over ERM is not established. The simulation cross-validation grids lie below the sufficient tuning threshold, which is at least 96 times each grid's maximum (Appendix O.5). Thus this theorem does not certify those fits. At $n=160,K=22$, intrinsic RERM/ERM ratios of mean integrated squared geodesic errors are 0.327, 0.539, 0.726 and 0.974 for $s=2,4,8,16$. Tangent LASSO has lower mean error at every level (Table 9). **3. Dimension and curvature constants (Q3)** Appendices E and O.1 specify the constants in the original dictionary corollary, retaining its realizability and geodesic-closure assumptions. With $$q_\tau=Q_{\mathcal H}(\tau)\ge\frac{(1-\tau^2)^2}{KM^2},\qquad 0<\tau<1,$$ the stated sample-size conditions yield, with probability at least $1-\delta-5e^{-c_Aq_\tau^2n}$, $$\|\widehat v-v^\star\|_{L_2}^2 \le\frac{8192L^2\kappa_B^6K}{\delta^2\varkappa_0^2n} +\frac{2\{\lambda\Omega(v^\star)+\epsilon_{\rm opt}\}}{\varkappa_0},$$ with $c_A,\varkappa_0$ as defined in the analysis. Geodesic conversion multiplies this bound by $\kappa_B^2$. Substituting the generic lower bound for $q_\tau$, with other parameters fixed, can produce a sufficient $K^5$ sample-size threshold and a $K^3/n$ tangent-radius term. This does not identify the actual small-ball probability or a necessary sample size. The curvature factors are $\kappa_B^6$ for the tangent-radius term and $\kappa_B^8$ after geodesic conversion. Appendix M separately bounds the Fourier contraction hull under realizability. Appendix L avoids the small-ball argument. An informative complete risk guarantee at the reported sample sizes remains unestablished. **4. Nearly constant EEG fits (Q4)** These results indicate strong shrinkage in within-recording interpolation. Of 308 test recordings from 22 subjects, 306 Fourier fits are constant. Subject-averaged squared geodesic errors are 2.563994 for intrinsic Fourier RERM, 2.563992 for tangent Fourier RERM and 2.563403 for constant Fréchet prediction. They do not establish temporal covariance recovery or an intrinsic-loss advantage. The EEG study remains exploratory: the available records do not establish that all final specifications preceded test inspection. Estimated base points and dependent windows also prevent direct application of the fixed-model i.i.d. guarantee.
AuthorsOpenReview ↗

Additional Clarification on Model Restriction and Realizability

To complement our response to your first concern, we clarify why the original small-ball analysis uses a restricted tangent model and identify where the separate realizability assumption enters its proof.

The restriction to a selected tangent model V⊆VpB\mathcal V\subseteq\mathcal V_p^B has a statistical purpose. As shown in Appendix D.4, when PT\mathbb P_T is nonatomic, the uniform small-ball condition fails on the full ambient difference class $ \mathcal H_{\mathrm{amb}} =\mathcal V_p^B-\mathcal V_p^B. $ Indeed, fix τ>0\tau>0, a unit tangent vector uu, and 0<a≤2B0<a\le2B. Nonatomicity allows measurable sets AA with arbitrarily small probability PT(A)=q>0\mathbb P_T(A)=q>0. Define h=a1Auh=a\mathbf 1_Au. Since both h/2h/2 and −h/2-h/2 belong to VpB\mathcal V_p^B, their difference hh belongs to Hamb\mathcal H_{\mathrm{amb}}. Moreover, $ ||h||_{L_2(\mathbb P_T)}=a\sqrt q, $ and, whenever τq≤1\tau\sqrt q\le1,

\||h(T)\||_{\mathfrak g_p} \ge\tau\||h\||_{L_2(\mathbb P_T)} \right)=q.$$ Letting $q\downarrow0$ therefore gives \$ Q_{\mathcal H_{\mathrm{amb}}}(\tau)=0\$ for every $\tau>0.$ Thus boundedness alone does not ensure the uniform small-ball condition. The original analysis requires this condition to be verified on the selected difference class $\mathcal V-\mathcal V$. This obstruction already occurs in $\mathbb R^D$, independently of curvature. The distinction between an oracle within a prescribed class and an ambient risk minimizer also appears in the learning frameworks of [Mendelson (2015)](https://arxiv.org/pdf/1401.0304), [Lecué and Mendelson (2018)](https://lecueguillaume.github.io/assets/sparse_lecue_mendelson.pdf), and [Alquier, Cottet and Lecué (2019)](https://lecueguillaume.github.io/assets/AOS1742.pdf). In common notation, their target within the class is \$ f_F^\star\in\operatorname*{arg\,min}_{f\in F}R(f), \$ when the risk is finite and the minimum is attained. Membership of this target in $F$ is part of its definition; this formulation does not require an unrestricted population risk minimizer to belong to $F$. Our original Theorem 3 makes the stronger assumption that the ambient oracle, represented by $v^\star$, belongs to $\mathcal V$. This membership permits comparison of the empirical objective with $v^\star$ in the basic optimality inequality (Appendix E.3, Eq. (26)) and gives \$ \mathcal V-v^\star\subseteq\mathcal V-\mathcal V. \$ Convexity of $\mathcal V$ additionally makes $\mathcal V-v^\star$ star-shaped around zero, which supports the radial localization in Lemma H.8. Ambient optimality also supplies the population first-order condition used in Lemma I.1. Model restriction and ambient-oracle realizability are therefore separate requirements of the original analysis. The new Theorem 5, stated fully in Theorem L.3, addresses model misspecification through a fixed feasible reference predictor and an explicit population-risk approximation term.

Thank you again for your careful review and constructive feedback. We hope that our responses and revisions have addressed your concerns and questions. If you find that they resolve your main concerns, we would be grateful if you would reconsider your assessment and score in light of these changes. We would be happy to clarify any outstanding points before the discussion period ends on 18 November.

Dear Reviewer,

As the discussion period is about to close, we would like to briefly follow up on our previous message. We have done our best to address each of your comments, both in our responses and in the revised manuscript. If you feel that your main concerns have now been resolved, we would be sincerely grateful if you could update your score to reflect this. Should any point still require clarification, please let us know and we will respond promptly.

Thank you again for your time and for engaging with our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comments and Overview of Revisions

We thank the reviewers for their careful reading and constructive feedback. Their comments helped us clarify the geometric assumptions, strengthen the analysis of sparse estimation, and make the scope of the empirical conclusions more explicit. The reviews highlight several strengths that the revision builds upon:

  • The geometric construction. All three reviewers recognize the role of the geodesic contraction hull. Their comments highlight how this construction addresses the distinction between tangent-space convexity and geodesic closure.

  • The connection between representation and prediction. Reviewers WFeG and ZKnt emphasize the explicit comparison between tangent-space and intrinsic errors, while xqyw recognizes the detailed treatment of geometric distortion, localized complexity, and decomposable penalties.

  • The transparent empirical comparisons. Reviewers ZKnt and xqyw appreciate the inclusion of tangent-space and intrinsic baselines and the reporting of unfavorable comparisons.

The main theoretical addition is a sparse oracle bound for a fixed feasible comparator (Theorem 5, Section 3.6; full statement and proof in Theorem L.3). It accounts separately for the approximation error of the chosen comparator, a sparsity-dependent estimation term, a coefficient-tail term for approximate sparsity, and global optimization error. The result does not require realizability or geodesic closure of the fitted dictionary class. For exactly sparse realizable targets, Corollary L.4 gives a sparsity-dependent leading term without a maximum involving the full-model localization radius.

Appendices M–O also bound the multiplier complexity of the Fourier geodesic contraction hull under realizability, make the dimension and curvature dependence explicit, and derive dual lower bounds that yield computable upper bounds on global objective gaps. We report numerical evaluations of these upper bounds for the simulation fits.

The revised manuscript clarifies how these results relate to the experiments. The simulation cross-validation grid does not satisfy the sufficient tuning condition of the new sparse theorem, and Fourier Gram-matrix conditioning alone does not provide a complete risk guarantee. We retain the empirical comparisons in which tangent LASSO performs better. The EEG analysis remains exploratory; Appendix O.3 explains that the available records do not establish that all model specifications were fixed before inspection of the test results.

Changes to the manuscript are marked in blue. The responses below explain the revised results, their assumptions, and the scope of the empirical conclusions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.