acceptodds
Under review as a conference paper at ICLR 2027

Stochastic Newton without Hessians: Estimation and Inference

Abstract

Stochastic Newton methods use population curvature, which may remain well defined when sample Hessians are unavailable or undefined. We develop an online stochastic Newton method based on symmetric score secants and Polyak-Ruppert averaging that allows discontinuous sample scores. A spectrally projected recursive average supplies the curvature matrix for subsequent parameter updates. Under stability, iterate rate and score regularity conditions, we derive restrictions on secant second moments, the perturbation radius and the auxiliary batch size under which curvature learning is asymptotically negligible in the Polyak-Ruppert expansion. For quantile regression, shared endpoint observations give an \(O(\delta^-1)\) secant second moment bound, whereas independent endpoints have variance of order \(\delta^-2\) under nondegeneracy, so sharing enlarges the sufficient tuning region for the curvature radius. For direct unbiased primary scores, the averaged estimator retains the usual first-order sandwich limit, and when the spectral projection set contains the population curvature the learned matrix also supports consistent online sandwich estimation. We separately analyze the smoothing bias and direction-dependent covariance of function-value primary updates with auxiliary score access retained, and under additional process conditions a centered functional limit yields iterate-based self-normalized inference. Simulations examine the predicted rates and finite-sample behavior, and an online news application illustrates streaming quantile inference.

Primary area:
optimization
Keywords:
Newton method, Nonsmoothed optimization, Secant approximation, Online inference

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hQreOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper focuses on giving an asymptotic analysis of Stochastic Newton method when Hessian are not available. The authors then suggest to consider the case where the curvature is learned via inference with perturbed gradient calls (also called "score secants"). The authors give an introduction to related work before defining their method, which involves a projection of the Hessian estimation via score secants onto the set of symmetric positive definite matrices with bounded eigenvalues. Them, authors describe their hypotheses and introduce their main theoretical results in section 3 and validate them experimentally in section 4.

Critical strengths

The paper extends the results from Am & Huo in the case of an indirect access to curvature estimation via "score secant", aka finite differences of gradient calls with perturbed model.

Critical weaknesses
  • The readability of the paper is not good. Many notations are not nicely introduced, the jargon used makes understanding complicated and certain sections like Section 2 hard to understand. The related work section is long and unclear. The problem is not introduced unlike in Leluc & Portier 2023, Godichon-Baggioni & Werge 2025 or An & Huo 2026. The algorithm is not clearly stated : not in the main paper and in a too verbose version in the appendix.
  • A clear comparison and discussion of the hypotheses and results from this paper and previous works is needed in the main paper. It would be better to highlight what is in common with An and Huo (2026) and more explicitly explain what is the added value of this work.
  • Assumption (T1) seems to be taken from an unpublished work An and Huo (2026) (cf their Assumption 5), yet it seems very restrictive and the scope of it's "applicability" should be discussed in the main paper, rather than in the last page of the appendix.
Questions for authors
  • Can the Proposition C.42 be applied to all the regression cases considered, including quantile regression which score function is not smooth? In other words, is it reasonable to assume (T1) for the method of this paper
  • Could you please highlight more explicitly the motivation of the paper? When does it happen one cannot directly access the Hessian? What are the other concurrent methods to solve these situations (first or second order)?
3 comments

Thank you for your time and valuable comments.

W1. Section 2 now places the update order beside the defining equations. Starting from the past-measurable θt−1\theta_{t-1} and Bt−1B_{t-1}, the method solves Bt−1st=G^t(θt−1)B_{t-1}s_t=\widehat G_t(\theta_{t-1}) and updates θt=θt−1−ηtst\theta_t=\theta_{t-1}-\eta_ts_t. Auxiliary score pairs are centered at θt−1\theta_{t-1}, and their projected recursive average forms BtB_t, first used at the next primary update. The lagged iterate average and score outer-product average are updated online. Detailed pseudocode remains in the appendix. The introduction states the quantile-regression motivation directly, and the related-work discussion separates online inference, curvature estimation and nonsmooth quantile methods.

W2. The comparison now separates shared assumptions from the secant calculation. The mean-square iterate bound (T1) corresponds to Assumption 5 of An and Huo; their decomposition and stabilization threshold supply the transfer argument. Their Proposition 5.1 also supplies an L2L^2-moment route to first-moment remainder control. Theorem 3.3 here verifies stabilization for the projected score-secant recursion using the exponent qq, radius δt\delta_t and batch size RtR_t. Quantile endpoint coupling gives the different sufficient restrictions for q=1q=1 and q=2q=2. The resulting limit shares the usual averaged covariance with stabilized Newton and SGD procedures, while the learned sensitivity is available for sandwich inference.

W3. (T1) is explicitly retained as a model-specific stability input in the main theorem. Secant moment bounds verify (T2), separately from the parameter-trajectory rate in (T1). The discussion next to (Q5) states both the deterministic-time localization and the mean-square rate required for the actual quantile recursion. Proposition C.42 provides a sufficient risk-descent route on a valid stability region. The general multivariate quantile result remains conditional on establishing that region and rate, or directly verifying (Q5).

Q1. Proposition C.42 applies through the population risk, so a discontinuous sample score is admissible. Its hypotheses are a convex region containing the actual trajectory and target, uniform strong convexity and smoothness of the population risk there, a predictable preconditioner with eigenvalues bounded away from zero and infinity, conditional score variance bounded by C1+∣g(θ)∣2C\\{1+\\|g(\theta)\\|^2\\}, and approximation bias bounded by Cηt1/2C\eta_t^{1/2}. For a direct unbiased score the bias is zero. For a sufficiently small initial stepsize, these conditions give

\le(1-c\eta_t)[R(\theta_{t-1})-R(\theta_\star)]+C\eta_t^2,$$ and hence the mean-square rate for $\eta_t\asymp t^{-\alpha}$, $1/2 < \alpha < 1$. Least squares satisfies a global quadratic-risk version under its stated moments. Logistic, Huber and quantile models require their stated trajectory conditions or a separate stability argument. The scalar compact-support quantile example verifies an invariant interval and the rate by its own drift calculation. Local positive population curvature therefore provides the local risk conditions, with trajectory stability handled separately. **Q2.** Quantile regression gives a concrete setting: the sample check loss is piecewise linear, with zero Hessian wherever that Hessian exists, whereas the population sensitivity is $A_\tau=\mathbb E\\{f_{Y\mid X}(X^T\theta_\tau\mid X)XX^T\\}$ and may be positive definite. Direct score SGD is a first-order alternative; sample-Hessian Newton is available for smooth losses; and stochastic score-difference or quasi-Newton methods use other curvature approximations. For inference, sandwich estimation, batching and iterate-based self-normalization are distinct choices. Our method uses paired score evaluations to learn and apply curvature, then reuses it in the sandwich estimator when the clipping set contains the population sensitivity. Its intended scope is fixed, moderate dimension with available auxiliary scores. Function-value primary updates retain that auxiliary-score requirement, and the additional observations, score calls and dense-matrix operations are part of the computational tradeoff. Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.

Thank you again for your careful review and constructive feedback. We hope that our responses and revisions have addressed your concerns and questions. If you find that they resolve your main concerns, we would be grateful if you would reconsider your assessment and score in light of these changes. We would be happy to clarify any outstanding points.

Reviewer_hQreOpenReview ↗

Concern about assumption T1

I thank the authors for their answers and their efforts.

I checked Prop C.42 and still do not understand why the assumption that the gradient bias bt(θ)b_t(\theta) shrinks to zero with the step size sequence. Can the authors justify this simply and clearly?

This proposition is the equivalent of Prop. 2.1 of An and Huo 2026, but in their case, authors do not need to impose their gradient bias to shrink, as they assume to unbiased gradient estimate.

Reviewer_bqhHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes an online stochastic Newton method for root-finding problems g(θ⋆)=0g(\theta^\star)=0 in which sample Hessians may not exist, such as quantile regression. They assume access to a noisy gradient estimator, which is used to estimate the Hessian using finite difference methods. The estimates are then used to update the parameter θt\theta_t using the Newton step, and the Polyak–Ruppert average is the final estimator.

The paper provides theoretical results, proving that under strong conditions, curvature learning preserves the averaged first-order covariance.

Critical strengths
  1. A well stated problem with a good answer

The paper shows that second-order information can be recovered from first-order information alone. Differences of sample gradients (score secants) estimate the population Hessian, even when the sample Hessian is zero or undefined. The paper proves that using this learned curvature in a Newton update leaves the limiting distribution of the averaged estimator unchanged.

Critical weaknesses
  1. Unclear writing

The notation is well defined in most cases, and the paper is easy to follow until section 2. However, section 3 is hard to follow, and many statements are left unexplained. For example, assumption 3.1 is a list of 6 assumptions, and only T1 and T2 are explained in the main body. The assumptions are also scattered, with the main body having 2 sets of assumptions, and referring to assumption C.4 in the appendix. This makes the paper harder to follow.

Moreover, while the results in section 3 are followed by proof sketches, they are not discussed - their importance and implications are left for the reader to interpret. This makes the paper hard to read, and it ends up feeling like a list of theorems.

  1. The experiments are weak, and do not test the paper’s motivating case

Where sample Hessians are available, the method shows no advantage. Though this is expected, what is surprising is that it costs much more. In the logistic regression experiment (Table 1), it matches SN on RMSE (0.079 vs. 0.081) while using 3× the observations and 5× the score evaluations.

More importantly, there is no simulation reporting RMSE or coverage for quantile regression. That is the setting the method is designed for, since sample Hessians do not exist there. The quantile simulations check only intermediate quantities: the variance of individual secant curvature estimates at a fixed parameter (Figure 1) and the rate at which the curvature matrix stabilizes (Figure 2). Neither shows whether the final estimator is accurate or whether the sandwich intervals achieve nominal coverage.

Questions for authors
  1. T1 is a very strong condition, and one expects this to be true under strong convexity, smoothness, etc. Is it true in settings the paper is interested in, like quantiles?
  2. Can the authors add a multivariate quantile simulation? This would be useful to understand the empirical effectiveness of the algorithm.
5 comments

Thank you for your time and valuable comments.

Note: Because of the 5000-character limit, we divided our response into two comments.

W1. Section 3 now explains all six transfer conditions together. (T1) supplies the iterate mean-square rate, (T2) controls variation of the preconditioner, (T3) controls mean-score linearization, (T4) controls replacement of the local score by its target value, (T5) controls local approximation bias, and (T6) supplies the target martingale limit. Assumption C.4 verifies generic auxiliary-secant moments, whereas (Q1)-(Q5) are the model-specific conditions used directly by the quantile theorem. This separation identifies which assumptions concern the parameter trajectory and which concern learning curvature.

Theorem 3.2 identifies when changing curvature preserves the averaged influence function. Theorem 3.3 turns score-secant moments into a sufficient radius/batch-size restriction. Theorem 3.5 specializes the calculation to quantile regression, and Theorem 3.6 permits the learned sensitivity to be reused in sandwich inference. The common first-order covariance is A−1QA−TA^{-1}QA^{-T}; curvature learning changes the transient trajectory and supplies a sensitivity estimate. The sandwich theorem assumes that the clipping set contains AA. Self-normalized inference instead uses a quadratic normalizer from partial averages and the additional centered functional-limit conditions in Appendix C.3.

W2. The logistic comparison illustrates the cost of replacing sample Hessians by score evaluations. At d=10d=10 and 20000 primary updates, the results are:

MethodRMSECoordinate coverageContrast coverageObservations (thousands)Scores (thousands)
Proposed0.0790.9320.91660100
Sample-Hessian Newton0.0810.9100.9522020
Population-curvature oracle Newton0.2450.0000.4622020
SGD0.2520.2820.4182020

The proposed method and sample-Hessian Newton have similar RMSE, with higher proposed coordinate coverage but lower proposed contrast coverage. The proposed method uses three times the observations and five times the score evaluations; sample-Hessian Newton additionally evaluates Hessians. These are equal-primary-update comparisons. The SGD intervals use population sensitivity as an oracle diagnostic. The resource comparison in the median experiment in Q2 (Part 2/2) favors observation-matched SGD in RMSE.

The new median-regression experiment evaluates the final averaged coefficient vector and its intervals, in addition to the original secant-variance and curvature-increment diagnostics. It uses ten regression coefficients, nominal 95% intervals and M=500M=500 replications per method. RMSE is the square root of the mean squared coefficient error over coordinates and replications; coverage is evaluated for the first coordinate and the contrast 10−1/2∑j=110θj10^{-1/2}\sum_{j=1}^{10}\theta_j. The proposed estimator has RMSE 0.028, coordinate coverage 0.910 and contrast coverage 0.926.

Continued in Part 2/2: Q1–Q2: trajectory stability and the median-regression experiment.

Part 2/2 of our response to Reviewer bqhH (continued): Q1–Q2: trajectory stability and the median-regression experiment.

Q1. Quantile-score discontinuity is compatible with the population-risk argument in Proposition C.42. Under the density conditions, the anchored population check-loss risk is differentiable with gradient gτg_\tau, and its Hessian is continuous and positive definite near the target. On a convex region containing the actual trajectory almost surely, uniform strong convexity and smoothness, predictable spectrally bounded preconditioners and the stated conditional noise bounds yield E∣θt−θτ∣2=O(ηt)\mathbb E\\|\theta_t-\theta_\tau\\|^2=O(\eta_t). Establishing that trajectory containment, or another valid global rate argument, is a separate model-specific requirement. Accordingly, (Q5) remains an explicit hypothesis of the multivariate quantile theorem. The appendix supplies a primitive least-squares proof and attainable scalar examples, including a quantile example with an invariant interval.

Q2. The added model is Y=XTθ⋆+εY=X^T\theta_\star+\varepsilon, with d=10d=10, independent standard Gaussian errors, Gaussian regressors with diagonal covariance eigenvalues ranging geometrically from 1 to 0.05, and τ=0.5\tau=0.5. The ten coefficients are obtained by repeating and truncating (0.5,−0.25,0.125,−0.0625)(0.5,-0.25,0.125,-0.0625). The proposed method uses ηt=0.8t−0.75\eta_t=0.8t^{-0.75}, δt=0.4t−0.2\delta_t=0.4t^{-0.2}, two auxiliary observations per update, clipping to [0.001,2][0.001,2], and 50 initial unpreconditioned updates while curvature learning continues. The gains for oracle Newton, equal-update SGD and observation-matched SGD are 0.4, 25.6 and 12.8, respectively, with the same exponent 0.75. Independent twelve-replication pilots fixed the gains before evaluation, using the smallest RMSE against the known coefficient target. At the evaluation horizons, the pilot grids were 0.4,0.8\\{0.4,0.8\\} for proposed, 0.2,0.4\\{0.2,0.4\\} for oracle Newton, and 3.2,6.4,12.8,25.6,51.2\\{3.2,6.4,12.8,25.6,51.2\\} for each SGD horizon. All methods start at zero and average the full lagged trajectory.

MethodPrimary updatesRMSECoordinate coverageContrast coverageObservations (thousands)Scores (thousands)
Proposed200000.0280.9100.92660100
Population-curvature oracle Newton200000.0260.9320.9082020
SGD, equal updates200000.0310.9200.8722020
SGD, equal observations600000.0150.9500.9206060

The table reports M=500M=500 per method. Proposed RMSE is lower than SGD at equal primary updates, while SGD is more accurate at equal observations and uses fewer score evaluations. Every method estimates score covariance from the primary-score outer products. Oracle Newton and SGD intervals use population sensitivity; the proposed intervals use learned sensitivity. Standard errors use each method's primary-update count and the normal critical value z0.975z_{0.975}. The equal-observation comparison has different score-query budgets, which are shown explicitly.

Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.

Thank you again for your careful review and constructive feedback. We hope that our responses and revisions have addressed your concerns and questions. If you find that they resolve your main concerns, we would be grateful if you would reconsider your assessment and score in light of these changes. We would be happy to clarify any outstanding points.

Reviewer_7Lj4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper suggests a method for estimating the asymptotic covariance matrix is a several SGD-related problems based on the discontinuous stochastic oracles. Here the Hessian at the optimal point is evaluated based on the differences of stochastic scores evaluated at the viscinity of the current point using the same observation. The main application of the proposed method concerns the quantile regression. Under appropriate regularity conditions, the paper obtains asymptotic consistency results for the proposed estimator and provides numerical results in the toyish environment.

Critical strengths

I actually would say "None".

Critical weaknesses
  1. I would say that the paper uses too much AI for writing. The resulting text is both heavy to read and completely not specific. Short example from main contributions:

"Paired score evaluations estimate the population Jacobian even for discontinuous sample scores, and their second moments determine the fluctuations of the recursively learned matrix"

While this sentence actually contains some information, at the same time, it is completely not specific. One can provide a large number of similar ambiguous phrases in the main text. There are also further issues: for example, the same part of contributions start discussing the exponent qq without introducing it. One can continue with the same list of AI-introduced issues.

  1. Literature review is incomplete. There are lot of recent papers e.g. by the group of Wei Biao Wu, e.g. [1-2] on the topic of the paper, which are not discussed. These are just quick references, at the same time, quantile regression and inference for it attracted lot of attention recently.

  2. The technical novelty is substantially low. The paper obtains only a rather simple asymptotic consistency result for the suggested estimator (theorems 3.5 and 3.6). The corresponding proofs are based on standard martingale CLT arguments. On the formal side, it remains unclear what is the respective rate of convergence, what is its dependence on dimension (the latter one is particularly important). Several elements of the analysis seems to follow closely [3], and it is unclear from the current writing, how substantial is the corresponding extension.

On the more philosophical level, I do not understand the main message of the paper. The current draft contains many theorems and auxiliary results, remarks, etc, while in my opinion it does not provide a reasonable central message and understanding on why all these results are needed. In this sense, I think that this is much more significant issue than the technical issues with asymptotic nature of the results.

References

[1] Ziyang Wei, Jiaqi Li, Likai Chen, Wei Biao Wu. Central Limit Theorems for Stochastic Gradient Descent Quantile Estimators. In IEEE Transactions on Information Theory, 2026.

[2] L Chen, G Keilbar, WB Wu. Recursive quantile estimation: Non-asymptotic confidence bounds. Journal of Machine Learning Research 24 (91), 1-25

[3] S. An and X. Huo. When does dynamic preconditioning preserve the Polyak–Ruppert CLT? A stabilization threshold, 2026. URL https://arxiv.org/abs/2604.23498.

Questions for authors
  1. Is it possible to quantify the rate of convergence in Theorems 3.5-3.6 and how it depends on dimension of the problem? I do not insist to include these results here, I would like just to understand if there are some technical difficulties here.

  2. Please specify your contribution in a more concrete and formal style, without summarizations of the paper itself.

  3. Could you please elaborate in more details what are the connections and novelty compared to the reference [3] above?

5 comments

Thank you for your time and valuable comments.

Note: Because of the 5000-character limit, we divided our response into two comments.

W1. The introduction now states the mechanism in terms of score differences and their variance. A shared observation at the two endpoints gives a quantile-secant second-moment bound O(δ−1)O(\delta^{-1}), whereas independent endpoints have variance of order δ−2\delta^{-2} under nondegeneracy. The contribution paragraph uses these orders before introducing the exponent qq. Section 2 gives the update order explicitly: the primary update uses Bt−1B_{t-1}, and the newly estimated curvature BtB_t is first used at iteration t+1t+1.

W2. The related-work discussion includes Chen, Keilbar and Wu (2023), who obtain nonasymptotic bounds for recursive averaged quantiles, including coordinatewise marginal quantiles, and Wei, Li, Chen and Wu (2026), who analyze constant-stepsize quantile SGD through its stationary distribution and a small-stepsize Gaussian limit. These results provide complementary quantile-estimation and inference guarantees. Our regression setting couples coefficient updates with a recursively learned population sensitivity matrix obtained from paired score evaluations.

W3. Theorem 3.5 gives a root-nn quantile-regression central limit theorem and consistent sandwich estimation; Theorem 3.6 gives covariance consistency and, together with the centered limit, Wald validity. These are fixed-dimensional results. The dynamic decomposition and stabilization threshold come from An and Huo. The secant-specific contribution is to derive the required stabilization from the score-increment moments, perturbation radius and auxiliary batch size, including the distinct shared- and independent-endpoint rates for discontinuous quantile scores. The quantitative remainder and its dimension-dependent constants are stated in Q1 (Part 2/2).

W4. The central question is how to learn and use population curvature when sample Hessians are zero or unavailable while retaining inference from the averaged iterates. The results follow one chain: score differences estimate curvature; their second moments control changes in the projected curvature recursion; sufficiently small changes preserve the averaged influence function; and the same curvature estimate enters the sandwich covariance when the clipping set contains the population sensitivity. The additional bias and functional-limit results specify the separate conditions for function-value primary updates and self-normalized inference.

Continued in Part 2/2: Q1–Q3: quantitative remainder, endpoint-sharing rates, and relation to An and Huo.

Part 2/2 of our response to Reviewer 7Lj4 (continued): Q1–Q3: quantitative remainder, endpoint-sharing rates, and relation to An and Huo.

Q1. Under all the hypotheses of Theorem 3.5, including the trajectory localization and mean-square rate in (Q5), let s=(qκH−r)+<1−αs=(q\kappa_{\mathrm H}-r)_+ < 1-\alpha and

En=n(θˉn−θτ)+Aτ−1n−1/2∑t=1nGτ(θτ;Zt).\mathcal E_n=\sqrt n(\bar\theta_n-\theta_\tau)+A_\tau^{-1}n^{-1/2}\sum_{t=1}^nG_\tau(\theta_\tau;Z_t).

The proof bounds yield

\mathbb E\\|\mathcal E_n\\|\le C_d\left\\{n^{-(1-\alpha-s)/2}+n^{-[\alpha(1+\nu)-1]/2}+n^{-\alpha/4}\right\\}.

Here q=1q=1 for shared endpoints and q=2q=2 for independent endpoints. The three terms bound the dynamic Abel remainder, population-score linearization and centered local score replacement. Appendix C.5.4 gives the calculation and its power-sum boundaries. Its scope is an L1L^1 asymptotic-linear-representation bound at fixed dimension. A coverage-error rate additionally requires quantitative Gaussian approximation and studentization control. The general hypotheses of Theorem 3.6 prescribe convergence, rather than rates, for the curvature and target second moments. A quantitative covariance bound requires separate control of those terms. The constant CdC_d depends on score and density-envelope moments, direction bounds, curvature eigenvalues and the iterate-rate constant; uniform growing-dimensional inference requires explicit control of their dependence on dd.

Q2. The concrete calculation is

E![∣H^t−E(H^t∣Ft−1)∣F2∣Ft−1]≤C1+(Rtδtq)−1,\mathbb E\\!\left[\\|\widehat H_t-\mathbb E(\widehat H_t\mid\mathcal F_{t-1})\\|_F^2\mid\mathcal F_{t-1}\right]\le C\\{1+(R_t\delta_t^q)^{-1}\\},

which, through the projected recursive average, gives

E∣Bt−Bt−1∣F21/2≤Ct−1+(qκH−r)+/2.\\{\mathbb E\\|B_t-B_{t-1}\\|_F^2\\}^{1/2}\le Ct^{-1+(q\kappa_{\mathrm H}-r)_+/2}.

Combining this bound with the inherited stabilization threshold gives (qκH−r)+<1−α(q\kappa_{\mathrm H}-r)_+ < 1-\alpha. For quantile scores, shared observations give q=1q=1, while independent endpoints give q=2q=2 under nondegeneracy. For example, with α=3/4\alpha=3/4 and a fixed auxiliary batch, the sufficient restrictions are κH<1/4\kappa_{\mathrm H} < 1/4 and κH<1/8\kappa_{\mathrm H} < 1/8, respectively. The direct primary quantile score remains unsmoothed, and the inferential conclusions retain the separate stability and score-regularity hypotheses.

Q3. The shared input (T1) corresponds to Assumption 5 of An and Huo. Their Lemma 3.1 supplies the preconditioner-isolating decomposition, Theorem 3.4 supplies the stabilization threshold, and Proposition 3.5 identifies its sharpness within the stated polynomial-rate hypothesis class. Their Proposition 5.1 already provides an L2L^2-moment route to first-moment control of the dynamic remainder. Our analysis derives the increment rate for the projected score-secant construction, allowing secant variability to increase as the radius shrinks, and connects it to endpoint coupling and auxiliary batching. The approximate-score extension also separates target bias, local bias variation and nonsmooth score replacement. These contributions retain model-specific iterate stability as an input.

Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.

Thank you again for your careful review and constructive feedback. We hope that our responses and revisions have addressed your concerns and questions. If you find that they resolve your main concerns, we would be grateful if you would reconsider your assessment and score in light of these changes. We would be happy to clarify any outstanding points.

Comments on the paper

1 comment

Thank you for your time and valuable comments.

We clarified the order of the parameter and curvature updates, the relation to existing dynamic-preconditioning theory, and the distinct roles of trajectory stability and secant stabilization. The discussion includes the quantitative quantile remainder, endpoint-sharing rates and feasible sandwich estimation. The end-to-end median-regression comparison now appears in the main text with its observation and score budgets, preserving the distinction between learned and oracle sensitivity.

These changes have been incorporated into the article, with the additions and modifications highlighted in blue. We hope the responses address your concerns and welcome further questions or discussion. Thank you again for your valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.