acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty Quantification of Next Generation Reservoir Computing with Applications to Memory-Driven Dynamical Systems

Abstract

Nonlinear dynamical systems with memory arise across science and engineering, yet uncertainty quantification for efficient forecasting methods such as Next Generation Reservoir Computing (NGRC) remains underdeveloped. We study Bayesian ridge and conformal prediction intervals for NGRC and characterize when their uncertainty estimates agree or differ. In low dimensions, their asymptotic widths are governed by different summaries of the residual distribution, so agreement depends on residual shape rather than dimensionality alone. In high dimensions, regularization introduces a further tradeoff between estimation variance, shrinkage bias, and posterior uncertainty, leading to an explicit transition between regimes where Bayesian intervals are wider or narrower than conformal intervals. We extend these results to quadratic NGRC feature maps and give sufficient conditions for transferring the analysis to temporally dependent forecast windows. Simulations and real-data experiments support the theoretical predictions and illustrate how residual distribution, regularization, dimensionality, and distribution shift affect interval calibration and efficiency. These results provide a principled framework for choosing and interpreting uncertainty quantification methods in reservoir-based forecasting.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Conformal prediction, Reservoir computing, Time series, Uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Agw8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
3
Read review
Paper summary

The paper compares Bayesian ridge and split-conformal prediction intervals built around the same NGRC predictor. For fixed-dimensional settings, it derives limiting interval widths based on the residual root-mean-square scale and an absolute-residual quantile. In high-dimensional settings, it separates the effects of ridge bias, estimation variance, and posterior uncertainty, and establishes an explicit condition for comparing interval widths under an isotropic Gaussian model.

Critical strengths
  • The analysis of fitted quadratic prediction errors is a key strength of the paper. Lemma 5 uses the quadratic component of the prediction error to establish a Gaussian approximation. Corollary 3 then bounds the contribution of training noise and identifies a remaining condition on the ridge-transformed bias.
  • The comparison of prediction intervals accounts for different sources of uncertainty. Using the same fitted predictor makes the comparison easier to interpret. The finite-sample analysis separates ridge bias, estimation variance, and posterior uncertainty, and the proofs establish concentration of the realized prediction error before deriving the conformal interval widths.
Critical weaknesses
  • The paper needs to clarify what it adds beyond existing work on ridge regression and quadratic features. Theorem 1 relies on consistency results for residual scales and quantiles, while the threshold in Corollary 2 is consistent with the established prior matching and optimal ridge scale, as the authors acknowledge in Section 4.4 and Appendix C.7.4. A relevant reference is Pandit, Wang, and Zhu, JMLR 2025. Their Lemma 21 establishes a similar order of quadratic-form concentration for the centered quadratic block considered in Lemma 4, and their paper also studies regression-risk limits in the quadratic scaling. The main contribution beyond this existing work therefore appears to be the analysis of the fitted-residual distribution and its use in deriving conformal quantiles. A more direct comparison with this paper would help clarify the novelty and significance of the contribution.

  • The connection between the main theoretical result and nonlinear forecasting applications is not fully established. The quadratic-feature theorem relies on several specific assumptions, including population-normalized features built from Gaussian base vectors, fixed regularization, an exact readout, and Gaussian noise. The deterministic extension also requires a condition involving the unknown signal and the fitted ridge matrix. Proposition 1 extends the results to sufficiently separated windows, but at a substantial sampling cost, and it retains the assumptions on the marginal model and readout. Meanwhile, the real data experiments use settings that do not fully satisfy the theorem's assumptions and serve mainly as qualitative illustrations. Although the authors clearly acknowledge these limitations, the paper would be stronger if it included a concrete forecasting model covered by the quadratic analysis or provided a clearer justification for how the theory relates to the applications.

Questions for authors
  • The paper would benefit from a clearer explanation of what Theorem 3 or Corollary 3 adds beyond existing work on quadratic feature concentration and ridge regression risk, particularly Pandit, Wang, and Zhu (JMLR 2025). Which part of the fitted-residual or quantile analysis provides the main new technical contribution?

  • What class of nonlinear forecasting processes is the quadratic benchmark, and its extension to dependent time windows, intended to capture? How do the Gaussian-base and exact-readout assumptions relate to this class?

  • Does the current analysis cover a specific class of nonzero deterministic quadratic signals for which Equation (5) can be verified after ridge fitting? Please clarify which cases are already covered by the existing proofs and which would require further analysis.

4 comments
AuthorsOpenReview ↗

Response 1 of 3: W1/Q1: contribution beyond PWZ

We thank the reviewer for a careful reading, for identifying the fitted-residual analysis as the core technical contribution, and in particular for the pointer to Pandit, Wang and Zhu (JMLR 20252025), which we had missed and now cite and discuss.

W1 / Q1 — What Theorem 3 and Corollary 3 add beyond Pandit, Wang and Zhu (PWZ).

The reviewer is right that there is overlap. (i) Lemma 44, which establishes quadratic-form concentration for the centered quadratic block, is of the same order as PWZ’s Lemma 2121. Our proof is a short Gaussian–Poincaré argument with explicit constants. We will state this and credit their result in the revision. (ii) When Σp=Ip\Sigma_{p}=I_{p}, the conformal residual variance TnT_{n} in Theorem 3 is the random-teacher ridge risk in the quadratic regime n≍d2n\asymp d^{2}, which PWZ characterize. In the revision, we will present this agreement as a consistency check rather than as a contribution. (iii) As the reviewer notes, Theorem 1 is a consistency-type statement and the original known-noise Corollary 2 threshold coincides with prior matching and the optimal ridge scale. The revised contribution list separates “new” from “benchmark consistent with known results”. What PWZ do not give, and what we believe is new, is the object that the conformal interval depends on. A risk limit is a second moment. The split-conformal radius is a quantile of the fitted residual distribution. The two coincide only when that distribution is Gaussian, and Appendix C.6.5 shows that this can fail at the same risk. A residual concentrated on one quadratic direction has a centered χ12\chi^{2}_{1} law whose 0.950.95 absolute quantile is not zz times its scale. The new technical pieces are as follows. (a) Lemma 55 provides a Stein-method Kolmogorov bound for a linear-plus-quadratic Gaussian functional whose rate is governed by the operator norm of the quadratic coefficient matrix. (b) Corollary 3, eq. 55, identifies the ridge-transformed direction Qd(λΣp1/2Aw0)Q_{d}(\lambda\Sigma_{p}^{1/2}Aw_{0}) as the relevant matrix rather than the raw signal Qd(w0)Q_{d}(w_{0}). The counterexample shows why raw diffuseness is not enough. (c) The Dvoretzky–Kiefer–Wolfowitz step converts the conditional Gaussian approximation into the conformal quantile expansion with its m−1/2m^{-1/2} term. (d) The concentration of r∗⊤Ar∗r_{\ast}^{\top}Ar_{\ast} for a dependent-coordinate test feature provides the control needed for the Bayesian width. Corollary 3 states a sufficient condition on the ridge-transformed quadratic direction for deterministic readouts. Under the assumptions of Corollary 3, suppose that the ridge-transformed quadratic direction satisfies ∣Dn∣op=OP(ρn)\\|D_n\\|_{\mathrm{op}}=O_P(\rho_n) with ρn→0\rho_n\to0. For mm independent calibration observations, the fitted-residual analysis gives q^1−α=z1−α/2σ2+Bn+Vn+OP!(n−1/4+ρn+m−1/2).\widehat q_{1-\alpha}=z_{1-\alpha/2}\sqrt{\sigma^2+B_n+V_n}+O_P\\!\left(n^{-1/4}+\rho_n+m^{-1/2}\right). This identifies the additional distributional step needed to turn a prediction-risk calculation into a conformal-width comparison. The approximation is expressed using finite-sample trace quantities. Its error rate is distinct from the rate at which those trace quantities converge to their spectral limits.

AuthorsOpenReview ↗

Response 2 of 3: W2/Q2: forecasting model class

W2 / Q2 — Which nonlinear forecasting processes the quadratic benchmark captures, and how the Gaussian-base and exact-readout assumptions relate to them.

We will make the model class explicit in the revision. Call it the Gaussian-linear-state quadratic Volterra (GLQV) model: a stationary Gaussian linear process (an ARMA process, or the discretized linear memory dynamics of Appendix B), observed through a delay vector dtd_{t} with covariance Σd\Sigma_{d}, and a response yt+H=a⊤dt+dt⊤Mdt+εt+Hy_{t+H}=a^{\top}d_{t}+d_{t}^{\top}Md_{t}+\varepsilon_{t+H} with independent Gaussian noise. After population whitening G=Σd−1/2dtG=\Sigma_{d}^{-1/2}d_{t}, the NGRC feature vector is ϕd(G)\phi_{d}(G) up to a fixed linear map, and the readout is exact with w0w_{0} determined by (a,M,Σd)(a,M,\Sigma_{d}). This is the discrete-time second-order Volterra expansion of a Gaussian-driven linear memory system, i.e. the second-order truncation of the memory-driven class in the introduction. In this class, “Gaussian base” means Gaussian innovations of the linear state and “exact readout” means that the second-order truncation is exact. Random readouts correspond to the random-effects version of Theorem 3. Fixed dynamical systems correspond to deterministic readouts, for which Corollary 3 additionally requires Equation (5)(5). The temporal experiment in Appendix E.3.2 is an instance with an AR⁡(1)\operatorname{AR}(1) state and diffuse MM. Its results are collected with the overlap diagnostics in our response to Reviewer qR69, W1/Q1. The Gaussian-input experiment in our response to Reviewer qR69, W1/Q1 provides an exact-readout example of this class and reports the existing AR⁡(1)\operatorname{AR}(1) temporal diagnostic alongside the new overlap sweep. That response also contains the autonomous nonlinear forecast, which tests departures from the Gaussian-base and exact-readout assumptions. For the nonlinear Volterra experiment with data-selected regularization, see Reviewer BwAY, Q1. For the theoretical coverage discussion under dependence, see Reviewer qR69, W1/Q1, which cites Oliveira et al. (2024)(2024) under their stated conditions.

AuthorsOpenReview ↗

Response 3 of 3: Q3: deterministic signals and diagnostics

Q3 — Which deterministic quadratic signals satisfy equation (5) after ridge fitting.

The following cases are currently covered. (a) Theorem 3 covers random-effects readouts under its stated assumptions. (b) Corollary 3 covers deterministic readouts for which Equation (5)(5) holds. Its proof already controls the noise-induced part of the quadratic direction at order n−1/4n^{-1/4}. The remaining condition concerns the deterministic bias after ridge fitting. Raw diffuseness alone does not establish this condition. The diagnostics below evaluate the transformed direction for specific fixed signal families. If needed, we can include an explicit sufficient class of deterministic quadratic signals satisfying Equation (5)(5) in the revision. To diagnose the transformed-signal condition directly, we held coefficient directions fixed across 100100 independent training-design/noise repetitions at each dimension. In the pure-quadratic diagnostic, the linear coefficient block is zero and the quadratic coefficient vector has squared norm 44. Features are independent population-normalized Gaussian linear-plus-quadratic rows with p=d(d+3)/2,nfit=ncal=2p,λ=0.5p=d(d+3)/2,n_{\mathrm{fit}}=n_{\mathrm{cal}}=2p,\lambda=0.5, and σ=0.25\sigma=0.25. Table R13. Fixed-strength, purely quadratic deterministic-readout diagnostic.

Base dimension ddTransformed quadratic normResidual Gaussian KS distanceMean SCP widthFinite-trace width
880.6596±0.01820.6596\pm0.01820.0669±0.00250.0669\pm0.00254.62454.62454.15644.1564
12120.5067±0.01030.5067\pm0.01030.0501±0.00200.0501\pm0.00204.36864.36864.04644.0464
16160.4706±0.00870.4706\pm0.00870.0447±0.00160.0447\pm0.00164.15244.15243.97943.9794
20200.4433±0.00740.4433\pm0.00740.0387±0.00190.0387\pm0.00194.09114.09113.93083.9308
24240.3708±0.00460.3708\pm0.00460.0348±0.00180.0348\pm0.00184.02454.02453.91513.9151
32320.3279±0.00370.3279\pm0.00370.0329±0.00150.0329\pm0.00153.93483.93483.85483.8548

Norm and KS entries are means ±1.96\pm1.96 Monte Carlo standard errors. Widths are replication means. From d=8d=8 to 3232, the transformed operator norm falls from 0.65960.6596 to 0.32790.3279 and the Gaussian KS distance from 0.06690.0669 to 0.03290.0329. The difference between mean SCP width and its finite-trace prediction falls from about 11.311.3\\% to 2.12.1\\%. At d=32d=32, SCP coverage is 0.94970.9497. The frozen full-feature, diffuse-diagonal, and concentrated controls have squared coefficient norm 44 and remain fixed across repetitions. The table shows the smallest and largest tested dimensions. Diffuse directions show improving Gaussian approximation, while the concentrated control retains larger transformed norms and KS distances. These diagnostics support the fitted-residual mechanism for the tested signal families. They do not establish a general sufficient-class theorem. Table R14. Ridge-transformed diffuseness and residual Gaussian approximation at the smallest and largest dimensions.

Base dimension ddSignalTransformed quadratic operator normResidual Gaussian KS distance
88Frozen spherical0.5210±0.01220.5210\pm0.01220.0562±0.00240.0562\pm0.0024
88Diffuse diagonal0.5233±0.00790.5233\pm0.00790.0794±0.00260.0794\pm0.0026
88Concentrated1.0033±0.02841.0033\pm0.02840.1406±0.00330.1406\pm0.0033
3232Frozen spherical0.3238±0.00310.3238\pm0.00310.0322±0.00180.0322\pm0.0018
3232Diffuse diagonal0.2721±0.00180.2721\pm0.00180.0476±0.00240.0476\pm0.0024
3232Concentrated0.8810±0.00950.8810\pm0.00950.1206±0.00260.1206\pm0.0026

We will place the necessary distributional diagnostics and model-class clarifications in the main manuscript within the conference page limit, with expanded numerical controls in the appendix. Thank you for your time and consideration!

References.

  • R. I. Oliveira, P. Orenstein, T. Ramos, and J. V. Romano. Split Conformal Prediction and Non-Exchangeable Data. Journal of Machine Learning Research, 25(225):1–38, 2024.
  • P. Pandit, Z. Wang, and Y. Zhu. Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regime. Journal of Machine Learning Research, 26(224):1–73, 2025.

Dear reviewer, thank you again for your constructive feedback and for the time and effort you have devoted to reviewing our paper.

As the author discussion period is coming to an end, we would be very grateful if you could take a look at our rebuttal responses and let us know whether they have addressed your concerns. In particular, we have:

  • Clarified the contribution beyond the prior work you identified, emphasizing the fitted-residual distribution and conformal-quantile analysis beyond quadratic-feature concentration and regression-risk limits.
  • Made the Gaussian-input quadratic forecasting model class explicit and explained its relationship to the nonlinear forecasting applications and additional experiments.
  • Clarified the condition required for deterministic quadratic signals after ridge fitting and provided numerical diagnostics, distinguishing the cases covered by the current results from possible extensions.

If any questions or concerns remain, we would be more than happy to discuss them further before the end of the discussion period. Thank you again for your time and consideration!

Reviewer_qR69OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates Bayesian ridge and split conformal uncertainty quantification for Next Generation Reservoir Computing (NGRC). The authors theoretically characterize the differences in prediction interval widths under low- and high-dimensional settings and extend their analysis to dependent quadratic NGRC features. Synthetic and real-world experiments are conducted to examine the theoretical findings.

Critical strengths
  1. Systematic theoretical characterization of uncertainty quantification methods. The paper provides a systematic theoretical comparison of Bayesian ridge and split conformal prediction across different dimensional regimes. In particular, the high-dimensional analysis characterizes how regularization bias, estimation variance, and posterior uncertainty determine the relative widths of the prediction intervals. These results offer useful insights into when and why the two uncertainty quantification methods produce different intervals.

  2. Technically meaningful extension to dependent quadratic NGRC features. The extension to quadratic NGRC features in Theorem 3 is a notable strength. By establishing quadratic-form concentration and identifying conditions for Gaussian approximation, the authors extend the high-dimensional analysis beyond standard Gaussian feature designs. The supporting theoretical derivations and controlled experiments provide a meaningful characterization of uncertainty behavior under nonlinear feature dependence.

Critical weaknesses
  1. Limited applicability of the theoretical results to practical temporal dependence. Although the paper is motivated by uncertainty quantification for memory-driven dynamical systems, the main high-dimensional theoretical results rely on assumptions that do not fully capture the temporal dependence encountered in practical NGRC forecasting. In particular, Theorem 3 assumes independent feature vectors, while Proposition 1 extends the analysis to temporally dependent observations only under sufficiently separated forecasting windows. This condition does not directly cover the overlapping windows commonly used in time-series forecasting. Although the authors acknowledge this limitation and provide empirical diagnostics, the gap limits the extent to which the theoretical guarantees can explain uncertainty behavior in practical memory-driven forecasting.

  2. Limited empirical connection to the high-dimensional theoretical regime. A central contribution of the paper is the high-dimensional characterization of Bayesian and conformal prediction intervals under proportional asymptotics. However, the real-world experiments in Section 6 operate in a substantially lower-dimensional regime, with feature-to-sample ratios of only 0.024–0.040. Furthermore, the theoretical analysis relies on specific assumptions regarding feature normalization and regularization, whereas the empirical implementation uses data-dependent regularization and empirical feature normalization. Although the synthetic experiments support the theoretical predictions under controlled settings, the extent to which these findings explain uncertainty behavior in realistic NGRC applications remains insufficiently established. This gap weakens the practical significance of the main theoretical contributions.

Questions for authors
  1. Applicability to overlapping forecasting windows. While Experiment 3 includes temporal-dependence diagnostics using an AR(1) process, could the authors further clarify whether the theoretical width comparisons remain informative under the overlapping forecasting windows typically used in practical NGRC applications? In particular, to what extent do the predicted interval-width relationships persist for nonlinear memory-driven systems with stronger temporal dependence?

  2. Relevance of the high-dimensional results to real-world applications. Given that the real-world experiments operate in a low-dimensional regime and use data-dependent regularization, how should the high-dimensional phase transition and interval-width approximations be interpreted in these settings? Could the authors provide additional empirical evidence or diagnostics connecting the theoretical predictions to the feature dimensions and regularization choices encountered in practical NGRC forecasting?

3 comments

Dear reviewer, thank you again for your constructive feedback and for the time and effort you have devoted to reviewing our paper.

As the author discussion period is coming to an end, we would be very grateful if you could take a look at our rebuttal responses and let us know whether they have addressed your concerns. In particular, we have:

  • Added experiments using consecutive overlapping forecasting windows, including strongly dependent nonlinear systems and regularization selected from fitting data.
  • Provided larger-dimensional, full-record real-data evaluations, with feature-dimension and regularization diagnostics connecting the theoretical comparisons to practical NGRC forecasting.
  • Clarified the distinction between existing coverage guarantees under dependence and the empirical evidence supporting our interval-width comparisons under overlapping windows.

If any questions or concerns remain, we would be more than happy to discuss them further before the end of the discussion period. Thank you again for your time and consideration!

Thank you for the thorough and thoughtful responses. I am satisfied with the additional experiments and clarifications, which have addressed most of my concerns and strengthened the manuscript. I appreciate the authors' efforts and will raise my rating from 2 to 3 (Weak Accept).

Dear reviewer, thank you for your constructive feedback and positive reassessment of our paper. Your comments helped us strengthen the connection between the theoretical benchmark and practical forecasting. We will incorporate the additional overlapping-window experiments and related clarifications into the revised manuscript. Thank you again for your time and consideration!

Reviewer_BwAYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper compares Bayesian and conformal prediction intervals for the specific case in which the forecasting model is an NGRC. Both methods use the same ridge regression model for the central prediction, but they estimate uncertainty differently. The paper shows that, when the number of features is small, the difference mainly depends on the shape of the prediction errors. When the number of features grows with the sample size, the comparison also depends on signal strength, noise, regularization, and model bias. The authors support these results with theoretical analysis, simulations, and experiments on four real time series.

Critical strengths
  • The paper studies an important practical question: when should Bayesian and conformal intervals give similar or different results?
  • The high-dimensional analysis gives a clear explanation of why either method can produce wider intervals.
  • The paper carefully discusses several limitations of the theory.
  • The empirical results support the theoretical results.
  • The scope is narrow (UQ for the specific case of NGRC forecasting), but the authors clearly acknowledge it and develop a large theoretical analysis for this exact setting.
Critical weaknesses

Major

  1. The main theory uses strong assumptions, including Gaussian data, independent samples, fixed regularization, and a known or accurately estimated noise level. These assumptions are quite different from a real NGRC forecasting problem.
  2. The high-dimensional theory assumes that the noise level is known or consistently estimated. However, the practical Bayesian method estimates it from the training errors. The paper itself explains that this estimate may be inaccurate when the number of features is large.
  3. The real-data experiments use relatively few effective features compared with the amount of training data. Therefore, they do not directly test the most interesting high-dimensional result of the paper.

Minor

  1. The results for experiment 1 are not in the main body of the paper.
  2. The main experimental comparisons focus primarily on a nominal coverage level of 95%. Although additional calibration curves are provided in Appendix F.6, a more comprehensive quantitative evaluation across multiple coverage levels (e.g., 80%, 90%, 95%, and 99%) would strengthen the empirical analysis. In particular, reporting coverage, interval width, and Winkler score across these levels would provide a clearer picture of the calibration–efficiency trade-off and the robustness of the conclusions.
  3. ResCP involves random reservoir initialization and stochastic residual sampling, making its performance potentially sensitive to the random seed. Although the authors provide a five-seed sensitivity analysis in Appendix F.8, the main experimental results are reported using a single seed. Reporting the mean and standard deviation across multiple independent seeds in the main comparison tables would provide a more reliable assessment of performance and improve the robustness of the empirical conclusions.
Questions for authors
  1. Could the authors add an experiment using a real high-dimensional NGRC model on a dependent nonlinear system, with overlapping windows and regularization selected from the data? Does the predicted transition between Bayesian and conformal intervals still appear?
  2. What happens to the theoretical results when the noise level is estimated from the training errors, as in the practical Bayesian method?

Comments on the paper

2 comments

Thank you for the detailed responses and additional experiments. The added comparison clarifies the contribution of the fitted-residual distribution and conformal-quantile analysis beyond regression-risk results. The Gaussian-state quadratic-response model and the consecutive-window experiments also make the connection between the theoretical benchmark and forecasting stronger. The deterministic-readout extension remains conditional on Equation (5). The new diagnostics are useful, but they do not establish a sufficient class satisfying this condition after ridge fitting. Please distinguish clearly between the proved random-effects result, the conditional deterministic result, and the numerical evidence. Please also preserve the distinction between the known-noise phase transition and the training-residual plug-in result, which predicts narrower Bayesian intervals under the benchmark assumptions. The overlapping-window experiments should remain described as empirical support for the width comparisons. These responses sufficiently address my main concerns about the contribution and its interpretation. I am increasing my rating from 2 to 3.

Dear reviewer, thank you for your thoughtful follow-up and positive reassessment of our paper. We will incorporate the clarifications and additional results into the revised manuscript, preserving the distinctions you highlighted between proved results, conditional extensions, and numerical evidence. Thank you again for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.