acceptodds
Under review as a conference paper at ICLR 2027

Task-Driven Selection and Sizing of Simple Cycle Reservoirs

Abstract

Sparse deterministic reservoirs can reduce deployment cost relative to random echo-state networks (ESNs), but it is unclear when a simple cycle is sufficient for a new forecasting task. We separate this architecture-selection problem from the subsequent choice of cycle size. For any fixed reservoir, optimal squared prediction risk decomposes into irreducible noise and projection error onto the reservoir feature span. For a linear simple cycle represented over a declared finite history, delays congruent modulo the cycle size share state coordinates, yielding an exact covariance-weighted representation loss and a model-conditional minimum-cost certificate. A conservative training-only screen based on this analysis recommends cycles on 23 of 72 benchmark configurations, with no observed recommendation exceeding a validation-selected nonlinear random ESN by more than the prespecified 2% held-out NRMSE margin. The certified regime is narrow, however: a reservoir-free diagnostic developed on controlled tasks fails to transfer reliably to 83 configurations from five real datasets. Importantly, this failure is primarily one of predicting cycle suitability rather than of the architecture itself: a cheaper cycle remains within 2% of the random ESN on 50/83 configurations. Separately, once the cycle family is chosen, a near-best validation rule reduces deployed cycle dimension by 42.0% on 24 held-out configurations while satisfying the 2% practical-equivalence criterion throughout. These results distinguish what can be certified from what must still be validated directly.

Primary area:
learning on time series and dynamical systems
Keywords:
reservoir computing, echo state networks, architecture selection, selective prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_VdvVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is using the reservoirs to decide. The experimental results are good, namely screen is not failed in 23 picks, and the sizing cut cycles is by 42%.

Critical strengths

Decisions are fixed before testing. Experiments have evidence that an upper bound on the error rate has been reported instead of just claiming the screen is safe.

Critical weaknesses

The baselines can be improved. For example, tuning the ESN, and compare the direct lag ridge on cost. The certificate seems to be trivial as well.

Questions for authors
  1. Which L does the screen use?
  2. Which role does the certificate play for N ≥ L?
  3. How do cycles compare with a tuned ESN and with direct lag ridge?
1 comment

Thank you for the concise review. We ran additional experiments addressing all three questions.

  1. History length (L). The broad screen uses (L=48). We will move this value, together with the input dimension, gain, (W_{\rm in}) construction, preprocessing, and state-standardization details, into the main method description.

  2. Role of the certificate once (N\ge L). For a scalar finite-history problem, we agree that if (N\ge L) and the finite map has full column rank, the ideal finite-history representation loss is zero. Our numerical audit revealed that the submitted Gram-pseudoinverse implementation could nevertheless drop geometrically attenuated directions, making some gate-2 failures numerically conservative. Stable SVD/QR projection confirms near-zero loss in these scalar full-span cases. We will explicitly state this boundary and distinguish it from the nontrivial compressed regime (N<dL). In new multivariate experiments with (dL) as large as 768 and (N\le128), 44/45 population-certified cases remain within 2% of unrestricted lag ridge, showing that the projection analysis is nontrivial when the cycle genuinely compresses the input history.

  3. Tuned ESN and direct lag. With a stronger validation-tuned ESN that also varies spectral radius, input scale, and leak rate, 21/23 original frozen cycle recommendations remain within the 2% test-NRMSE margin; the mean online-multiplication reduction remains approximately 86.6%. Direct-lag ridge is essentially tied with the cycle in the original scalar certified regime (mean NRMSE approximately 0.29501 vs. 0.29512 for the cycle). We therefore revise the claim: the cycle should be viewed as a structured compressed history representation, not as universally superior to lag regression.

We will include these comparisons and the numerical-conditioning boundary in the revision. We hope these results address the baseline and certificate concerns and would appreciate reconsideration of the score.

Reviewer_AffwOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work asks when a sparse linear simple cycle reservoir is sufficient for forecasting tasks (rather than a full ESN), and separates this problem from the choice of a cycle size whenever the cycle is adopted.  On the theory side, it starts by decomposing the (fixed) reservoir risk into irreducible noise and projection error on its features and shows that, for linear cycles, delays congruent mod N share a states coordinates. The authors use this to provide an exact covariance-weighted representation loss which is used as a model-conditional minimum-cost criterium. The theoretical investigation is converted into a training-only architecture screen that, in three steps, either recommends a cycle or abstains. A separate rule determines the cycle size.

Critical strengths

The authors set the predictions, margins and bounds before testing, and the prospective and post-hoc analyses are clearly separated. The results looks honestly reported, including negative results as well. Theory and derivations looks correct.

Critical weaknesses
  1. Equivalence with a lagged regression Within the defined setting, the cycle is basically a lagged regression. With g=0.7, g^N \le 3.3\times10^{-3} for N\ge16, the "aliasing" is negligible. Therefore, the ring effectively just holds the last N inputs, and a linear readout on that is ordinary linear regression on lagged values. So, E_N = 0 for all N \ge L [538], making the certificate vacuous. A direct lag ridge baseline is mentioned [292] but never reported. I might have misunderstood this, but, I think that since the envelope resamples only H, Cor. 6’s probability guarantee does not apply to the implemented procedure.

  2. Weak baseline The tested ESN, with \rho=0.7, input scale 0.1, and no leak rate is basically operating close to linearity. Indeed, NARMA is a standard benchmark where nonlinear reservoirs should win clearly, while the performance is suspiciously on par with the linear model.  The base rates [342-343, 355-356, 365], should be taken into account when reporting the net advantage of the proposed algorithm.

  3. Scope and motivation for the screen are unclear While it is true that the screen saves some costs by avoiding fitting competing models, ridge-readout ESN fits are cheap. The paper’s own conclusion is to validate candidates directly, so the screen adds value only where its assumptions hold, which is where a linear lag model is the natural choice anyway. The motivation also conflates two costs: while deployment cost is cited as the reason to care [011, 039–040], the screen economizes selection cost, which is a separate axis. The screen recommends only 23 of the 60 configurations where a cycle was suitable (38% recall; [342–346]), and, even if every result is correct, it is unclear why anyone would use the screen instead of simply fitting both models and picking the better one. A better motivated framing would greatly improve this research question. Lastly, several important details are still unspecified, some major examples being:

values of g and L for the broad benchmark whether states are standardized how multivariate inputs are handled.

Questions for authors

Clarifications

  1. What are d, W_{in}, and the target dimension on the multivariate datasets? Are reservoir states standardized before ridge regression? What values of g and L are used in the broad benchmark? If d>1 with genuine compression of \mathbb{R}^{dL}\to\mathbb{R}^N, the lag-regression equivalence mentioned above would not apply, and that would substantially change my view of the theory’s relevance.
  2. Please report the direct-lag ridge baseline you already ran, at matched cost, on all three experiments. Do cycles beat direct-lag ridge at matched cost?
  3. How many abstentions does each gate produce in the 72-configs benchmark? Does gate 2 ever abstain when \max\mathcal{N}\ge L?
  4. Is the nonlinear guard a non-inferiority test (G.1) or a failure-to-reject test (§6.1)?

Suggestions

Tune the ESN baseline over \rho, input scaling, and leak rate. It would be interesting to extend the analysis to d>1 and to fixed linear bases such as HiPPO.

Satisfactory answers to Q1 and Q2 could move my rating.

2 comments

Thank you for the technically detailed review. Several of your concerns identified genuine boundary cases in our original presentation, and the additional experiments have helped us sharpen the scope substantially.

  1. Relation to lagged regression and the (N\ge L) case. We agree that, for a scalar declared finite history of length (L), if (N\ge L) and the finite cycle map has full column rank, the ideal finite-history representation has zero excess representation loss. We now state this explicitly. Our numerical audit also revealed that the submitted Gram-pseudoinverse implementation can discard geometrically attenuated directions and therefore report positive loss even in such full-rank cases. Stable SVD/QR projection gives essentially zero loss for all scalar (N\ge L) cases in the audit. We therefore distinguish the exact population result from numerical conditioning in the implementation, and we will no longer interpret all high-(N) gate-2 failures as genuine representational insufficiency.

This does not make the projection analysis vacuous in the compressed regime. We ran additional controlled multivariate experiments with (L=48), (d\in{2,4,8,16}), and raw history dimension (dL) up to 768 while (N\le128). Here the cycle genuinely maps (\mathbb R^{dL}) into a much smaller recurrent representation. Among 45 population-certified cases, 44 remain within 2% of unrestricted direct-lag ridge and 44 remain within 2% of the strong ESN. We will use this to clarify that the theory is most meaningful when (N<dL), not in the trivial full-span scalar case.

  1. Direct-lag ridge. We added the requested baseline. On the 23 original recommendations, mean test NRMSE is approximately 0.29512 for the cycle and 0.29501 for unrestricted direct-lag ridge, i.e. they are essentially tied. We therefore agree that, in the scalar screened regime, the linear cycle behaves very similarly to a structured lag representation. We will not claim otherwise. We also added matched-feature and matched-online-multiply lag baselines; these remain competitive, so our revised claim is that the cycle is a structured recurrent compression of history, not that it universally dominates direct lag regression.

  2. Stronger ESN baseline. We tuned spectral radius, input scale, leak rate, size, seed, and ridge using validation data only. Under this stronger comparator, 21/23 original cycle recommendations remain within the 2% margin, with approximately 86.6% lower online-multiplication cost on average. Two recommendations fail the margin, and we will report these explicitly.

  3. Gate counts and role of the nonlinear guard. The 72 configurations split as follows: 27 fail linear adequacy; 17 additional cases fail the certificate; 28 pass the certificate; 5 of those are rejected by the nonlinear guard; 23 are recommended. Several certificate-pass cases on Lorenz/Rössler show a large nonlinear-ESN advantage, which demonstrates that linear representational sufficiency is not sufficient for architecture selection.

  4. Finite-sample envelope. We agree that the implemented uncertainty calculation does not establish an unconditional finite-sample (1-\alpha) guarantee: it resamples coefficient uncertainty while treating the covariance and oracle-risk estimates as plug-in quantities. The abstract corollary remains valid conditional on (U_N) actually being a simultaneous upper envelope, but we will revise the implementation language to “approximate/conditional plug-in upper envelope” rather than claiming unconditional finite-sample coverage.

  5. Practical use case and cost. We separately measured selection and deployment cost. The E22 procedure averages approximately 0.714 s per task, compared with approximately 11.85 s per task for the broad naive cycle+strong-ESN architecture search used in the rebuttal experiment, i.e. roughly 16.6× lower selection time. We will also separate this from deployment cost in the revised paper.

Finally, we will clarify the main-text values of (L), (d), (W_{\rm in}), gain, standardization, and the direction of the nonlinear test. The guard is implemented as a one-sided practical non-inferiority-style check using the paired loss (e_{\rm cyc}^2-(1+\delta)^2e_{\rm ESN}^2), requiring the upper confidence bound to be nonpositive.

You noted that satisfactory answers to the direct-lag and multivariate-compression questions could change your assessment. We hope the new results and the corrected scope address those concerns and would appreciate reconsideration of the score.

Dear authors, I will consider the rebuttal, but given the time constraints (the rebuttal was submitted on the last day of the author-reviewer discussion period), I will make my updates during AC-reviewer discussion period.

Reviewer_4PidOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The work presents a new model selection technique for choosing the size of simple cycle reservoir, a reservoir computing model whose recurrence matrix is costrained to be a cyclic permutation, and concurrently an algorithm to decide whether to use the SCR at all, in place of a more complex tanh-ESN. The algorithm is based on three steps

  1. fitting a linear model, that should predict the data adequately.
  2. trying different reservoir sizes, and choosing the smallest one that performs not worse than the linear model, plus a theoretically derived threshold.
  3. Compare it with a classic tanh-ESN

The method is tested on both synthetic and real-world datasets.

Critical strengths

Broad motivation: Model selection in RC is a burden. Any attempt at creating new ways to reduce it, be it with theoretical results on the expected performance or algorithms, are welcome.

Critical weaknesses

I have many issues with this paper. Since only two are allowed at most, I am stating the two I believe are the most critical

Clarity: The AI usage paragraph states that AI assisted with polishing the writing of this paper in terms of flow and English grammar. If so, that was done poorly, as some sentences of the work result very difficult to be interpreted, even after careful reading. Since I'm familiar with the topic, I believe this should not be the case. The method itself is not explained clearly. Perhaps the most important quantity for understanding the method, UNU_N, is only defined informally and in natural language as "a simultaneous upper confidence envelope estimated from training data". I understand it as the simultaneous confidence bound satisfying P(max⁡RN<UN)>1−aP(\max R_N<U_N)>1-a. How to estimate the bound is not completely clear from the main text. In the experimental section, most of the results are discussed briefly, and the discussion on the main text often refers to figures in the appendix; conversely, the only figure in the main text seems not to be discussed properly. As a result understanding the empirical benefit of the method is also not easy. More references should be included in various points, as the current ones are not sufficient, when referring to previous methods, datasets. A broader related work discussion would help, especially on model selection for reservoir computing and related forecasting methods.

In general, I value clarity as the most important aspect when reading and evaluating a paper, therefore my assessment of the work is severely and negatively impacted. The work needs to go through a substantial, hopefully humanly-assisted, rewrite.

Impact: It saddens me to say it as a practitioner of reservoir computing, however I think the work fundamentally lacks the impact to be published in a top-tier generalist deep learning venue. It tackles a very narrow problem on a very specific architecture (SCR) of reservoir computing. I believe that a submission to a more neural nets or computational neuroscience-focused venue (after the aforementioned rewriting) would be better aimed.

Questions for authors

No question could significantly affect my assessment. However, I believe that clarifying some critical points could make the work in a better shape for further submissions.

## Clarification requests

  • Can the authors clarify the practical use-case of the selection algorithm? Since the majority of the states sizes NN must, to my understanding, be swept anyway to compute the risk and the envelope, it seems to me that just sweeping the sizes would effectively be the same thing. Can you provide timings on how much the model selection performed saves w.r.t. a grid search on the best NN that simply looks at the validation metrics? In other words, I think the way that right now is used, to reasonable success, to determine the two questions 1. "what size to use" and 2. "what architecture to use" is

    1. Sweep the reservoir size for SCR and ESN, and take the best.
    2. Compare with an ESN.

    Does this method achieve better perfomance, or better timings than the naive baseline for model selection?

  • Is the reservoir considered purely linear, as suggested from equations (3), (4)? While the original (Rodan and Tino 2011) consider both linear and tanh reservoirs, in practice in their experiments they use mostly the tanh version, that is stated to usually perform better. Furthermore, (Li et al. 2024) prove that (linear) SCRs and ESNs are universal in the same family, provided that the readout of the SCR is expressive enough (e.g. MLP or polynomial), while linear reservoir with a linear readout is less expressive in general. In the specific case, the model seems very close to an autoregressive moving average. For these models, a number of well established techniques for choosing the correct model order (which is equivalent to the reservoir size NN) are available classicaly (Salau 2003, Stoica 2004). This intuition of the equivalence is consolidated by section 5.2, that explicitly assumes linear dependence + noise betweeen input and output observations to derive the theoretical results. Can the authors clarify similarities and differences with those other model selection methods?

### references Li, Boyu, Robert Simon Fong, and Peter Tiňo. 2024. ‘Simple Cycle Reservoirs Are Universal’. arXiv:2308.10793. Preprint, arXiv, June 4. https://doi.org/10.48550/arXiv.2308.10793.

Salau, M. O. 2003. ‘The Effects of Different Choices of Order for Autoregressive Approximation on the Gaussian Likelihood Estimates for ARMA Models’. Statistical Papers 44 (1): 89–105. https://doi.org/10.1007/s00362-002-0135-6.

Rodan, A., and P. Tino. 2011. ‘Minimum Complexity Echo State Network’. IEEE Transactions on Neural Networks 22 (1): 131–44. https://doi.org/10.1109/TNN.2010.2089641.

Stoica, P., and Y. Selen. 2004. ‘Model-Order Selection: A Review of Information Criterion Rules’. IEEE Signal Processing Magazine 21 (4): 36–47. https://doi.org/10.1109/MSP.2004.1311138.

2 comments

Thank you for the detailed comments. We agree that the main presentation did not make the practical role of the method, the construction of (U_N), or the relation to classical lag/model-order selection clear enough. We have substantially revised the presentation and run additional experiments aimed directly at these points.

Practical use case and comparison with naive model selection. The intended use is selective architecture screening before a broad reservoir search: the method either recommends a cheap cycle under a restricted model-conditional regime or abstains. We measured this directly. Across the 72-task benchmark, the screening procedure requires approximately 0.714 s/task, whereas the broad naive search over cycle and strongly tuned ESN candidates requires approximately 11.85 s/task, about a 16.6× difference in selection time. We will make clear that this is a selection-cost result; deployment cost is a separate axis.

Clarity of (U_N) and the algorithm. We agree that the submitted description was too informal. The revised method section explicitly defines (R_N), the plug-in upper envelope (U_N), the data available to each gate, and the nonlinear guard. Importantly, the implemented (U_N) is an approximate/conditional plug-in envelope: coefficient uncertainty is propagated while the covariance and oracle-risk estimates are treated as fixed. We therefore remove any implication of an unconditional finite-sample coverage guarantee.

Relation to lag regression and classical model-order selection. We agree that this connection should have been discussed explicitly. In the scalar finite-history setting, once (N\ge L) and the finite map is full rank, the cycle can span the declared lag history and behaves similarly to a lagged linear predictor. Our new direct-lag baseline confirms this empirically: on the 23 original recommendations, mean test NRMSE is approximately 0.29512 for the cycle and 0.29501 for direct-lag ridge. We will add the classical AR/ARMA model-order-selection literature, including the references you suggested, and clarify that our contribution is not an alternative information criterion for scalar AR order.

The distinction becomes meaningful under genuine compression. We therefore added controlled multivariate experiments with history dimension (dL) up to 768 and cycle dimension at most 128. In that regime the cycle maps a high-dimensional lag vector into a much smaller recurrent representation. Among 45 population-certified cases, 44 stay within 2% of unrestricted direct-lag ridge and 44 within 2% of a strong ESN. We will use this result to motivate the projection analysis as structured recurrent compression rather than ordinary scalar lag-order selection.

Baseline strength. We also repeated the comparison with a stronger validation-tuned ESN over spectral radius, input scale, leak rate, size, seed, and ridge. Of the 23 original frozen recommendations, 21 remain within the prespecified 2% test-NRMSE margin, with approximately 86.6% lower online-multiplication cost on average. We will report the two failures rather than retain the original 0/23 result without qualification.

Finally, we are rewriting the main method section to state (L), (d), gain, (W_{\rm in}), standardization, gate flow, and all selection data explicitly, and to move the key empirical comparisons into the main text rather than relying on appendix references.

We appreciate the criticism on clarity and the connection to classical model-order selection; both led to concrete changes that substantially sharpen the scope of the paper.

Thank you for the detailed rebuttal.

I appreciate the effort spent in trying to address my concerns. Some computational gain in time spent is now given, which I believe could strenghten the work's positioning. However, a search over two architectures and the hyperparameters of the two should take substantially more than 11 seconds, unless a high number of jobs is run in parallel. Without hardware and more detailed search specification, the baseline and the corresponding gain are hard to interpret. The manuscript faced a lot of changes. Some of those, such as better defined variabels are welcome; still, the readability did not improve significantly. I think that it is not proper mentioning that results are a product of the rebuttal or "reviewer requests", since the revisions are presumed to go in the camera-ready version in case of acceptance. Finally, I believe it is not fair to change that much of the main part between revisions in a short time period rebuttal as this. Overall the style and readability do not still meet the bar.

For the reasons highlighted above, I am maintaining my score.

Reviewer_eAT2OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when a simple cycle reservoir is sufficient for a forecasting task and, once the cycle architecture is chosen, how small the reservoir can be. The main technical idea is that delays congruent modulo the cycle size are aliased into the same state coordinates, which motivates a covariance-weighted, task-aligned representation loss and a model-conditional certificate. The paper further proposes a conservative screening procedure and a near-best validation rule for cycle sizing.

Critical strengths
  1. Clear and memorable technical insight.The modulo-NN delay-aliasing view gives a concrete interpretation of what cycle size controls, and the task-aligned representation loss turns this mechanism into a principled selection criterion.

  2. Well-calibrated claims. The paper clearly distinguishes model-conditional guarantees from empirical screening and does not overstate abstention or zero observed violations as universal guarantees.

  3. Clean evaluation protocol. Architecture and sizing decisions are frozen before held-out evaluation, while post-hoc analyses are explicitly labeled as exploratory, reducing concerns about test leakage.

  4. Convincing sizing results. The near-best validation rule substantially reduces deployed cycle size while remaining within the prespecified 2% test-NRMSE margin on all 24 held-out configurations.

Critical weaknesses
  1. The contribution of the analytic certificate is not isolated.Since the final screen also contains a direct nonlinear-ESN guard, it is unclear whether the certificate itself materially improves screening over a simpler guard-only procedure.

  2. The ESN comparator may be under-tuned.The audit searches size, seed, and ridge, but fixes important dynamical hyperparameters such as spectral rescaling and input scale, which could affect the reported safety margin.

  3. The certified regime is fairly narrow. The screen abstains on most benchmark configurations and makes no recommendations on the tested chaotic systems, limiting the practical scope of the theory-derived selection rule.

Questions for authors
  1. What happens if the analytic certificate is removed? Could you report the coverage and unsafe rate of a baseline using the same adequacy check and nonlinear ESN guard but without the certificate?

  2. How robust are the 23 recommendations to stronger ESN tuning, particularly variations in spectral radius/rescaling and input scale?

  3. How many configurations are rejected at each stage of the screening pipeline? In particular, how many pass the analytic certificate but are subsequently rejected by the nonlinear guard?

1 comment

Thank you for the careful review and for highlighting the modulo-(N) interpretation, the separation between architecture selection and sizing, and the held-out sizing protocol. We ran additional experiments addressing all three questions.

  1. Contribution of the analytic certificate. We ablated the three-stage screen on the frozen 72-configuration benchmark. The full rule (adequacy + certificate + nonlinear guard) recommends 23/72 configurations with 0 unsafe recommendations against the original prespecified comparator. Removing the certificate and using the same adequacy check plus a fixed-(N=64) nonlinear guard increases coverage to 30/72 but produces 1 unsafe recommendation. Conversely, adequacy + certificate without the nonlinear guard recommends 28/72 and produces 3 unsafe recommendations. A guard-only fixed-(N=64) baseline recommends 52/72 with 1 unsafe recommendation. Thus, the certificate is not uniquely responsible for safety; rather, it supplies a size-specific, model-conditional compression criterion, while the nonlinear guard is needed to reject cases in which linear representational sufficiency does not imply nonlinear architectural sufficiency. We will revise the paper to state this more precisely.

  2. Stronger ESN tuning. We repeated the audit with a substantially stronger validation-tuned ESN that additionally varies spectral radius, input scale, and leak rate, as well as size, seed, and ridge. Of the original 23 frozen cycle recommendations, 21 remain within the prespecified 2% test-NRMSE margin. The two exceptions are Electricity (h=6) and ETTm1 (h=6). Across the 23 recommendations, the mean relative test-NRMSE difference remains essentially zero, while the cycle uses approximately 86.6% fewer online multiplications on average. We will report this stress test and no longer present 0/23 as robust to arbitrary ESN tuning.

  3. Gate counts. Of 72 configurations, 27 fail the linear-adequacy gate. Of the remaining 45, 17 fail the analytic certificate, leaving 28 certificate passers. Five of those are rejected by the nonlinear guard, leaving 23 final recommendations. The five certificate-pass/guard-fail cases are Lorenz (h=5), Rössler (h=10), Rössler (h=25), Electricity (h=1), and Weather (h=6). In particular, the chaotic examples show why the nonlinear guard is not redundant.

We have also revised the manuscript to make these roles explicit and to separate the exact model-conditional theory from the empirical screening procedure. We hope these additional ablations and stronger baseline address the concerns and would appreciate reconsideration of the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.