acceptodds
Desk rejected at ICLR 2027

Finite-width recurrent kernels: From shared-weight convergence to task-relevant prediction risk and dynamical stability

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Reservoir computing (RC) uses fixed random recurrent dynamics with a lightweight trained readout, but practical reservoirs operate at finite width and reuse the same random operator across time. We study how finite-width reservoirs with shared recurrent randomness approach the recurrent-kernel limit and when improved kernel fidelity translates into improved prediction performance. Shared weights generate structured off-diagonal two-time correlations that are strongly suppressed by independent temporal resampling, showing that equal-time agreement alone does not capture the full temporal kernel structure. Finite shared-weight reservoirs nevertheless converge systematically toward the analytic kernel as width increases. Bias–variance analysis further shows that orthogonal and SORF constructions reduce finite-width error primarily by lowering realization variance, while SORF offers a favorable accuracy–memory trade-off through its implicit structured transform. Downstream experiments show that global kernel discrepancy only partially explains prediction risk. A task-aligned discrepancy based on regularized readout-relevant directions yields stronger within-method correlations with excess risk. Stateful Lorenz-63 forecasting reveals an additional dynamical constraint: wider reservoirs can extend valid horizons, but the benefit depends strongly on the recurrent operating point. Overall, finite-width reservoir performance depends on temporal covariance, task relevance, and dynamical error amplification in addition to kernel fidelity.

Primary area:
learning on time series and dynamical systems
Keywords:
reservoir computing, recurrent kernels, finite-width effects, structured random features, SORF, kernel alignment

Related papers

Discussion (0)

Sign in to comment.