Finite-width recurrent kernels: From shared-weight convergence to task-relevant prediction risk and dynamical stability
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Reservoir computing (RC) uses fixed random recurrent dynamics with a lightweight trained readout, but practical reservoirs operate at finite width and reuse the same random operator across time. We study how finite-width reservoirs with shared recurrent randomness approach the recurrent-kernel limit and when improved kernel fidelity translates into improved prediction performance. Shared weights generate structured off-diagonal two-time correlations that are strongly suppressed by independent temporal resampling, showing that equal-time agreement alone does not capture the full temporal kernel structure. Finite shared-weight reservoirs nevertheless converge systematically toward the analytic kernel as width increases. Bias–variance analysis further shows that orthogonal and SORF constructions reduce finite-width error primarily by lowering realization variance, while SORF offers a favorable accuracy–memory trade-off through its implicit structured transform. Downstream experiments show that global kernel discrepancy only partially explains prediction risk. A task-aligned discrepancy based on regularized readout-relevant directions yields stronger within-method correlations with excess risk. Stateful Lorenz-63 forecasting reveals an additional dynamical constraint: wider reservoirs can extend valid horizons, but the benefit depends strongly on the recurrent operating point. Overall, finite-width reservoir performance depends on temporal covariance, task relevance, and dynamical error amplification in addition to kernel fidelity.
- Primary area:
- learning on time series and dynamical systems