Finite-width recurrent kernels: From shared-weight convergence to task-relevant prediction risk and dynamical stability

Reservoir computing (RC) uses fixed random recurrent dynamics with a lightweight trained readout, but practical reservoirs operate at finite width and reuse the same random operator across time. We study how finite-width reservoirs with shared recurrent randomness approach the recurrent-kernel limit and when improved kernel fidelity translates into improved prediction performance. Shared weights generate structured off-diagonal two-time correlations that are strongly suppressed by independent temporal resampling, showing that equal-time agreement alone does not capture the full temporal kernel structure. Finite shared-weight reservoirs nevertheless converge systematically toward the analytic kernel as width increases. Bias–variance analysis further shows that orthogonal and SORF constructions reduce finite-width error primarily by lowering realization variance, while SORF offers a favorable accuracy–memory trade-off through its implicit structured transform. Downstream experiments show that global kernel discrepancy only partially explains prediction risk. A task-aligned discrepancy based on regularized readout-relevant directions yields stronger within-method correlations with excess risk. Stateful Lorenz-63 forecasting reveals an additional dynamical constraint: wider reservoirs can extend valid horizons, but the benefit depends strongly on the recurrent operating point. Overall, finite-width reservoir performance depends on temporal covariance, task relevance, and dynamical error amplification in addition to kernel fidelity.