Temporal Pooling Shapes Covariance Geometry and Readout Risk in Random Recurrent Networks

Temporal pooling determines the covariance seen by a downstream learner, yet this covariance is often reduced to total variance or participation dimension. We study a stationary linear random recurrent system with stability gap $\epsilon =1-g^2$. In the ordered limit where width tends to infinity first, $T\sqrt{\epsilon }\to \infty$ and $T\epsilon \to 0$ define an intermediate observation window. For a broad class of normalized pooling kernels, every fixed moment of the variance-carrying part of the spectrum converges, and its covariance-size-biased limit is $Y_h=(M/2)|\hat{h}(U_h){|}^2$. Here $M$ has the aspect-one Marchenko–Pastur law, while the independent filter-gain factor depends on the kernel. Pooling rules can therefore match trace and participation dimension while retaining different higher-order geometry. To test whether this distinction matters for a common prediction task, we study 56 independent real Ginibre networks at widths 128 and 256. For each network, a calibration based only on feature covariances matches the trace and second spectral moment of boxcar and causal-exponential pooling to numerical precision. Both pooled views then predict the same future recurrent state. At width 256 and $T={\epsilon }^{-3/4}$, causal-exponential pooling reduced population normalized mean-squared error by 5.86–7.14 percentage points across three stability gaps. Aligning the kernels' mean amplitude-weighted sample ages reduced these differences to 1.71–2.43 points but did not eliminate them. Validation-selected finite-sample ridge differences were 1.93–3.91 points in the same direction, although two width-128 fits reached the edge of the penalty grid, making that comparison descriptive. A separate Gaussian-design analysis shows why no kernel ranks universally: higher-order geometry raises ridge bias but can lower variance enough to reverse the ordering under minimum-norm regression. These finite experiments measure task behavior at fixed parameters; they do not validate the ordered asymptotic law.