What Does a Fitted Model Represent? Graded Orders and Sampling Cost in Echo-State Model Spaces

Representing a time series by the parameters of a model fitted to it is a common approach, from model-space learning to weight-space learning, yet little is known about what such a representation retains. We analyse echo-state networks, in which a ridge readout of a fixed random reservoir is fitted to forecast each sequence and serves as its representation. The fitted readout is a statistic of the law of the process. Its linear part depends only on the autocovariance, and higher-order structure enters order by order: for scalar input, once lower-order moments agree, a difference of order $2k+2$ enters at order ${\lambda }^{2k+1}$ in the input scale $\lambda$, generically when it is a joint cumulant of the forecasting target and the current input. Detecting it from one trajectory of length $L$ is governed by ${\lambda }^{4k}L$, which we establish for bounded moving averages at a fixed ridge with a shared non-degenerate linear noise as a local Gaussian limit, with a lower bound on the error of every classifier of bounded complexity that population LDA attains. The same analysis reveals blind spots created by the forecasting target and by the symmetry of the reservoir, and shows how to remove them. Together, these results explain when a fitted-model representation distinguishes time series from different processes, and what to change when it does not.