- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how recurrent neural networks represent and perform sequential probabilistic inference. The authors train GRUs to carry out sequential Gaussian process (GP) regression, where observations are presented one at a time and the network predicts both the mean and uncertainty at each query location. By comparing network behavior and hidden-state representations with an analytically specified Bayesian filter, the paper argues that the trained networks approximate GP inference through reduced-rank recursive Bayesian regression over the leading eigenfunctions of the GP kernel. The paper further reports that the posterior mean is approximately linearly represented in the hidden state, that the corresponding eigenmodes emerge sequentially during training, and that uncertainty is better explained by covariance-like rather than precision-like parameterizations. Finally, hidden-state interventions are used to test whether the identified representations causally influence the network’s predictions.
Critical strengths
-
Sequential GP regression provides a controlled setting in which the target Bayesian computation is known exactly, making it particularly suitable for studying whether recurrent networks learn probabilistic representations.
-
The paper does not rely on a single diagnostic. It estimates the network’s effective rank using predictive behavior, the spectrum of its effective kernel, and the dimensionality of hidden representations. The agreement among these measurements in the main GRU setting is encouraging.
-
The hidden-state perturbations go beyond standard linear probing. Editing the inferred posterior-mean representation produces the expected change in predictive means, while the uncertainty interventions affect predictive variance after controlling for changes in the mean. These experiments provide meaningful evidence that the identified representations are functionally relevant.
-
Interpolating between covariance- and precision-based representations using the power-Euclidean family is an interesting methodological contribution for studying uncertainty representations in artificial and potentially biological neural systems.
Critical weaknesses
- The experimental setting is too narrow to establish the generality of the conclusions.
The main experiments use a one-dimensional input on a fixed interval, a stationary RBF kernel with mostly fixed hyperparameters, Gaussian observation noise, and a relatively small set of recurrent architectures. It is therefore unclear whether the conclusions extend to multidimensional inputs, other kernels, nonuniform input distributions, non-Gaussian or heteroscedastic noise, or more complex sequential-inference problems. The neuroscience implications should consequently be stated more cautiously.
- The interpretation that the GRU “implements” reduced-rank recursive Bayesian regression is stronger than the current evidence supports.
The experiments establish that the network’s behavior and hidden representations are consistent with a reduced-rank Bayesian solution. However, they do not directly show that the GRU performs the specific recursive updates derived in the paper. In particular, the work does not identify neural variables corresponding to the innovation or Kalman gain, nor does it demonstrate that one-step hidden-state transitions match the analytical Bayesian update. Alternative learned algorithms could produce similar outputs and linearly decodable belief states.
- The effective rank of approximately seven is insufficiently characterized.
The agreement among rank estimates in the main setting is interesting, but there is no reason for seven to be a universal or structurally special value. The learned rank should depend on the GP eigenspectrum, kernel lengthscale, noise level, context length, hidden-state capacity, and training budget. Although the appendix varies some of these factors, it primarily reports predictive loss rather than how the learned rank changes. Controlled experiments with known finite-rank kernels or systematically varied spectra are needed to show that the network adaptively selects the relevant rank rather than reaching an architecture- or optimization-dependent limit.
- The precise form of the uncertainty representation is not convincingly identified.
The encoding scores for , , , and are nearly identical. Thus, the shallow optimum at should not be interpreted as evidence that the network specifically encodes . The stronger conclusion supported by the results is only that covariance/expectation-side parameterizations are more linearly aligned with the hidden state than precision/natural-side parameterizations. Moreover, direct covariance interventions also alter the predictive mean and require an additional mean-clamping operation, indicating that the two representations are not cleanly separated.
Questions for authors
-
Can the authors directly test whether the GRU’s one-step state transition implements the analytical recursive Bayesian update—for example, by identifying representations of the prediction error/innovation and Kalman gain, or by testing whether an edited belief state evolves according to the correct update on subsequent observations?
-
How does the learned effective rank vary with kernel lengthscale, observation noise, context length, hidden size, and training duration? In particular, would the network recover different ranks on synthetic GP tasks whose true kernel ranks are explicitly set to, for example, 3, 7, and 12?
-
Do the principal conclusions hold for multidimensional inputs, nonuniform or out-of-distribution query distributions, different kernel families, and heteroscedastic or non-Gaussian observation noise?
2 comments
(2) The interpretation that the GRU “implements” reduced-rank recursive Bayesian regression is stronger than the current evidence supports.
We agree with the reviewer's assessment that our work does not explicitly identify e.g. where the Kalman gain is implemented in trained networks, and have accordingly softened our language somewhat in light of this concern (e.g., we have refrained from using the word "implements"). However, we do believe our analysis demonstrates that the networks learn a reduced-rank Bayesian regression solution insofar as the Bayesian sufficient statistics can be recovered from linear encoding and can be used to linearly steer the network's internal representation. By definition, if the Bayesian sufficient statistics can be linearly read-out at each timestep (which they can be), the decoded trajectory itself satisfies the Kalman recursion approximately. While we do not direct causal manipulations at these identified Kalman gain representations (and concede that as an alternate methodology), we believe our correlational and causal results on the rank- Bayesian weight-space sufficient statistics adequately establish that the network approximate reduced-rank Bayesian regression.
We would greatly appreciate the reviewer's thoughts on these arguments and look forward to their feedback.
We greatly appreciate the reviewer's thoughtful feedback on our manuscript and took great care to incorporate many of their criticisms. Our response is separated into two comments.
We thank the reviewer for this feedback and have accordingly added new analyses to the manuscript. Specifically, Figure 5 (new) now shows key analyses on a broader range of recurrent architectures. This analysis strengthens our results because it suggests a natural dichotomy in the belief state parameterisation: We find that nonlinear (gated) RNNs learn expectation parameter representations (since expectation parameter update dynamics are inherently nonlinear), while modern structure state space models (SSMs, which have linear dynamics) learn natural parameter representations. As suggested by the reviewer, we also include Figure 8 (new) in the appendix, which varies RBF kernel hyperparameters, Gaussian observation noise, hidden state capacity, and the true (truncated) rank of the prior kernel used during training and demonstrates approximate robustness in our three rank estimation procedures. Crucially, when training samples are generated from rank-M truncated GPs (rather than the full GP), the three rank estimates seem to recover this rank decently well. We believe the architectural generalisation in Figure 5 and the hyperparameter generalisation in Figure 8 address the reviewer's primary concerns and help to strengthen our underlying arguments. The reviewer also recommended multidimensional inputs, other kernels, and non-Gaussian observation models, which we believe are valuable extensions of this work that we intend to investigate in future work; however, we were not able to include those extensions in this work.
We direct the reviewer to Section 4.3 and Figure 5 for our extended results across architectures. While we concede that the exact value α⋆ should not be overinterpreted given the flatness of the maximum (as now explicitly articulated in footnote 3), the fact that gated nonlinear networks and SSMs reveal optima at different ends of the power α spectrum strengthens our claim that the particular choice of parameterisation is important in identifying the format of the network's internal representation. Crucially, we now not only report encoding R2 as a function of power α (and a new power β which controls how much the mean component of the belief state is transformed by the covariance, i.e. Σβμ) but also report the steerability R2 as a function of those parameters (showing how effective linear steering is at each α,β point). Thus, our covariance analysis is not simply correlational---it directly impacts how effectively one can causally manipulate the network's internal representation.