- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper provides a theory of noisy linear RNN learning dynamics under a simplified setting and show that noise can decisively shape representation learning. The key result is that in the regime where the weights are initialized with small values, noise strongly modulates the directions in which the read-in, readout, and recurrent weights align. Moreover, noise can produce a "phase transition": for sufficiently high noise, only a trivial solution is learned.
Critical strengths
- Overall the paper is well written and the mathematical derivations seem to be correct.
- The findings and detailed analysis on how noise can shape representation learning and lead to the phase transition behavior are interesting.
Critical weaknesses
- The setting studied is relatively simple and the assumptions are overly restrictive. For instance, the target filter is assumed to be exponential, the RNN update equation is assumed to be linear in the state, input and noise, all of which are not realistic in practical settings.
- The paper shows that noise can decisively shape representation learning in the simple linear RNN setting but does not provide empirical results to test whether similar claims also hold for nonlinear RNNs and the more realistic settings.
- It is not clear what are the practical gains from the studies on the simple RNN setting. There are no detailed analysis of the implications of the findings on downstream tasks such as next-value prediction and classification for time series data. The paper also seems to be better suited for a physics journal than ICLR.
Questions for authors
- Would similar findings/claims still hold for linear RNNs with different structures, e.g. the linear antisymmetric RNNs [1] plus noise?
- What are the implications of the findings on downstream tasks such as next-value prediction for real-world time series data and chaotic dynamical system data?
- In addition to shaping the representation learned, how would the noise and its magnitude affect trainability? How are the impacts of noise on trainability related to those on representation learning?
[1] Moe, S., Remonato, F., Grøtli, E. I., & Gravdahl, J. T. (2020, July). Linear antisymmetric recurrent neural networks. In Learning for Dynamics and Control (pp. 170-178). PMLR.
5 comments
Rebuttal (part 2)
(continued)
Numerical validation
The paper shows that noise can decisively shape representation learning in the simple linear RNN setting but does not provide empirical results to test whether similar claims also hold for nonlinear RNNs and the more realistic settings.
There are no detailed analysis of the implications of the findings on downstream tasks such as next-value prediction and classification for time series data.
What are the implications of the findings on downstream tasks such as next-value prediction for real-world time series data and chaotic dynamical system data?
We think that it is likely that certain qualitative insights survive in the nonlinear setting---in particular, networks should preferentially compute with non-noisy state space directions, and too much noise pushes networks into a 'trivial' regime---and are currently checking these claims in very simple experiments. On the other hand, we do not expect that our exact expressions for post-alignment dynamics hold anymore, since these are more closely tied to our specific setting. We will add brief comments on the nonlinear setting to the revised version.
Also, we have now made a serious effort to compare our theory with numerical BPTT experiments. We are happy to report that we have now conducted extensive experiments with moderate-sized linear RNNs (). Our high-level conclusion is that our theory is correct within its domain of validity. See the very end of the updated paper PDF for some illustrative (but still not 100% polished) figures.
-
1. BPTT agrees with the gradient descent on the analytic objective in the large-batch, small-time-step limit. One reason linear RNNs are nice is that we can avoid doing BPTT on the original objective (Eq. 6) and instead perform gradient descent on a more explicit equivalent objective that doesn't require BPTT (Eq. 7), i.e., which doesn't require differentiating through time steps. While this is mathematically true, is it numerically true? Yes, at least when one expects: when the batch size per SGD step is taken to be very large ( is generally high enough) and the time discretization is small enough. Here, we used and . These values should be understood relative to the 'overall' time scale set by the target leaky integration rate, which we set to in these experiments.
-
2. Our theory works given small weight initializations. We compared gradient descent on the analytic objective (Eq. 7) with our theory's predictions. (We can also do this with BPTT and the Eq. 6 objective, but training networks this way is much slower.) We find that SGD empirically matches our predictions provided parameter initializations are small. In particular, there is an alignment phase followed by effectively low-dimensional dynamics.
-
3. Phase transitions occur where the theory predicts. The previous two points illustrate that our theory works when its assumptions hold. As expected, there are also phase transitions where we predict.
In the process of doing this validation, we also learned other interesting things:
- Training with SGD and a fixed learning rate is generally extremely unstable (or takes a prohibitively long time). To make training stable, we had to use an adaptive learning rate; without this trick, we couldn't get BPTT-based training to work at all. In this specific setting, we know a lot about the mathematical structure of the objective and its gradients, so we could use our math to periodically approximate the Hessian and use it to decide whether the learning rate should be made larger or smaller. To be conservative, we set the learning rate to be proportional to 1/(largest eigenvalue).
- Using too-large initial weights makes our theory still qualitatively work (e.g., there is an alignment phase), but the precise dynamics are a little different from what the theory predicts, probably because some interactions with the non-aligned parameters can survive and affect dynamics.
- Near a phase transition (i.e., noise too high), training really slows down and requires many more epochs for convergence. This is empirically annoying, but it's consistent with our theoretical insight that the objective gets much flatter (see, e.g., Fig. 5c).
- Something interesting happens if BPTT with a small-to-moderate (less than , say) batch size is used for training. Noise associated with using small batches can cause updates to that are not aligned with the dominant rank-one component. This can actually cause the alignment between and (and ) to steadily go down after initially becoming high during the alignment phase. The effect completely goes away if either the analytically simplified (Eq. 7), non-BPTT-requiring objective is used, or if there's a small amount of regularization on .
Rebuttal (part 3)
(continued)
On noise and trainability
In addition to shaping the representation learned, how would the noise and its magnitude affect trainability? How are the impacts of noise on trainability related to those on representation learning?
This is something we address in the original paper, although we will think about how to communicate it more clearly in the revised version. In Sec. 4 and Appendix C of the paper, we characterize the initial 'alignment phase' of dynamics. During this phase, , , and align in the direction of the least-noisy eigenvector of , the matrix that controls the RNN's intrinsic noise.
One result we derive indicates how the convergence time during this phase depends on the noise matrix . Eq. 14 shows that, as the amount of noise gets larger, training really slows down. One way of stating this fact is that these linear RNNs get harder to train, since they require many more epochs for convergence.
For example, during our numerical experiments reproducing the phase transition discussed in Sec. 6 and Fig. 5 (discussed in part 2 above), we found that
- high noise levels close to the phase transition required a large number of training epochs to converge (~ 30k);
- lower noise levels, which are farther from the phase transition, required a much smaller number of training epochs to converge (~ 3k).
Hence, we find that noise has consequences for both representation learning (parameters align in the least noisy directions) and trainability (training takes much longer as the noise level gets higher). Importantly, this is true even in the limit of infinite-sized batches; these conclusions apply to the noise-averaged, simplified objective (Eq. 7).
Linear asymmetric RNNs
Would similar findings/claims still hold for linear RNNs with different structures, e.g. the linear antisymmetric RNNs [1] plus noise?
This is a great question. You've inspired us to think about two possible extensions: to symmetric matrices (), and to antisymmetric matrices ().
Both constraints yield slightly different gradient descent updates for (see Eq. 8 from Sec. 2). If we use to denote the unconstrained right-hand side of , in the symmetric case this becomes and in the antisymmetric case it becomes \frac{d W_{i j}}{d\tau} = F_{i j} - F_{j i} \ In the symmetric case, the alignment-phase ODE for is nearly the same as its unconstrained counterpart. The answer turns out almost the same, which is unsurprising since of the form (where denotes the alignment direction) is already symmetric. The subsequent dynamics are the same as before. Hence, choosing to be symmetric doesn't change much.
Meanwhile, using an antisymmetric yields a very different answer, since is not antisymmetric. In fact, the lowest-order contribution to alignment-phase dynamics actually vanishes here because of the antisymmetry constraint, and we need to approximate the right-hand side of using more terms. We end up getting both a different alignment phase (whose result is that has rank rather than rank one) and different subsequent dynamics. On the other hand, this shouldn't be that surprising, since antisymmetric matrices can't have rank one.
A more thorough examination of this phenomenon would probably be out of scope. An antisymmetric linear network can't even solve our task, since the eigenvalues of can only be imaginary (and the task requires developing a specific real eigenvalue). In any case, we will comment on these generalizations in the revised Discussion (and possibly a new appendix if we run out of space). We will also cite the interesting paper you mentioned.
Rebuttal (part 1)
Thank you for your close reading of our paper and for your helpful comments. We hope to convincingly address your concerns below, but please let us know if we fall short of doing so!
On the simplicity of the setting
This is absolutely a good point, and it is something we try to explicitly highlight in the last paragraph of the Discussion. We do indeed make a large number of simplifying assumptions. In our view, this is unfortunate, but also necessary given what we are trying to accomplish. Our goal is not to construct a state-of-the-art model or to qualitatively analyze RNNs through experiments. Rather, we aim to construct a specific kind of mathematically detailed explanation for learning dynamics: we aim to characterize, in a mathematically rigorous way, full learning dynamics trajectories and key qualitative features of them (e.g., whether there is a trivial or nontrivial solution given the noise level).
We are heavily inspired by Bordelon et al. (https://arxiv.org/abs/2503.18754) and Proca et al. (https://proceedings.mlr.press/v267/proca25a.html), and a large body of prior literature (e.g., the Saxe et al. characterizations of learning dynamics in linear feedforward networks; https://arxiv.org/abs/1312.6120) that tries to accomplish something similar. Because it's very hard to exactly solve the ODEs associated with neural network learning dynamics, all of these past papers had to make strong simplifying assumptions. While our assumptions are strong, we think this is typical of previous work that proceeds along similar lines.
We agree that the practical relevance of our work is not immediately clear. In our defense, the same is true for the above tradition of mathematical work on learning dynamics: given various strong assumptions, it is sometimes unclear what one learns about 'realistic' scenarios. However, in our view, the big-picture idea is to use these simple settings to build an understanding of how to model and think about learning dynamics. The hope is that concepts learned by analyzing simple, tractable networks can eventually be used to understand bigger and more complex networks. Or it may be possible to leverage these concepts to speed up training, something that has arguably proven true in work related to scaling laws and the μP parameterization.
Finally, we note that fixing the scope to linear networks (and not simulating any nonlinear networks) is not unusual for this type of theoretical work. Bordelon et al. and Proca et al. (the latter of which was published in ICML) both do this. We think this is just a matter of effort: although the settings of these papers are limited, it's a lot of work to mathematically characterize them thoroughly.