- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Direct State Feedback (DSF), a training method for recurrent networks that replaces the time-varying recurrent Jacobians in the backward pass with a fixed diagonal feedback operator. This makes the backward recurrence stationary and allows it to be computed using a parallel prefix scan. The paper also introduces T-DSF, which combines exact short-horizon BPTT with DSF for longer-range credit assignment.
The experiments cover several recurrent architectures, language modeling, and memory tasks. DSF consistently improves over fully truncated BPTT, while T-DSF offers a useful trade-off between training cost and performance. My main concerns are the extent of the technical novelty, the relevance of the theoretical bound to the observed behavior, and the justification for the fixed feedback operator.
Critical strengths
-
The method is simple, and the experiments show that it is useful. Replacing the backward recurrence with a stationary diagonal operator gives a clear route to parallel computation. DSF consistently improves over fully truncated BPTT, and T-DSF offers a practical compromise between exact local gradients and cheaper long-range credit assignment.
-
The evaluation goes beyond a single architecture or task. The paper tests multiple recurrent architectures, sequence lengths, memory tasks, and model scales. I also appreciate the gradient diagnostics in Section 4.7. They help identify what DSF is actually doing and show that its effectiveness cannot simply be attributed to closely reproducing the BPTT gradient.
Critical weaknesses
-
The technical advance over existing ideas appears limited. Fixed feedback operators have a substantial history in Feedback Alignment, and stationary diagonal recurrences with scan-based computation are established tools in state-space models. Applying these ideas to the temporal backward dynamics of nonlinear RNNs is interesting, but the paper needs to make clearer what new algorithmic insight this combination provides. The empirical usefulness is more convincing to me than the claimed methodological advance.
-
The theory and the choice of feedback operator do not yet explain the observed results. Proposition 1 bounds the gradient difference under contractivity and bounded Jacobian-deviation assumptions, but the paper does not establish whether these assumptions hold in the trained models. Section 4.7 reports a cosine similarity of only 0.114 between the DSF and BPTT gradients for recurrent parameters at sequence length 4096. This does not contradict the proposition, but it leaves unclear why the bound helps explain successful training at long horizons.
The choice of the feedback operator is also central here. The default spectrum, linearly spaced from 0.1 to 0.9, is selected in one six-layer GRU/OpenWebText setting and then reused across substantially different settings. Since this spectrum controls how quickly the backward signal decays, broader sensitivity evidence is needed to justify the choice. Together, these issues leave the success of the surrogate gradient insufficiently explained.
Questions for authors
-
Can you report measurements of the recurrent Jacobian norms and their deviations from the fixed feedback operator during training? These measurements would help determine whether the assumptions in Proposition 1 describe the models used in the experiments.
-
How should Proposition 1 be interpreted in light of the low gradient alignment at long sequence lengths? In particular, what does the bound tell us about training effectiveness when the DSF gradient differs substantially from the BPTT gradient?
-
How sensitive are the results to the feedback spectrum across architectures and tasks? Does the spectrum selected in the GRU/OpenWebText experiment remain competitive with other choices in the remaining settings, and is there a criterion for choosing it without task-specific tuning?
4 comments
[2/3]
W.2, Q.1, Q.2 — Proposition 1, Jacobian assumptions, and low gradient alignment
W.2 "The theory and the choice of feedback operator do not yet explain the observed results. Proposition 1 bounds the gradient difference under contractivity and bounded Jacobian-deviation assumptions, but the paper does not establish whether these assumptions hold in the trained models. Section 4.7 reports a cosine similarity of only 0.114 between the DSF and BPTT gradients for recurrent parameters at sequence length 4096."
Q.1 "Can you report measurements of the recurrent Jacobian norms and their deviations from the fixed feedback operator during training?"
Q.2 "How should Proposition 1 be interpreted in light of the low gradient alignment at long sequence lengths? [...] What does the bound tell us about training effectiveness when the DSF gradient differs substantially from the BPTT gradient?"
Response. We agree that Proposition 1 alone does not explain the observed training behavior, and that its assumptions needed to be checked. We revised the proposition and measured all its assumptions on the trained models.
Revised Proposition 1. The original assumption of contractive Jacobians does not hold in practice (see below). The revised Proposition 1 (Section 3.3) only requires the products of the true Jacobians, , to be summable: , which allows . With , , and , it gives, for any sequence length ,
\leq \frac{\varepsilon E\kappa}{1-\|A\|}.$$ **Measurements (Q.1).** The new Appendix C reports the Jacobian norms, their deviation from $A$, and the summability constant $\kappa$ on the final 75M, 166M, and 1B models, and along training for the 75M model (Tables 13–14, Figures 8–9): | Rule | $\max_t\|J_t\|$ (75M / 166M / 1B) | Contractive steps | $\kappa$ | Deviation $\|J_t-A\|$ | Deviation along the DSF signal | |---|---|---|---|---|---| | BPTT | 4.82 / 3.04 / 1.68 | 0% / 0% / 12% | 269 / 279 / 263 | 2.44 / 1.67 / 1.12 | 0.55 / 0.49 / 0.44 | | DSF | 3.46 / 3.91 / 1.69 | 0% / 0% / 10% | 304 / 399 / 215 | 2.06 / 1.71 / 0.90 | **0.32 / 0.26 / 0.25** | | FT-BPTT | 3.38 / 2.41 / 1.28 | 0% / 0% / 41% | 71 / 98 / 129 | 2.16 / 1.26 / 1.02 | 0.57 / 0.45 / 0.44 | All steps are contractive at initialization, but almost none are after a few percent of training, which is why we replaced this assumption. The constant $\kappa$, in contrast, stays finite, and on 8192-token streams the partial sums saturate before the training length (they increase by at most 2.1% from 1028 to 7295 steps). These are measurements on finite samples, not a proof that the assumption holds for all trajectories. The deviation $\|J_t-A\|$ is not small, mostly because of the off-diagonal part of the Jacobians. However, the proof only uses the deviation along the DSF signal, $\|\widetilde g_t(J_t-A)\|/\|\widetilde g_t\|$, which is 40–47% lower for DSF than for BPTT and FT-BPTT. This is explained by a new finding: training with DSF makes the Jacobians align with $A$ (Section 4.7). **Interpretation of the low gradient cosine (Q.2).** The cosine of 0.114 (now Section 4.8) was measured on a single recurrent layer at initialization, with synthetic inputs and five seeds, not on trained models; the revised Section 4.8 and Appendix A.10 now state this protocol explicitly. At initialization, the Jacobians are not yet aligned with $A$, so this setting is closer to a worst case. Moreover, the cosine stays positive and 85–89% of the coordinates have the same sign as the BPTT gradient at all lengths, so the DSF update remains a descent direction to first order, even when its angle to the BPTT gradient is large. We agree that the proposition bounds the error on the hidden-state gradients, not on the parameter gradients, and does not by itself guarantee descent or convergence; we renamed it accordingly ("Uniform State-Adjoint Approximation Error").[3/3]
W.2, Q.3 — Spectrum selection and generalization across architectures
W.2 "The default spectrum, linearly spaced from 0.1 to 0.9, is selected in one six-layer GRU/OpenWebText setting and then reused across substantially different settings. Since this spectrum controls how quickly the backward signal decays, broader sensitivity evidence is needed to justify the choice."
Q.3 "How sensitive are the results to the feedback spectrum across architectures and tasks? Does the spectrum selected in the GRU/OpenWebText experiment remain competitive with other choices in the remaining settings, and is there a criterion for choosing it without task-specific tuning?"
Response. We agree that the choice of the spectrum is important, as it sets how far back in time errors are propagated: an error steps later is weighted by . The default was selected on the six-layer GRU on OpenWebText (Appendix E, Table 16), where it was the best of the tested spectra. We then kept it unchanged for all architectures (GRU, LSTM, sLSTM, vanilla RNN) and scales (up to the 1B multi-head GRU, where each head uses ), and it gave consistent results without any tuning. We did not run a full spectrum sweep for every architecture, however.
Sensitivity across tasks (Q.3). To study sensitivity on a very different task, we added a spectrum ablation on the copy task with delays up to 8192 (new Appendix E.1, Table 17, five seeds), next to the language-modeling ablation of Table 16:
| Copy task: choice of | Exact match |
|---|---|
| BPTT (reference) | 99.78 ± 0.05 |
| (default) | 43.5 ± 6.3 |
| 79.6 ± 6.2 | |
| 81.2 ± 3.3 | |
| Log-spaced time scales, | 99.18 ± 0.20 |
| Language modeling (6-layer GRU): choice of | Validation PPL |
|---|---|
| BPTT (reference) | 40.78 |
| 49.14 | |
| 46.22 | |
| (default) | 44.60 |
| 45.59 | |
| 58.59 | |
| 201.58 |
Here .
The two tasks prefer opposite spectra: long time scales help a lot on the copy task, where log-spaced time scales match BPTT, but entries close to one hurt language modeling, and even caused training to diverge in the original setup of our 1B experiment.
A criterion for choosing . Proposition 1 explains these opposite trends: the DSF signal satisfies , which becomes large when approaches one and errors arrive at every time step, as in language modeling. In the copy task, errors are sparse, so the DSF signal stays bounded and long time scales can carry credit across long delays. This suggests a simple rule without task-specific tuning: a contractive spectrum spread over short and medium time scales, such as our default, for densely supervised tasks, and longer, log-spaced time scales for sparsely supervised tasks with long delays. We do not claim this rule is optimal for every architecture.
Thank you for the clarifications and additional experiments. These responses largely address my concerns regarding the theoretical assumptions and the selection of the feedback operator . I also appreciate the clearer positioning of T-DSF as a practical hybrid and the added Jacobian alignment experiments supporting its connection to Feedback Alignment. However, my main concern about methodological novelty remains: BPTT and FT-BPTT do not use the fixed feedback operator , and their lack of alignment with is therefore largely expected, so this comparison provides only limited evidence about the role of the alignment mechanism. Overall, the rebuttal substantially strengthens the paper, although I still have some reservations about the methodological contribution. I therefore appreciate the progress but will keep my current score.
Rebuttal [1/3]
We thank the reviewer for the thoughtful review and for recognizing the practical value of the approach. We agree that our contribution should be more clearly separated from existing feedback-alignment methods and parallel recurrence algorithms.
W.1 — Novelty of combining fixed feedback and parallel scans
Response. We agree that fixed feedback matrices, diagonal linear recurrences, and parallel scans are well-established tools, and we do not claim novelty for these individual ingredients. Our contribution is the way they are combined to train nonlinear RNNs: DSF keeps the nonlinear forward pass unchanged, and replaces only the true Jacobians in the backward recurrence of the hidden-state gradients with a fixed diagonal matrix,
\quad\longrightarrow\quad \widetilde g_t=e_t+\widetilde g_{t+1}A.$$ This matters computationally: exact BPTT multiplies time-dependent, generally dense Jacobians, whereas DSF only needs powers of a single diagonal matrix, i.e., element-wise products. The backward pass then becomes a linear recurrence that a parallel scan computes with $O(\log T)$ sequential depth. Our implementation uses $O(Td\log T)$ work (Table 1), and work-efficient scans reduce this to $O(Td)$. Unlike state-space models, DSF does not change the architecture: it is a training method for existing nonlinear RNNs, here GRUs, LSTMs, sLSTMs, and vanilla RNNs, and up to 1B parameters for GRUs and sLSTMs. We also acknowledge important predecessors: Han et al. (2020) already use temporal feedback with matrix powers, ModProp (Liu et al., 2022) uses fixed temporal filters with eligibility traces, and RFLO (Murray, 2019) uses fixed leaky eligibility traces. DSF can indeed be written exactly in a forward, eligibility-trace form, $$R_t=AR_{t-1}+P_t, \qquad \sum_tP_t^\top\widetilde g_t = \sum_tR_t^\top e_t,$$ with $P_t$ the local parameter Jacobian, so we do not claim a new class of temporal credit filters. The practical difference is that DSF propagates $d$-dimensional hidden-state gradients backward, instead of storing the potentially large $d\times p$ eligibility traces. T-DSF additionally combines exact BPTT over short horizons with DSF beyond them; this is a practical hybrid, not a new principle. **New finding: alignment in time.** The revision also shows that DSF behaves like Feedback Alignment beyond the backward computation itself: as in Feedback Alignment, the network adapts to the fixed feedback. When trained with DSF, the true Jacobians become aligned with $A$ (new Section 4.7, Appendix D). The correlation between their diagonals and $A$ reaches 0.89, 0.79, and 0.60 at 75M, 166M, and 1B parameters, against roughly 0 for BPTT and FT-BPTT, and it appears early in training. We will revise the presentation to state this contribution clearly, rather than suggesting novelty for the individual ingredients.