acceptodds
Under review as a conference paper at ICLR 2027

Hybrid Temporal Credit Assignment via Internal and External Recurrence Decomposition

Abstract

Recurrent neural networks process sequential input through their internal dynamics, but deploying them in neuromorphic settings requires training methods that are both memory efficient and capable of accurate temporal credit assignment. Backpropagation through time (BPTT) computes exact gradients, but its memory demands rise linearly with sequence length, making it unsuitable for long sequences and on-chip learning. Truncated BPTT (TBPTT) approximates BPTT while reducing memory demands, but it discards long-range dependencies. Approximate forward-propagation methods such as e-prop and HYPR, on the other hand, can retain long-range gradient information, but they restrict gradient computations to neuron-internal recurrence while neglecting pathways through recurrent connections. We propose Forward-Backward Propagation Through Time (FBPTT), an algorithm that computes gradients by combining forward propagation of neuron-internal sensitivities through the entire sequence with TBPTT within sequence chunks. Forward-propagated sensitivities preserve long-range learning signals, while TBPTT captures short-term dependencies across recurrent connections. This results in a semi-online algorithm with chunk-wise updates and bounded memory. We evaluate FBPTT on both standard artificial recurrent networks and spiking neural network models. Across all settings, FBPTT consistently outperforms pure forward approximations and standard TBPTT, showing that the combination of long-term neuron-internal credit with short-term intra-layer credit is an effective strategy for scalable semi-online learning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Recurrent Neural Networks, Temporal Credit Assignment, Spiking Neural Networks, Neuromorphic Computing, Backpropagation Through Time, Real-Time Recurrent Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_18uiOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how to train recurrent neural networks in an online fashion, without using backpropagating error signals through long periods of times. The paper combines locally backpropagated errors (as in truncated backpropagation through time) with diagonal approximations to the sensitivity matrix of real-time recurrent learning (à la e-prop / RFLO). The paper provides analysis that this performs better than doing any of the two individually, on a diverse set of tasks and architectures.

Critical strengths

The paper is nicely written and well executed; from introduction everything makes sense.

Critical weaknesses

I am afraid that the algorithmic innovation of the papers are more limited than what is claimed. The idea that we can use truncated backpropagated error signals to couple with the eligibility traces of eprop rather some instantaneous errors is already present in Bellec et al. 2019 (variation 3 of the eprop algorithm, they additionally use synthetic gradients to estimate errors outside of that window) and to some extent in Schmidhuber 1992, Neural Computation (combines RTRL sensitivity with truncated backprop errors). In addition to that, the paper is motivated from an on-chip learning perspective. The truncated backprop errors that is added compared to eprop is what particularly hard to estimate on chip (which the authors acknowledge in the limitations section). It is unlikely that there is any practical setting in which this algorithm would be favored against e-prop or BPTT.

Questions for authors

Please let me know if my assessment of the contributions is unfair, in which case I am happy to revise it.

4 comments

Weaknesses

[W.1]

I am afraid that the algorithmic innovation of the papers are more limited than what is claimed. The idea that we can use truncated backpropagated error signals to couple with the eligibility traces of eprop rather some instantaneous errors is already present in Bellec et al. 2019 (variation 3 of the eprop algorithm, they additionally use synthetic gradients to estimate errors outside of that window) and to some extent in Schmidhuber 1992, Neural Computation (combines RTRL sensitivity with truncated backprop errors).

Yes, the FBPTT gradient estimator is equivalent to e-prop 3 without the synthetic-gradient boundary term. We agree that the idea of combining truncated backpropagation with forward-propagated sensitivities goes back to Schmidhuber (1992). We now state this explicitly in the revised manuscript, give a proof of the equivalence in the appendix, and no longer present FBPTT as a new learning rule. The manuscript nevertheless makes two contributions, on which the revised version is focused.

The first is a boundary formulation of this estimator, which computes the same gradient as e-prop 3 in a more efficient way. In e-prop 3, the gradient contribution of a chunk is obtained by multiplying, at every time step, the learning signal with the eligibility trace, which contains the parameter influence from all previous chunks as well as from the current chunk. FBPTT handles these two parts separately: the contribution of the current chunk is computed exactly by standard truncated BPTT, and the forward-propagated sensitivity G~\tilde{G} enters only through the parameter influence of the previous chunks, once per chunk, via the boundary term δiλ+1iλ+1:(i+1)λHI,iλ+1G~iλ\delta_{i\lambda+1}^{i\lambda+1:(i+1)\lambda} H^{I,i\lambda+1} \tilde{G}^{i\lambda}. Because G~\tilde{G} is needed only at chunk boundaries, it can be computed by a parallel associative scan, which reduces the number of sequential steps for this computation from O(λ)\mathcal{O}(\lambda) to O(log⁡λ)\mathcal{O}(\log\lambda); the intermediate sensitivities within a chunk need not be computed explicitly, and only a single contraction with the error signal is performed per chunk instead of one per time step.

The second contribution is an evaluation of this estimator on modern spiking neuron models (BRF, SE-adLIF) as well as on GRU, LSTM and HORN networks on typical benchmark tasks, together with a gradient similarity analysis. Bellec et al. evaluated the variant without synthetic gradients only as an ablation on two LSTM tasks.

We have revised the manuscript accordingly. In the Introduction, we now discuss e-prop 3 and the hybrid BPTT-RTRL algorithm of Schmidhuber (1992) and state our contributions with respect to this prior work. The Related Work section contains a new paragraph on combinations of backward- and forward-propagation methods, including e-prop 3. In Methods, we added a new paragraph about the relation to e-prop 3 following Eq. (6), which explains the equivalence and the computational differences described above.

[W.2]

In addition to that, the paper is motivated from an on-chip learning perspective. The truncated backprop errors that is added compared to eprop is what particularly hard to estimate on chip (which the authors acknowledge in the limitations section).

We agree that the backpropagation within chunks is difficult to implement on neuromorphic hardware with local, online learning rules, and we did not claim otherwise in our submitted manuscript. We have revised the motivation to clarify this more directly. FBPTT is not intended as an on-chip learning rule. Its use case is memory-bounded training of recurrent and spiking networks on long sequences on conventional hardware, where full BPTT is limited by memory that grows linearly with the sequence length, and where purely forward approximations such as e-prop and HYPR lose the credit that flows through recurrent connections.

[W.3]

It is unlikely that there is any practical setting in which this algorithm would be favored against e-prop or BPTT.

We have made the intended use case more specific in the revised manuscript. Compared with e-prop and HYPR, FBPTT additionally captures credit through recurrent connections within chunks. In the paper, this makes a large difference on several tasks; for example, on NeuroMorse HYPR remains at chance level (2.0%), whereas FBPTT reaches 62–69% (BRF) and 49–50% (SE-adLIF), and on sCIFAR with SE-adLIF FBPTT reaches 52–56% compared with 23% for HYPR (Tables 3 and 4 in the submitted manuscript). Compared with BPTT, the memory of FBPTT does not grow with the sequence length. We therefore position FBPTT as an option for long sequences with long-range dependencies under a fixed memory budget, in particular for networks whose internal dynamics can carry information over time, rather than as a general replacement for BPTT.

Questions for Authors

[Q.1]

Please let me know if my assessment of the contributions is unfair, in which case I am happy to revise it.

We hope that the revised framing, the explicit equivalence to e-prop 3, and the new experiments, which identify both the regime in which the forward-propagated neuron-internal credit is decisive and the regime in which it is not, make the contribution of the paper clearer, and we would be grateful if the reviewer would take them into account.

I thank the authors for their rebuttal. The clarifications and additional results strengthen the paper and better situate it within the literature.

However, the rebuttal has not changed my view on the method’s relevance to online learning:

  1. As the authors themselves acknowledge in the paper, the method has limited potential for on-chip learning, or, equivalently, for explaining learning in the brain, which is the other main motivation for online learning algorithms. The paper offers no insight that would help address these problems in the future, so it appears to be something of a dead end in that direction.
  2. On current hardware, memory is rarely a bottleneck, even for very long context lengths. And where memory is a concern, BPTT with gradient checkpointing offers cheap solution while still computing exact gradients.

I therefore maintain my score, and I leave it to the AC to weigh whether a solid experimental investigation outweighs a relatively narrow methodological contribution.

We thank the reviewer for considering our responses.

We note that we respectfully disagree with the reviewer regarding the statement "On current hardware, memory is rarely a bottleneck, even for long context lengths." After all, the whole point of truncated BPTT and also checkpointed BPTT is to overcome the memory demand of BPTT. In particular for large SNNs with thousands of simulation steps, memory demand is an important consideration.

Regarding gradient checkpointing, we note that it does not remove the limitations that motivate our approach. Checkpointed BPTT trades memory for recomputation: it requires at least one additional forward pass, and reducing memory further increases the compute cost, as parts of the forward pass must then be recomputed multiple times. For very long sequences, the cost of recomputation becomes substantial. Furthermore, memory still grows with sequence length: For checkpoints every λ\lambda-th step, memory cost in terms of sequence length is O(Tλ+λ)O(\frac{T}{\lambda}+\lambda). More fundamentally, checkpointing does not change the offline nature of BPTT: gradients are only available after the full sequence (or truncation window) has been processed, so parameter updates cannot be made online as data streams in. In contrast, our method requires constant memory, independent of sequence length, which we believe is relevant for unbounded sequences where BPTT, with or without checkpointing, is not applicable.

We agree however that checkpoined BPTT deserves a discussion in the paper, which we will add to Discussion in the camera ready version in case of acceptance.

Reviewer_eXKwOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a new mechanism for training recurrent models called forward-backward propagation through time (FBPTT). The approach can be thought of as combining truncated backpropagation through time (TBPTT) with an approximate forward propagation method, where TBPTT is used to propagate exact gradients within a time chunk, and the approximate forward method is used to propagate information across chunks. The authors motivate this theoretically by showing that the total derivative of the current state with respect its previous value can be split into contributions from self-recurrence per neuron and contribution from the previous output which depends on all neurons. The former is easy to compute with forward propagation, so FBPTT only propagates this term across chunks. The authors evaluate their method on several RNN tasks like sMNIST and sCIFAR across a variety of models against TBPTT and HYPR baselines. They also study spiking neural networks, where their semi-online approach is most promising. Finally, they test tasks that require long range dependencies such as the AddTask and state-tracking. Across experiments, FBPTT outperforms baselines and nearly matches BPTT. Additionally, FBPTT gradients align best with BPTT gradients.

Critical strengths
  • Scalable algorithms for training recurrent models has been a long-standing challenge. It is particularly timely as transformer-based architectures move to involving recurrent components, or as a memory-bounded alternative to transformers.
  • The authors do excellent job of presenting the prior literature on this topic, situating their algorithm in the prior work, and making their core idea simple and intuitive.
  • The evaluation is thorough, ranging across a variety of models (different RNN architectures and SNNs) and task types.
  • The appendix does a good job discussing the computational complexity of their approach and other approaches
Critical weaknesses
  • While the authors' contribution is unique and novel, they do not seem to cite the most relevant prior work: BPTT combined with RTRL (i.e., not an approximate forward method). See Section A.4 of https://arxiv.org/abs/2305.19044 for a more recent summary of this.
Questions for authors
  1. Can you add a citation to the hybrid BPTT-RTRL work?
  2. How does the number of hidden neurons affect your algorithm? In particular, I think that having more complex neurons allows single neurons to carry more state information, potentially making your forward approximation more useful for a task.
2 comments

Weaknesses

[W.1]

While the authors' contribution is unique and novel, they do not seem to cite the most relevant prior work: BPTT combined with RTRL (i.e., not an approximate forward method). See Section A.4 of https://arxiv.org/abs/2305.19044 for a more recent summary of this.

We now cite and discuss the hybrid BPTT–RTRL algorithm of Schmidhuber (1992, Neural Computation), which combines truncated backpropagation within blocks with exact RTRL sensitivities carried across block boundaries.

Questions for Authors

[Q.1]

Can you add a citation to the hybrid BPTT-RTRL work?

Yes, please see our response to W.1.

[Q.2]

How does the number of hidden neurons affect your algorithm? In particular, I think that having more complex neurons allows single neurons to carry more state information, potentially making your forward approximation more useful for a task.

Yes, the number of neurons NN and the complexity of individual neurons (their state dimension kk) influence FBPTT in different ways. To quantify the effect on accuracy, we performed new simulations, running a sweep over NN on psMNIST with BRF networks and λ=196\lambda = 196 (Table R1). One sees that the number of neurons influences all algorithms in a similar manner in this task, where FBPTT keeps a constant advantage over TBPTT. It is hard to evaluate the influence of the hidden dimension, since this would necessitate the definition of multi-state (more than two states) neuron models which is out of the scope of this study. We have added these results to the revised manuscript in the new Appendix A.10.3 ("Effect of the Number of Neurons", Table A6 and Figure A1), and they are summarized in the main text in the new paragraph at the end of the Experiments section.

Table R1. psMNIST test accuracy (%) for different numbers of neurons NN (BRF, λ=196\lambda=196, mean ± standard deviation over 3 runs).

NNBPTTTBPTTFBPTT
3275.92 ± 0.3163.58 ± 0.4672.27 ± 0.38
6484.51 ± 0.3774.86 ± 0.3280.61 ± 0.35
12892.16 ± 0.1781.63 ± 0.3786.82 ± 0.29
25694.27 ± 0.1884.49 ± 0.3390.21 ± 0.24

Thank you to the authors for addressing my concerns. I am leaning to keeping my score as-is, while noting the novelty concerns raised by the other reviewers. I believe the authors have done an adequate job addressing these concerns by a) re-scoping the claims and acknowledging the prior work, b) showing the equivalence to e-prop as a gradient estimator, and c) making a case for how they are practically different. I will wait for the other reviewers' response as well.

Reviewer_CWHnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a hybrid temporal credit assignment system that breaks the training into chunks. They implement standard truncated backprop through time within those chunks, keep an approximation of the gradient-updates at the chunk boundaries, and applies those in a "forward-only" fashion across chunks. Additionally, the authors find this decomposition in a way that prevents double-counting gradient contributions. Crucially, the authors achieve this with bounded memory (set by the truncation+approximation). Finally, the authors indicate that their algorithm can work in a "semi-online" fashion.

Critical strengths
  • I think the authors provided a practical algorithm that shows performance improvements across a set of benchmarks and a set of different models. The experiments cover multiple topologies: LSTM, GRU, BRF, and SE-adLIF networks.
  • I appreciate that the authors provide details beyond just final accuracy. In particular, I did like the approach taken for Fig 4. this was a nice way to isolate everything.
  • This is a useful problem to address in my opinion!
Critical weaknesses

Note the authors need a quick proofread, e.g., "achieving bounded memory and improved gradient" seems like an incomplete sentence to me? I could be wrong and have totally missed the intended meaning here?

  • There's one key element I'd like to better understand, if I understand the author's work correctly, their trace is (G~) ^t = H^t (G~)^{t-1} + F^t across chunks, while using BPTT within each chunk. You'd specifically have (G~)^0 = 0 ? This keeps the internal recurrence, but not external recurrence from earlier chunks. In my opinion this resembles Bellec et al.'s e-prop 3, whichtalks about truncated BPTT + eligibility traces and carries a similar internal/external split. Would the authors be able to contextualize details of their contribution vs. this related work? Specifically, please see https://arxiv.org/html/1901.09049v2 equations (8) and (9) (this is eqn (6) for the authors). This would really help me in figuring out the specifics with respect to your first bullet point re: contribution.

  • in the add task -- the network has to add marked input values and output the sum after the sequence, this seems to be quite favorable depending on how across-chunk breakdown occurs? Can we look at some other tasks that are possibly more comprehensive? https://elifesciences.org/articles/65459 ideally, would want some idea of accuracy vs. dependency delay and chunk length similarly, you could try something like SHD-Adding from https://arxiv.org/pdf/2306.16922v3 (specifically section 4.2 describes the SHD-Adding dataset ).

  • Broadly, I think the comparisons ought be on the basis of identical memory/compute budget, e.g., isn't TBPTT getting a lower budget vs. your approach right now? In particular, I'd like for the authors to include some probes that can isolate the interaction between temporal gradient flow and the approximations. As an example: given the same delay how would the model performance change if recurrent transfer was moved from one chunk to instead straddle chunks (i.e., earlier chunk)?

Questions for authors

Particularly interested in details on how your formulation is different from eprop3, thoughts on performance vs. Chunked e-prop 3? Would you be able to provide those details on the across-chunk transfer -- an effective sensitivity analysis on the loss contribution from those terms?

11 comments

Weaknesses

[W.1]

Note the authors need a quick proofread, e.g., "achieving bounded memory and improved gradient" seems like an incomplete sentence to me? I could be wrong and have totally missed the intended meaning here?

The intended meaning was that FBPTT achieves bounded memory while producing gradient estimates that are closer to the exact BPTT gradient than those of TBPTT and HYPR, as measured in Fig. 4 and Fig. A1. We have clarified this in the revised version.

[W.2]

There's one key element I'd like to better understand, if I understand the author's work correctly, their trace is (G)t=Ht(G)t−1+Ft(G)^t = H^t (G)^{t-1} + F^t across chunks, while using BPTT within each chunk. You'd specifically have (G~)0=0(\tilde{G})^0 = 0? This keeps the internal recurrence, but not external recurrence from earlier chunks. In my opinion this resembles Bellec et al.'s eprop 3, which talks about truncated BPTT + eligibility traces and carries a similar internal/external split. Would the authors be able to contextualize details of their contribution vs. this related work? Specifically, please see https://arxiv.org/html/1901.09049v2 equations (8) and (9) (this is eqn (6) for the authors). This would really help me in figuring out the specifics with respect to your first bullet point re: contribution.

As a gradient estimator, FBPTT is equivalent to e-prop 3 without the synthetic-gradient boundary term. The two methods differ, however, in how this gradient is computed, and this difference has practical consequences.

In e-prop 3, the gradient contribution of chunk mm is obtained by multiplying, at every time step tt of the chunk, the learning signal LmtL^t_m with the eligibility trace ϵt\epsilon^t (Eqs. (68)–(69) in Bellec et al.). The eligibility trace ϵt\epsilon^t is accumulated from the beginning of the sequence and therefore contains two parts: the parameter influence from all previous chunks, propagated forward through the neuron-internal dynamics, and the parameter influence from the current chunk itself.

FBPTT handles these two parts separately. The contribution of the current chunk is computed exactly by standard truncated BPTT within the chunk, i.e., by the term ∑tδtiλ+1:(i+1)λFt\sum_{t} \delta_t^{i\lambda+1:(i+1)\lambda} F^t in Eq. (6). Consequently, FBPTT never multiplies the sensitivities built up inside the current chunk with the error signal, since this product is already contained in the TBPTT gradient. The forward-propagated sensitivity G~\tilde{G} enters only through the parameter influence of the previous chunks, and only once per chunk, via the single boundary term δiλ+1iλ+1:(i+1)λHI,iλ+1G~iλ\delta_{i\lambda+1}^{i\lambda+1:(i+1)\lambda} H^{I,i\lambda+1} \tilde{G}^{i\lambda}.

The boundary formulation has a direct computational advantage. Because G~\tilde{G} is needed only at chunk boundaries, it is sufficient to compute its value at the end of each chunk. All immediate parameter influences FtF^t and internal Jacobians HI,tH^{I,t} within a chunk can be computed in parallel and combined by an associative scan (Blelloch, 1990), as in HYPR (Baronig et al., 2026), which reduces the time complexity from O(λ)\mathcal{O}(\lambda) to O(log⁡λ)\mathcal{O}(\log\lambda) (Appendix A.5.3). Unlike the per-step formulation of e-prop 3, FBPTT does not require the intermediate sensitivities G~t\tilde{G}^t within a chunk. Therefore, they are not explicitly computed and only a single contraction with the error signal is performed per chunk instead of one contraction per time step.

We have revised the manuscript accordingly. In the Introduction, we now discuss e-prop 3 and the hybrid BPTT-RTRL algorithm of Schmidhuber (1992) and state our contributions with respect to this prior work. The Related Work section contains a new paragraph on combinations of backward- and forward-propagation methods, including e-prop 3. In Methods, we added a new paragraph about the relation to e-prop 3 following Eq. (6), which explains the equivalence and the computational differences described above.

[W.3]

in the add task, the network has to add marked input values and output the sum after the sequence, this seems to be quite favorable depending on how across-chunk breakdown occurs? Can we look at some other tasks that are possibly more comprehensive? https://elifesciences.org/articles/65459 ideally, would want some idea of accuracy vs. dependency delay and chunk length similarly, you could try something like SHD-Adding from https://arxiv.org/pdf/2306.16922v3 (specifically section 4.2 describes the SHD-Adding dataset).

To evaluate FBPTT on a task with controllable temporal structure, we performed additional experiments on a version of the STORE-RECALL task of Salaj et al. (2021).

Task and setup: The network receives four input populations (value 0, value 1, STORE, RECALL). Each input sequence has a length of T=600T=600 time steps and is divided into 12 consecutive segments of 50 time steps each. In every segment, a random binary value (0 or 1) is presented by the corresponding value population for the full 50 time steps. During one randomly chosen segment, the STORE population is active as well, and exactly DD time steps later, the RECALL population is activated. The target is the value that was presented during the STORE segment, and the loss is restricted to the recall window. In the original task, the delay between STORE and RECALL is random; fixing it allows us to vary the dependency length systematically relative to the chunk length λ\lambda. We trained BRF networks for delays D∈100,200,300,400,500D \in \\{100, 200, 300, 400, 500\\} and chunk lengths λ∈300,150,50,10\lambda \in \\{300, 150, 50, 10\\}. Results are shown in Table R1.

Table R1. STORE-RECALL test accuracy (%), TBPTT and FBPTT, BRF network (mean ± standard deviation over 3 runs). Bold indicates chunk lengths shorter than the delay.

Delay DDMethodλ=300\lambda=300λ=150\lambda=150λ=50\lambda=50λ=10\lambda=10
100TBPTT100.00 ± 0.00100.00 ± 0.0084.23 ± 0.2979.42 ± 0.41
FBPTT100.00 ± 0.00100.00 ± 0.0099.82 ± 0.1199.96 ± 0.07
200TBPTT99.67 ± 0.1877.53 ± 0.3980.86 ± 0.4283.41 ± 0.47
FBPTT99.91 ± 0.0999.76 ± 0.1499.43 ± 0.2899.36 ± 0.29
300TBPTT75.72 ± 0.4673.31 ± 0.3370.61 ± 0.5170.69 ± 0.48
FBPTT99.88 ± 0.1299.71 ± 0.1899.78 ± 0.1499.73 ± 0.17
400TBPTT76.43 ± 0.4468.82 ± 0.4969.76 ± 0.3658.21 ± 0.57
FBPTT99.69 ± 0.1999.77 ± 0.1299.91 ± 0.0899.78 ± 0.25
500TBPTT73.86 ± 0.5265.24 ± 0.4757.61 ± 0.6651.53 ± 0.74
FBPTT99.82 ± 0.1399.51 ± 0.2899.91 ± 0.0999.68 ± 0.22

FBPTT maintains high accuracy across all tested delays and chunk lengths, whereas TBPTT performance decreases substantially as the delay increases and the chunk length becomes shorter. This highlights the benefit of FBPTT for learning long-range temporal dependencies under truncated training. We have added this experiment to the revised manuscript: the task, the experimental setup and the full results are described in the new Appendix A.10.1 ("Experiments with Controlled Delay", Table A4), and the results are summarized in the main text in the new paragraph "Controlled delays, matched memory, and network size" at the end of the Experiments section.

[W.4]

Broadly, I think the comparisons ought be on the basis of identical memory/compute budget, e.g., isn't TBPTT getting a lower budget vs. your approach right now? In particular, I'd like for the authors to include some probes that can isolate the interaction between temporal gradient flow and the approximations. As an example: given the same delay how would the model performance change if recurrent transfer was moved from one chunk to instead straddle chunks (i.e., earlier chunk)?

According to Table A1 in the manuscript, TBPTT requires O(λmk)\mathcal{O}(\lambda mk) memory for the states within a chunk, while FBPTT additionally stores the sensitivity G~\tilde{G}, giving O(λmk+mkp)\mathcal{O}(\lambda mk + mkp). The additional term is independent of the sequence length TT and the chunk length λ\lambda. Both methods additionally store the mp=O(m2)mp = \mathcal{O}(m^2) network weights. We performed new experiments where we compared FBPTT with TBPTT at a memory-matched chunk length, chosen such that both methods require approximately the same memory (measured during training). Table R2 compares FBPTT with TBPTT at chunk lengths with the same or larger memory usage.

Table R2. Comparison at matched memory. STORE-RECALL and state tracking: BRF networks, test accuracy. Add task (T=5000T=5000): LSTM networks, percentage of correctly processed test sequences. In all rows, TBPTT uses a chunk length with the same or larger memory usage than FBPTT (mean ± standard deviation over 3 runs).

TaskFBPTT λ\lambdaFBPTT (%)TBPTT λ\lambdaTBPTT (%)
STORE-RECALL, D=300D=3001099.73 ± 0.1730075.72 ± 0.46
STORE-RECALL, D=500D=5001099.68 ± 0.2230073.86 ± 0.52
Add task (LSTM)1097.81 ± 0.3450089.64 ± 0.41
State tracking1099.58 ± 0.1920097.46 ± 0.31

On STORE-RECALL, FBPTT with λ=10\lambda = 10 clearly outperformed TBPTT with λ=300\lambda = 300. On the Add task with sequence length 5000, FBPTT with λ=10\lambda = 10 also outperformed TBPTT with λ=500\lambda = 500. Here, the two marked inputs can be thousands of time steps apart, so that the dependency spans many chunks even for the long TBPTT chunk length. On the state tracking task, the performance of TBPTT at λ=200\lambda = 200 is comparable to FBPTT at λ=10\lambda = 10. This is probably due to the dense per-time-step supervision in this task, which does not demand long-term gradient flows across chunks. We conclude that, at the same memory budget, FBPTT outperforms TBPTT when the temporal dependencies of a task are longer than the chunk length that TBPTT can afford with this budget, because the forward-propagated sensitivities carry credit across chunk boundaries independently of the dependency length. We have added this comparison to the revised manuscript: the results are reported in the new Appendix A.10.2 ("Matched Memory Experiments", Table A5) and summarized in the main text in the new paragraph at the end of the Experiments section.

Questions for Authors

[Q.1]

Particularly interested in details on how your formulation is different from eprop3.

Please see our response to W1.2. In short, the FBPTT estimator is equivalent to e-prop 3 without synthetic gradients. The formulations differ in (i) the absence of a synthetic-gradient module in FBPTT and (ii) the organization of the computation: e-prop 3 multiplies the eligibility trace with the learning signal at every time step, whereas FBPTT computes the contribution of the current chunk by backpropagation and uses the forward-propagated sensitivity only once per chunk boundary, which allows it to be computed by a parallel scan without storing intermediate sensitivities.

[Q.2]

thoughts on performance vs. Chunked e-prop 3?

Since FBPTT computes the same gradient as e-prop 3 without synthetic gradients, all FBPTT results in the paper and in this response also hold for this variant of chunked e-prop 3.

[Q.3]

Would you be able to provide those details on the across-chunk transfer -- an effective sensitivity analysis on the loss contribution from those terms?

Because FBPTT is exactly TBPTT plus the boundary correction terms, the difference between FBPTT and TBPTT at equal chunk length measures the effect of the across-chunk terms directly. On STORE-RECALL, this difference is zero when the dependency lies within a chunk (λ≥D\lambda \ge D and DD small) and grows with longer delays (Table R1). At the level of the gradient itself, Fig. 4 and Fig. A2 in the submitted manuscript show that adding the boundary terms increases the cosine similarity to the BPTT gradient compared with TBPTT in all considered settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.