- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors suggest to extend standard encoding analyses of forward activations to map backpropagated gradients onto neural data. They show that backpropagation signals can predict neural activation, beyond the predictable information provided by forward activations. They validate it across 8 conventional AI models predicted on fMRI and MEG signals.
Critical strengths
The paper propose to extend standard encoding analyses of forward activations. Motivated by literature on biologically plausible learning methods, arguing that top-down neural activities can propagate backpropagation-like signals, the authors suggest to test this hypothesis by using backpropagation signals to predict neural activities. This is a novel approach, and can provide valuable information to the open question of how learning mechanisms are implemented in the brain. Their results suggest, that backpropagation signals indeed provide information beyond can be explained solely by forward activation. They analyze temporal and hierarchical behavior, and demonstrate interesting insights on the predictive patterns and the training progress.
Critical weaknesses
I find the interpretation of Fig 2 (E,F) as an overclaim - the textual description of the results doesn't fully aligned with the visual results. I prefer more accurate description of the visible results, and supporting claims like: "shallower layers peak earlier, whereas deeper layers align later with brain responses (Fig. 2E)." with a numeric evaluation since the time of the two peaks seems similar across depth.
The paper oversimplify existing claims for biologically learning models - most do not argue for a simple clean forward - backward separation in the brain (a forward phase followed by a backward propagation), and many suggest more complex time dynamics with interacting bottom-up and top-down activities.
Questions for authors
How can these results be interpret when relaxing the assumption of a clean forward backward separation?
2 comments
Thank you again for your thoughtful review. We hope our revisions have addressed your feedback.
Would you have any remaining questions or points you’d like us to address or clarify?
Layer-wise peak-time statistics clarify the temporal hierarchy
We thank the reviewer for this thoughtful review and for highlighting the novel approach, successful isolation of gradient signals, and the temporal and hierarchical analyses.
W1 — Peak-time hierarchy is now quantified. We now quantify peak times in a new column (Fig. 2G–H) and compare forward and backward timing (Supp. Fig. 11). Forward peak time increases with depth (r = .96, p = 1e-6); backward peak time does not (r = .19, p = .5). We improved the text accordingly.
W2 — Scope now targets sequential backpropagation. We agree. We test only the standard, sequential form of backpropagation, and we now state this clearly in the Introduction and Discussion.
Q1 — The spatial result does not require a separate backward phase. Our predictions assume that the backward pass follows the forward pass. Backward layers do peak later on average, though no layer peak differs significantly between forward and backward (paired Wilcoxon, BH-FDR q ≥ .5; Supp. Fig. 11). This latency complements the result that residual backprop becomes significantly positive at later latencies (0.5–1.7 s; Fig. 4F). But crucially, backward peaks show no depth order (Fig. 2H: r = .19, p = .5), unlike forward peaks (Fig. 2G: r = .96, p = 1e-6). Together, these results are consistent with learning-related signals emerging after initial image processing, but not with the specific reverse-ordered temporal hierarchy predicted by sequential backpropagation.
If we relax the sequential assumption, gradients still predict brain responses (Fig. 3) and the reversed fMRI hierarchy remains (Fig. 2D; Supp. Fig. 10H), this reversed fMRI hierarchy is also now reproduced for a new benchmark of 6 video models like VJEPA-2 (Supp. Fig. 10). The MEG result therefore argues only against a strict, sequential temporal hierarchy of gradient backpropagation; it does not rule out simultaneous or recurrent learning models. We revised the text to make this scope explicit.
This review helped us improve the paper by:
All new figures and text are in the paper; we tried to keep this response brief, but are of course happy to discuss any detail.