- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper converts a pretrained Transformer into a spiking network without retraining it. From the same forward pass, it extracts continuous, sparse, and hybrid continuous–sparse representations. Using naturalistic fMRI data from 49 people who listened to the same audiobook, the authors study how the number of integration steps, the leak rate, and the representation format affect brain-signal prediction. The results suggest that the model aligns better with language-cortex activity within a limited integration window, and that the hybrid representation outperforms a random sparse control across 13 language-related brain regions. However, this result should be interpreted as evidence about the model’s temporal dynamics rather than as a direct measurement of the brain’s actual time scale.
Critical strengths
The experiments are carefully designed, with random sparse controls, a second conversion method, and multiple temporal settings.
The paper connects spiking neural networks, representation formats, and brain-signal prediction, making it interesting from an interdisciplinary perspective.
Critical weaknesses
The conclusions are sensitive to the evaluation metric. The Pearson analysis can lead to a different conclusion from the main metric. Also, the second conversion experiment changes both the backbone model and the conversion method, so their effects cannot be fully separated.
fMRI has relatively low temporal resolution, so the model’s integration window cannot be directly interpreted as the actual temporal scale of the human cortex.
Questions for authors
The conclusions are sensitive to the evaluation metric. The Pearson analysis can lead to a different conclusion from the main metric. Also, the second conversion experiment changes both the backbone model and the conversion method, so their effects cannot be fully separated.
fMRI has relatively low temporal resolution, so the model’s integration window cannot be directly interpreted as the actual temporal scale of the human cortex.
4 comments
Response to Reviewer BFsj (2/3): W1, backbone versus conversion
This is right for the submission, and the paper lists the separation as a next step ("a factorial design that separates backbone from conversion", Future work). We have now run it as far as the conversions allow (point 3). The rise and plateau replicate when only the conversion changes, and when the backbone changes under a fixed conversion (Table B2).
1. Same backbone, second conversion. We converted the paper's backbone, MA-BERT, with the second conversion, LAS, without any training or calibration. Alignment gains +0.052 from T=4 to T=13 in 48 of 49 listeners. From T=13 to T=16 it is equivalent to zero within ±10% of that gain, a bound set before the test, and at T=16 the converted network differs from its parent ANN by −0.0002. The advantage of layer 8 over layer 4 also grows from T=11 to T=16 under the correlation metric (40/49 listeners).
2. Same conversion, other backbones. With LAS fixed, we replaced MA-BERT by standard BERT: the paper's fine-tuned LAS model and three checkpoints that were never fine-tuned, one of them Chinese and tested on the 35 Chinese listeners of the paper's corpus. The rise appears in every listener for all four.
Table B2. Rise of layer-8 alignment from T=4 to T=13 and change from T=13 to T=16 (Little Prince fMRI, correlation metric, final pipeline).
| Backbone | Conversion | Rise [95% CI] | Listeners positive | T=16 minus T=13 (share of rise) |
|---|---|---|---|---|
| MA-BERT (paper's model), τ=1.30 | SpikedAttention | +0.067 [+0.059, +0.075] | 49/49 | +0.0004 (0.6%) |
| MA-BERT | LAS | +0.052 [+0.045, +0.059] | 48/49 | −0.00003 (0.05%) |
| BERT fine-tuned on SST-2 (paper's LAS model) | LAS | +0.0623 | 49/49 | +0.0034 (5.4%) |
| BERT-base-cased, not fine-tuned | LAS | +0.0646 [+0.0569, +0.0722] | 49/49 | +0.0020 (3.1%) |
| BERT-base-uncased, not fine-tuned | LAS | +0.0653 | 49/49 | +0.0025 (3.8%) |
| bert-base-chinese, not fine-tuned (Chinese cohort) | LAS | +0.0967 [+0.0878, +0.1058] | 35/35 | +0.0017 (1.7%) |
3. Why these comparisons. A standard BERT is not directly compatible with SpikedAttention (§3.1), so the backbone can be varied only under LAS, and the conversion can be varied only on MA-BERT. Rows 1–2 change the conversion alone; rows 2–6 change the backbone under one conversion. The profile appears in all of them, so it belongs neither to one backbone nor to one conversion. This is the replication that Reviewer 23qG describes as strengthening the investigation, now with the two factors varied separately.
In the revision. §5.1 replaces the single second-conversion replication by Table B2 and states which factor each row changes; the Limitations paragraph on the second conversion is updated accordingly; per-listener values go into new Appendix C.3.
Response to Reviewer BFsj (3/3): W2, the temporal resolution of fMRI
This is the reading the paper intends, and the review's summary states it well: the result is "evidence about the model's temporal dynamics rather than as a direct measurement of the brain's actual time scale". The window is a property of the model. What the new data add is that it does not depend on the slow fMRI signal: it replicates on recordings with millisecond resolution, with the same onsets under the correlation metric.
1. The paper's position. T and τ "are model-side computational variables and are not a proxy for cortical temporal receptive windows", and "the bounded plateau is a property of the model's encoding performance and is not a tested cortical claim" (Limitations). The abstract and introduction list a correspondence with cortical temporal receptive windows among the findings; the revision deletes that sentence from both, and no result depends on it.
2. Time-resolved recordings. The paper names "ECoG or MEG with finer temporal resolution" as future work. We ran the same sweep of T and τ on two independent intracranial datasets with word-locked, millisecond-resolution responses: Podcast electrocorticography (ECoG, 9 participants) and Brain Treebank stereo-EEG (10 participants). Alignment rises from T=4 to T=13 at both leaks in every participant (Table B3). Under the correlation metric the plateau starts at T=10 (τ=1.30) and T=6 (τ=1.60), the same cells as on fMRI. At T=16 the converted network reaches its parent ANN's level (ECoG: r = 0.142 and 0.138, against 0.140).
3. A second fMRI dataset. On LeBel et al. (2023; 8 participants, a different stimulus) the rise appears in all eight participants and the plateau starts at the same two cells.
Table B3. Paper's model: rise of alignment from T=4 to T=13 (correlation metric; 95% CI where computed) and start of the plateau.
| Data | Resolution | Rise, τ=1.30 | Rise, τ=1.60 | Positive | Plateau starts (τ=1.30; 1.60) |
|---|---|---|---|---|---|
| Little Prince fMRI (paper) | 2 s | +0.067 [+0.059, +0.075] | +0.042 [+0.037, +0.046] | 49/49; 49/49 | T=10; T=6 |
| LeBel et al. fMRI | 2 s | +0.124 | +0.075 | 8/8; 8/8 | T=10; T=6 |
| Podcast ECoG | milliseconds | +0.137 [+0.110, +0.161] | +0.093 [+0.070, +0.112] | 9/9; 9/9 | T=10; T=6 |
| Brain Treebank stereo-EEG | milliseconds | +0.082 [+0.068, +0.096] | +0.063 [+0.054, +0.071] | 10/10; 10/10 | T=10; T=6 |
So the bounded window is a model-side property that is stable across recording methods, stimuli and participants. Whether it corresponds to a cortical timescale is a separate question that the paper does not claim to answer.
In the revision. §5.1 adds Table B3; the sentence on cortical temporal receptive windows leaves the abstract and introduction; §5.1 and the conclusion describe the window as a property of the converted network; per-participant values go into new Appendix C.4 (second fMRI dataset) and C.5 (intracranial datasets).
We thank the reviewer again. The three points led to new experiments, two of them steps the paper had listed as future work, and with them its claims are better separated: a rise and plateau that hold across metrics, conversions, backbones and recording methods, and a hybrid result stated with the metric under which it was measured, and significant under both in three of four independent intracranial tests.
Response to Reviewer BFsj
We sincerely appreciate your time and constructive feedback. With the discussion period approaching its end, we would like to gently follow up and see whether our rebuttal has sufficiently addressed your comments and concerns. Should you have any remaining questions or require further clarification, we would be more than happy to provide additional details before the discussion closes.
Response to Reviewer BFsj (1/3): overview; W1, sensitivity to the metric
We thank the reviewer for recognizing the motivation and direction of this work, and for noting that the experiments are "carefully designed, with random sparse controls, a second conversion method, and multiple temporal settings". The review raises three points. Two of them, separating backbone from conversion and using time-resolved recordings, are steps the paper lists as future work; we have now carried them out. Replies 1–3 take the points in order, with results and the change in the paper; we will upload one revised PDF after replying to all reviewers.
W1 (first part): sensitivity to the metric
The paper's first conclusion, the rise and plateau of alignment, holds under both metric families in every dataset. For the hybrid contrast the paper uses held-out R²; on independent intracranial data, with both metrics fixed in advance, the contrast holds under both in three of four tests.
1. Rise and plateau hold under both metrics (Table B1, top). With the paper's model, alignment rises from T=4 to T=13 in every participant of every dataset under the correlation metric, and in all but one listener under held-out R², and it levels off under both.
2. On independent intracranial data the hybrid contrast holds under both metrics in three of four tests (Table B1, bottom). For the two intracranial datasets, held-out R² and a correlation metric were named before any hybrid result existed. The contrast is significant under both at both peak cells on stereo-EEG and at T6/τ=1.60 on ECoG.
Table B1. Paper's model. Top: participants with a positive rise from T=4 to T=13 (τ=1.30; τ=1.60). Bottom: matched contrast after the simulation null, q over the two cells.
3. Why held-out R² for the hybrid contrast. It scores the prediction error directly, the quantity the encoder is fitted to minimize. The paper reports the divergence from Pearson² itself (App. A.7), which Reviewer 23qG lists as a strength ("transparently reports cases where different metrics support different conclusions"). Because held-out R² was adopted after that divergence was seen, the appropriate check is the one in point 2: new data, metrics fixed beforehand.
In the revision. §5.1 reports the rise and plateau under both metrics; §5.2–5.3 state the fMRI hybrid result for held-out R² and name the metrics with every result; Table B1 goes into new Appendix C.5.