- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ConDenFormer, a spiking Transformer that incorporates source-specific dendritic memory and adaptive conductance-mediated exchange into two transition stages of MaxFormer. Each modified neuron maintains four dendritic states that persist across somatic resets. Input activity and dendritic context regulate reciprocal exchange between these states and the soma. The paper provides a stability analysis and a constructive demonstration of source–time selectivity, followed by visual recognition experiments, component ablations, and temporal interventions.
Critical strengths
-
The method has a clear motivation: preserving subthreshold evidence and source-specific temporal information that can be lost through immediate input aggregation and somatic resets. The proposed updates explicitly address these mechanisms.
-
The theoretical analysis provides useful structure beyond biological motivation. The dissipative exchange identity characterizes the stability of the interaction, while the source–time analysis demonstrates an expressivity advantage over a scalar LIF neuron receiving aggregated inputs.
Critical weaknesses
-
The empirical distinction from existing dendritic and multi-compartment neurons remains insufficient. The related-work section acknowledges that DH-LIF preserves branch-specific currents across somatic resets, while TC-LIF and TS-LIF introduce coupled dendritic–somatic dynamics. However, these models are not directly evaluated under the same backbone and training protocol. The one-way memory control is useful, but does not establish the advantage of the proposed exchange over existing coupled-neuron designs. Matched comparisons would better isolate the methodological contribution.
-
The strongest controlled evidence is limited to one dataset and one training seed. Appendix E specifies a single paired seed for the CIFAR10-DVS ablations. The “Last 20” metric averages checkpoints from the same run and therefore does not quantify variability across independent training runs. Moreover, the complete model improves peak accuracy over learnable-time-constant LIF by 2.00 percentage points, whereas the corresponding Last-20 improvement is 0.72 points. This makes repeated-run evaluation particularly important. Matched neuronal comparisons on a static-image dataset would also strengthen the evidence for generality, since the main benchmark tables use baseline results from the original papers.
-
The computational cost is not sufficiently characterized. The small parameter increase does not capture the cost of maintaining four additional states, computing pre-cancellation activity, applying context-dependent gates, and performing reciprocal updates. Appendix D reports 1.5 MiB of additional dendritic state per event-model sample, but does not provide end-to-end latency, throughput, peak training memory, or hardware-energy measurements. Such measurements are needed to assess the accuracy–efficiency trade-off. The mathematical dissipation result should also be distinguished from computational energy efficiency.
-
The practical importance of the theoretical source–time nonseparability remains only partially established. Proposition 1 demonstrates an expressivity difference relative to scalar LIF under fixed-conductance, subthreshold dynamics. However, Appendix F.4 reports median nonseparable response-energy fractions of only 0.00762 at D3 and 0.00157 at D4, suggesting that the sampled local responses are largely approximated by rank-one kernels. The intervention results support the use of temporal information, but do not isolate nonseparability as the cause of the accuracy gains. A stronger connection between this theoretical property and task performance would improve the central argument.
Questions for authors
-
How does ConDenFormer compare with DH-LIF, TC-LIF, and TS-LIF when inserted at the same locations and trained with the same recipe?
-
Can the main neuronal comparisons be repeated across multiple seeds and extended to CIFAR-100 or ImageNet? Please also clarify the checkpoint-selection protocol.
-
What are the end-to-end training and inference overheads relative to learnable-time-constant LIF, including source-activity computation and recurrent-state storage?
-
Can activity-dependent gating and dendritic-context gating be ablated separately? How sensitive are the results to the number of branches and the contrast-write adjustment in Equation 30?
8 comments
Response to Reviewer w8ME (W2 & Q2)
W2 & Q2. Checkpoint selection and robustness across random seeds
For checkpoint selection, we followed the protocol used in the official MaxFormer implementation: we evaluated the model after each epoch using the official test set and selected the checkpoint with the highest test accuracy during training. We applied the same selection rule to the corresponding baselines to maintain consistency in the comparison.
Regarding the Last-20-epoch results, we agree that this measure characterizes the final phase of an individual training run rather than variability across independent runs. It provides complementary information about performance stability within a run but cannot establish robustness to random initialization.
To assess the robustness of the performance improvements to random initialization, we conducted additional multi-seed experiments comparing ConDenFormer with two matched-backbone baselines: MaxFormer with conventional fixed- LIF neurons and a variant using learnable- LIF neurons throughout the network. We evaluated all three models on CIFAR10-DVS, DVS128 Gesture, and CIFAR-100 using five, ten, and two random seeds, respectively. The results are summarized in Tables 2–4. ConDenFormer achieves the highest mean Top-1 accuracy on all three datasets, outperforming both baselines across the five CIFAR10-DVS seeds and the two CIFAR-100 seeds. These results provide additional evidence that the performance improvements are robust to random initialization across both event-based and static-image recognition tasks.
Table 2: Top-1 accuracy (%) across five random seeds on CIFAR10-DVS
| Model | Random seed | Mean SD | ||||
|---|---|---|---|---|---|---|
| 618137 | 35307 | 1282524290 | 1944488102 | 15482 | ||
| MaxFormer | 83.90 | 83.30 | 83.70 | 84.10 | 82.40 | |
| Learnable- LIF | 83.80 | 83.40 | 84.10 | 84.30 | 82.30 | |
| ConDenFormer | 84.60 | 85.40 | 84.80 | 84.90 | 83.60 |
Table 3: Top-1 accuracy (%) across ten random seeds on DVS128 Gesture
| Model | Random seed | Measured mean SD | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 618137 | 35307 | 1282524290 | 1944488102 | 15482 | 27183 | 42671 | 57239 | 68417 | 91643 | ||
| MaxFormer | 98.26 | 98.61 | 98.26 | 98.61 | 97.92 | 98.61 | 97.57 | 98.61 | 98.26 | 98.96 | |
| Learnable- LIF | 98.61 | 97.92 | 98.61 | 98.61 | 98.61 | 98.61 | 98.26 | 98.61 | 98.61 | 98.61 | |
| ConDenFormer | 98.61 | 98.96 | 98.61 | 98.96 | 98.96 | 98.96 | 98.61 | 98.96 | 98.61 | 98.61 |
Table 4: Top-1 accuracy (%) across two random seeds on CIFAR-100
| Model | Random seed | Mean SD | |
|---|---|---|---|
| 35307 | 618137 | ||
| Learnable- LIF | 82.38 | 82.86 | |
| ConDenFormer | 83.42 | 83.03 |
Response to Reviewer w8ME (W3 & Q3)
W3 & Q3. Training cost
ConDenFormer introduces additional computational overhead. Relative to MaxFormer, ConDenFormer requires approximately and the total training time on CIFAR10-DVS and DVS128 Gesture, respectively (Table 5). The dendritic states at D3 and D4 together require approximately 1.5 MiB of storage per sample in FP32. On an RTX 5090, the complete model requires approximately 15.48 GiB of peak GPU memory for a forward and backward pass with the specified batch size.
Table 5: Comparison of training cost
| Dataset | Model | Median epoch time (s) | Full loop (h:mm:ss) |
|---|---|---|---|
| CIFAR10-DVS (106 epochs) | MaxFormer | 35.49 | 1:03:50 |
| Learnable- LIF | 35.15 | 1:02:48 | |
| ConDenFormer | 106.26 | 3:07:48 | |
| DVS128 Gesture (160 epochs) | MaxFormer | 7.60 | 0:21:12 |
| Learnable- LIF | 7.72 | 0:21:31 | |
| ConDenFormer | 15.63 | 0:41:53 |
Response to Reviewer w8ME (W1 & Q1)
Thank you for your detailed and constructive comments. We have substantially revised the manuscript and figures to improve clarity and readability. In response to W4, we have replaced the original example with a clearer numerical illustration of the proposed mechanism. Furthermore, we have conducted supplementary experiments to address the questions concerning comparisons with existing neuronal models, sensitivity to random initialization, computational cost, and the role of spatiotemporal non-separability in the trained network. Our point-by-point responses are provided below.
W1 & Q1. Relationship to DH-LIF, TC-LIF, and TS-LIF
We agree that our work builds on several established ideas in multi-compartment neuronal modeling, including persistent dendritic memory, conductance-inspired computation, and somatic–dendritic coupling. As discussed in the Related Work section, DH-LIF employs heterogeneous dendritic time constants to process information over multiple timescales; TC-LIF introduces feedback between two compartments to improve long-sequence modeling; and TS-LIF combines the temporal properties and spike outputs of two compartments.
Our approach differs in how these components interact. In ConDenFormer, current pre-cancellation input activity and retained dendritic context jointly regulate reciprocal soma–dendrite exchange, while dendritic capacity controls the extent to which this exchange modifies the stored dendritic states. This provides an activity- and context-dependent mechanism for accessing and updating source-specific temporal memory. The relationship to other models has been included in the revised manuscript.
To assess the practical differences under a common setting, we adapted the DH-LIF, TC-LIF, and TS-LIF neurons to our backbone and evaluated them on CIFAR10-DVS. Since the original methods were developed using different architectures and task settings, these experiments provide a controlled comparison within our evaluation framework rather than a direct reproduction of their original results. As shown in Table 1, ConDenFormer achieves the highest accuracy among the evaluated models.
Table 1: Performance across neuron configurations on CIFAR10-DVS