acceptodds
Under review as a conference paper at ICLR 2027

ConDenFormer: A Conductance-Inspired Spiking Transformer with Dendritic Memory and Adaptive Neuronal Dynamics

Abstract

Spiking neural networks (SNNs) process information through discrete spikes, offering a promising approach to energy-efficient, brain-inspired computation. However, conventional current-based leaky integrate-and-fire (LIF) neurons face two limitations. First, weak inputs may fail to elicit spikes within a finite simulation window, causing subthreshold evidence to be lost from the emitted spike sequence. Second, their responses depend strongly on the temporal synchrony of excitatory inputs: dispersing a fixed number of spikes over time can substantially reduce the output firing rate compared with synchronized inputs. We address these limitations with ConDenFormer, a spiking Transformer inspired by conductance-dependent integration and dendritic–somatic computation. ConDenFormer integrates adaptive conductance exchange with persistent, source-specific dendritic memory. Learnable nonnegative conductances, modulated by pre-cancellation input activity and dendritic context, control how retained evidence contributes to the somatic membrane potential and its effective response timescale. Dendritic states persist across somatic spikes, preserving input history that would otherwise be discarded. We establish a dissipative exchange law, bounded driven dynamics, and a constructive separation of input sequences with identical source and temporal marginals. Across multiple benchmarks, ConDenFormer achieves 97.21% accuracy on CIFAR-10, 83.42% on CIFAR-100, 85.4% on CIFAR10-DVS, 99.0% on DVS128 Gesture, and 79.45% on ImageNet-1K with only 16.29M parameters. These results demonstrate that source-specific memory and adaptive conductance can improve the preservation and utilization of temporal evidence in spiking visual computation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Neural Networks, Spiking Transformers, Brain-Inspired Computation, Conductance-Based Neuron Models, Excitation–Inhibition Balance

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_w8MEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ConDenFormer, a spiking Transformer that incorporates source-specific dendritic memory and adaptive conductance-mediated exchange into two transition stages of MaxFormer. Each modified neuron maintains four dendritic states that persist across somatic resets. Input activity and dendritic context regulate reciprocal exchange between these states and the soma. The paper provides a stability analysis and a constructive demonstration of source–time selectivity, followed by visual recognition experiments, component ablations, and temporal interventions.

Critical strengths
  1. The method has a clear motivation: preserving subthreshold evidence and source-specific temporal information that can be lost through immediate input aggregation and somatic resets. The proposed updates explicitly address these mechanisms.

  2. The theoretical analysis provides useful structure beyond biological motivation. The dissipative exchange identity characterizes the stability of the interaction, while the source–time analysis demonstrates an expressivity advantage over a scalar LIF neuron receiving aggregated inputs.

Critical weaknesses
  1. The empirical distinction from existing dendritic and multi-compartment neurons remains insufficient. The related-work section acknowledges that DH-LIF preserves branch-specific currents across somatic resets, while TC-LIF and TS-LIF introduce coupled dendritic–somatic dynamics. However, these models are not directly evaluated under the same backbone and training protocol. The one-way memory control is useful, but does not establish the advantage of the proposed exchange over existing coupled-neuron designs. Matched comparisons would better isolate the methodological contribution.

  2. The strongest controlled evidence is limited to one dataset and one training seed. Appendix E specifies a single paired seed for the CIFAR10-DVS ablations. The “Last 20” metric averages checkpoints from the same run and therefore does not quantify variability across independent training runs. Moreover, the complete model improves peak accuracy over learnable-time-constant LIF by 2.00 percentage points, whereas the corresponding Last-20 improvement is 0.72 points. This makes repeated-run evaluation particularly important. Matched neuronal comparisons on a static-image dataset would also strengthen the evidence for generality, since the main benchmark tables use baseline results from the original papers.

  3. The computational cost is not sufficiently characterized. The small parameter increase does not capture the cost of maintaining four additional states, computing pre-cancellation activity, applying context-dependent gates, and performing reciprocal updates. Appendix D reports 1.5 MiB of additional dendritic state per event-model sample, but does not provide end-to-end latency, throughput, peak training memory, or hardware-energy measurements. Such measurements are needed to assess the accuracy–efficiency trade-off. The mathematical dissipation result should also be distinguished from computational energy efficiency.

  4. The practical importance of the theoretical source–time nonseparability remains only partially established. Proposition 1 demonstrates an expressivity difference relative to scalar LIF under fixed-conductance, subthreshold dynamics. However, Appendix F.4 reports median nonseparable response-energy fractions of only 0.00762 at D3 and 0.00157 at D4, suggesting that the sampled local responses are largely approximated by rank-one kernels. The intervention results support the use of temporal information, but do not isolate nonseparability as the cause of the accuracy gains. A stronger connection between this theoretical property and task performance would improve the central argument.

Questions for authors
  1. How does ConDenFormer compare with DH-LIF, TC-LIF, and TS-LIF when inserted at the same locations and trained with the same recipe?

  2. Can the main neuronal comparisons be repeated across multiple seeds and extended to CIFAR-100 or ImageNet? Please also clarify the checkpoint-selection protocol.

  3. What are the end-to-end training and inference overheads relative to learnable-time-constant LIF, including source-activity computation and recurrent-state storage?

  4. Can activity-dependent gating and dendritic-context gating be ablated separately? How sensitive are the results to the number of branches and the contrast-write adjustment in Equation 30?

8 comments
AuthorsOpenReview ↗

Response to Reviewer w8ME (W1 & Q1)

Thank you for your detailed and constructive comments. We have substantially revised the manuscript and figures to improve clarity and readability. In response to W4, we have replaced the original example with a clearer numerical illustration of the proposed mechanism. Furthermore, we have conducted supplementary experiments to address the questions concerning comparisons with existing neuronal models, sensitivity to random initialization, computational cost, and the role of spatiotemporal non-separability in the trained network. Our point-by-point responses are provided below.

W1 & Q1. Relationship to DH-LIF, TC-LIF, and TS-LIF

We agree that our work builds on several established ideas in multi-compartment neuronal modeling, including persistent dendritic memory, conductance-inspired computation, and somatic–dendritic coupling. As discussed in the Related Work section, DH-LIF employs heterogeneous dendritic time constants to process information over multiple timescales; TC-LIF introduces feedback between two compartments to improve long-sequence modeling; and TS-LIF combines the temporal properties and spike outputs of two compartments.

Our approach differs in how these components interact. In ConDenFormer, current pre-cancellation input activity and retained dendritic context jointly regulate reciprocal soma–dendrite exchange, while dendritic capacity controls the extent to which this exchange modifies the stored dendritic states. This provides an activity- and context-dependent mechanism for accessing and updating source-specific temporal memory. The relationship to other models has been included in the revised manuscript.

To assess the practical differences under a common setting, we adapted the DH-LIF, TC-LIF, and TS-LIF neurons to our backbone and evaluated them on CIFAR10-DVS. Since the original methods were developed using different architectures and task settings, these experiments provide a controlled comparison within our evaluation framework rather than a direct reproduction of their original results. As shown in Table 1, ConDenFormer achieves the highest accuracy among the evaluated models.

Table 1: Performance across neuron configurations on CIFAR10-DVS

Neuron configurationTop-1 (%)
MaxFormer83.30%
Learnable-τ\tau LIF83.40%
DH-LIF, four-branch adaptation82.10%
TC-LIF adaptation81.80%
TS-LIF adaptation83.30%
ConDenFormer85.40%
AuthorsOpenReview ↗

Response to Reviewer w8ME (W2 & Q2)

W2 & Q2. Checkpoint selection and robustness across random seeds

For checkpoint selection, we followed the protocol used in the official MaxFormer implementation: we evaluated the model after each epoch using the official test set and selected the checkpoint with the highest test accuracy during training. We applied the same selection rule to the corresponding baselines to maintain consistency in the comparison.

Regarding the Last-20-epoch results, we agree that this measure characterizes the final phase of an individual training run rather than variability across independent runs. It provides complementary information about performance stability within a run but cannot establish robustness to random initialization.

To assess the robustness of the performance improvements to random initialization, we conducted additional multi-seed experiments comparing ConDenFormer with two matched-backbone baselines: MaxFormer with conventional fixed-τ\tau LIF neurons and a variant using learnable-τ\tau LIF neurons throughout the network. We evaluated all three models on CIFAR10-DVS, DVS128 Gesture, and CIFAR-100 using five, ten, and two random seeds, respectively. The results are summarized in Tables 2–4. ConDenFormer achieves the highest mean Top-1 accuracy on all three datasets, outperforming both baselines across the five CIFAR10-DVS seeds and the two CIFAR-100 seeds. These results provide additional evidence that the performance improvements are robust to random initialization across both event-based and static-image recognition tasks.

Table 2: Top-1 accuracy (%) across five random seeds on CIFAR10-DVS

ModelRandom seedMean ±\pm SD
618137353071282524290194448810215482
MaxFormer83.9083.3083.7084.1082.4083.48±0.6783.48\pm0.67
Learnable-τ\tau LIF83.8083.4084.1084.3082.3083.58±0.7983.58\pm0.79
ConDenFormer84.6085.4084.8084.9083.6084.66±0.66\mathbf{84.66\pm0.66}

Table 3: Top-1 accuracy (%) across ten random seeds on DVS128 Gesture

ModelRandom seedMeasured mean ±\pm SD
6181373530712825242901944488102154822718342671572396841791643
MaxFormer98.2698.6198.2698.6197.9298.6197.5798.6198.2698.9698.37±0.4098.37 \pm 0.40
Learnable-τ\tau LIF98.6197.9298.6198.6198.6198.6198.2698.6198.6198.6198.51±0.2398.51 \pm 0.23
ConDenFormer98.6198.9698.6198.9698.9698.9698.6198.9698.6198.6198.79±0.18\mathbf{98.79 \pm 0.18}

Table 4: Top-1 accuracy (%) across two random seeds on CIFAR-100

ModelRandom seedMean ±\pm SD
35307618137
Learnable-τ\tau LIF82.3882.8682.62±0.3482.62\pm0.34
ConDenFormer83.4283.0383.23±0.28\mathbf{83.23\pm0.28}
AuthorsOpenReview ↗

Response to Reviewer w8ME (W3 & Q3)

W3 & Q3. Training cost

ConDenFormer introduces additional computational overhead. Relative to MaxFormer, ConDenFormer requires approximately 2.94×2.94\times and 1.98×1.98\times the total training time on CIFAR10-DVS and DVS128 Gesture, respectively (Table 5). The dendritic states at D3 and D4 together require approximately 1.5 MiB of storage per sample in FP32. On an RTX 5090, the complete model requires approximately 15.48 GiB of peak GPU memory for a forward and backward pass with the specified batch size.

Table 5: Comparison of training cost

DatasetModelMedian epoch time (s)Full loop (h:mm:ss)
CIFAR10-DVS (106 epochs)MaxFormer35.491:03:50
Learnable-τ\tau LIF35.151:02:48
ConDenFormer106.263:07:48
DVS128 Gesture (160 epochs)MaxFormer7.600:21:12
Learnable-τ\tau LIF7.720:21:31
ConDenFormer15.630:41:53
Reviewer_aTboOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ConDenFormer introduces four persistent source-specific dendritic states and adaptive reciprocal conductance exchange at two transitions of a MaxFormer backbone. The paper derives charge conservation, a dissipative exchange identity and a bounded driven-state result, and constructs a source–time response that scalar aggregation cannot distinguish. It reports static-image and event-recognition results, matched neuronal ablations, and frozen-checkpoint temporal interventions.

Critical strengths
  1. The exchange mechanism is specified sufficiently precisely to analyze. The charge/energy identities and their assumptions are internally consistent, and the constructive source-order example is useful. Importantly, Appendix B.1 also shows that a one-way multistate control can distinguish the example: the paper does not attribute that expressivity exclusively to reciprocity.

  2. The mechanism experiments go beyond a leaderboard comparison. The one-way, spike-reset, static-conductance, and shared-write controls target distinct design choices, while count-preserving timing shifts and signed-write interventions connect retained history to downstream predictions. The appendix reports both local response measurements and their limited nonseparability, rather than presenting only a favorable illustrative neuron. Source code is included in the supplementary ZIP.

Critical weaknesses
  1. The empirical support for the component-level gains is narrower than the main recognition claims. The controlled evidence is confined to one CIFAR10-DVS training seed (Appendix E), with 1,000 test examples and peak-checkpoint selection. Relative to learnable-tau LIF, the gain is 2.00 percentage points at the peak but 0.72 points in the last-20 average (Table 3); the latter is an average of correlated checkpoints, not independent training runs. The supplied default CIFAR10-DVS trainer evaluates the official test split each epoch and saves its maximum (train_dvs.py, lines 1150–1162 and 1194–1197; the optional internal-validation path is disabled by the supplied default). This makes the checkpoint-selection protocol important to the interpretation of the reported peak. The ImageNet comparison, meanwhile, uses literature MaxFormer values and also changes ordinary neurons to channel-wise learnable retention. A matched ImageNet learnable-tau baseline is needed to attribute the 1.63-point improvement specifically to the proposed source-memory/exchange mechanism. I do not require state-of-the-art accuracy, but the present comparisons leave the size and source of the improvement uncertain.

  2. The provenance of the reported ImageNet result needs reconciliation with the supplied reproduction package. The manuscript reports a completed 79.45% result and a detailed training recipe. In the attached ZIP, README.md lines 78–81 describe the ImageNet implementation as a reproducible port, and PACKAGE_AUDIT.md lines 16–20 state that no completed historical ConDenFormer ImageNet checkpoint was available. This may reflect an earlier package; it is not evidence by itself that the reported result is incorrect. However, the current materials do not establish which implementation/run produced the Table 2 value, which matters because ImageNet is the largest-scale empirical support for the method.

The mechanistic analysis is promising. The recommendation remains tentative because the evaluation/selection protocol and the provenance of the main large-scale result need clarification; the theoretical mechanism itself is not the reason for rejection.

Questions for authors
  1. Were the CIFAR10-DVS peak checkpoints selected on a separate validation split or on the official test set? Please provide validation-selected, frozen-test results for the complete model and the strongest matched control across independent training seeds. Can you also give a matched ImageNet learnable-tau baseline under the same recipe to isolate the proposed exchange mechanism?

  2. Which source version and completed run produced the 79.45% ImageNet result? Can you reconcile the manuscript with the package statements above and provide the corresponding training/evaluation record or checkpoint identity and exact evaluation procedure?

2 comments
AuthorsOpenReview ↗

Response to Reviewer aTbo (W1 & Q1)

Thank you for your thoughtful assessment of our theoretical analysis, experimental protocol, and reproducibility, as well as for your constructive comments. We appreciate your recognition of the consistency of our theoretical results and the value of the mechanistic experiments. We have substantially revised the manuscript and figures to improve the clarity, consistency, and overall readability of the paper.

Furthermore, we have conducted additional experiments to address your concerns regarding checkpoint selection, robustness across random initializations, and the attribution of performance improvements to the proposed neuronal mechanism. In addition, we have rechecked the ImageNet-1K training records and prepared an updated reproducibility package. Our point-by-point responses follow.

W1 & Q1. Checkpoint selection and robustness across random seeds

Thank you for highlighting the distinction between peak accuracy, the Last-20-epoch average, and variability across independent training runs. We agree that these metrics provide different information and that the Last-20-epoch average cannot substitute for multi-seed evaluation.

We followed the checkpoint-selection protocol of the official MaxFormer implementation. Specifically, we evaluated the models on the official test set after each epoch and selected the checkpoint with the highest test accuracy, without using a separate validation split. We applied the same protocol to the corresponding baselines to ensure procedural consistency.

As you correctly note, these checkpoints are correlated observations from a single run and therefore do not measure variability across independent initializations. To assess robustness to random initialization, we conducted additional multi-seed experiments comparing ConDenFormer with two matched-backbone baselines: MaxFormer with conventional fixed-τ\tau LIF neurons and a variant using learnable-τ\tau LIF neurons throughout the network. We evaluated the three models on CIFAR10-DVS, DVS128 Gesture, and CIFAR-100 using five, ten, and two random seeds, respectively. Tables report the individual results and mean ±\pm standard deviation.

Table 1: Top-1 accuracy (%) across five random seeds on CIFAR10-DVS

ModelRandom seedMean ±\pm SD
618137353071282524290194448810215482
MaxFormer83.9083.3083.7084.1082.4083.48±0.6783.48\pm0.67
Learnable-τ\tau LIF83.8083.4084.1084.3082.3083.58±0.7983.58\pm0.79
ConDenFormer84.6085.4084.8084.9083.6084.66±0.66\mathbf{84.66\pm0.66}

Table 2: Top-1 accuracy (%) across ten random seeds on DVS128 Gesture

ModelRandom seedMeasured mean ±\pm SD
6181373530712825242901944488102154822718342671572396841791643
MaxFormer98.2698.6198.2698.6197.9298.6197.5798.6198.2698.9698.37±0.4098.37 \pm 0.40
Learnable-τ\tau LIF98.6197.9298.6198.6198.6198.6198.2698.6198.6198.6198.51±0.2398.51 \pm 0.23
ConDenFormer98.6198.9698.6198.9698.9698.9698.6198.9698.6198.6198.79±0.18\mathbf{98.79 \pm 0.18}

Table 3: Top-1 accuracy (%) across two random seeds on CIFAR-100

ModelRandom seedMean ±\pm SD
35307618137
Learnable-τ\tau LIF82.3882.8682.62±0.3482.62\pm0.34
ConDenFormer83.4283.0383.23±0.28\mathbf{83.23\pm0.28}

We also considered the requested matched comparison on ImageNet-1K. However, given the available computational resources and rebuttal timeline, we could not complete a learnable-τ\tau LIF experiment using the same backbone and training recipe. We acknowledge this limitation. However, given that CIFAR-100 is also a static-image dataset, albeit at a smaller scale than ImageNet-1K, we believe this comparison provides qualitative evidence that our multi-compartment dendritic structure can improve performance over learnable-τ\tau LIF neurons.

AuthorsOpenReview ↗

Response to Reviewer aTbo (W2 & Q2)

W2 & Q2. Provenance and reproducibility of the ImageNet-1K result

Thank you for identifying the discrepancy between the manuscript and the previously supplied reproduction package. We apologize for this oversight. The discrepancy arose because we inadvertently uploaded an earlier version of the code package, while the accompanying PACKAGE_AUDIT.md retained an outdated statement.

We have rechecked the original training records and prepared an updated reproducibility package containing the implementation, configuration files, and training log for the run that produced the 79.4579.45\\% Top-1 accuracy reported in Table 2 in the manuscript. The reported result and the underlying trained model are unchanged; the issue was confined to the previously uploaded package and its outdated audit description.

The ImageNet-1K model was trained from scratch for 200 epochs on eight NVIDIA A800 GPUs, using a per-GPU batch size of 64 and a global batch size of 512. We used AdamW with a five-epoch linear learning-rate warmup from zero to 0.00270.0027, followed by cosine decay to a minimum learning rate of 2.5×10−62.5\times10^{-6}.

Evaluation was performed on all 50,000 images in the standard ImageNet-1K validation set. Each image was resized so that its shorter side measured 256 pixels, followed by a single 224×224224\times224 center crop. Top-1 accuracy was evaluated using four simulation time steps (T=4T=4) in FP32, without test-time augmentation. The checkpoint was selected based on validation Top-1 accuracy, and the checkpoint from epoch 192 achieved 79.4579.45\\%.

The updated supplementary package includes the training and evaluation code corresponding to this run, the launch command, complete configuration files, and the associated training log. These materials allow the implementation, training procedure, and evaluation protocol underlying the reported ImageNet-1K result to be independently inspected and reproduced.

Reviewer_9xfBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper provides a novel architecture for Spiking Transformers which builds on the MaxFormer architecture. Specifically, they introduce a novel method which integrates adaptive conductance exchange with persistent dendritic memory to overcome weaknesses posed by weak inputs and temporal asynchrony.

Critical strengths
  1. This paper provides a well-motivated approach towards mitigating known issues with SNNs by drawing inspiration from dendritic branches and conductance modulation.
  2. The paper includes strong theoretical analysis to support their proposed architecture and the specific claims made
  3. The paper include extensive empirical analysis including ablations studies to analyze the contribution of different proposed components, a thorough comparison to existing methods, among others.
  4. The paper's method provides genuine gains as well as a novel conceptual contribution to existing wor.
Critical weaknesses
  1. Could the authors include additional experiments to test how sensitive the improvements gained from the ConDenFormer architecture are to the specific number of dendritic branches and their placements? This would help to test the robustness of the achieved benefits.
  2. In a similar vein, to test the confidence one should have in the empirical conclusions given in the paper, could the authors provide results across multiple seeds?
  3. Although the paper reports parameter counts and additional dendritic-state memory, the practical computational overhead seems insufficiently clarified. Can the authors provide clarity on what the overhead cost of this method is? For example, what is the additional cost in terms of training time, peak memory, and inference latency and what components seem to add to this cost the most.
Questions for authors
  1. Can the authors provide clarity on the functional distinction between their proposed model and an adaptive filtering approach (e.g. SAFformer)? Is the proposed method complementary to filtering approaches (e.g. could they be used together) or a completely disjoint methodology?
  2. From the appendix, it seems as though the one-way model was also able to distinguish the source-order example. The authors provide evidence showing an accuracy advantage for reciprocal exchange, but could the authors provide a further explanation (whether intuition or analysis) on what such feedback contributes and why it helps?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 9xfB (W1)

Thank you for recognizing the motivation and theoretical analysis of our work. In response to the comments, we have substantially revised the manuscript and figures to improve clarity and readability. We have also conducted additional experiments to investigate branch configurations, performance across random seeds, training cost, and the contributions of reciprocal exchange and persistent dendritic memory. Our point-by-point responses follow.

W1. Effect of the number and placement of dendritic branches on performance

We first investigated the number of dendritic branches by evaluating several configurations at D3 and D4 using the same random seed. As shown in Table 1, the configuration with four branches at each stage achieved the highest CIFAR10-DVS accuracy among those tested. Increasing the number to eight branches at both stages did not yield further improvement. We therefore adopted four branches per receiving neuron as the best-performing configuration in this comparison.

Table 1: Effect of the number of dendritic branches at D3 and D4 on CIFAR10-DVS performance.

D3 branchesD4 branchesTop-1 Acc (%)
1183.20
2283.10
2483.50
4283.10
4485.40
8882.90

We restrict the deployment of multi-compartment neurons to D3 and D4 for two complementary reasons. First, the earlier stages have fewer input channels, providing less scope for partitioning inputs into distinct source banks and potentially limiting the benefit of source-specific dendritic memory. Second, their larger spatial resolutions make multi-compartment processing substantially more expensive. As shown in Table 2, D1 and D2 together contain four times as many receiving neurons as D3 and D4. Extending the same four-branch design to D1 and D2 would add four times the dendritic-state storage and branch-update cost of the current D3/D4 deployment, increasing the overall cost of these operations by approximately fivefold.

Table 2: Somatic states across stages

Receiving stageTensor shape (C×H×WC\times H\times W)Somatic states
D116×128×12816\times128\times128262,144
D232×64×6432\times64\times64131,072
D364×32×3264\times32\times3265,536
D4128×16×16128\times16\times1632,768
AuthorsOpenReview ↗

Response to Reviewer 9xfB (W2)

W2. Performance across random seeds

To assess the robustness of the performance improvements to random initialization, we conducted additional multi-seed experiments comparing ConDenFormer with two matched-backbone baselines: MaxFormer with conventional fixed-τ\tau LIF neurons and a variant using learnable-τ\tau LIF neurons throughout the network. We evaluated all three models on CIFAR10-DVS, DVS128 Gesture, and CIFAR-100 using five, ten, and two random seeds, respectively. The results are summarized in Tables 3–5. ConDenFormer achieves the highest mean Top-1 accuracy on all three datasets, outperforming both baselines across the five CIFAR10-DVS seeds and the two CIFAR-100 seeds. These results provide additional evidence that the performance improvements are robust to random initialization across both event-based and static-image recognition tasks.

Table 3: Top-1 accuracy (%) across five random seeds on CIFAR10-DVS

ModelRandom seedMean ±\pm SD
618137353071282524290194448810215482
MaxFormer83.9083.3083.7084.1082.4083.48±0.6783.48\pm0.67
Learnable-τ\tau LIF83.8083.4084.1084.3082.3083.58±0.7983.58\pm0.79
ConDenFormer84.6085.4084.8084.9083.6084.66±0.66\mathbf{84.66\pm0.66}

Table 4: Top-1 accuracy (%) across ten random seeds on DVS128 Gesture

ModelRandom seedMeasured mean ±\pm SD
6181373530712825242901944488102154822718342671572396841791643
MaxFormer98.2698.6198.2698.6197.9298.6197.5798.6198.2698.9698.37±0.4098.37 \pm 0.40
Learnable-τ\tau LIF98.6197.9298.6198.6198.6198.6198.2698.6198.6198.6198.51±0.2398.51 \pm 0.23
ConDenFormer98.6198.9698.6198.9698.9698.9698.6198.9698.6198.6198.79±0.18\mathbf{98.79 \pm 0.18}

Table 5: Top-1 accuracy (%) across two random seeds on CIFAR-100

ModelRandom seedMean ±\pm SD
35307618137
Learnable-τ\tau LIF82.3882.8682.62±0.3482.62\pm0.34
ConDenFormer83.4283.0383.23±0.28\mathbf{83.23\pm0.28}
AuthorsOpenReview ↗

Response to Reviewer 9xfB (W3)

W3. Training cost

ConDenFormer introduces additional computational overhead. Relative to MaxFormer, ConDenFormer requires approximately 2.94×2.94\times and 1.98×1.98\times the total training time on CIFAR10-DVS and DVS128 Gesture, respectively (Table 6).

Table 6: Comparison of training cost

DatasetModelMedian epoch time (s)Full loop (h:mm:ss)
CIFAR10-DVS (106 epochs)MaxFormer35.491:03:50
Learnable-τ\tau LIF35.151:02:48
ConDenFormer106.263:07:48
DVS128 Gesture (160 epochs)MaxFormer7.600:21:12
Learnable-τ\tau LIF7.720:21:31
ConDenFormer15.630:41:53

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response To Reviewers

We thank all three reviewers for their evaluation of our work and for recognizing its motivation, theoretical analysis, and potential contributions. We are encouraged by their positive assessment of the proposed integration of adaptive conductance exchange and persistent dendritic memory, as well as their recognition of the value of our stability analysis and characterization of source--time response properties.

Meanwhile, we want to reiterate that our architecture builds on prior work in conductance-inspired computation and dendritic modeling and leaves room for further simplification and optimization. The central contribution we attempt to bring to the community is showing such a procedure: We analyze systematically how mechanisms intrinsic to SNNs can cause information loss and degrade performance, then use that analysis to develop a biologically inspired architecture that addresses those limitations and yields empirical gains. We hope this work encourages researchers to use the principles of SNNs when designing network architectures, optimizers, neuron models, encoding schemes, and training procedures, so that these designs can better exploit the mechanisms distinctive to SNNs and address the limitations that arise from them.

The reviewers also identified several aspects of the empirical validation and presentation that would benefit from further clarification and strengthening. We have carefully considered all comments and revised the manuscript and additional analyses accordingly. Below, we provide detailed point-by-point responses to the reviewers' comments and describe the corresponding revisions to the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.