acceptodds
Withdrawn from ICLR 2027

Learning Local Mixtures of Temporal Codes for Spiking Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Temporal codes determine how a static image drives a spiking neural network (SNN) over time. A pointwise encoder applies the same coding rule across an image, although equal normalized intensities can occur in different local structures. This raises an allocation question: how should a bank of temporal codes be combined across spatial locations? We introduce FAPE, a framework that learns this allocation from local image context. FAPE constructs binary Burst, time-to-first-spike (TTFS), and Rate candidates in parallel and uses multiscale features to predict normalized mixture weights at each location and simulation step. The resulting continuous currents drive an unchanged SNN backbone, with optional temporal conditioning parameterized separately. On validation-selected CIFAR-100 models, spatial-only FAPE achieves 69.21 ± 0.19% official-test top-1 accuracy across three training seeds, exceeding parameter-count-matched input-global and fixed-input gates by 1.90 and 3.18 percentage points. Both paired gains are positive for every seed. On the 1.28-million-image ImageNet32 benchmark, spatial-only FAPE exceeds the globally learned mixture by 2.86 points under a shared 90-epoch protocol. Direct min–max current reaches 74.24 ± 0.41% on CIFAR-100, identifying a stronger alternative input representation. Follow-up retraining with time-averaged, repeated inputs improves selected validation accuracy in all nine matched method–seed comparisons, while reducing the mean allocation contrasts. Together, these results connect local-allocation gains to the candidate representation and its presentation to the backbone.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
spiking neural networks, input encoding, temporal coding, spatially adaptive allocation, conditional mixtures, image classification

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_L94eOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces FAPE, an input encoding framework for static images in SNNs. FAPE integrates multiple encoding schemes: bursts, TTFS, and rate coding, together with leverages multi-scale local image features to predict corresponding mixing weights at each spatial location and simulation time step. The resulting continuous-valued current drives a fixed SNN backbone. Experiments on the CIFAR-100 and ImageNet32 datasets demonstrate that this spatial allocation strategy offers significant advantages over baseline methods employing global or static gating. Furthermore, by comparing this approach with direct encoding, the study explores the extent to which these performance gains depend on the chosen representational forms and their temporal presentation.

Critical strengths
  1. The strategy of hybrid encoding has been proven effective in many studies; here, the authors further emphasize that applying hybrid encoding to the image modality is beneficial. FAPE uses local image context to learn spatially varying mixtures of Burst, TTFS, and Rate codes while preserving the SNN backbone. This provides a concrete mechanism for adapting input encoding to local image content and examining its contribution independently of backbone modifications. (importance 7/10)
Critical weaknesses
  1. The scope of the benchmarking is limited, and the results lack sufficient comparability. This evaluation is restricted to the CIFAR-100 and ImageNet32 datasets, both of which utilize an input resolution of 32×32. Although ImageNet32 contains a large number of images, the evaluation results cannot be directly compared with those from standard ImageNet benchmarks commonly used in representative research within the field of Spiking Neural Networks (SNNs). Furthermore, as the study focuses exclusively on low-resolution inputs, the effectiveness of the proposed "local spatial allocation" mechanism remains unverified for high-resolution data. (importance 9/10)
  2. Unclear research motivation: The introduction notes that pointwise encoders overlook the local structure of images, yet it fails to explain why different structures should necessitate different temporal encodings, nor does it propose a clear, testable hypothesis. Consequently, the proposed allocation mechanism lacks a well-defined scientific question. Furthermore, the strong performance of the DC baseline—along with the observed performance gains when using time-averaged repeated inputs—underscores the need to disentangle the respective contributions of spatial allocation, candidate representations, and temporal variation. In the abstract and introduction, the authors should explicitly articulate the specific scientific question this study aims to address and clarify how the experiments are designed to investigate that question. (importance 10/10)
Questions for authors

larification requests: None

suggestions for future work:

  1. The authors did not provide certain supplementary visualizations that would highlight the advantages of their encoding scheme—visualizations that would have been highly beneficial for the paper and its novel contributions.
  2. The information presented in the author’s main figure is overly complex, making it difficult to quickly grasp the core innovations and theoretical contributions; consequently, the figure does not effectively support the text. The author should reconsider the design of the figure to ensure that readers can promptly identify the key elements that highlight the paper’s contributions.
Reviewer_47izOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FAPE, a learnable image input encoder for SNNs. It combines rate, time-to-first-spike (TTFS) and burst coding with spatially adaptive mixture weights and optional temporal conditioning. The continuous encoded values are fed directly into a SNN backbone. Experiments show that FAPE improves image classification accuracies over fixed and global mixtures. However, FAPE's performance falls far behind that of direct coding, the most commonly used input encoding method for modern deep SNNs.

Critical strengths
  1. The idea of exploring adaptive combinations of different input encoding methods is interesting.
  2. The authors honestly report unfavorable results.
Critical weaknesses
  1. Limited practical value compared with simpler input coding methods due to low performance. As shown in Table 1, direct coding substantially outperform FAPE.
  2. Insufficient evaluation. Experiments are conducted only on low-resolution image classification tasks using convolutional SNNs.
  3. Different FAPE variants and output interfaces are used for different experiments, but the rationale is not properly justified.
  4. Poor writing styles. To name a few of them,
    • The abbreviation FAPE is repeatedly used while its full name is not presented.
    • zz in Equation (2) is not defined (maybe the complex variable for z-transform?). Also, the purpose of Equation (2) is unclear, as it has nothing to do with the subsequent method design.
Questions for authors
  1. For direct coding, the first linear/convolutional layer and subsequent spiking neurons effectively form a learnable spike encoder that can capture local structures of an image. Then, what specific limitation of direct coding does FAPE address?
  2. Given the significantly higher accuracy of direct coding, under what practical circumstances would FAPE be preferred?
  3. PAC-inspired conditioning provides inconsistent gains. Please clarify its necessity.
Reviewer_webQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FAPE, a framework for learning spatially adaptive mixtures of temporal input codes for spiking neural networks (SNNs). The method constructs three binary candidate encodings-Burst, time-to-first-spike (TTFS), and Rate-and employs multiscale spatial convolutions to predict code-mixture weights at each spatial location and simulation step. The resulting weighted mixtures form generally continuous input currents that are passed to an otherwise unchanged SNN backbone. The authors additionally investigate generic temporal conditioning and phase–amplitude coupling (PAC)-inspired modulation. The paper also reports two important observations. First, direct continuous-current inputs substantially outperform the proposed temporal-code mixtures on CIFAR-100. Second, retraining with temporally averaged and repeated inputs improves validation accuracy while reducing the performance differences between spatially adaptive and control allocations. Overall, the paper provides useful empirical evidence for input-conditioned, spatially varying code allocation. However, the methodological novelty, the role of temporal structure, and the practical advantages over simpler continuous-input representations remain insufficiently established.

Critical strengths

The paper addresses a meaningful limitation of conventional pointwise temporal encoding: coding decisions generally do not depend on local image structure. Formulating temporal-code selection as a context-dependent allocation problem is a reasonable research direction for SNN input representations. The experiments include ImageNet32, with approximately 1.28 million training images, as well as a non-residual Spiking VGG-11 backbone. These evaluations strengthen the evidence beyond small-scale CIFAR experiments. The authors explicitly report that direct continuous-current inputs outperform the proposed mixtures and investigate temporal reversal, permutation, averaging, and matched retraining. The distinction between binary candidate events and the continuous mixture boundary is also clearly documented. I appreciate this transparency, which makes the empirical contribution more informative.

Critical weaknesses

The core FAPE mechanism combines fixed candidate encoders, multiscale convolutional features, softmax gating, and weighted aggregation. Although sensible, these components are conceptually related to established ideas in conditional mixtures, spatial attention, and adaptive input encoding. The related-work section discusses Gated Attention Coding (GAC) and mixture-of-experts approaches, but the paper provides limited direct evidence distinguishing FAPE from existing adaptive coding strategies.

Questions for authors

1.If simpler continuous inputs provide better recognition accuracy, what advantages justify the additional encoding and gating mechanism?

2.Could the authors clarify the primary methodological differences between FAPE and existing adaptive encoding approaches, particularly GAC? Is a matched comparison with an established adaptive encoder feasible using the same backbone and training protocol?

Reviewer_yb25OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FAPE, a learnable framework that mixes multiple temporal input codes (Burst, TTFS, Rate) locally across spatial positions and time steps to drive spiking neural networks. A multiscale spatial feature extractor predicts per-location, per-step softmax gates over a fixed set of binary candidate codes; optional temporal conditioning (generic or PAC-inspired) modulates branch weights. Across controlled protocols, FAPE’s spatially varying allocation improves accuracy over globally learned and fixed mixtures on CIFAR-100 and ImageNet32, while continuous-current inputs outperform all candidate mixtures, highlighting the importance of input representation in SNN encoders.

Critical strengths

1. Well-structured temporal encoding framework.

The paper proposes a modular approach to learning local mixtures of temporal codes, separating encoding decisions from the SNN backbone. The multiscale spatial filters and spatial-temporal gating design provide a flexible way to adapt encoding to local image content. This offers a useful perspective on temporal encoding for static-image SNNs.

2. Careful experimental design and analysis.

The paper includes seed-matched experiments, parameter-matched controls, and comparisons with continuous-input alternatives. These comparisons help isolate the benefits of local encoding allocation from other design choices. The detailed training protocols and diagnostic analyses also make the results easier to interpret and reproduce.

Critical weaknesses

1. Limited practical advantages over continuous-input methods.

The continuous-current alternatives achieve higher accuracy than the proposed temporal code mixtures on CIFAR-100. Although local encoding allocation improves over fixed or global mixtures, its practical advantage remains unclear. The lack of hardware energy or latency measurements also makes it difficult to assess whether the additional encoding complexity is justified.

2. Limited exploration of encoding choices.

The proposed framework relies on a fixed bank of three relatively simple encoding methods, which may limit its applicability to more diverse inputs. In addition, the improvements from PAC-inspired temporal modulation appear modest and setting-dependent. Further analysis would help clarify when temporal modulation provides meaningful benefits beyond spatial gating alone.

Questions for authors
  1. Practical advantages: Since continuous-current inputs outperform the proposed code mixtures on CIFAR-100, could the authors clarify in which scenarios FAPE would be preferable? Are there any latency or energy benefits that could justify the additional encoding complexity?

  2. Encoding choices: How sensitive is FAPE to the choice of the three candidate codes (Burst, TTFS, and Rate) and the per-image min–max normalization? Would the method still provide similar improvements with different or learned encoding candidates?

  3. Temporal modulation: The PAC-inspired temporal modulation provides relatively small and inconsistent improvements over spatial gating alone. Could the authors clarify when this component is particularly useful and whether simpler temporal modulation strategies achieve comparable results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.