acceptodds
Under review as a conference paper at ICLR 2027

Piecewise Few-Spikes Neurons: Nonlinear Function Approximation for Provable and Efficient ANN-to-SNN Conversion

Abstract

Due to their potential for energy-efficient deployment on neuromorphic hardware, Spiking Neural Networks (SNNs) have gained significant attention. While conversion from pre-trained Artificial Neural Networks (ANNs) serves as a popular route for constructing deep spiking models, its theoretical capabilities and underlying approximation dynamics remain insufficiently understood. In this paper, we categorize common conversion methods into two main paradigms and analyze their core mechanics, exposing their primary limitations concerning non-linear representations and activation resolution. To address these constraints, we propose Piecewise Few-Spike (PWFS) neurons, which introduce time-dependent parameter dynamics to enable accurate nonlinear function approximation without excessive spike counts. Unlike existing approaches that rely on heuristic optimizations, PWFS provides an explicit theoretical convergence guarantee with provable error bounds, establishing a precise mathematical trade-off between approximation accuracy and inference latency. We conduct comprehensive experiments to validate the efficacy of PWFS, showcasing its favorable accuracy in ANN-to-SNN conversion.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Spiking neural networks, SNNs, ANN-SNN-conversion, approximation, energy efficiency

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_NRCAOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Piecewise Few-Spike (PWFS) neurons for accurate and efficient ANN-to-SNN conversion. Building upon the Few-Spike (FS) neuron framework, the authors introduce a piecewise approximation strategy with time-dependent neuron parameters to better approximate nonlinear activation functions. The paper also provides theoretical convergence guarantees and explicit approximation error bounds. Extensive experiments across vision models and language models, including Qwen3, demonstrate that PWFS can effectively preserve the performance of the original ANNs.

Critical strengths

The research idea to extend the FS-neuron (Stöckl & Maass 2021) for ANN-to-SNN conversion is interesting and novel. And the paper provides a solid theoretical analysis of existing conversion methods and establishes error bounds estimation for the proposed PWFS neuron and conversion framework.

Further more, experiments span across different archs and datasets, showing excellent nonlinear approximation performance. The proposed PWFS achieves accurate approximations of complex nonlinear activation functions, including GELU, SiLU, and Sigmoid, which are important components in modern Transformer architectures.

Critical weaknesses

The proposed method builds upon the FS neuron introduced by Stöckl and Maass (2021). It would be better if the authors could explicitly clarify the key modifications compared with the original FS formulation.

The experimental results on large-scale architectures (Qwen3) are impressive. However, the paper would benefit from a more detailed explanation of how each components are actually converted into spiking operations. For example, modern architectures contain many operations that are challenging to implement with SNNs, such as Layer normalization, large GEMM in attention and FNN, and softmax. It is therefore important to clarify the converison detail and the architecture after conversion. Since parameter adjustments is required, can authors discuss how these mechanisms could be efficiently implemented on existing neuromorphic hardware.

A final concern is code availability and reproducibility. It would be beneficial if the authors could release their implementation for reproducibility. At least providing the inference code.

Questions for authors

See in weaknesses

1 comment

We thank Reviewer NRCA for the time taken to assess our work and for the valuable feedback. Below we address the remaining points raised specifically by this reviewer:

  • Key modifications to original FS neurons. Standard FS neurons use a single schedule of thresholds, reset values, and spike weights across the entire input domain. Although these parameters vary over time, their schedule is fixed independently of the input. PWFS introduces an interval-dependent schedule through a two-phase computation. Offline, we construct a piecewise-linear approximation of the target function. During inference, Phase 1 identifies the interval containing the input and subtracts its left endpoint from the membrane potential. Phase 2 then applies FS dynamics to this residual input, with thresholds and reset values scaled by the selected interval’s width, spike weights scaled by its output increment, and an output bias corresponding to its left endpoint. Thus, PWFS uses FS dynamics to approximate each local affine segment, allowing the effective slope and offset to vary across the input domain. The two phases require MM interval-search steps followed by KK FS approximation steps.

  • Code availability and reproducibility. We agree on this point. We are finalizing the codebase and plan to release it publicly upon the paper’s acceptance.

  • Implementation of parameter adjustment on existing neuromorphic hardware. The parameter adjustment mechanism proposed in this work involves event-driven updates and is straightforward to map onto the neuromorphic hardware platforms currently available to us. While a comprehensive analysis of hardware implementation is outside the immediate scope of this paper, we acknowledge its importance, alongside the latency and efficiency comparisons mentioned by other reviewers and plan to address this in future work.

Reviewer_RAdgOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces piecewise few-spikes neurons (PWFS) for approximating nonlinear activations in two stages: first selecting an input interval, then using few-spikes coding to realize a local linear approximation on that interval. For twice continuously differentiable target functions, Theorem 4.2 gives an upper bound that combines piecewise approximation error and coding error, with a total of M+K time steps. The experiments replace activations in vision and language models with this construction and evaluate how well performance is retained relative to the original ANNs.

Critical strengths

I appreciate the constructive approach. Separating nonlinear approximation into interval selection and within-interval coding makes the relationship between precision and the number of time steps explicit. The M^{-2} and 2^{-K} terms in the error bound correspond to these two sources of approximation error, and they give a basis for allocating computation rather than relying only on empirical tuning of neuron parameters.

The downstream results also support the construction as more than a curve-fitting exercise. EfficientNet-B0 reaches 77.43% accuracy, close to the original ANN’s 77.72%; GPT-2 obtains a WikiText-103 perplexity of 37.47, compared with 37.38 for the original model. Performance is well preserved across different vision architectures and language models, so the value of the method is not limited to fitting a few scalar activation curves.

Critical weaknesses

The statements in Section 3.2 about the expressiveness of general FS neurons need qualification. Under Definition 3.3, it appears possible to set the reset amount h(t)=0 and then use increasing thresholds, together with differences between adjacent function values, to construct a staircase approximation. For Lipschitz-continuous functions, the error of such a construction can decrease toward zero as the number of time steps increases. The claim that local slope variation necessarily causes FS error to saturate is therefore not supported by the definition given in the main text. The authors should distinguish three different issues: the general expressive capacity of FS neurons, the approximation rate of a particular coding scheme, and optimization difficulties that arise in gradient-based fitting. This concern does not directly invalidate the positive PWFS construction, but it does affect how the paper explains its theoretical advance over FS.

The efficiency evidence is also incomplete. In Tables 1 and 3, PWFS uses 26-32 time steps while FS uses 12-16, and the methods handle input ranges differently as well. These results strongly support accuracy retention, but they do not directly establish an advantage at matched latency or computational cost. The second stage depends on parameters chosen for each interval, and the paper should clarify which operations in the full network are actually converted to spike-based computation. Claims of hardware friendliness and end-to-end efficiency should line up with the costs that have actually been counted.

Overall assessment: I lean toward acceptance, primarily because of the interpretable piecewise construction, the explicit approximation-error analysis, and the performance retention across architectures. The claims about FS limitations and the efficiency comparison still need careful clarification, and I do not regard those advantages as established. The constructive and empirical value currently shown is enough for me to give a 3. My direct research experience in this specific area of SNN conversion is limited, so my confidence is 2.

Questions for authors

For Section 3.2, what additional restrictions on the FS model family under analysis rule out the staircase approximation above? Should the theoretical advantage be understood mainly as a better approximation rate, as a stable constructive procedure, or as an expressive limitation of a particular FS code?

Are there existing comparisons at the same total number of time steps and with comparable input ranges? Please also clarify the implementation cost of selecting interval parameters, and which downstream network operators remain continuous-valued?

4 comments

We thank Reviewer RAdg for the time taken to assess our work and for the valuable feedback. Below we address the remaining points raised specifically by this reviewer:

  • Statements about the limitations of the FS model in Section 3.2. First of all, we clarify that by 'saturating' (line 215), we actually mean that the FS approximation error may decrease slowly as TT grows and not that it stops decreasing altogether. We modified the corresponding paragraph in the paper to avoid confusion.

    We agree that the three issues of FS raised by the reviewer should be distinguished: (1) the expressive capacity of the neuron model, (2) the approximation rate of the coding scheme, and (3) the optimization difficulties arising from surrogate-gradient-based approaches. However, rigorous statements are difficult to obtain for each of them separately.

    An FS neuron with TT time steps represents a piecewise constant function with up to 2T2^T constant pieces, yet it has only 3T3T free parameters, i.e., (d(t),h(t),θ(t))t=1T(d(t),h(t),\theta(t))^T_{t=1}. Because of the nontrivial temporal dynamics, which related works often overlook, it is not straightforward to characterize the precise expressivity of this representation quantitatively, in particular which functions it can approximate. Even setting the optimization aside, the optimal approximation rate is hard to analyze, since it ultimately depends on the number and placement of the realized constant subintervals. For this reason, we gave in Section 3.2 an intuitive explanation of the difficulties of approximating with FS, and provided in Proposition C.3 a rigorous analysis in a simplified setting which, in our view, illustrates the limitation of the FS approximation capacity. Proving the limitations in a stronger sense seems considerable more difficult mathematically.

    We now relate these points to the staircase construction described by the reviewer. We confirm that this construction is valid; in fact, it is equivalent to the first inference phase of PWFS. It achieves a convergence rate of O(T−1)\mathcal{O}(T^{-1}), but it does not exploit the fine input partition that FS can in principle realize. Proposition C.3, in contrast, considers the maximal partition into 2T2^T input subintervals and therefore excludes the staircase construction. In practice, we expect trained FS neurons to lie between these two extremes. Our intuitive argument based on local slopes in Section 3.2 applies to spikes that switch on and off across several input regions, i.e. requiring a sufficiently fine input partition; it does not apply to the staircase construction, where each spike is active on a single contiguous region. Our analysis suggests that reducing the FS approximation error requires a coarse, non-uniform input partition, which may offset the benefit of having a very large number of constant pieces.

    To answer the reviewer's questions directly: the additional restriction that rules out the staircase construction in our analysis is the assumption of a fine (or in Proposition C.3, a maximal) input partition. Accordingly, the theoretical advantage of PWFS should be understood primarily as a constructive procedure with a provable approximation rate, rather than as a proof that general FS neurons are less expressive.

Thank you for the clear response; my theoretical concern is resolved. My main remaining concern is the practical benefit: at the same latency, PWFS does not clearly outperform FS, so the efficiency claim should be stated more carefully. If you upload a revised PDF, I will take a look.

We thank Reviewer RAdg for the response, and are glad to hear that the theoretical concern has been resolved. Regarding the practical concern, we would refer the reviewer to our comment addressed to all reviewers. In particular, we acknowledge that PWFS is not yet optimized, especially in this respect: when applied to a large input interval, a substantial share of the latency budget is spent resolving the negative tail of the activations, which is in any case mapped to approximately zero. Even accounting for this inefficiency, PWFS’s provable error bound and its consistent ability to improve with additional latency (unlike FS, which can plateau) mean it remains the preferable choice when conversion accuracy is the priority. As noted both in our comment and in the revised paper (in blue), we see incorporating the activation statistics into the conversion as a promising direction for further improving our method.

Reviewer_RAdgOpenReview ↗

Thanks for the clarification; rating raised

Thank you for the clarification and the revision. The theoretical concern is resolved, and the matched-latency comparison together with the more careful discussion of PWFS's limitations addresses my remaining concern. I have raised my rating to 4.

Reviewer_UpUiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper focusses on the problem of pretrained ANN to SNN conversion for constructing deep spiking models. They mathematically analyzed IF and FS methods to expose their limitations related to non-linear representations and activation resolution. To overcome the limitations, they proposed Piecewise Few-Spike (PWFS) neuron model to approximate nonlinear activations with provable error bounds. Experiments across various tasks and architectures show PWFS consistently achieves higher accuracy than baselines.

Critical strengths
  • This paper focusses on an important problem of energy-efficient AI. I feel this is going to be increasing important given model size and their usage increasing exponentially.

  • I like the approach authors took -- analyze existing method theoretically to understand their gaps (propositions 3.2 and 3.4), and then propose solution inspired by this analysis (PWFS Theorem 4.2).

  • Spiking activity analysis in Figure 2 is also very informative.

  • Its great to see PWFS achieving near lossless performance on challenging vision dataset Imagenet.

  • I also found application of SNNs to NLU task and language models very interesting. This could enable of a lot of interesting future research directions.

  • Experiments are sufficiently comprehensive and PWFS consistently achieves better performance than baseline FS.

Critical weaknesses

I did not find any Critical Weaknesses. I do however have some questions and suggestions for the authors in the section below.

Questions for authors
  • EfficientNet-B0 FS accuracy on ImageNet seems very low 15.69±0.16. Could you explain why?

  • I appreciate imagenet results given that I have mostly see SNN papers reporting CIFAR-10 results. But, I want to check if authors performed any CIFAR-10 analysis and comparison with SNN SOTA?

  • Are PWFS and FS comparison is at matched total latency budget?

  • What are PWFS’s benefits from temporal quantized computation eg typical in Q-ANN?

  • I am curious if end-to-end latency and energy comparisons between PWFS versus FS can be reported?

  • Does PWFS reconstruct the real-valued output between layers? Or are intermediate values communicated solely as binary spikes? Basically, do you foresee any challenges in implementing it on neuromorphic hardware?

  • You reported perplexity values of only GPT-2. What are bottlenecks in testing SOTA open-weight LLMs?

  • Any justification for only using Few-Spike (FS) neurons (Stöckl & Maass, 2021) as a baseline?

2 comments

We thank Reviewer UpUi for the time taken to assess our work and for the valuable feedback. Below we address the remaining points raised specifically by this reviewer:

  • Low accuracy of FS-converted EfficientNet-B0. The range of activations in the FS has to be hand-fitted, while the activations in the source EfficientNet-B0 were approximately between [−250,250][-250,250], which is also the largest activation value interval among the models we considered. We have tried different input intervals and optimization configurations but have never managed to get an acceptable conversion accuracy in this case.

  • Additional experiments with CIFAR10 and open-weight LLMs. We added the results for Qwen3 models to Table 2 in the paper, for CIFAR10 please see our comment to all reviewers.

  • PWFS's benefits from temporal quantized computations e.g. typical in QANN. We are not entirely certain if we interpret this question correctly, but we understand it as what benefits PWFS’s temporal spike coding offers relative to QANN-style bit-width quantization. Our answer: the benefit is not precision-per-bit efficiency, but hardware realizability. Each spike in PWFS triggers an accumulate (AC) operation rather than a multiply between two runtime-dependent values, matching the accumulate-only, event-driven primitives of neuromorphic hardware, whereas QANNs still involve multiply (MAC) operations. Moreover, the outputs in case of PWFS or FS can be used directly as initial potential to the subsequent layer's neurons, whereas a QANN's output would require conversion into spikes to interface with an SNN pipeline.

  • Justification for only using FS neurons as baseline. We selected FS neurons because they directly address nonlinear-function approximation and have a documented implementation pathway on neuromorphic platforms (Stöckl & Maass, 2021). They therefore provide a relevant reference for the computational setting studied here.

    The other cited approaches address different aspects of the problem or introduce different computational resources. Zheng et al. (2026) do not directly address nonlinear-function approximation, while the multi-threshold or multi-neuron constructions of Chen et al. (2026) and Wang et al. (2026) change the resources and operations available to represent each function. A comparison with these methods would need to account explicitly for neuron count, threshold and state storage, and computation steps, alongside approximation accuracy.

    Our theoretical analysis provides guarantees for both FS and PWFS under the assumptions stated in the paper. The framework may extend to other neuron constructions, although establishing corresponding guarantees would require verifying these assumptions and accounting for differences in their computational structure and resource requirements. We acknowledge that our empirical comparison is limited to FS; broader evaluation under explicitly specified resource budgets remains an important direction for future work.

Thank you for your responses. I remain positive in my assessment (now with more confidence) of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.