- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SpikeInpaint, a hybrid ANN-SNN inpainting generator built on an FFC encoder-decoder trained with a GAN objective. Spiking computation is used only in the low-resolution bottleneck. It includes: four FFC ResBlocks whose local and global activations are replaced by TriHardLIF, a single-timestep ternary hard-reset neuron; two SNN Bottleneck modules that compress channels before spiking; a channel-wise gate that blends ANN and ternary features in the decoder.
Critical strengths
Detailed, honest, well-scoped reporting. The paper presents that FID/LPIPS lag behind dense baselines, that sparsity gives no wall-clock speedup on GPUs, and that the 19.3% energy figure is an upper bound. It also includes useful per-branch firing analyses, that makes the actual claim (low PSNR cost of sparsifying the bottleneck relative to a matched dense model) easy to evaluate, although it does not by itself justify acceptance.
Critical weaknesses
-
The core contribution amounts to ternary activation quantization. Under the single-timestep setting with a reset after every iteration, the neuron keeps no state. TriHardLIF therefore reduces to a simple ternary threshold function: the leak and the reset rule have no effect, and the stated train/inference-mismatch motivation does not apply. Ternary spikes and SNN+FFC hybrids have also been explored before [1, 2]. Without a comparison to an equally quantized ANN, it is unclear what the spiking formulation adds.
-
The results do not show practical value. Reconstruction quality is clearly below the dense baselines, especially on the perceptual metrics (FID/LPIPS) and on CelebA-HQ. The small PSNR gap is measured against a weak all-ANN reference, and perceptual quality still degrades noticeably after sparsification. The efficiency gains are marginal: most computation remains dense.
[1]Guo, Y., Chen, Y., Liu, X., Peng, W., Zhang, Y., Huang, X., & Ma, Z. (2024, March). Ternary spike: Learning ternary spikes for spiking neural networks. In Proceedings of the AAAI conference on artificial intelligence (Vol. 38, No. 11, pp. 12244-12252). [2]Miao, W., Shen, J., Xu, Q., Hamalainen, T., Xu, Y., & Cong, F. (2025, April). Spikingyolox: Improved yolox object detection with fast fourier convolution and spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 2, pp. 1465-1473).
Questions for authors
- At T=1 with a per-iteration reset, do and the reset rule have any effect on the forward pass or the gradients? If not, how does TriHardLIF differ from a ternary threshold activation trained with a surrogate gradient?
- How does "Signed IF" differ from TriHardLIF at T=1? If the two are functionally equivalent, the 0.36 dB gap suggests that run-to-run variance exceeds the 0.09 dB claim.
- Table 3 reports 3.65T operations per image, yet the measured latency is 27.96 ms on an RTX 4070. Please clarify how operations are counted, since the two numbers appear inconsistent with the GPU's peak throughput.
7 comments
Q2 SignedIF and TriHardLIF are not equivalent under the submitted T=1 execution. SignedIF is designed to support signed ternary responses through an internal sign state. Under the submitted execution, this state begins inactive and is reset after each emitted output. At initialization, this state is inactive. Starting from v[0]=0, the state-dependent charge correction is zero and v[1]=x[1]. The first emitted output is:
s[1] = H(x[1]-1),
where H is the Heaviside forward function used by the spiking surrogate. Since the sign state has not yet been advanced by an earlier firing event, the negative branch is unavailable and the output takes values in {0,+1}. The corresponding surrogate gradient is:
∂s/∂x = g'(x-1),
where g' is the derivative of the arctan surrogate. The reset changes the stored state after the output, so it does not alter s[1] or its gradient at this timestep.
TriHardLIF uses a symmetric, simultaneous threshold. With v[0]=0 and threshold θ=1.5 in the FFC blocks, it emits:
s[1] = sgn(x[1])·1{|x[1]|>1.5},
which takes values in {-1,0,+1} within the first timestep. Its surrogate path uses both thresholds:
∂s/∂x = g'(x-1.5) + g'(-x-1.5).
SignedIF is therefore designed to represent {-1,0,+1}, but its negative response requires state from an earlier event. At T=1 from the initial state, it acts as a positive-only binary IF activation and can emit only 0 or +1. TriHardLIF immediately applies both signs and emits a ternary output. The two also use different thresholds and different surrogate-gradient expressions.
The 0.36dB difference is not a residual between equivalent operations and cannot be used to estimate run-to-run variation. It compares two single-step activation functions with different thresholds in separately trained models, so it isolates neither the neuron definition nor seed noise. The 0.09dB result is likewise a matched single-run All-ANN comparison, not a seed-variance estimate.
Q3 The reviewer is correct that the submitted operation count and latency cannot be reconciled. We traced the discrepancy to two errors in the submitted version. The 3.65T value came from a static counter that walked the modules in definition order and reused stale shape information across residual and FFC branches. It overcounted the FFC blocks by 16$\times$. We recounted the released checkpoint with forward hooks that read the actual input and output shapes of every convolution and linear layer, covering 69 calls per 512$\times$512 image. The corrected total is 351.5G MAC-equivalent operations per image. Of these, 307.7G are in the dense ANN path (87.5%) and 43.8G are inside the spiking modules (12.5%). The count reports arithmetic opportunities before spike skipping: 21.9G spiking-module operations receive spike tensors and are eligible for AC execution (6.2% of all operations), while the measured 4.11% firing rate corresponds to approximately 0.90G executed AC-equivalents. FFT and inverse FFT, complex spectral multiplication, normalization, and the remaining dense projections remain floating-point or dense. The 27.96ms value came from the legacy process-level CPU timing path, which did not synchronize CUDA and therefore did not measure end-to-end GPU latency. We withdraw 3.65T and 27.96ms as paired hardware-throughput evidence. We re-measured the same 2000 image-mask pairs with a synchronized wall-clock protocol in two passes using forward and reverse image order. Each pass used batch size 1 and 200 excluded warmup forwards. Across the two passes, SpikeInpaint has a mean latency of 63.581ms, while the matched all-ANN generator has a mean latency of 31.084ms. SpikeInpaint therefore incurs an additional 32.497ms, corresponding to approximately 2.045$\times$ the latency of its matched dense counterpart. At the corrected 351.5G count, the SpikeInpaint timing implies approximately 5.53TMAC/s, or 11.06TFLOP/s if one MAC is counted as two FLOPs. This is about 38% of the RTX 4070 FP32 peak and is consistent with a batch-size-1 implementation that also executes FFTs, normalization, nonlinearities, and memory transfers outside the MAC count. The timing result is a consistency check and a hardware measurement; it does not imply that the sparse arithmetic produces a GPU speedup. These corrections also invalidate the submitted 19.3% energy estimate, which was computed from the inflated count. With the corrected operation graph, the compute-only bounds are 10.0% if all 43.8G spiking-module operations are charged as ACs, 5.0% if all 21.9G eligible operations are charged as ACs, and 0.2% when the measured 4.11% firing rate is applied. These values exclude memory traffic, buffers, the NoC, and the floating-point FFT; they are not measured system energy. The revision will report measured latency, operation counts, and compute-only estimates separately. The old 27.96ms, 3.65T, 19.3%, and their derived speedup claims will be marked as superseded and retained only in an audit note. The revised paper will not claim a GPU speedup or measured system-energy saving.
We will correct the SignedIF distinction, operation count, latency and compute-only energy table in the revision, and remove the claims that the corrected results do not support.
We appreciate the precious comments. Here are our comments.
Q1 For the submitted T=1 configuration, neither the leak term nor the reset changes the forward output or the surrogate-gradient path. Eq. (1) gives v[1] = x[1], and the hard reset runs after the only output has already been emitted. The deployed operator is therefore:
s[1] = sgn(v[1]) · 1{|v[1]| > θ},
where 1{·} denotes the indicator function. This is a signed ternary threshold with a silent state. We replaced every TriHardLIF membrane forward in the released checkpoint with:
f(x) = sgn(x) · 1{|x| > θ}
and compared both implementations on a 512$\times$512 Places2 image and its paired mask. The full generator outputs were bit-identical, with maximum and mean absolute differences of 0.0. We therefore make no claim that temporal neuron dynamics account for the reported performance. Within SpikeInpaint, TriHardLIF is a signed ternary activation, and its role is defined at the architecture level by where the signed ternary representation is placed, how it is compressed, and how it is fused with the dense path.
To determine whether the learned ternary mapping can be replaced by an ordinary quantizer or sparsifier at inference, we applied two post-training controls at every TriHardLIF location. INT4 uses symmetric per-image activation quantization. Sparsification preserves the largest activation magnitudes and calibrates each module's threshold to its measured firing rate. Across the same image-mask pairs, INT4 lowers PSNR by 6.52dB and SSIM by 0.0701 relative to TriHardLIF, while sparsification lowers PSNR by 8.93dB and SSIM by 0.1421. These controls use one released checkpoint and are not retrained equal-quantized models. They provide a post-training diagnostic of replacing the learned ternary mapping at inference. They do not establish a retrained equal-quantized baseline, and the architectural claim does not rely on a temporal-neuron advantage.
The corrected operation breakdown separates spike sparsity from the original operation-count estimate. Only 6.2% of all operations are eligible for accumulate execution, and applying the measured firing rate leaves approximately 0.90G executed AC-equivalents. The original 19.3% value used the uncorrected module-level count; for the corrected operation graph, the event-aware compute-only estimate is 0.2%. Most operations remain dense, and the FFT, inverse FFT, complex spectral multiplication, normalization, gate MLP, and dense projections remain floating-point or dense. Memory traffic, buffers, and interconnect are not included in this arithmetic estimate.
The matched latency measurement shows that this sparsity does not produce a GPU advantage in the current implementation. Under the synchronized two-pass protocol, SpikeInpaint has a mean latency of 63.581ms and the matched All-ANN generator has a mean latency of 31.084ms. SpikeInpaint is therefore approximately 2.045× slower. The matched quality comparison changes PSNR from 24.50dB to 24.41dB, SSIM from 0.8261 to 0.8222, FID from 10.26 to 12.41, and LPIPS from 0.1718 to 0.1802. We therefore report efficiency only as a compute-level boundary and make no GPU-speedup or measured-system-energy claim. The supported contribution is the architecture-level placement of a sparse signed ternary representation and the measured quality-sparsity cost of bottleneck ternarization. It is not a new temporal neuron, a spiking-specific quality advantage, or a measured system-energy saving.