acceptodds
Under review as a conference paper at ICLR 2027

SpikeUAV-DETR: Energy-Efficient End-to-End Spike-Driven Object Detection for Unmanned Aerial Vehicle Imagery

Abstract

Onboard detection for unmanned aerial vehicles (UAVs) can provide reliable, real-time detection results when network connectivity is limited. However, limited onboard resources require detectors to balance accuracy and energy consumption. Detectors based on spiking neural networks (SNNs) exploit spike-driven computation, resulting in lower energy consumption and making them promising for UAV applications. However, most real-time SNN detectors retain non-maximum suppression (NMS), which requires extensive tuning. Moreover, the discrete spike encoding inherent to SNNs leads to information loss and accuracy degradation. To address these challenges, we propose SpikeUAV-DETR, an end-to-end spike-driven detector that enables NMS-free inference. First, we employ time-channel adaptive firing thresholds that adjust to membrane-potential distributions, thereby improving spike information transmission. Second, we introduce high-resolution cross-level feature fusion to preserve fine-grained details and lightweight gating to inject high-level semantics, thereby enhancing small object detection. Third, we employ hybrid query selection and deformable attention modules to preserve precision with limited dense computation. Experimental results on two UAV imagery datasets show that SpikeUAV-DETR achieves a favorable accuracy-energy trade-off over representative detectors while retaining real-time inference, specifically improving mAP@50 by 1.1 percentage points over the ANN baseline RT-DETR-R18 on VisDrone2019-DET while reducing energy consumption by 52.4%.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Unmanned aerial vehicles, Spiking Neural Networks, End-to-End Object Detection, Spiking neuron, Energy-Efficient

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HHVZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SpikeUAV-DETR, an end-to-end spiking detector for object detection in UAV imagery. Built on RT-DETR, the method introduces temporal–channel adaptive multi-level integrate-and-fire neurons (AT-LIMF), high-resolution cross-layer feature fusion, and gated semantic enhancement modules (S2FD/CSGF). It also adopts SNN–ANN hybrid structures for query selection and deformable attention. The method is evaluated on the VisDrone2019-DET and UAVVaste datasets, with detection accuracy, theoretical energy consumption, and inference speed reported, along with component ablation studies and NMS-threshold sensitivity analysis. In the experiments, the T=3 configuration achieves 46.0% mAP@50 on VisDrone2019-DET, and 49.0% mAP@50:95 and 80.4% mAP@50 on UAVVaste, with an inference speed of 65 FPS.

Critical strengths
  1. The paper reports detection accuracy, theoretical energy consumption, and inference speed on two UAV image datasets, and compares the method with ANN and SNN detectors, providing quantitative results across multiple performance metrics.

  2. The paper uses AT-LIMF, S2FD/CSGF, and SNN–ANN hybrid modules to address spike information encoding, fine-grained feature preservation, and accuracy-sensitive operations, respectively. Ablation experiments further quantify the effects of each module on mAP, energy consumption, parameter count, and FPS.

Critical weaknesses
  1. The proposed method does not appear to be spike-driven. As shown in Fig. 2, CSGF contains a Sigmoid activation function. Moreover, the use of ANN–SNN hybrid modules contradicts the claim that the method is spike-driven. To my knowledge, Deformable Attention also involves floating-point operations arising from bilinear interpolation.
  2. Does each block in Fig. 2 have an AT-LIMF module before it? If so, the annotation below Fig. 2 is problematic. If not, then the network is fully ANN-based.
  3. Since the proposed method is a hybrid network, I believe its energy consumption cannot be calculated simply by following the practices used in other SNN papers. ANN and SNN components cannot run simultaneously on a chip supporting spike-driven computation; otherwise, the efficiency of the SNN would be substantially compromised. Therefore, the proposed method would need to run its SNN component on a neuromorphic chip and its ANN component on a conventional GPU or another edge-computing device, which would introduce additional I/O costs. The authors should include these costs in their inference-cost analysis.
Questions for authors

See Weakness, if authors can resolve my concerns or correct my misunderstandings, I will revise my score.

1 comment

Dear authors, the AC has reminded us reviewers to respond to your rebuttal. If there is anything you would like to clarify or discuss, please leave a comment here, and I will respond as soon as possible.

Reviewer_EAAGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents SpikeUAV-DETR, a spike-driven end-to-end object detector for UAV imagery built on RT-DETR, aiming to combine NMS-free set prediction with low energy consumption. Its components are: (1) AT-LIMF, a multi-level firing neuron whose threshold ladder is scaled by a learned time-channel matrix (T x C parameters per layer), adapting quantization granularity and range to membrane-potential distributions while preserving lossless binary decomposition at inference; (2) encoder changes that bring the stride-4 S2 features into cross-scale fusion (S2FD, with channel-expanded downsampling) and inject high-level semantics through lightweight gating (CSGF); (3) SNN-ANN hybrid designs in which spike-driven blocks perform dense transformations (value projection, spiking MLP) while ANN branches handle precision-sensitive operations (class scores, sampling offsets, attention weights, deformable sampling). On VisDrone2019-DET, the T=2 and T=3 configurations reach 45.6 and 46.0 mAP@50 at 65.7 and 93.8 mJ theoretical energy, exceeding RT-DETR-R18 (44.5) and using 52.4-32% less energy, while measured FPS on an H800 is 87.5/65.0. On UAVVaste, T=3 achieves the best mAP@50:95 (49.0) among compared ANN and SNN detectors. Ablations attribute +0.6 to AT-LIMF, +5.3 to S2FD+CSGF, and +1.4 to the hybrid query selection and attention.

Critical strengths
  1. The combination is well chosen for the gap it identifies: prior real-time SNN detectors retain NMS while Spiking-DETR is computationally heavy, and this work delivers the first real-time end-to-end spike-driven NMS-free detector for UAV imagery. The Appendix E study quantifying NMS threshold sensitivity and latency fluctuation substantiates the motivation rather than assuming it.
  2. Efficiency reporting is more honest than is typical for SNN detection papers: measured GPU FPS is reported alongside theoretical energy, the 42.3% slowdown relative to RT-DETR-R18 is stated plainly, the hybrid MAC/SOP energy formulation is derived module by module, and the energy-accuracy Pareto analysis locates both configurations explicitly.
  3. The SNN-ANN allocation is principled and its accounting is credible: dense transformations are spike-driven while precision-sensitive operations stay ANN, and the hybrid design adds only 0.009M parameters while reducing energy by 13.2% in the ablation.
Critical weaknesses
  1. Energy remains a 45 nm theoretical estimate, and neuromorphic deployment is deferred to future work. The measured FPS reflects simulated spiking on an H800 rather than spike-driven execution, and the hybrid design leaves a substantial ANN portion (3.2-3.7 GFLOPs) alongside 64.7-94.9 GSOPs. The headline 52.4% energy saving is computed against RT-DETR-R18 specifically.
  2. Baseline comparability in Table 2 is loose: several entries are quoted reproductions from other studies (footnote a), evaluated at different input resolutions (QueryDet at 2400 x 2400, original DETR at 1333 x 750) and different training recipes, while all VisDrone results are on the 548-image validation set. Cross-resolution comparison particularly affects the small-object claims, which are central to the paper.
  3. Component attribution is uneven: S2FD+CSGF contributes +5.3 of the total +7.3 mAP@50 while AT-LIMF, the most distinctive SNN component, contributes +0.6. The benefit of AT-LIMF over simpler adaptive-threshold alternatives (per-layer learnable thresholds, time-only or channel-only scaling, or I-LIF with per-layer calibration) is not ablated, so the neuron-level contribution is not isolated.
  4. The temporal operating point is under-explored: only T=2 and T=3 are reported, although energy scales with T and the threshold ladder parameters (n=2, m=3, D=6) are fixed throughout; a sweep over T and the ladder configuration is needed to justify the selected operating points.
Questions for authors
  1. Ablate AT-LIMF against simpler adaptive-threshold variants (per-layer shared scale, time-only, channel-only, and a recalibrated fixed LIMF) under identical training. How much of the +0.6 mAP@50 specifically requires time-channel adaptation?
  2. What fraction of the 93.8 mJ at T=3 comes from ANN MACs versus spike-driven SOPs, and how would the hybrid modules (ANN offset/weight heads, softmax, query selection, deformable sampling) map onto current neuromorphic platforms that lack efficient dense MAC support?
  3. Can you provide a comparison at matched input resolution (e.g., all methods at 640 x 640, or yours at the baselines' native resolutions), and ideally results on the VisDrone test-dev server rather than the validation set?
Reviewer_Bg9jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SpikeUAV-DETR for UAV object detection. The method is based on spiking neural networks and DETR. It uses adaptive firing thresholds, high-resolution feature fusion, and hybrid SNN-ANN modules. The goal is to improve detection accuracy while reducing energy use. The method is tested on VisDrone2019-DET and UAVVaste.

Critical strengths
  1. The paper studies a practical problem. UAV detection needs good accuracy, low delay, and low energy use. The motivation of this paper is clear.
  2. The method shows a good accuracy–energy trade-off. SpikeUAV-DETR gets competitive detection results while using lower estimated energy than several ANN-based methods.
Critical weaknesses
  1. The energy result is based on estimation, not real hardware measurement. The paper estimates energy from FLOPs, SOPs, and fixed MAC/AC energy values. It does not measure the real energy use on UAV hardware or neuromorphic hardware. Since energy efficiency is a main claim, this makes the result less convincing.
  2. The main novelty is not very clear. The method combines several parts, including adaptive thresholds, feature fusion, gating, query selection, and hybrid SNN-ANN attention. Many of these ideas are related to existing designs. Moreover, the compared methods use different backbones, input sizes, model designs, and training settings. Some results are also taken from previous work. Because of this, it is hard to know how much of the gain comes from the proposed method itself.
Questions for authors
  1. Can the authors provide real energy measurements on UAV hardware or neuromorphic hardware?
  2. What is the main new idea of this paper compared with existing SNN detectors?
  3. Can the authors provide a more controlled comparison using the same backbone, input size, and training setting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.