- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses out-of-distribution (corruption) detection for SNN-based event-camera perception under low-compute constraints. It observes that subthreshold membrane potentials Vmem, already computed by a spiking front end, carry a usable corruption signal, and builds two descriptors: a pooled per-channel moment descriptor (mean, variance, excess kurtosis) and a spatial descriptor (dispersion, local flatness, persistence). The Multi-Descriptor Deviation (MDD) detector combines a PCA-residual branch, a deep-layer Mahalanobis branch, and a spatial branch via calibrated maximum fusion, fitted only on clean data. The authors also release Gen1-C, a six-family, five-severity corruption benchmark on Prophesee Gen1. On a hybrid SNN-ANN detector, MDD reaches AUROC above 0.88 on five of six corruptions at the highest severity using a 3.2 s observation window, outperforming an adapted suite of post-hoc OOD baselines (mean AUROC 0.809 vs. 0.624 for the next best), while polarity-flip detection remains near chance. An MCU implementation runs within the 50 ms frame budget on physical M4F boards with modest AUROC loss, and transfer to Gen4 and SpikeYOLO is reported.
Critical strengths
- The problem setting is real and the core idea is economical: reading membrane states adds no forward/backward pass to the host network, which is exactly the constraint that makes standard OOD methods unusable on spiking detectors. This is a sensible and previously unexploited internal signal for this setting.
- The evaluation protocol is unusually disciplined: corruption-blind fitting with a separate sensitivity split not used for selection, a matched baseline suite under identical sequences and calibration, cross-dataset (Gen4) and cross-model (SpikeYOLO) runs, and measured per-frame latency on five physical MCU boards rather than claimed efficiency.
- The treatment of failure is exemplary: the polarity-flip failure is decomposed into mixture dilution, moment cancellation, and an explicit descriptor-collision witness (Appendix F), and all limitations are stated rather than hidden. Gen1-C itself is a reusable contribution for event-camera robustness research.
Critical weaknesses
- The "corruption-blind" property holds only at the calibration stage: the descriptors themselves (local flatness, persistence, dispersion) are evidently designed with these corruption families in mind, and no experiment tests a corruption family that was unknown during design. The stated motivation for clean-only fitting — robustness to corruption types beyond the suite — is therefore asserted but never evaluated, e.g., via a leave-one-family-out design.
- The signal's effectiveness is host- and pipeline-specific. Headline results use one hybrid host; on Gen4 and SpikeYOLO the corruption ranking changes substantially and absolute AUROC is much weaker (e.g., event flood 0.670 and spatial dropout 0.450-0.770 across severities on SpikeYOLO). Combined with near-chance polarity-flip detection and weak low-severity performance on several families (rate shift 0.519, flood 0.556 at S1), the practical monitoring envelope is narrower than the abstract suggests.
- The matched baseline suite compares only network-derived features (membrane-kNN, spike-count-kNN, DIME, PatchCore, etc.). A monitor built directly on input-space event statistics (event rate, spatial support, ON/OFF asymmetry) requires no network at all and would show how much detection power is unique to membrane dynamics rather than trivially available from the raw histogram; this control is missing.
- Presentation needs editing: Section 3.3.3 contains a duplicated sentence, Appendix A.3-A.5 explicitly disclaim that the analysis establishes nothing about the detector while the main text cites these results as motivation, and there are typos (e.g., "failiure" in Section 5).
Questions for authors
- Event flood at S5 collapses detection mAP to 0.003 within a few frames, yet fused per-frame AUROC is only 0.551 and the deployable operating point needs a 3.2 s window. What is the detection-delay distribution for acute failures, and would a two-tier scheme (fast input-space alarm plus slow membrane monitor) cover both acute and sustained degradation?
- How does MDD compare, under the same protocol, with a corruption-blind monitor built directly on raw event-histogram statistics? This would isolate how much of the performance comes from membrane dynamics versus information already present in the input.
- Can you evaluate MDD on a corruption family excluded from the descriptor design process (e.g., v2e-style sensor artifacts such as contrast-threshold mismatch or bandwidth limiting) to substantiate the claim that the clean-only fitting generalizes beyond the suite?
- Table 10 shows event flood reaching 1.000 AUROC at full-recording aggregation versus 0.892 at W=64. What aggregation length would you actually recommend for a given alarm-latency budget, and how should a practitioner pick W without corruption labels?
4 comments
Thanks - Reviewer NngW
Dear Reviewer NngW,
As the discussion period is about to end in less than a day, we hope our responses have adequately addressed your concerns. Please let us know if you have further questions; we look forward to the discussion.
Thank you again for your valuable time and consideration.
Best Regards, The Authors
I thank the authors for a thorough and responsive rebuttal. The three additions directly resolve my main concerns: frozen MDD detects unseen v2e-inspired corruptions that cause real task degradation (background counts .929, packet loss .920 AUROC) without any refitting, the input-statistic control confirms that membrane features carry detection power beyond raw event statistics (mean S5 AUROC .877 vs. .792), and the two-tier alarm covers all tested acute floods at 50 ms delay with quantified false-alarm costs. The window-selection guidance and the presentation fixes are also appreciated. The remaining limitations — readout-throughput thinning and polarity flip — are now explicitly scoped rather than hidden, and are documented in the paper itself. My concerns about generalization and the missing control are resolved; I raise my rating to 4.
Thanks - Reviewer NngW
Thank you for your careful consideration of our responses and additional experiments, and for updating your rating. We appreciate your constructive feedback throughout the discussion, which helped us strengthen and clarify the paper.
Best Regards, The Authors
Response to Reviewer NngW
Thank you for your insightful review and helpful suggestions. All section references below correspond to the revised manuscript (where changes are highlighted in blue).
Frozen MDD on additional corruptions
We tested v2e-inspired histogram perturbations plus readout and packet-loss stress tests. Host weights, descriptors, MDD parameters, calibration and score directions remained fixed. On seven matched 256-frame prefixes, clean mAP was 35.03%. At S5:
At S4, both background counts and packet loss cause at least five points of mAP loss:
Frozen MDD detects harmful background noise and packet loss beyond Gen1-C. Throughput thinning is not reliably detected; threshold mismatch and refractory caps cause little task degradation at the tested settings.
These are v2e-inspired histogram tests, not runs of v2e.
Dependence on the host model and pipeline
Section 4.3 and Appendix D discuss how Gen4 and SpikeYOLO change performance and corruption rankings. Section 5 retains the polarity-flip and low-severity limitations. These results identify the conditions MDD monitors effectively and those needing additional signals.
Comparison with an input-only monitor
Section 4.5 adds the input-statistic control with the same MDD scoring method, separate clean-data fitting and the same test protocol. At W=64:
Membranes lead overall; inputs lead on polarity flip. Section 4.5 includes spikes, random membranes and ANN features. Input/membrane descriptor dimensions differ; Appendix C.5 tests cross-frame PLIF state separately.
Acute failures and two-tier alarms
Section 4.7 and Appendix C.7 add an input alarm (W=1), a trailing-64-frame membrane alarm and their joint OR alarm. Monitors and thresholds remain frozen; thresholds use separate clean calibration recordings with a 1% alarm-frame budget. Testing uses 50 additional complete held-out recordings disjoint from fitting and calibration.
Coverage includes warnings active before onset. For new warnings, median/P90 observation delays are both 50 ms, excluding compute and communication. The fast path covers all tested S5 floods, at a higher false-alarm rate; held-out rates can exceed the calibration target.
A separate 12-recording sustained-corruption test found the combined alarm active on 100% of hot-pixel frames and 38.1% of rate-shift frames, versus 94.5% and 34.0% for membranes. It is a tradeoff between false alarms and detection.
Choosing the averaging window without corruption labels
Choose the smallest W that meets the desired clean false-alarm rate within the allowed averaging time. Calibrate thresholds on clean calibration recordings and select W on separate clean validation recordings, without corruption labels. At 50 ms per frame, W=64 spans 3.2 s and W=128 spans 6.4 s; these are averaging spans, not measured alarm delays. Full-recording aggregation is an offline diagnostic.
Window averaging mainly reduces score noise; persistence separately tracks recording history. We added this practical selection guidance in Section 4.7.
Presentation and interpretation of Appendix A
We removed duplicate text and corrected typos.
Thank you again. Please let us know if you have further questions; we look forward to the discussion.