- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces CF-TS-SNN, an optical-flow-free two-stream spiking network for action recognition. The temporal pathway processes chronologically sampled video frames or event windows, while the spatial pathway repeatedly encodes a center frame or a full-clip event summary. The pathways interact through intermediate addition and final fusion. Spatial features also generate temporal, channel, and spatial attention weights and regulate neuronal thresholds in the final temporal stage. At 20 time steps, the model achieves 98.15% accuracy on DVS128Gesture and 94.05% on UCF-50. The paper provides fusion-operator comparisons and component ablations, but the experimental protocol, external baseline comparisons, and computational cost analysis remain insufficient to fully establish its contribution.
Critical strengths
The paper presents a clear two-stream design motivated by the complementarity of persistent spatial context and dynamic temporal information, and evaluates the same framework on both event and RGB inputs. Structurally matched spiking backbones facilitate interactions between corresponding stages, and the method avoids additional optical-flow preprocessing. Experiments compare fusion operators, preservation of an independent spatial output, and components of attention and threshold regulation, providing initial support for the design choices. The complete model improves over the temporal-only baseline by 1.25 and 7.75 percentage points on the two datasets, respectively. The larger UCF-50 improvement suggests that spatial context can provide useful complementary information for temporal representations.
Critical weaknesses
Comment 1: The experimental protocol and implementation details are insufficient for reproduction and reliable comparison. Section 4.1 mainly specifies the datasets, time-step count, training hardware, and use of identical settings across variants. It does not clearly specify training, validation, and test splits, optimizer, learning rate, training epochs, batch size, or augmentation. The splitting unit and clip-sampling procedure for UCF-50 are particularly important for assessing whether correlated clips from the same source video appear across splits. The attention mappings, threshold bounds, and basic neuronal parameters also require specification. Stating that settings are identical does not replace a complete experimental protocol.
Comment 2: Comparisons with related methods are limited and do not sufficiently establish competitiveness. The UCF-50 experiments primarily compare internal variants, without direct comparisons against SVFormer, ReSpike, or representative optical-flow-free video methods discussed in the related work. Table 6 on DVS128Gesture also mainly includes earlier methods and has limited coverage. The results establish improvements over the paper's own baselines, but do not adequately establish advantages over related approaches. Key external baselines should be added, with differences in pretraining, input length, and testing strategy explicitly documented.
Comment 3: Single-stream and two-stream comparisons lack matched capacity and computational budgets. The two pathways have independent parameters, so the two-stream model increases both representation capacity and computation relative to the temporal-only baseline. The current comparison does not exclude additional network capacity as a partial explanation for the gains. Although comparisons between simple two-stream fusion and the complete model support the fusion module, a single-stream baseline with matched parameter count or computation would strengthen the analysis. Parameter counts, operation counts, inference latency, and firing rates should also be reported to assess the cost of repeatedly encoding spatial inputs and processing both pathways.
Comment 4: The interpretation and necessity of threshold regulation need stronger evidence. Equation (19) combines activity, spatial entropy, and attention into a threshold multiplier, but the signs and relative scales of these terms are primarily justified through intuition. Removing individual terms supports their empirical usefulness, but does not distinguish context-dependent regulation from general threshold adjustment. Comparisons against tuned fixed thresholds, learnable thresholds, and shuffled spatial guidance maps would help isolate the proposed mechanism. Analysis of threshold distributions, clipping frequency, and firing-rate changes would also be useful. The paper should additionally explain how the threshold map generated from the final spatial feature is mapped to neuronal dimensions across layers within temporal Block 3.
Questions for authors
See Weaknesses.