- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper the authors propose a method for improving the accuracy of object detections in RGB video by pairing RGB frames with event camera data. This way there is no "lag" between predictions and the next frame. The authors show significant improvements in detection accuracy this way. The approach works by using events to predict how features extracted from the previous RGB frame would look in the next. To train this the authors use two-stage disdtallation approach, whereby a teacher network has access to the "next" RGB frame and the student has access only to the previous frame and the events. The paper also introduces a synthetic dataset that is used for training and benchmarking the reuslts.
Critical strengths
- Paper is resonably clear and easy to understand.
- Qualitative results show large improvements w.r.t compared methods.
- I appreciate the fact that the authors did not produce AI slop in this paper, which is sadly rather common nowadays.
- Extensive ablation studies are great,
- The promised synthetic dataset could be quite useful to others.
Critical weaknesses
- How is this paper fundamentally different from event camera + RGB temporal super resolution? I image I could just run such a model to upsample RGB frames temporally, then get my predictions on that higher frequency stream.
- The authors use 5x temporal bins per polarity, like is done in RVT. It’s not really clear how sensitive the proposed latent feature prediction is to this discretization. To me, this is just basically fusing a low framerate RGB camera with a slightly higher framerate event image camera. The authors say there are two approaches for dealing with events, sparse operations (e.g SNNs) and binning. But there are also approaches for image reconstruction where an image state is maintained and updated on a per event basis (e.g see Continuous-time intensity estimation using event cameras). Why not use a continuous time image reconstruction to generate these features rather than binning? That way the authors would eliminate motion blur and not need to choose an arbitrary binning rate. I think this would be interesting to compare and not too difficult to implement. Also a plot of how performance changes with number of bins.
- Figure 2 is confusing - the figure shows just two frames and then says it is running at 25x. Not sure what is going on. I think it is not a bad figure but could be clearer about the relationship between vent representation and the operating frequency.
- The proposed architecture uses an RF-DETR-L DINOv2 image encoder with YOLO26 components. Several compared methods have substantially different backbones, pretraining and training procedures. So the contribution of the temporal feature prediction is somewhat entangled with the benefits of distillation. e.g the no-adapter baseline is evaluated only without frozen distillation. A no-adapter baseline trained with frozen teacher supervision would show whether the improvements come form the event sensor latent feature prediction rather than the additional supervision signal. Does that make sense?
Questions for authors
Nits:
-
There are some instances of what I call “gratuitous math”, i.e equations that neither help explain, prove or motivate the work. What is the point of such formalism other than to confuse the reader and make the work seem more complex? I would recommend:
- Remove Eq(1). EDI is not implemented or solved in this paper at all. Just say on one sentence “Events encode intensity changes over time. The high temporal resolution can help update features extracted from an earlier RGB image” or something.
- Eq 11 is a bit trivial no?
- The rest is up to the authors, but I don’t think all of that is needed to understand or reproduce.
-
Why do the authors use ESIM as their backbone simulator? Since this paper, there have been significant improvements in video-to-event simulation. It’s fine, but a sentence to motivate the use of this particular simulator would be nice.
4 comments
Dear Reviewer 1xXr,
Thank you again for your insightful comments. We have responded to each of your points and included the suggested comparisons in our response and the revised paper. If you have any remaining questions or concerns, we would very much appreciate hearing them and would be happy to discuss further. Thank you for your time and consideration.
Best regards, Authors
Rebuttal Reply
Dear authors - thank you for your replies. Fair enough - given what I have learned here I would be happy to update my recommendation to a "Weak Accept". Best of luck!
Dear Reviewer 1xXr,
Thank you for your follow-up and for updating your recommendation. Your suggestions have helped improve the paper, and we sincerely appreciate your time.
We thank the reviewer for the constructive feedback. We have added the comparison with continuous-time image reconstruction and the ablations on event binning and distillation, and revised the paper and Figure 2 as suggested below.
1. Difference from event camera + RGB temporal super-resolution (W1). Event-guided temporal super-resolution (e.g., TimeLens) interpolates between the previous and next RGB frames, but the next frame is unavailable at query time. More fundamentally, any pixel-space upsampling must re-run the image encoder at every query, which nearly doubles the per-query latency on a T4 even before counting the reconstruction time Trecon. PRED encodes each RGB frame once and updates representations in feature space with the light event pathway.
Per-query latency on a Tesla T4 (TensorRT FP16, 320×224).
2. Comparison with continuous-time image reconstruction (W1, W2). We ran the suggested comparison: we reconstructed the current frame from the previous frame and events with the cited continuous-time intensity estimation method (complementary filter, tuned on validation), and evaluated it with our frozen current-image teacher and with a student without the adapter trained on reconstructed frames. In both cases, the reconstructed frame brings no clear gain over the previous frame It0, and all remain below PRED-FD.
Detection on DSEC-Det with complementary-filter reconstructions.
3. Sensitivity to temporal event binning (W2). The event tensor is not a fixed-rate event frame: each query builds it from all events since the last RGB frame, so queries can be issued at any time. Accuracy is insensitive to the discretization between 3 and 10 bins per polarity and drops slightly only with a single bin, which discards the temporal order of events.
Number of temporal bins per polarity on DSEC-Det (PRED-FD).
4. Figure 2 (W3). Thank you for pointing this out. The two RGB frames in Figure 2 are consecutive 20-Hz frames, the two event images mark the same time points with the continuous event stream between them, and the ticks in between mark query times, at which the current frame is unavailable. "25×" referred to queries at 500 Hz, 25 times the RGB rate. We have revised the figure to read "queries up to 500 Hz" and noted in the caption that each query bins all events since the last RGB frame, so the query rate is independent of both the RGB rate and the event representation.
5. Feature prediction vs. extra supervision (W4). Yes, this makes sense; thank you for suggesting this control, which makes the role of each component clearer. We trained the student without the adapter under the same frozen-teacher supervision (now in Table 5). Supervision alone brings some improvement, while adding the adapter for feature prediction increases the gain substantially.
Adapter vs. frozen-teacher distillation on DSEC-Det.
6. Equations (Nits). We agree. We have removed Eq. (1), keeping the EDI motivation as one sentence in the problem formulation, and replaced Eq. (11) with a one-sentence definition of recovery.
7. Choice of ESIM (Nits). We follow "Event Quality Score (EQS): Assessing the Realism of Simulated Event Camera Streams via Distance in Latent Space" (Chanda et al., CVPRW 2025), which compared event simulators against real DSEC data and found ESIM closest to the real events (EQS 0.866 vs. 0.773 for v2e and 0.501 for PIX2NVS). We have added this motivation to Section 4, and we will explore newer video-to-event simulators in future work.