- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an efficient method of pretraining an event encoder. The proposed method is straightforward: leveraging the aligned RGB-Event pairs, the method learns to distill the information from DINOv3 into a backbone specific to event streams. The pretraining is supervised by global cosine loss, patch cosine loss and patch InfoNCE loss. Experiments on several datasets demonstrate the representation ability of the resulting models under in-domain conditions, cross-dataset transferability, and cross-task generalization ability.
Critical strengths
- Using paired RGB event data to transfer representations from RGB foundation models to event encoders is a promising idea. The paired images provide strong prior information that could be used to improve efficiency. Future work can build on this idea to scale up the training of event foundation models with reduced computational overhead.
- The proposed method is simple but effective. Experiments on different datasets show significant improvements with the pretrained encoder under in-domain conditions (DSEC-Detection), cross-dataset transferability (GEN1), and cross-task generalization ability (DSEC-Seg).
- This paper is clearly structured and well-written.
Critical weaknesses
- Although experiments involve two different dense tasks, the scope of tasks validated was still limited, especially considering that training and evaluation were mainly based on DSEC dataset. For example, it remains unclear how the model performs on image recognition and depth estimation, which were covered in the GEP paper.
- The evaluation of the model’s generalization ability can be improved. Although GEN1 is a new dataset that has not been seen during pretraining, the object categories (car, pedestrian) still fall within the in-domain scope. Therefore, it is unclear how the pretrained model transfers to categories that have not been seen during pretraining.
- I am wondering how the proposed method performs compared to fusion-based methods, as the pretraining has gone through both modalities. Does the proposed method have an advantage even compared with fusion-based methods?
- The pretrained models can be a good contribution to the community, while it remains unclear if they will be released.
Questions for authors
None
2 comments
Thank you for your replies, the additional experiments on depth estimation look promising, I will keep my score.
Response to Reviewer NmU1
We sincerely thank the reviewer for the positive assessment of our work and for recognizing the effectiveness and efficiency of RGB-guided event representation learning. We appreciate the constructive suggestions and have conducted additional experiments to further strengthen the evaluation of our method and address the concerns raised.
W1. Extension to Additional Downstream Tasks
To evaluate whether our learned representations generalize beyond semantic perception tasks, we conducted additional monocular depth estimation experiments on DSEC-Depth and MVSEC-Depth. Unlike object detection and semantic segmentation, depth estimation requires geometric understanding, providing a complementary evaluation of the learned representations.
As shown in the table below, our method achieves the best performance across all three metrics on DSEC-Depth, and the best δ₁ and Abs Rel on MVSEC-Depth. Furthermore, compared with the same architecture without our proposed pretraining, our method consistently improves all three metrics on both datasets. These results further demonstrate the effectiveness and generalizability of our RGB-guided pre-training across different downstream tasks and datasets. We will also update the manuscript to include these additional depth estimation results and a detailed analysis.
W2. Generalization to Unseen Semantic Categories
We appreciate the reviewer's suggestion to evaluate generalization to unseen semantic categories. However, a direct evaluation remains challenging due to the limited availability of suitable event-based benchmarks. Specifically, we evaluate our method on three representative downstream tasks: object detection, semantic segmentation, and depth estimation. In event-based vision, established benchmarks for these tasks with high-quality annotations and standardized evaluation protocols remain limited in domain diversity, with most focusing on autonomous driving and road scenes. We therefore adopt these widely used benchmarks to ensure fair and direct comparisons with prior related work.
Meanwhile, the reviewer raises an important point that reflects a fundamental challenge in event-based vision. As discussed in our Introduction, event datasets remain substantially more limited in scale and diversity than RGB image datasets. This is precisely one of the key motivations for our work: leveraging the rich semantic knowledge of large-scale RGB foundation models to learn more transferable event representations with limited event data.
W3. Comparison with Fusion-Based Methods
RGB-event fusion methods and our approach operate under different inference settings. Fusion-based methods may achieve higher accuracy by combining complementary information from both modalities during inference, whereas all our reported downstream results are achieved using event inputs alone. Our research objective therefore differs from fusion-based approaches: we aim to utilize the rich semantic knowledge from RGB foundation models through RGB-guided pretraining to develop a more powerful event encoder, enabling downstream tasks to rely solely on event inputs without requiring paired RGB data.
Therefore, the key advantage of our design is modality independence during inference. When RGB observations are unavailable or unreliable, such as in low-light or high-speed scenarios, our model can still operate independently, whereas fusion-based methods that rely on both modalities may be limited. Moreover, training fusion-based models often requires substantial paired RGB-event data to achieve strong performance, yet such data remain scarce. In contrast, our approach can learn effective event representations with only limited paired RGB-event data by transferring rich semantic knowledge from RGB foundation models during pre-training.
W4. Release of Pretrained Models
Yes, we will publicly release our pretrained model checkpoints, training and evaluation code, and configurations to facilitate future research in event-based vision.