- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies paired image-event instance retrieval and asks whether a training-free verification step can fix the top-1 errors left by learned image-event aligners. Keeping each trained model and its top-five shortlist fixed, it reranks the shortlist by comparing hand-crafted responses, either a simulation of the benchmark's documented camera motion or static image gradients matched against event-count gradients, and it also fuses this response score with the original correspondence score. Experiments with three trained rankers on two display-recorded event benchmarks report gains from reranking, an advantage for static gradients on mini N-ImageNet, and further gains from fusion.
Critical strengths
-
The controlled protocol is the right way to ask this question. Freezing each checkpoint and its shortlist means every change comes from the response representation alone, and the paper reports per-seed and per-direction results, including the case where verification hurts.
-
A training-free comparison of image edges with event activity corrects many errors of CLIP-based aligners. It needs no training, and the rescue and damage accounting shows why fusion helps where response-only ranking does not.
Critical weaknesses
-
The evidence does not reach beyond display-recorded benchmarks, and the verifier is built from those benchmarks' acquisition protocols. Both datasets were made by showing an image on a monitor and moving an event camera along a documented path. The N-Caltech101 response replays that saccade path with a training-calibrated raster scale (Eq. 51 to 54), and the mini N-ImageNet response uses a display transform and translation extent fitted on training pairs (Eq. 66). Here the event stream is close to a deterministic function of the displayed image, so rendering the image, or just taking its gradients, should match the events well. The title and abstract speak about image-event retrieval in general, and natural recordings are left to future work. I am not convinced the takeaway survives outside this regime. Supporting the general framing needs real paired data, for example frames and events from a DAVIS-type sensor with unknown motion. If the claims are instead restricted to display-recorded benchmarks, I am unsure what the community learns beyond them.
-
The acquisition-informed response, presented as the main verifier, is not shown to beat simple static baselines. On mini N-ImageNet the static gradient response G beats S by 3.57 and 4.61 pp (Table 2), and under controlled motion the plain spatial activity score A is the best response-only ranker on both subsets (Table 20). On N-Caltech101, G and A are not reported on the test set, although A is defined for that dataset in Sec. B.5. All compared verifiers are the authors' own hand-crafted responses, with no learned reranker such as the pairwise classifier in [1], which the submission cites. Predicting brightness increments from image gradients and matching them to events is standard in event-based tracking, for example EKLT [2].
-
The writing makes the method look heavier than it is. A few lines of method are spread over 94 numbered equations, some repeated (Eq. 13 to 15 reappear as Eq. 41, 47 and 48), and PP-UOT, which is not a contribution, fills much of the method section.
[1] Cross-modal Place Recognition in Image Databases using Event-based Sensors, arXiv 2023.
[2] EKLT, Asynchronous Photometric Feature Tracking Using Events and Frames, IJCV 2020.
Questions for authors
-
How do G and A perform on the N-Caltech101 test set with the same shortlists, alone and fused with correspondence? If they match or exceed S there, the case for acquisition-informed verification rests on mini N-ImageNet controls that already favor static responses. This one matters most for my rating.
-
PP-UOT drops from 78.9 on internal validation to 41.0 on Original Test (Table 10), while EventBind and CEIA do not drop. What differs between the training-pool recordings and the released validation recordings, and does this shift affect the fusion weight, which was selected on the internal validation split?