- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies 3D object rotation estimation given a reference and a corresponding query image under exposure degradation, with additional event stream. It introduces ExPO-Shape, a synthetic benchmark rendered from ShapeNetCore-v2 with 48,145 RGB-event pairs of 9,629 objects in 19 categories, where event streams are simulated from frames interpolated along the trajectories under 6 under-/over-exposure levels. It also proposes EVENPose, a framework with three components: an Event-Stabilized Embedding that fuses DINOv3 features of the reference image and three event maps via cross-attention, an event-injected U-Net for novel-view embedding synthesis, and an Event Consistency Loss that compares events induced between the reference and the decoded synthesized view with the ground-truth events. EVENPose outperforms RGB-only, event-augmented, and exposure-correction baselines on five held-out ExPO-Shape categories.
Critical strengths
- The benchmark substantially increases object diversity over existing RGB-event pose datasets (9,629 objects in 19 categories vs. 1-21 objects), with six controlled under-/over-exposure levels. This enables evaluation of unseen-category generalization under exposure degradations, which existing RGB-event datasets do not support.
- The Event Consistency Loss is simple and motivated by the event triggering mechanism. It supervises novel-view synthesis in the event domain instead of exposure-sensitive pixel space. The ablation indicates that it is the most influential component (+13.3-point ACC30 on ExPO-Shape).
Critical weaknesses
- The event stream may leak relative motion information from reference to query. Based on the construction of ExPO-Shape, the transition path is deterministic given reference and query pose, and the number of interpolated frames is proportional to the angular distance. Therefore, the Event Count Map C, may directly indicate , and the spatial distribution of events may indicate the rotation direction. This differs from existing RGB-event pose benchmarks such as YCB-Ev and YCB-Ev SD.
- The synthetic data may not reflect how RGB and event camera work. In the real world, RGB under-/over-exposure is due to inappropriate camera settings, and does not directly affect event camera. However, in this work, intermediate frames are first rendered with under-/over-exposure and then converted into event streams. This may lead to event streams to inherent RGB failures, e.g.: under over-exposure, the saturated region of RGB image is almost white and produce no events, whereas a real event camera would still respond.
- The paper mix under-exposure and low-light scenes in some discussions within method and experiment sections. Real low-light scenes introduce photon shot noise, read noise, and quantization noise [1], and event cameras may also degrade in low light, neither of which is modeled by the noise-free exposure scaling in ExPO-Shape. Therefore, the results on ExPO-Shape may support robustness to camera exposure settings rather than to low-light conditions.
[1] Kaixuan Wei et al., Physics-based noise modeling for extreme low-light photography. IEEE TPAMI, 2022.
Questions for authors
- In YCB-Ev SD, each pose has its own short event sequence, and the views within a scene do not form a continuous trajectory. What event stream is used as the events from Ir to Iq when evaluating on YCB-Ev SD?
- Are the intermediate frames stored as 8-bit images or as linear float/HDR values, and how is each exposure level implemented? Are the events generated from the exposure-degraded frames or from clean frames?
- Since NOPE is the primary baseline, why is NOPE+E not reported in Table 1?
- In Fig. 5, the baseline on ExPO-Shape (25.2/61.8) matches RelPose in Table 1, while the baseline on YCB-Ev SD (15.4/69.1) matches NOPE. Which method is the baseline?
- The embedding is built on DINOv3 features with learnable adapters. Which pretrained VAE decoder maps it to an image?