- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces GRACE (Guided evidential Regression for Adversarial and Covariate uncertainty Estimation), a localisation-aware post-hoc evidential meta-model for object detection that provides uncertainty estimates for each predicted object. One of the benefits of GRACE is the absence of retraining the model for object detection. The current approach was evaluated based on a single object detector (Yolov11n) against 4 datasets and 3 baselines model.
Critical strengths
I believe GRACE meta-model approach has its merit: having a meta-model that works for all object detection model is a valuable contribution. GRACE shows some promising results on CelebA and VisDrone dataset. However, the current approach is not generalizable to other object detectors and other datasets (performance-wise).
Critical weaknesses
-
The evaluation focus on a single object detector model (YOLOv11). Therefore, it is difficult to assess the generalizability of the proposed method.
-
Looking at the numbers from Table 1 and Table 3, GRACE does not always outperforms the baseline methods. In the case where GRACE outperforms the baseline methods, the performance gain is marginal. For Table 1, on the CelebA dataset, GRACE is vastly superior to baseline methods. However, things start to fall apart on the other datasets.
-
I do not think Figure 3, FIgure 4, and Figure 5 provide a truthful representation of GRACE performance (robustness) to (non-adversarial and adversarial) perturbations targeting the image. As seen in Table 1, under normal conditions (no perturbation), GRACE is performing well on CelebA compared to baseline methods. A better approach would be to show Figure 3, 4, and 5 on datasets such as TT100k or COCO where GRACE is on par with the baselines (in absence of perturbations).
Questions for authors
- Have you evaluated GRACE using a model different than YOLOV11n?
- How fast is GRACE on a edge device CPU (smartphone or laptop)?