- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors investigate image-level uncertainty in multi-object detection. They extend a framework for Conformal Risk Control, i.e., they introduce the Conditional Adaptive Boundary Extension (CABE) framework. The framework leverages a differentiable surrogate loss based on soft Intersection-over-Ground-Truth (IoG) to directly optimize for improved conditional coverage. The method is derived theoretically, and it builds upon the CRC method and generally outperforms it. The tests are performed on three datasets, and there are several ablation studies.
Critical strengths
-
The authors address an important topic related to the safety of object detection models and introduce a risk control framework. I think it makes sense to have bounds for both: correctly classified and spatially contained. The method is also theoretically justified.
-
The authors conducted many experiments. They tested three datasets and identified a suitable baseline method. They also carried out various ablation studies.
Critical weaknesses
-
The theoretical explanations are sometimes a bit vague. “To make this differentiable, we define the Intersection-over-Ground-Truth (IoG) ratio.” This statement is somewhat imprecise. The IoG itself is not necessarily smooth or differentiable everywhere due to the box intersection, especially at the boundaries. "As β increases, this soft score provides a tighter upper bound on the true coverage count, penalizing..." As β increases, the function becomes stricter and approaches a binary function, in other words, it is more of a soft approximation of the binary coverage indicator.
-
The related work section is a bit hard to follow, especially the part explaining why more methods are not considered as baselines.
Questions for authors
- The experiments are generally quite broad, but I think the selection of models is too limited (only YOLOv8 and YOLOv10 were tested). I'd be interested to see how the method performs on other models as well. Were other models tested?
- Was oracle expansion (using ground truth) for δ also tested to see how far CABE was from the optimal result?
- Which other methods described in the related work could still be usefully employed as baselines, or perhaps not (and why)?
Remarks:
- Typo in line 84: Ginven
- Some figures and captions are very small, e.g., Figure 3 and 5.
- In the tables, up/down arrows would help indicate whether a evaluation metric should be high or low for good performance.
I thank the authors for the detailed response. The clarifications on the calibration rule, the nonnegative offset, and hyperparameter selection address my earlier concerns. Before finalizing my assessment, I would appreciate clarification on two points.
1. Conditional reliability. The per-image gap has a nonzero floor even under exact conditional control (e.g., 2α(1−α) for single-object images), so the stratified gap remains hard to interpret. Could the authors report the pooled coverage per stratum (object count and size) versus the target, together with the expected gap floor per group?
2. Joint coverage. CABE's mean joint coverage is below 0.90 in several settings (e.g., 0.894 ± 0.004 on DroneVehicle). What is the joint target 1−α1−α2, and is this consistent with the guarantee?
Apologies for the late reply. My final assessment will depend on these clarifications.