acceptodds
Under review as a conference paper at ICLR 2027

Towards Trustworthy Object Detection: Adaptive Boundary Optimization with Joint Classification and Localization Risk Control

Abstract

Quantifying image-level uncertainty in multi-object detection is paramount for safety-critical applications, such as autonomous driving. While Conformal Risk Control (CRC) provides marginal coverage guarantees, achieving rigorous risk control for individual images remains a significant challenge. To address this problem, we propose a novel approach that learns image-adaptive boundary extension thresholds to ensure consistent detection reliability. Specifically, we introduce the Conditional Adaptive Boundary Extension (CABE) framework, which leverages a differentiable surrogate loss based on soft Intersection-over-Ground-Truth (IoG) to directly optimize for improved conditional coverage. Experiments across three diverse safety-critical domains demonstrate that CABE not only achieves superior conditional coverage compared to standard methods but also significantly reduces the magnitude of boundary expansion. Our approach offers a robust pathway for reliable and interpretable uncertainty estimation in object detection, providing vital conditional guarantees required for high-stakes applications.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
conformal prediction, object detection, conformal risk control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_R7jeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors investigate image-level uncertainty in multi-object detection. They extend a framework for Conformal Risk Control, i.e., they introduce the Conditional Adaptive Boundary Extension (CABE) framework. The framework leverages a differentiable surrogate loss based on soft Intersection-over-Ground-Truth (IoG) to directly optimize for improved conditional coverage. The method is derived theoretically, and it builds upon the CRC method and generally outperforms it. The tests are performed on three datasets, and there are several ablation studies.

Critical strengths
  1. The authors address an important topic related to the safety of object detection models and introduce a risk control framework. I think it makes sense to have bounds for both: correctly classified and spatially contained. The method is also theoretically justified.

  2. The authors conducted many experiments. They tested three datasets and identified a suitable baseline method. They also carried out various ablation studies.

Critical weaknesses
  1. The theoretical explanations are sometimes a bit vague. “To make this differentiable, we define the Intersection-over-Ground-Truth (IoG) ratio.” This statement is somewhat imprecise. The IoG itself is not necessarily smooth or differentiable everywhere due to the box intersection, especially at the boundaries. "As β increases, this soft score provides a tighter upper bound on the true coverage count, penalizing..." As β increases, the function becomes stricter and approaches a binary function, in other words, it is more of a soft approximation of the binary coverage indicator.

  2. The related work section is a bit hard to follow, especially the part explaining why more methods are not considered as baselines.

Questions for authors
  • The experiments are generally quite broad, but I think the selection of models is too limited (only YOLOv8 and YOLOv10 were tested). I'd be interested to see how the method performs on other models as well. Were other models tested?
  • Was oracle expansion (using ground truth) for δ also tested to see how far CABE was from the optimal result?
  • Which other methods described in the related work could still be usefully employed as baselines, or perhaps not (and why)?

Remarks:

  • Typo in line 84: Ginven
  • Some figures and captions are very small, e.g., Figure 3 and 5.
  • In the tables, up/down arrows would help indicate whether a evaluation metric should be high or low for good performance.
Reviewer_Acs2OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses image-level reliability of conformal risk control (CRC) for object detection, where a single global box margin can over-cover easy images and under-cover hard ones. It proposes CABE, which predicts an image-specific anisotropic box expansion with a ResNet-50 trained on a differentiable soft IoG coverage loss, then calibrates a global offset with CRC on a held-out split. Combined with a CRC-calibrated classification threshold, a union bound yields a finite-sample marginal joint risk guarantee for classification and localization. Experiments on three safety-critical datasets with YOLOv8 and YOLOv10 show that CABE consistently reduces the image-level coverage gap compared with global and stratified CRC while maintaining the target marginal coverage.

Critical strengths
  1. The paper tackles a relevant problem: marginal guarantees can hide large per-image variation, which matters in safety-critical detection, and Fig. 2 illustrates this clearly.
  2. Learning the expansion and then recalibrating with CRC on a held-out split preserves a valid marginal guarantee regardless of how well the network is trained.
  3. The experimental evaluation is broad, covering three datasets, two detectors, 20 random splits, several risk levels, ablations on output dimensionality, and sensitivity analyses.
Critical weaknesses

The method follows Luo et al. (2026), which learns image-adaptive thresholds through a differentiable miscoverage loss for segmentation, and the contribution beyond extending this approach to detection should be made explicit.

Conditional reliability is measured only through the image-level coverage gap. Because per-image coverage is a realized fraction over few objects, this metric remains nonzero in expectation even under exact conditional control. Results stratified by object count or object size could give more direct evidence.

Questions for authors

The calibration code in Appendix A.18.3 (Algorithm 4) selects the offset when empirical coverage reaches 1−α₂, without the finite-sample correction of Eq. 17 and Algorithm 1. Which rule was used in the reported experiments? The same code searches over negative offsets, whereas Proposition A.2 requires t ≥ 0. Were β and λ_reg selected on a split disjoint from the test sets used for Table 1? Since a single expansion is predicted per image, could conditioning on object-level features further reduce the coverage gap?

1 comment

I thank the authors for the detailed response. The clarifications on the calibration rule, the nonnegative offset, and hyperparameter selection address my earlier concerns. Before finalizing my assessment, I would appreciate clarification on two points.

1. Conditional reliability. The per-image gap has a nonzero floor even under exact conditional control (e.g., 2α(1−α)2\alpha(1-\alpha) for single-object images), so the stratified gap remains hard to interpret. Could the authors report the pooled coverage per stratum (object count and size) versus the target, together with the expected gap floor per group?

2. Joint coverage. CABE's mean joint coverage is below 0.90 in several settings (e.g., 0.894 ± 0.004 on DroneVehicle). What is the joint target 1−α1−α21 - \alpha_1 - \alpha_2, and is this consistent with the guarantee?

Apologies for the late reply. My final assessment will depend on these clarifications.

Reviewer_zTJwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the application of Conformal Risk Control (CRC) to object detection, with the objective of simultaneously controlling classification and localization errors for matched detections. The proposed approach constructs conformal prediction sets for object classes and adjusts predicted bounding boxes to achieve a target coverage level.

The main contribution is Conditional Adaptive Boundary Extension (CABE), which introduces an image-dependent bounding box expansion mechanism. Specifically, a neural network is trained using a differentiable soft-IoU-based loss to estimate expansion factors shared across the detections of a given image. A subsequent conformal calibration step determines an additional correction to provide the desired statistical guarantees.

The approach is evaluated on three object detection datasets (AFO, DroneVehicle, and Rescue) using YOLOv8 and YOLOv10. The experimental analysis compares CABE against standard CRC with non-adaptive bounding box expansion, focusing on coverage and localization performance.

Critical strengths

1/ Relevance of the problem: The paper addresses an important problem in safety-critical object detection, where controlling both classification and localization errors is particularly relevant. Providing and improving statistical guarantees for these two aspects is valuable for improving the reliability of object detection systems.

2/ Interesting incremental contribution: The proposed approach combines existing ideas from Conformal Risk Control for object detection (Andéol et al., 2023) and adaptive risk control (Blot et al., 2025). The introduction of a NN to provide an image-dependent boundary expansion mechanism is an interesting application of (Blot et al., 2025) in object detection use cases, allowing the bounding box corrections to account for variations between images rather than relying exclusively on a global correction.

Critical weaknesses

1/. Overstated theoretical guarantees and unclear scope of the contribution The paper does not sufficiently distinguish between marginal risk control and conditional coverage. Several statements (abstract l24 “on, providing vital conditional guarantees required for high-stakes applications”, conclusion l478 “achieving conditional risk control”) suggest that CABE provides conditional guarantees at the image or instance level, whereas the formal CRC guarantees remain marginal, as acknowledged in Appendix A.2.2. While the proposed method aims to improve conditional coverage empirically, this should not be presented as a image or instance level guarantee. This distinction is particularly important given the paper's motivation around safety-critical applications. In addition, the claim on the joint classification-localization guarantee requires a stronger justification since the adaptive bounding box expansion and the experimental analysis primarily focus on spatial coverage only (Table 1 caption only refer to a single \alpha value).

2/. Limited methodological novelty and insufficient justification of design choices The approach builds upon existing work on conformal risk control for object detection (Andéol et al., 2023) and adaptive risk control (Blot et al., 2025). Although their combination is interesting in the object detection setup, the paper does not sufficiently establish what makes this particular formulation distinctive compared with previous approaches. Some metric choices also deserve stronger justification, notably the use of a box-wise coverage risk (l126) rather than the pixel-wise formulation discussed by Andéol et al., and the choice of a symmetric coverage gap (eq 33) and training loss (Eq 14) that penalizes both under-coverage and over-coverage (an asymmetric one would better align to the safety objective). These points make it difficult to assess whether the proposed formulation is particularly well suited to the stated safety-critical objectives.

3/. Experimental comparisons and unclear practical significance of the improvements The experimental evaluation does not fully establish the advantage of CABE over existing conformal approaches. In particular, a comparison with a baseline using separate corrections for the bounding box dimensions, a 2-dimensional correction using Bonferroni as in De Grancey et al, would help isolate the contribution of the 2-dimensional adaptive expansion mechanism of CABE (it can also be shown by a 1-d CABE version as an ablation study). Moreover, some of the reported improvements in bounding box size and IoU appear relatively small (except for DroneVehicle), and their practical significance is not sufficiently discussed. Finally, the additional computational cost introduced by the expansion network is not reported, making it difficult to evaluate the practical benefits of the method.

other weaknesses:

  • Discrepancy between objective FNR <= 0.1 and samples provided for CRC_add in Fig 3 and 4 (FNR>0.1)
  • l86 The definition of the CRC guarantee is false: it should be stated in average on a new X_{n+1} point and not "for a new test point"
  • Eq12 is similar to the pixel-wise definition of (Andéol et al., 2023) and should be detailed. The \epsilon on the numerator should be justified also
  • l191 mention "uncoverred ground truth objects" but the paper only address the matched ground truth objects , and won't improve the recall
  • several typos and notations that are defined after their usage
Questions for authors

1/. Scope of the theoretical guarantees (clarification) The paper repeatedly refers to conditional risk control, while Appendix A.2.2 acknowledges that distribution-free conditional guarantees cannot be established in general. Could the authors clarify precisely which guarantees are theoretically established by CABE and which improvements in conditional coverage are only empirical? In particular, how should the claims about image-level reliability be interpreted?

2/. Justification of the risk and optimization objectives (clarification) Could the authors further motivate the choice of a box-wise coverage risk rather than the pixel-wise formulation proposed by Andéol et al. (2023)? Additionally, why does the CABE training objective penalize both under-coverage and over-coverage, given the safety-critical motivation? Would an asymmetric objective focusing on under-coverage be more appropriate?

3/. Experimental comparison and practical significance (additional evidence) Could the authors compare CABE with a conformal baseline using separate corrections for the bounding box dimensions (e.g., a Bonferroni-based approach)? This would help determine whether the reported improvements are attributable to image-adaptive expansion rather than the additional flexibility of the correction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.