acceptodds
Withdrawn from ICLR 2027

CertOD: Class-Conditional Pair Certificates for Matched Object Detections in Autonomous Driving

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

16 views

Abstract

Object detection combines a discrete class and a continuous box. Certifying both under class-specific error budgets requires a joint guarantee conditional on the true class, which is unknown at inference, so selecting a geometric margin by the detector's possibly incorrect top-1 label does not establish that guarantee. CertOD instead emits per-class pair certificates: one calibrated box for each surviving class hypothesis. Under within-class exchangeability it provides finite-sample coverage of the event that the true class is retained and its associated box contains the object, at each class's prescribed level for matched detections of certifiable classes. Against a matched single-box control, pairs reduce retained true-class pedestrian area by (median) and (p90) on BDD100K at a exclusion rate; CODA2022 reproduces the tail benefit (– across four common classes). The gain requires a consumer that retains class–box associations: the smallest axis-aligned envelope of all pairs is exactly the conservative single box. Supremum scoring lowers the finite-quantile sample threshold roughly fourfold against coordinate Bonferroni without shrinking pedestrian boxes, and unified joint scoring is larger in median area wherever the class set is held fixed. We also report three boundaries: of BDD100K pedestrians match no proposal, zero-shot transfer of BDD thresholds to CODA2022 misses all eight tested score–class targets, and the repair weakens as the detector sharpens—on the misclassified slices its joint coverage falls from to across four backbones, because a better classifier yields smaller class sets.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Conformal Prediction, Uncertainty Quantification, Object Detection, Distribution-Free Guarantees, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_xT8BOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces CertOD, an application of conformal prediction to object detection. The motivation is clear: autonomous driving requires precise classification to define tolerable error margins, which are more critical on some classes than others. Validation is done via coverage evaluation on object detection task on the BDD100K and CODA2022 datasets.

Critical strengths

It is clear that the method works as a wrapper for frozen object detectors that provide multi-class scores. The main idea is explained clearly: use conformal prediction to strive for the predicted class to be correct and the predicted box to contain the labeled box, in an effort to maximize object detection performance, especially for vulnerable road users, such as pedestrians.

Critical weaknesses

The main critical weaknesses concern (1) explanation of background and method and (2) interpretation and relevance of results. The two are explained in detail below with concrete examples of what can be improved:

(1) The paper assumes that the reader is an expert in conformal prediction, as it does not attempt to even briefly explain what it is about, nor any of its central concepts, such as calibration data. The authors could improve readability by introducing conformal prediction and related concepts in the introduction. The work "Ries et al. 2026" that the authors cited provides a good example for how to introduce the topic nicely, and even the remainder of that paper is well written. In the same line, the authors did not cite the work that claims to be the first work to apply conformal prediction to the object detection task: De Grancey, Florence, et al. "Object detection with probabilistic guarantees: A conformal prediction approach." International Conference on Computer Safety, Reliability, and Security. Cham: Springer International Publishing, 2022. Phrases like ”Conformal prediction supplies finite-sample coverage under exchangeability” are unclear. The introduction does not clearly explain terminology, e.g. ”The uncertainty creates a calibration-inference mismatch” is hard to understand. The concept of ”margin” appears quite central to this work, but is not defined nor is any reference provided for it. The introduction reads like a combination of methodology and results sections, as a method overview is given, and some results are already presented; it would be better if the introduction set the background and described the landscape where this work lies, as well as the motivation for this work (which is already somewhat mentioned). The contributions at the end of the introduction read like results, instead of highlighting the method together with its results as a contribution compared to prior art. Some acronyms are not defined, such as CP and IoU. The same goes for some variables: in Section 3.1; M, K, and J are not defined, neither are \Delta^K, x in S(x), \alpha_c, nor box coordinates x and y. Clearly defining these would help the reader understand instead of having them figure them out. The pairing mechanism is not explained; is it Hungarian matching, greedy matching, or something else? It might not be needed to explain it in detail, but at least a reference or a few words would help. From the introduction and Figure 1 I got the feeling that argmax and set-max methods were prior art, but in section 3.2 it seems that they are the authors’ contributions, this should be made clearer. In Section 3.3 it seems that the pair certificate is the main contribution, however, details that are central to understanding the contribution, such as how the enlargement q is done or how many pairs there are, are missing. The certifiability mask and budget are not clearly explained, even though they are central to the contribution.

(2) The experiments do not show comparisons with prior art like Ries et al., 2026 or De Grancey, Florence, et al., 2022, nor is it clearly discussed anywhere how this work contributes additional knowledge to the one that the mentioned prior art contributed. Some phrases appear out of context, such as: ”On BDD100K, the frozen detector misclassifies 1.27% of matched pedestrians.” - which frozen detector are the authors talking about, and where does the number come from? Also, it is not clear if ”The geometric certificate enlarges the detector box outward to contain the true box.” means that the enlargement is a part of the method, or an intended consequence of the method. The citations in line 366 for model and dataset have the wrong order and can confuse readers. Section 4.1 presents results but does not interpret them. Is it positive that Single box shows larger coverage than CertOD? Is the fact that Single box and CertOD have a much larger joint coverage than Nominal a positive result of the method? It is unclear what role the Area metric has in evaluating the method; in particular, is the fact that that the area is larger than in "Nominal" a negative result? Also, I am confused by the bold letters; in Table 3 it would seem that best results are in bold, but the same does not apply in Table 1, this should be clarified. The meaning of "Bad strata" in Table 3 is not defined. In general, the results are just shown but not interpreted, it would be preferred if something was said about the main results and why they are valuable and worth publishing. I didn’t understand why the focus is on the pedestrian class in the conclusion (but also in the abstract and introduction), and other classes are left out of the main thesis; if the focus of the paper was to improve performance for pedestrian object detection, this should be made clearer from the beginning.

Questions for authors

Following the "Critical Weaknesses" section, I would ask the authors to make following main clarifications in the paper: (a) What is conformal prediction, how has it been used in object detection before, and how does your work contribute additional knowledge in that area compared to prior art? (b) How do your results compare to prior art results, and based on the outcome, what is the conclusion? I would recommend the authors to read the "Critical Weaknesses" section for details.

Reviewer_cLf2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies conformal certificates for object detection in autonomous driving when the class of a detected object is uncertain. It argues that selecting a box margin by the predicted class does not ensure coverage conditional on the true class. It then compares two outputs built from the same class-conditional calibration. A single box enlarged by the largest margin among the classes in a conformal class set serves as the control (Set-Max). CertOD instead emits one enlarged box for each retained class. A union bound over class exclusion and box non-containment yields finite-sample coverage of the true-class pair under within-class exchangeability. With a frozen COCO-pretrained DETR on BDD100K, the pairs reduce the retained pedestrian area by 6.8% at the median and 22.2% at p90, and CODA2022 shows p90 reductions of 6.6% to 35.6%.

Critical strengths

S1. The paper states the joint, true-class-conditional target clearly (Equation 1), and its comparison holds the quantiles, class sets, budgets and scores fixed so that the area difference reflects only the output representation. It also reports the limits of the approach openly, such as the proposal-recall ceiling and the failure of cross-dataset transfer. These are the main points in the paper's favour.

Critical weaknesses

W1. The central problem and the single-box construction already appear in published work that the paper does not cite. Timans et al. (ECCV 2024) address the same issue, namely that a box interval conditioned on a predicted class loses validity when that class is wrong. Their two-step method propagates the uncertainty of a conformal label set into the box intervals with class-conditional calibration. It takes the coordinatewise maximum of the class-specific margins over the label set and is evaluated on BDD100K. This is the construction the paper calls Set-Max (Equation 6), and the paper relates it only to concurrent work from 2026 (Ries et al.). Timans et al. also adapt the intervals to object size, which the affine score and the size audit of Section 4.3 revisit. Once this work is accounted for, the contribution reduces mainly to emitting one box per retained class instead of their maximum. The resulting gain is modest. A class-aware consumer saves at most 1.31 percentage points of frame area, and 0.11 points for pedestrians (Section 4.4). The summed area of the emitted pedestrian boxes is 35.3% larger at the median than the single box (Appendix A.2).

W2. The evaluation relies on outdated detectors and compares with no published conformal detection method. All four detectors date from 2020 and 2021 (DETR-R50, DETR-R101, Deformable DETR and YOLOS-small). The main detector is a COCO-pretrained DETR-R50 applied to BDD100K without fine-tuning, and the four detectors reach a pedestrian recall of only 53% to 62% (Table 6). Current detectors that the paper itself cites, such as DINO and RT-DETR, are not evaluated, and YOLO-style detectors cannot be used at all (Appendix A.5). The weak detectors also shape the reported findings. A third of BDD100K pedestrians match no proposal, and the misclassification that motivates the method shrinks as the detector improves. With Deformable DETR, the joint coverage on misclassified pedestrians falls to 12.7% (Table 6). The main comparison is against the paper's own Set-Max control, with Ries et al. (2026) only in the appendix. Neither Timans et al. (ECCV 2024) nor any other published conformal detection method is compared. The results therefore do not show how CertOD performs with a current detector or against prior methods.

Questions for authors

Q1 (clarification). How does CertOD differ from the two-step method of Timans et al. (ECCV 2024) beyond replacing the maximum over the label set with one box per class? A direct comparison on BDD100K with their size-adaptive intervals is needed.

Q2 (requested experiments). Can the authors evaluate CertOD with current detectors, such as DINO or RT-DETR, fine-tuned on driving data? Such results would show whether the misclassification that motivates the pair output, and the reported area gains, persist.

Q3 (clarification). The main tables pool several rows per image, which the paper notes does not establish within-class exchangeability (Section 3.1). Do the coverage and the area gains hold under the one-row-per-image-class design of Appendix B.1?

Reviewer_jLwMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies conformal certification of object detections in autonomous driving under class-specific risk budgets. The target is a joint guarantee, conditional on the true class, that the true class is retained in a candidate set and that the box attached to it contains the object. The authors argue that choosing the box margin by the predicted label does not certify this true-class event. They also argue that inflating a single box by the largest margin over a conformal class set (Set-Max) is overly conservative. CertOD instead calibrates class thresholds and box margins per true class and emits one calibrated box for each retained class. The box score uses a supremum over edges and scale normalization. On BDD100K and CODA2022 with a frozen DETR, the pair output meets the class-level targets and reduces the certified area of the true-class box relative to Set-Max, mainly in the tail. The authors also report limitations, including the detector's recall ceiling, failure of cross-dataset transfer, and a weaker benefit on stronger detectors.

Critical strengths

1.The paper makes precise that, under heterogeneous per-class budgets, selecting the margin by the predicted class certifies P(· | ĉ = c) rather than the required P(· | c* = c), and it gives a pair certificate that satisfies the true-class joint guarantee without test-time labels. The main experiment holds class sets, quantiles, scores and budgets fixed, so the resulting tail-area reduction (22.2% at p90 for pedestrians on BDD100K, replicated on CODA2022) can be cleanly attributed to retaining class–box associations. This is the paper's central contribution.

2.The box score and the certifiability rule make strict per-class budgets practical when calibration data are scarce. Scale-relative normalization reduces median pedestrian certified area from ×11.32 to ×4.44 of ground-truth area at unchanged coverage. The supremum score lowers the finite-quantile sample requirement fourfold, which makes motorcycle certifiable on all splits instead of none. The mask also prevents an under-sampled class from making every box vacuous. These matter because rare, safety-relevant classes are exactly where per-class calibration is data-limited.

Critical weaknesses

1.Novelty is limited relative to closely related work that the paper does not cite. Timans et al. (ECCV 2024) already calibrate box coordinates per true class so that coverage holds for misclassified objects. They take, for each coordinate, the maximum class-specific quantile over a conformal label set (their Sec. 5.2), which is exactly Set-Max, and they combine coordinates with a max-rank correction instead of Bonferroni, which overlaps with the supremum score. Mukama et al. (2024), de Grancey et al. (2022) and Andéol et al. (2023) are also missing. What remains new is essentially not taking that maximum, plus heterogeneous budgets and the mask, and the supporting results (Corollary 2, Proposition 7) follow directly from the union bound.

2.The practical value of the pair representation is not established. A downstream consumer does not know which pair is true, and a safety-critical planner that must respect all retained hypotheses effectively uses their envelope, which the authors prove equals Set-Max. Consistent with this, summed pair area exceeds Set-Max by 35%–150% for pedestrians, and a class-aware consumer recovers only 0.11 percentage points of pedestrian frame area (Appendix D.4). No decision rule that exploits multiple hypotheses is evaluated. The benefit also fades with stronger detectors (12.7% joint coverage on misclassified objects with Deformable DETR), which suggests it mainly helps weak zero-shot detectors like the one used here.

Questions for authors

1.How does CertOD differ, methodologically and in its guarantees, from Timans et al. (ECCV 2024)? Their Sec. 5.2 appears identical to Set-Max, and their max-rank correction appears to address the same issue as the supremum score. 2.Please specify a concrete downstream decision rule that consumes multiple class–box pairs, preserves the certificate's safety semantics, and benefits from the per-class reduction without knowing which pair is true. Even an open-loop planning or collision-check proxy would strengthen the significance claim substantially. 3.Can the guarantee be restated over an event observable at test time, for example all proposals above a score threshold with an explicit "no object" outcome? How do coverage and area behave under that definition?

Reviewer_8XHNOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper clearly argues that in object detection the uncertainty of a box prediction is tied to the predicted class, and thus the uncertain margins for the predicted class can be too small. Thus, the authors suggest to consider a multi-modal distribution constituted of per-class calibrated box uncertainties, to obtain the smallest safe box margin. Candidate boxes with strong ground truth overlap are used to define a candidate set of classes and respective (safety-margin-widened) boxes. In essence, the box consumer is presented with these margin-widened candidate safe boxes.

A demo-like setup with two datasets and one model showcase that the obtained safe boxes are slightly but (in that setting) consistently smaller than using a conservative common safety margin.

Critical strengths
  1. The practical problem of too small / too large safety margins for per-predicted class / all-class box uncertainty is well explained and argued.
  2. The suggested solution of per-class calibration is fairly simple but elegant (if it proves to have a reasonable safety-conservativeness tradeoff).
  3. A real-world use-case is chosen for the demo.
Critical weaknesses

My key concerns summarize as:

  1. The experimental evaluation considers two datasets, but only automated driving and only on a single pretrained model. This leaves completely unclear, whether the results generalize to other models or even other weights for the same architecture; and the lack of metric definition leaves unclear, whether the safety margins truly serve their purpose to avoid misses compared to more conservative techniques. This is reflected in the key contribution claims, that just list claims about the two specific datasets. This lack of convincing experimental evidence is the main reason for my low soundness rating and the reject decision: If I haven't completely overlooked something that is easy to squeeze from appendix into the paper, then adding sufficient results is a major revision in my opinion.
  2. The paper is not self-contained, as lots of relevant definitions (e.g., metrics, algorithms) and background about key related work (Mondrian calibration; Ries et al. 2026) are omitted or moved to the appendix.
  3. The notation is partly overly complex and one of the key ideas are left unmotivated (Eq. (7)).

In summary, I cannot properly judge the innovativeness of the ideas, nor the practical usefulness of the approach as it is presented and evaluated right now. More details are provided below.

Where I experienced a lack in self-containment:

  • It does not become clear from ll. 124ff what is the difference of the current approach to (Andeol et al., 2025) and (Ries et al., 2026).
  • l. 171: The one-row-per-image-class audit is not clear from the term alone. The definition in Appendix B.1 only is provided very late and heavily relies on the exchangeability notion, which is not defined before to what I saw.
  • The metrics of the experimental section are not defined in the main paper.
  • Algorithm 1 is fairly trivial and can be summarized in one sentence (return the candidate boxes as defined in Eq. (7), but expanded by their static relative safety margin). However, the actual interesting contribution of how to obtain the per-class calibrated safety margins is hidden in Algorithm 2 in the appendix.

Where I found notation and explanations could be improved:

  • The key definition of the class candidate set construction is unclear, since the practical difference between S and S^+ remains uncommented. To what I understand, they are only different when the winning class of an instance has a justifiable budget at all, but the instance's confidence is too low. Why that would be included is not clear to me.
  • The conditional notation P( some condition on c' | c' = c) (e.g., in Eq. (1), (3), (4), (5)) can be simplified to just P(some condition on c). This would ease readability a lot, as the roles of c and c^* seem to swap, e.g., between Eq. (1) and (3).
  • The paper heavily expands on the problem formulation, which largely only introduces the formalization used later. It does not become clear (a) in how far this formalization has been used in or applies to prior work (justifying that the claimed shortcoming is a flaw in literature), (b) where the problem that "a guarantee under the predicted class does not establish the required guarantee under the true class" (l. 165) is actually resolved. It should be strictly separated between notation, target definition, and an analysis of the flawed solutions in literature so far.
  • The "two-sided score" and "inward errors" in ll. 295f are undefined.
Questions for authors
  1. How are matched detections defined? When is a detection matched?
  2. What is the definition (formula) of the used metrics joint coverage, and the two areas? Does any of them represent the error rate of how often a ground truth box on the test set was not fully covered by the predicted box + class-specific safety margin?
  3. What are the standard deviation values for tab. 1? Are the differences significant? How well do results generalize to other models and datasets?
  4. What is the motivation for the definition of S^+ (instead of S)?
  5. How does this work differ from those of the mentioned ones by Andeol et al. (2025)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.