- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper formulates OOD object detection as (n+1)-way detection. Synthetic outliers are generated by text-guided inpainting, verified and re-localized with an open-vocabulary detector, and used to fine-tune a detector with an additional OOD class. On OoDIS, the method reports AP50:95 of 57.6/29.4/39.2 on RoadObstacle/RoadAnomaly/Fishyscapes, compared with 48.5/22.7/22.7 for ZS-OVOD, and ablates vocabulary overlap, outlier proportion, annotation refinement, and ID-data diversity.
Critical strengths
- The formulation is simple and operationally attractive. Treating OOD as an additional class removes the need for an auxiliary branch at inference and avoids dependence on real OOD data, while remaining compatible with both closed-set and open-vocabulary detectors.
- The annotation refinement procedure is the most substantive technical contribution. Box refinement, label reassignment, and removal of empty inpaintings address a failure mode that is easy to overlook: without refinement, OOD AP falls to 1.3/4.2/1.9, compared with 46.9/24.4/29.3 after refinement. The 1,307 reassigned car instances provide a concrete estimate of how often inpainting preserves the original ID object.
- The ablation suite is broader than is typical for this literature. The vocabulary-overlap experiment includes a matched non-overlapping control, and the outlier-proportion, per-class ID trade-off, and domain-diversity analyses each isolate a design choice.
Critical weaknesses
- The comparison set is incomplete for a state-of-the-art claim. SIREN, SAFE, Unknown Sniffer, SyncOOD, and Dream-Box are discussed as related work but are not evaluated on OoDIS, despite being the closest trained OOD-detection methods. Without them, the empirical contribution cannot be positioned relative to the current literature.
- The headline comparison conflates training regimes. ZS-OVOD is zero-shot, whereas SynOE-OD is fine-tuned on NuImages and optionally Cityscapes or BDD. The abstract's 57.6 versus 48.5 therefore does not isolate the effect of synthetic outlier exposure. The appropriate control—ID-only fine-tuning—is reported in Table 2 and the 0% row of Table 8, but it is not foregrounded.
- The operating point and model-selection protocol are not adequately controlled. On RoadAnomaly, Table 2 reports 419 true positives against 1,222 false positives, implying precision of approximately 26%; yet the paper dismisses FPR95 and AUROC as inapplicable and does not report false positives per image or a threshold sweep. The OOD class name and training configuration were also selected using the OoDIS test sets, and several cells differ across tables (48.8, 48.7, and 47.6 for the same setting), which further weakens the reported comparison.
Questions for authors
NA
2 comments
W3.2: Class name, configuration, and differing cells
Thank you for checking the tables so carefully. This comment has three parts, and we answer them separately:
-
Class name. We would like to clarify that we did not select the class name on the test sets. SynOE-OD (GDINO:SwinT) is text-conditioned, so the class name must be a word that the text encoder can represent. "OOD" is split into meaningless tokens (Table 4a). "anomaly" is a single token. The training loss confirms this before any evaluation: it starts at a lower value. For SynOE-OD (DD:R50 and DD:Swin-L) the class name is irrelevant, because these models have no text encoder.
-
Training configuration. We selected the training configurations on LostAndFound, based on OOD performance and its balance with ID performance.
-
Numbers across tables. We agree that the paper does not make it clear enough, but they do not come from the same setting: 48.8 trained on NuImages and Cityscapes (domain-gap study), 48.7 trained on NuImages (standard setting), 47.6 schedule matched with the ID-only baseline so it is comparable to it in that table. Each table has one fixed setting and answers one fixed question. The numbers are therefore not meant to be compared across the tables. We will state the setting in each table caption clearly and use proper indicators to avoid confusion.
We hope that we sufficiently addressed the concerns raised, and we are open to further discussion. We thank the reviewer for their time.
We thank the reviewer for the thorough review and the precise comments. We answer them point by point.
We agree that these comparisons were missing. We extended the comparison set and now evaluate Unknown Sniffer, SIREN, Dream-Box, SAFE and SyncOOD. We also included a copy-paste baseline proposed by reviewer 1.
Results: Of the five methods, four underperform the detector they are built on. SAFE keeps the detector frozen and trains a classifier on sensitivity-aware features, using perturbed in-distribution samples and scores the boxes the detector already produced. SIREN also leaves the detection stage unchanged for unknown objects and shapes the ID class representations and scores each box by its fit to that representation. SyncOOD likewise trains a classifier on boxes of a frozen detector. Therefore these methods underperform their base detector. Dream-Box trains the region proposal network, but on outliers that are regenerated ID objects with their original annotations and it also underperforms its base detector.
Unknown Sniffer learns a generalized object confidence score from known objects, and with negative energy suppression, keeps unknown objects from being discarded as background. It is the only method that exceeds its base detector in recovering objects, which is what it is designed to do. In addition, the Unknown Sniffer paper reports a performance drop on images that contain both known and unknown objects. This is the most common case in street scenes.
Thank you for this observation. We agree. The comparison in the abstract does not isolate the effect of synthetic outlier exposure. The correct control is the identically trained ID-only model. It reaches 11.3, compared with 57.6 for SynOE-OD. This is also below the ZS-OVOD baseline (48.5). We will foreground this comparison in the main results and add the ID-only model as a row to Table 1.
Thank you for this comment. We agree that false positives per image and a threshold sweep were missing, and we add them below.
We would like to clarify why FPR95 and AUROC are not applicable in our setting. FPR95 and AUROC measure how well two sets of scores are separated: scores on ID data and scores on OOD data. An object that is not detected appears in neither set. These metrics therefore cannot penalize a missed detection, so AP is the right metric.
The benchmark returns the number of predictions per frame, (TP + FP) / #images. We report it for the three OoDIS tracks. For LostAndFound we report false positives per image and a precision-recall curve from a threshold sweep.
False positives (FP) per image on LostAndFound. Over LostAndFound scenes, SynOE-OD produces 2.13 FP per image, the ID-only 3.27, and VOS 0.77. SynOE-OD therefore detects more than twice as many objects as the ID-only control (1,245 against 553 true positives) with fewer false positives per image, while VOS's low rate comes with only 334 true positives.
Precision-recall curve on LostAndFound. SynOE-OD reaches a recall of 0.646, while the ID-only model and VOS stop at 0.287 and 0.173.