acceptodds
Withdrawn from ICLR 2027

Correcting the Selector: Training-Free Proposal-Field Calibration for Open-Vocabulary Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A frozen open-vocabulary detector can already contain a correct candidate and still fail to return it. Adding labels can evict an unchanged detection from a finite output list; within a class, confidence can favor a poorly localized box over a better alternative. We identify the selector as a distinct intervention point: contextual and geometric relations remain available in the dense proposal field even when scalar-score selection discards them. We introduce VoPR, a training-free correction of the proposal field for fixed-vocabulary accuracy before the unchanged selector. Proposal-Conditioned Vocabulary Routing (PVR) adjusts category admission, and Candidate Quality Calibration (CQC) combines semantic competition, frozen distributional box redecoding, and proposal agreement to refine candidate ordering. For class-separable scoring, we localize vocabulary coupling to shared finite caps. Under fixed-order selection, class-broadcast routing admits only a prefix of each class's NMS-survivor sequence; candidate-specific calibration supplies the complementary ability to change the sequence itself. VoPR improves WeDetect Tiny/Base/Large by \(+0.35/+0.24/+0.20\) AP on COCO; frozen profiles also improve all nine LVIS and COCO-O transfer settings. With distractor labels kept active, VoPR gains \(+0.344\)–\(+0.637\) target-label AP under held-out distractor content. These paired results connect class admission and candidate refinement to improved detection without additional training, parameter updates, or another visual forward pass.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Open-Vocabulary Detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EikDOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies selection failures in frozen open-vocabulary detectors. A correct candidate may already exist in the proposal field yet never appear in the final output. The authors identify two causes: appended labels can evict unchanged candidates through shared finite caps, and scalar confidence can misrank localization quality. They verify that preserved scores stay unchanged when labels are added, and show that relaxing the final cap alone removes most of the vocabulary-induced loss. Based on this diagnosis, the paper proposes VoPR, a training-free calibration applied between one frozen forward pass and an unchanged selector. PVR adjusts class admission via bounded, image-conditioned class offsets derived from top-M proposal support and spatial consistency. CQC refines individual candidates via semantic competition against nearby text classes, top-L redecoding of the native DFL box distributions, and non-self peer agreement fused with confidence. A selector-level theorem shows that class-broadcast offsets can only admit prefixes of the native NMS-survivor sequences, which motivates the candidate-specific second stage. Experiments use paired same-process comparisons on WeDetect at three scales, covering component comparisons, a held-out composition audit, distractor-vocabulary tests, cross-architecture ports, and latency measurements. The method yields consistent positive gains on COCO, LVIS, and COCO-O under fixed output budgets.

Critical strengths

1.The paper addresses a recognized problem in open-vocabulary detection. Vocabulary-induced degradation and confidence–localization misranking are both documented in prior work (e.g., LOVD, VocAda, and the localization-confidence literature). The paper gives them a unified account at the selector level, which is a clear and useful framing. 2. The method has practical value as a genuine plug-in. Many deployments must keep the detector frozen: no training data, no retraining budget, and a fixed output budget. VoPR operates between one frozen forward pass and an unchanged selector. It needs no training and no parameter updates, and it leaves the native post-processing pipeline intact. It is orthogonal to representation learning and can be attached to existing checkpoints that expose dense class logits and box distributions. 3. The reasoning chain from diagnosis to design is complete. Cap-relaxation interventions verify the claimed failure pathway. The class-wise prefix-constraint theorem identifies what class-level routing cannot do, which motivates the candidate-specific second stage. Component experiments then test each stage separately. Diagnosis, theory, and method are internally consistent.

Critical weaknesses
  1. No comparison with same-field methods. Vocabulary-induced degradation is documented in LOVD and VocAda. Confidence–localization misranking is documented in the localization-confidence literature. Training-free test-time corrections for detection already exist: ViTPrompt (CVPR 2026) refines boxes and scores at test time, and BCA+ (Zhou et al., 2025) adapts class scores without training. None of these is cited or compared. Neither are ProCal, FRACAL, or faithful AggDet. All gains are measured only against the paper's own pre-trained baseline. The cross-architecture ports were "selected on their evaluation splits," in the authors' own words. The advantage over existing training-free corrections is not established.
  2. Marginal gains at a non-negligible cost. Filtering and refining candidate boxes is known to help detection (Soft-NMS, Fitness NMS, IoU-Net, AggDet). Against this backdrop, the gains here are small: +0.20–0.35 COCO AP, no change in relative standing, LVIS deltas down to +0.06 AP without uncertainty quantification. The cost is heavy for a test-time method. The exact variant adds roughly 85% inference latency; the sparse variant adds about 30%. For training-free test-time correction, lightweight deployment is the main appeal. This trade-off is never weighed against cheaper options (PVR-only routing, relaxed output caps).
  3. Too many hyperparameters for a training-free method. The sensitivity audit alone sweeps ten hyperparameters per model scale (Appendix H, Table 17). Tiny adds two more (J and δ; Table 15). Several frozen values differ across scales, with no stated principle. All were tuned on COCO val2017 — the same split used for the headline results. The five-fold audit keeps the hyperparameters fixed and corrects only the composition choice, on one scale. Its estimate (+0.16) falls below the headline (+0.20). For a method claiming "no training, no parameter updates," this hand-tuning is a hidden cost.
Questions for authors
  1. In TTA, nearly every method improves over the frozen pre-trained baseline. Baseline-relative gains alone therefore cannot establish effectiveness. Can you compare VoPR against TTAOD baselines under the same protocol (e.g., ViTPrompt, BCA+, ProCal, FRACAL, or AggDet)? And can you state the method's scope of application — which distribution shifts, vocabulary regimes, and detector families it targets?
  2. The many hyperparameters ease reproduction but reduce the method's practical value. Can you justify the necessity of each hyperparameter? Is there a minimal configuration that retains most of the gain?
  3. The full inference pipeline nearly doubles the computation cost. At this price, many alternative methods become viable substitutes. Can you add orthogonality tests with other modules (e.g., existing test-time corrections) and verify that the gains stack? Such results would markedly strengthen the paper.
Reviewer_DNFtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how a frozen open-vocabulary detector selects its final detections. On WeDetect, adding class names leaves the original scores and boxes unchanged to numerical precision. If this invariance is exact, any loss on the original classes must come from the shared pre-NMS and final caps, a failure the paper calls finite-cap eviction. Within a class, confidence can also favour a poorly localized box over a better one. The proposed VoPR corrects the dense outputs before the unchanged threshold, caps and NMS, without training. PVR adds a bounded per-image, per-class logit offset, and CQC adds a margin against semantically similar classes, re-decodes boxes from the top DFL bins and fuses confidence with agreement among overlapping boxes. On COCO, where its settings were tuned, VoPR improves WeDetect Tiny/Base/Large by 0.35/0.24/0.20 AP; with those settings frozen, the gains stay positive on LVIS, COCO-O and ODinW.

Critical strengths

The diagnosis is the most useful part of the paper. If added names leave the original scores and boxes unchanged, they can only hurt the original classes through the shared caps. The paper checks both halves of this argument: preserved scores move by less than 2 × 10⁻⁶ with identical boxes, and relaxing the caps recovers 84–91% of the lost AP. Theorem 1 also states cleanly what a per-class offset can and cannot change. This is worth knowing for anyone who runs a detector with a large vocabulary.

The evaluation is careful, and the paper is open about its limitations. Comparisons are paired, COCO results have bootstrap intervals, and the appendix includes a full component factorial, hyperparameter sweeps, reproduced baselines and timing. With one frozen configuration per model size, every benchmark reported improves, so I believe the effect is real, if small.

Critical weaknesses

My main concern is the link between the cap diagnosis and VoPR's gains. AP at a fixed vocabulary and output budget does improve, but the experiments do not show how much of that improvement comes from preventing eviction. PVR alone gains about as much with relaxed caps as with the native ones (0.19–0.28 vs. 0.20–0.28 AP, Table 10). With 1,120 hard (semantically close) distractors, it removes only about 0.03 of the 0.675 AP that the caps cost the baseline. Within an image, it cannot reorder candidates of the same class (Theorem 1). Sec. 4.2 notes that it can also act through threshold admission and ranking across images, but neither route is isolated, and a lower-threshold baseline is missing. Full VoPR is never run with relaxed caps. It gains most with no distractors (0.57/0.80 AP on the two slices of Table 11, against 0.42/0.44 with the 1,120 hard ones). It also makes the drop caused by the distractors larger (0.73 → 0.87 and 0.85 → 1.21 AP, App. F).

Without a clearer link, the contribution is modest. Dave et al. (2021) already showed that the per-image detection limit makes categories compete; raising it from 300 to 5,000 lifted LVIS AP from 22.6 to 26.0. On WeDetect the loss is under 1 AP and is largely recovered by raising the caps. The components adapt known ideas, such as agreement among overlapping boxes (AggDet) and image-level class evidence (SIC-CADS, where it is learned).

The gains are also small where they are best measured: 0.20–0.35 AP on COCO and 0.06–0.23 on LVIS. Every interval for the full method is computed on COCO val2017, where all settings were developed. The five-fold audit shows a 0.16 AP gain but re-selects only the component set. WeDetect's transfer gains are larger under domain shift but have no intervals. LVIS minival reuses COCO val2017 images, and ODinW uses a substitute prompt. The only interval on new images with a new vocabulary is for a reduced variant without PVR or the semantic margin (+0.098 AP, [+0.0006, +0.195]).

The full method is also expensive. On WeDetect-Large, PVR adds 1 ms for 0.09 AP, while full VoPR adds 243 ms for 0.20 AP (Table 19). Table 20's end-to-end benchmark shows an 85% overhead, or 31% for a sparse variant that keeps 97% of the gain on its 1,000-image slice. The main text does not report latency.

Questions for authors

Questions 1 and 2 are clarifications; 3 and 4 ask for experiments that could change my assessment.

  1. At C = 80, WeDetect-Large reaches 47.5 and 46.8 AP on the slices of Tables 10–11, but 56.92 on the slice of Table 20 and 54.49 on all of val2017. Which images are in these slices, and do the resolution, score threshold or prompts differ? The released WeDetect expects Chinese class names; in what language were the target and distractor names given?
  2. The 17-AP drop on OWLv2 in Fig. 1(a) motivates the paper. Is it a finite-cap effect, and how much of it is recovered when that model's caps are relaxed?
  3. Please run full VoPR under the relaxed caps of Table 10, and the baseline with a lower score threshold. For Base and Large, the fused test p⋆ > τ is equivalent to p̄·q̄ > τ² in the notation of Proposition 4, so thresholds down to τ² are the natural comparison.
  4. Please report full VoPR and PVR alone, with image-level intervals, on the 15,000 LVIS val images outside COCO val2017. COCO results with all settings chosen on disjoint images, such as part of train2017, would also help.
Reviewer_RompOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces VoPR: Vocabulary-to-Proposal Rectification, an offline method to improve the selection of prediction in open-vocabulary object detectors. The VoPR targets two failure modes:

  1. Including additional category names can result in contaminating the current detections as the new candidates compete for the same limited output slots.

  2. A poorly localized box may receive a higher score as compared to the accurate one and will get selected.

VoPR adjusts candidate scores and box coordinates before the detector’s normal selection process. The method has two stages, the first is Proposal-Conditioned Vocabulary Routing (PVR) that estimates how strongly a category is supported in the current image. It averages the category’s strongest proposal scores and checks whether surrounding image locations show similar responses. The second is Candidate Quality Calibration (CQC) that refines individual predictions. It checks if a candidate favors its assigned category over related categories, refines box boundaries, and measures how closely other candidate boxes agree with its location.

Experiments mainly use the WeDetect detector family and show minor improvements over the reproduced baseline on COCO, LVIS, and COCO-O.

Critical strengths

The critical strength of the paper is that its proposed VoPR is inference time optimization and there is no re-training required. This means that it can be applied to multiple detectors, although the portability of the transfer requires stronger evidence.

Critical weaknesses

The major weakness of the VoPR is that the gains over the baseline are minimal. On average, the improvements are below 0.5 points which limits the practical significance. Further, this improvement is likely to come as a result of added inference latency.

Further, the paper does not provide any analysis on how often the previously correct predictions become incorrect after VoPR. There may be a reasonable number of predictions shifting from correct to incorrect after applying VoPR. Both recovered and damaged detections should be reported.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.