acceptodds
Withdrawn from ICLR 2027

The False-Negative Trap: Stable Source-Free Instance Adaptation with Recall-Prior Masking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Source-free unsupervised domain adaptation for instance-based detection and segmentation relies heavily on self-training with pseudo-labels. Because target labels are unavailable under the unsupervised setting, a deployable method must deliver strong terminal performance under a fixed training schedule rather than relying on peak checkpoint selection. However, in practice, self-training is often unstable: noisy pseudo-labels can accumulate harmful supervision and degrade terminal performance. We study a detector-specific failure mode of this paradigm: when a real target object is omitted from the pseudo-label set, the detector matcher can convert that omission into explicit background supervision. We refer to this mechanism as the false-negative trap. We introduce Recall-Prior Masking (\rpm), a conservative strategy that uses the frozen teacher's region proposal network (RPN) as a class-agnostic recall prior. Rather than converting uncertain proposals into additional pseudo-positive supervision, \rpm treats them only as evidence that background supervision may be unsafe. By suppressing possibly unlabelled visual evidence via Informed Cutout and masking localized background losses, \rpm aims to prevent harmful optimization. Under a predeclared common-budget protocol on three driving-scene adaptation benchmarks, \rpm improves terminal object-detection and instance-segmentation performance and reduces late-training degradation. In a strictly matched comparison with \adabn + Fixed SF-FM, adding \rpm improves Last AP by / segm/bbox on SYNTHIACityscapes, / on CityscapesFoggy Cityscapes, and / on KITTICityscapes. These results show that explicitly protecting potentially omitted objects from erroneous background supervision substantially improves terminal self-training performance and reduces late-training degradation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Unsupervised Domain Adaptation, Source-free, Instance segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hpSuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the false-negative trap in source-free instance adaptation, where omitted objects in pseudo-labels may be incorrectly supervised as background. To address this issue, the authors propose Recall-Prior Masking (RPM), which combines RPN-guided Informed Cutout with Gradient Masking to suppress potentially harmful supervision. Experiments on three domain adaptation benchmarks demonstrate improved terminal detection and instance segmentation performance, along with reduced late-training degradation.

Critical strengths
  1. The manuscript is clearly written, with a well-motivated problem formulation and a logical presentation of the proposed method. The distinction between false-positive and false-negative pseudo-label errors is particularly easy to follow.

  2. Clear illustrations and straightforward methodology.

  3. Reasonable design of Gradient Masking.

Critical weaknesses

My primary concern lies in the first component, Informed Cutout, whose design appears highly counterintuitive. According to Section 3.2.1, the method masks all selected high-objectness RPN proposals, including regions corresponding to existing positive pseudo-labels, while retaining the original pseudo-label supervision. Consequently, the student may receive images in which a substantial portion of the foreground evidence has been removed, yet it is still expected to accurately classify and localize the corresponding objects. The current explanation does not convincingly justify why this potentially inconsistent supervision should improve detection.

More importantly, Table 4 indicates that Informed Cutout contributes substantially more to the performance improvement than Gradient Masking. However, the authors provide limited analysis of why such aggressive foreground masking is beneficial. I strongly encourage the authors to extend Table 4 by introducing controlled comparisons with standard Random Cutout and the Stochastic Complementary Masking strategy in SOCCER (Stochastic Context Consistency Reasoning for Domain Adaptive Object Detection, ACM MM 2024). Similar to SOCCER, the masking ratio should be explicitly controlled and analyzed to determine whether the improvements primarily depend on masking locations, masking proportions, or the proposed RPN-guided strategy.

Another important concern is the potential shortcut introduced by proposal-guided masking. Since the masked regions directly correspond to RPN proposals, the student may learn to exploit the locations and shapes of the gray-filled regions as artificial localization cues rather than learning meaningful object representations. Furthermore, when discriminative foreground evidence is largely removed, it becomes unclear how the student can reliably distinguish visually similar categories, such as cars, trucks, and buses, based mainly on the remaining background context.

Questions for authors
  1. How do the authors justify the effectiveness of Informed Cutout when positive objects are heavily masked but their pseudo-labels remain unchanged? Please consider extending Table 4 with Random Cutout and SOCCER-style stochastic complementary masking under matched masking ratios. How sensitive is performance to masking location and proportion? Additionally, how can the authors rule out shortcut learning from gray-filled proposal regions and explain reliable classification of visually similar categories when foreground evidence is largely removed?

  2. Why is the Cutout filling value fixed at 127? Is there any specific motivation for this choice? Have the authors evaluated other filling values, and how sensitive is the final performance to this parameter?

Reviewer_zZhWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates training instability in source-free domain adaptation for object detection and instance segmentation, focusing on false-negative pseudo-labels that cause genuine objects to receive erroneous background supervision. It proposes Recall-Prior Masking (RPM), combining RPN-guided Informed Cutout with Gradient Masking to suppress potentially harmful supervision.

Critical strengths
  • The paper addresses a relevant practical problem in source-free domain adaptation, namely the instability of pseudo-label self-training and the lack of labeled target data for checkpoint selection.

  • The proposed method is relatively simple, computationally inexpensive, and can be integrated into an existing detection framework without modifying the underlying architecture.

  • The experiments demonstrate encouraging improvements in terminal detection and segmentation performance across three domain adaptation benchmarks.

  • The authors provide ablations separating the contributions of Informed Cutout and Gradient Masking, alongside analyses of RPN recall and training dynamics.

Critical weaknesses
  • The proposed contributions have limited novelty and are insufficiently positioned against very similar prior work. The "false-negative trap", presented as contribution C1, is a well-known and studied consequence of incomplete annotations in object detection. Combating Noise (NeurIPS 2021) for instance describes how proposals covering omitted objects are incorrectly assigned to background, A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data (AAAI 2021) studies false-negative pseudo-label noise in source-free detection. Even regardless of these works, the false-negative trap seems like a manifestation of precision-recall tradeoff, and I don't believe identifying it is as important or novel as the authors claim. The proposed RPM method also overlaps substantially with established approaches. LPLD (ECCV 2024) identifies potentially omitted objects using RPN proposals, Soft Teacher (ICCV 2021) reduces unreliable background classification losses, and Label, Verify, Correct (CVPR 2022) uses ignore regions to prevent erroneous supervision of potentially missed objects. The manuscript does not adequately distinguish its conceptual or methodological contributions from these predecessors (even though some are cited). Its terminal-evaluation motivation also builds on prior work, particularly Hao et al. (2024).

  • The experimental evaluation is insufficiently convincing. The authors mostly compare RPM against variants of the Hao et al. (2024) framework, with limited controlled comparisons against alternative approaches addressing the same problem. In particular, the experiments omit simple alternatives that directly address the proposed failure mechanism. The authors attribute instability to false-negative pseudo-labels induced by conservative confidence filtering, yet fix the detection confidence threshold at 0.8 without examining its effect (i.e. why is RPM needed? The motivation seems to be Precision-Recall tradeoff, so perhaps tuning the threshold would give similar performance?). A confidence-threshold sweep, a dual-threshold strategy that ignores uncertain detections, or teacher-confidence-based background-loss weighting would test whether RPM offers benefits beyond simpler confidence-based supervision. Comparisons against the supervisory mechanisms of Soft Teacher and existing ignore-region approaches are particularly important given their large similarity to RPM. Without these comparisons, the added value of the proposed method is unclear.

  • The dominant component of RPM, Informed Cutout, is inadequately evaluated. Table 4 shows that Informed Cutout alone improves terminal bbox AP50 from 29.95 to 37.59, compared with 33.46 using Gradient Masking alone. Despite accounting for most of the improvement, Informed Cutout is never compared against standard random Cutout, area-matched random masking, or other conventional spatial augmentation strategies. Moreover, Informed Cutout also occludes regions corresponding to correctly pseudo-labeled objects, suggesting that its gains may arise from general spatial regularization rather than specifically mitigating false-negative supervision.

  • The paper provides insufficient evidence supporting its central causal explanation. Although RPM improves AP and terminal stability, the experiments do not show that these gains result from preventing FN-induced gradients. The authors should measure pseudo-label false-negative rates, erroneous background assignments, the precision of the selected ignore regions, or the evolution of missed-object recall during training. Figure 3 shows higher RPN recall than ROI recall at selected confidence thresholds, but doesn't show that the additional proposals reliably identify omitted objects or explain the performance gains. Furthermore, the method does not clearly specify whether Gradient Masking suppresses erroneous background supervision in both the RPN and RoI heads, which use distinct matching and loss computations.

  • The experiments lack statistical validation of the reported improvements and stability claims. The manuscript provides no independent-seed results, standard deviations, confidence intervals, or statistical significance analysis. The number of independent training runs is not specified, and the reported training trajectories appear to represent individual runs. This is particularly concerning because improved training stability is one of the paper's principal contributions - this makes it crucial to report consistency across stochastic training runs. The smaller gains, such as +2.2 segmentation AP50 and +3.1 detection AP50 on KITTI → Cityscapes, are difficult to assess without estimates of run-to-run variability. Even the larger improvements lack uncertainty quantification, limiting confidence in the robustness of the reported findings.

  • The evaluation lacks controls needed to interpret the reported adaptation and stability gains. Results would be much more convincing with an AdaBN-only baseline evaluated before student optimization. It is currently unclear whether RPM substantially improves over simply deploying the adapted teacher, or whether it mostly prevents degradation caused by subsequent self-training. Additionally, the stability evaluation relies on a fixed 40k-iteration schedule, although several baselines peak much earlier. For example, on B1, the matched baseline reaches its peak detection AP50 at 2k iterations, while RPM peaks at 40k. Although the authors cite Hoyer et al. (2022) as precedent for this budget, that work concerns semantic segmentation under a different training configuration. The authors do not explain why 40k iterations is appropriate for their setting, how the budget was selected, or whether it was chosen independently of target-domain performance. This is particularly concerning because RPM's advantage is substantially larger at the selected terminal checkpoint than at the best intermediate checkpoints. Experiments with alternative predetermined budgets and learning-rate schedules are needed to establish whether the reported stability gains generalize beyond the chosen evaluation protocol. This point is especially important because the authors also do not report statistical significance of their results.

Questions for authors
  • How was the 40k-iteration training budget selected? Was it determined independently of target-domain performance, and (most-importantly) are the stability gains consistent across other predetermined budgets?

  • How does RPM compare against simpler alternatives, e.g. dual-threshold ignore-region supervision, teacher-confidence-weighted background losses, and standard random Cutout?

  • Does RPM improve over the AdaBN-adapted teacher before self-training? Please report the corresponding baseline results.

  • Can the authors provide multi-seed results and direct evidence that suppressing erroneous background supervision on genuinely omitted objects explains the observed improvements?

  • Can the authors clarify the novelty of both the proposed "false-negative trap" and RPM relative to closely related prior work? In particular, how does C1 differ from the false-negative supervision mechanisms identified in Combating Noise (NeurIPS 2021) and A Free Lunch (AAAI 2021), and how does RPM conceptually differ from existing ignore-region, background-loss weighting, and RPN-based proposal selection methods?

Reviewer_Tb12OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper challenges the source-free adaptation for detection models. The key insight is about the false negative regions, i.e., object that are not assigned with pseudo-labels and thus receive background supervision by the detector's matcher (the "false-negative trap"). It proposes RPM, which uses a frozen AdaBN teacher's class-agnostic RPN proposals in two ways: Informed Cutout (IC) occludes proposal regions in the input, and Gradient Masking (GM) ignores background losses on proposals not covered by pseudo-labels. Under a fixed 40k-iteration budget, RPM improves LAST AP50 over AdaBN+Fixed SF-FM† on three driving-scene shifts, for both detection and instance segmentation.

Critical strengths
  • Evaluating terminal performance without target-label checkpoint selection is a good protocol, and BEST/LAST/∆ are reported consistently.
  • The main comparison is tightly matched, and the method is simple.
  • Gains are consistent across three shifts and two tasks.
Critical weaknesses
  • IC is poorly motivated and may not work through the claimed mechanism. By design, IC also occludes pseudo-labelled objects while keeping their targets. This design is very confusing: how could a region without any actual object be supervised with positive signal? Yet IC gives the largest gain (Table 4a), and protecting labelled regions hurts (Table 4b).
  • Positioning and clarity. GM is essentially an ignore (gray) label, a well-known idea in detection (e.g., ignore bands in anchor assignment, background-loss reweighting for missing annotations), but this is neither stated nor discussed. LPLD is not reproduced under the same protocol. The introduction would be much clearer if it stated this ignore-label idea up front and then explained how the ignore regions are identified.
Questions for authors
  • Under default IC, how are occluded pseudo-labelled objects supervised? What fraction of object area is occluded, and are losses still applied to fully occluded objects?
  • Figure 2 appears to show pseudo-labelled objects left unoccluded, with a black fill instead of κ=127. Which IC variant does it depict?
  • What are the results of a random Cutout control matched in count and area?
  • What do the component and τo ablations look like on the main RPM† configuration?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.