The False-Negative Trap: Stable Source-Free Instance Adaptation with Recall-Prior Masking

Source-free unsupervised domain adaptation for instance-based detection and segmentation relies heavily on self-training with pseudo-labels. Because target labels are unavailable under the unsupervised setting, a deployable method must deliver strong terminal performance under a fixed training schedule rather than relying on peak checkpoint selection. However, in practice, self-training is often unstable: noisy pseudo-labels can accumulate harmful supervision and degrade terminal performance. We study a detector-specific failure mode of this paradigm: when a real target object is omitted from the pseudo-label set, the detector matcher can convert that omission into explicit background supervision. We refer to this mechanism as the false-negative trap. We introduce Recall-Prior Masking (\rpm), a conservative strategy that uses the frozen teacher's region proposal network (RPN) as a class-agnostic recall prior. Rather than converting uncertain proposals into additional pseudo-positive supervision, \rpm treats them only as evidence that background supervision may be unsafe. By suppressing possibly unlabelled visual evidence via Informed Cutout and masking localized background losses, \rpm aims to prevent harmful optimization. Under a predeclared common-budget protocol on three driving-scene adaptation benchmarks, \rpm improves terminal object-detection and instance-segmentation performance and reduces late-training degradation. In a strictly matched comparison with \adabn + Fixed SF-FM, adding \rpm improves Last AP${}_{50}$ by $+8.9$/$+11.6$ segm/bbox on SYNTHIA$\to$Cityscapes, $+4.8$/$+4.8$ on Cityscapes$\to$Foggy Cityscapes, and $+2.2$/$+3.1$ on KITTI$\to$Cityscapes. These results show that explicitly protecting potentially omitted objects from erroneous background supervision substantially improves terminal self-training performance and reduces late-training degradation.