Controlling Spurious Foreground Expansion in Open-Vocabulary Segmentation

Open-vocabulary segmentation identifies image regions corresponding to text queries, even when the queried classes are absent from the image. During single-image test-time adaptation, confidence sharpening can suppress true foreground, while anchor self-training can turn a weak false foreground into its own supervision. The resulting update enlarges the false mask, which then supplies more anchors. This second failure is termed spurious foreground expansion (SFE). To control this feedback loop, the proposed Spurious Foreground Expansion Guard (SFE-Guard) provides a two-stage guard. Credit-Denominated Issuance (CDI) freezes zero-shot semantic evidence and bounds the pseudo supervision issued to each foreground class. Hierarchical Risk Correction (HRC) reviews aggregate and background risks and applies a feasible correction before a candidate update proposed by the optimizer is committed and reused as supervision. Across all three datasets, SFE-Guard improves state-of-the-art Present Dice by 3.066 and 3.177 points for one and multiple queries, respectively. It removes 77.9% of the excess false foreground caused by anchor-based self-training in one-query OVS and restores multi-query absent class area to the zero-shot level.