Calibrated Concept Denoising for Open-Vocabulary Semantic Segmentation

Frozen vision-language encoders support open-vocabulary segmentation, but their dense features often mix target evidence with correlated background and compet- ing concepts. This motivates a query-dependent scoring operation: the same chan- nel can be useful for one concept and misleading for another. We propose Cali- brated Concept Denoising (CCD), which reweights visual channels for each text query while retaining dense cosine scoring. CCD transfers concept masks from SmartCLIP, refines them with a prior-preserving residual adapter, and estimates masks directly from labeled competitor errors. A labeled set of 1,000 training images per dataset supports adaptation and branch selection; the selected branch is fixed for validation. Under the released ClearCLIP evaluator, CCD improves ViT-B/16 average mIoU by 0.48 points across six benchmarks, including a 2.20- point gain on Cityscapes. Controlled experiments across three feature families and a ViT-L/14 extension characterize where query-conditioned channel reweighting helps.