VIRA: Semantic Attribute-Guided Adaptation for Continual Open-Vocabulary Segmentation

Open-vocabulary segmentation (OVS) must incorporate newly observed data while preserving its ability to respond to changing query vocabularies. However, continual fine-tuning can couple adaptation to the training vocabulary and degrade performance across other domains and class sets. We introduce \method (Vocabulary-Induced Relation-Guided Adaptation), which represents runtime classes through shared semantic attribute coordinates. Class–attribute relations are grounded with region-level visual evidence and converted into a fixed-dimensional condition that controls low-rank residual composition. For continual deployment, completed adaptations are frozen and retained, while compatibility retrieval and region-wise arbitration determine when to use them or fall back to the frozen base. We evaluate \method on a COCO-pretrained $\to$ A-150 $\to$ Cityscapes $\to$ Mapillary Vistas stream, together with retention benchmarks, expanded taxonomies, and cross-architecture transfer. Capacity-matched controls, coordinate-inventory comparisons, mechanistic interventions, and routing diagnostics examine the role of semantic attributes and deployment cues. Across the three incremental domains, \method improves macro mIoU/PQ by +4.33/+2.84 over the frozen source, while preserving access to frozen-source behavior on retention benchmarks.