Textual Residual Prompt for Fine-Grained Real-Time Open-Vocabulary Detection

Real-time open-vocabulary detectors combine open-category recognition with efficient inference, but still lag significantly behind heavy open-vocabulary detectors on fine-grained detection tasks. We observe an interesting property in the textual embedding space: semantic residuals induced by the same fine-grained attribute remain highly consistent across different base categories, suggesting that fine-grained attributes correspond to relatively stable and transferable semantic directions. Motivated by this observation, we propose $TextualResidualPrompt(TRP)$, a lightweight adaptation method for fine-grained real-time open-vocabulary detection. Instead of modifying the detector or relearning complete textual representations, TRP learns textual residuals to enhance fine-grained discriminability. The resulting textual representations can still be cached before inference, preserving the efficient prompt-then-detect paradigm. Experiments show that TRP achieves superior fine-grained detection performance among parameter-efficient fine-tuning methods while maintaining real-time inference efficiency.