Preserving Manifold Capacity: A Geometry-Aware Dual-Prompt Framework for Open-World Cross-Modal Retrieval

In large-scale cross-modal data mining, constructing an embedding space that generalizes to open-set distributions is a fundamental challenge. We investigate a geometric failure mode, termed “Feature Space Cannibalization," in which excessive adaptation to seen categories may reduce the representational flexibility required for generalization to unseen categories. This paper proposes a Geometry-Aware Dual-Prompting (GADP) framework to address this topology distortion. We introduce Modality-Adaptive Contextual Tokens as distribution anchors to adapt heterogeneous inputs (e.g., sparse sketches and dense images) to a pre-trained manifold without catastrophic forgetting. Crucially, we reinterpret an established angular-margin objective as a compactness regularizer for zero-shot cross-modal retrieval. By synergizing global alignment with angular margin regularization, we actively compact the distributions of seen classes, encouraging compact seen-class distributions and preserving representational flexibility for unseen categories. Experiments on three datasets under four standard evaluation settings demonstrate the effectiveness of GADP. In particular, the method reports 34.5% mAP@all on QuickDraw Ext. under our evaluation setup.