Make Every Bit Count: Residual Capacity Reallocation for Unsupervised Cross-Modal Hashing

Unsupervised cross-modal hashing (UCMH) maps heterogeneous modalities into compact Hamming spaces for efficient retrieval. However, existing methods typically optimize a fixed-length hash code as a whole, overlooking that individual bits may differ substantially in marginal utility while some cross-modal relations remain insufficiently represented. This leads to capacity misallocation under a fixed hash budget. To address this issue, we propose ReCapHash, a plug-in framework for fixed-budget residual capacity reallocation. ReCapHash first estimates the bidirectional marginal utility of individual hash dimensions and adaptively identifies reclaimable low-utility bits while preserving the remaining dimensions as an informative core. It then contrasts pretrained semantic affinity with the Hamming affinity captured by the retained core to mine unresolved cross-modal relations and organize them into a paired residual graph. Global spectral partitioning and diversity-aware binary recovery transform these relations into complementary residual targets, which are distilled into modality-specific predictors and reallocated to the reclaimed positions. Extensive experiments on MIRFlickr, NUS-WIDE, and IAPR-TC12 demonstrate strong retrieval performance across multiple code lengths and retrieval directions, while consistent improvements over diverse base hashing methods further validate the generality of ReCapHash.