HoRA: Rethinking Multimodal Low-Rank Adaptation as Implicit Parameterization

While multimodal low-rank adaptation (LoRA) has achieved strong empirical success, its explicit parameterization makes adaptation capacity closely tied to the dimensions and design of low-rank factors, making modality-wise parameter allocation less directly controllable once the rank is fixed. To address this limitation, we propose feature h ashing-based multimodal l o w-r ank adaptation (HoRA), a novel framework that reformulates multimodal LoRA as an implicit parameterization problem. HoRA constructs low-rank adaptations via feature hashing-based implicit parameterization, generating matrix entries on-the-fly from compact parameter buckets instead of explicitly storing them. By independently controlling the bucket size of each modality, HoRA introduces an additional degree of freedom for fine-grained control over modality-wise adaptation parameters within a fixed-rank formulation. We theoretically characterize the feature hashing-based low-rank parameterization through its unbiased reconstruction and collision-dependent approximation bound under the assumed hashing scheme. We further establish the convergence property of HoRA, with the convergence bound characterized by modality-specific bucket sizes. Extensive experiments across diverse multimodal benchmarks demonstrate that HoRA achieves competitive or superior performance with comparable or fewer trainable parameters. The code is available at https://anonymous.4open.science/r/HoRA-E512.