GDPLoRA: Partitioned Low-Rank Adaptation with Gap-Driven Local Rank Allocation

Low-Rank Adaptation (LoRA) applies a uniform rank to each weight projection, implicitly assuming that all parameters of a projection require equal adaptation capacity. We show that this assumption does not hold at a finer granularity: local submatrices within a single projection exhibit markedly different spectral concentration, so a monolithic low-rank factorization fits them unevenly. Crucially, this structural signal is already present in the frozen pretrained weights, before any fine-tuning. Building on this observation, we propose **Gap-Driven Partitioned LoRA (GDPLoRA)**, a fine-grained adaptation framework that tailors rank allocation to local weight sub-structures. GDPLoRA partitions each frozen weight matrix into functionally cohesive domains via direction-based clustering, then redistributes a fixed per-projection rank budget according to local spectral concentration. Because domains and rank assignments are determined offline from the frozen weights, GDPLoRA requires no input-dependent routing and preserves exact mergeability into the pretrained weights, leaving inference overhead unchanged. Experiments demonstrate that GDPLoRA outperforms competitive PEFT baselines across multiple backbones and tasks.