Share Once, Adapt Everywhere: High-Rank Adaptation Across Layers and Modules

Recent high-rank parameter-efficient fine-tuning (PEFT) methods improve the expressiveness of weight updates while retaining a separate set of trainable parameters for each target weight matrix. This leaves open whether high-rank adaptation can accommodate more extensive parameter sharing across layers and modules. We introduce SHARE, a high-rank adaptation method that organizes target weight matrices according to their update shapes and reuses a small set of trainable parameter banks across layers and modules. Weight matrices assigned to the same bank reuse the corresponding high-rank update. We provide a first-order analysis showing that fixed parameter sharing defines a reparameterization of the weight updates and derive the resulting update to the model weights. The shared structure also allows each high-rank update to be constructed once and reused by all target weight matrices assigned to the same bank during training. Extensive experiments on commonsense reasoning, mathematical reasoning, and code generation across multiple backbone models show that SHARE consistently outperforms strong PEFT baselines, while achieving competitive training throughput.