Low-Rank Adapter Structure Should Be Read Off, Not Written In

Low-Rank Adaptation (LoRA) efficiently represents task-specific updates, but an important question remains: how much structure does a task actually need? Standard LoRA fixes the number of rank-one components in advance and uses dense factors. We propose SPAR (SParsity And Rank, adaptively read), a two-stage framework that uses task information to determine component counts and factor sparsity. During fine-tuning, Adaptive Rank compares each module's task-loss gradient signal with its historical peak to guide component growth and stopping. After fine-tuning, Adaptive Sparsity refits each learned adapter update with sparse factors to obtain a compact representation. Component counts and coordinate supports are selected through input-aware reconstruction and model-level task-quality checks, without a prescribed coordinate keep ratio. Thus, SPAR uses task information to determine how much capacity to build during learning and how much structure to retain afterward. On eight GLUE tasks with DeBERTaV3-Base, Adaptive Rank achieves a higher average score than existing LoRA baselines with fewer factor parameters. Adaptive Sparsity further reduces the mean number of nonzero factor parameters by 33.4% while maintaining task performance. We also demonstrate that SPAR is effective with Qwen3-4B-Base on the mathematical reasoning benchmarks MATH500 and AIME2024. The two stages also apply to other adapter methods, as illustrated by adaptive capacity allocation in LoRA and LoRA+ and sparse refitting of AdaLoRA and IncreLoRA updates. These results support our central view: adapter structure can be read off from task-dependent signals rather than written in as a prescribed final rank and coordinate keep ratio. Source code is available at https://anonymous.4open.science/r/SPAR-6F72/.