UaRA: Uncertainty-aware Rank Allocation for Parameter-Efficient Fine-Tuning of Diffusion Large Language Models

Diffusion large language models (dLLMs) have recently attracted increasing attention for their iterative denoising paradigm and potential for flexible parallel generation. Unlike autoregressive generation, diffusion decoding operates over continuously changing noise states, leading to varying adaptation demands throughout the generation process. However, existing parameter-efficient fine-tuning (PEFT) methods such as LoRA are primarily designed for autoregressive models and do not account for the dynamic adaptation demands of diffusion decoding. In particular, standard LoRA adopts a fixed rank, allocating the same adaptation capacity to all tokens and noise conditions despite their varying prediction difficulty. Existing adaptive-rank methods such as AdaLoRA are also not directly suited to dLLMs, because their gradient-based importance estimates can be distorted by noise-dependent loss scaling rather than reflecting intrinsic parameter importance. To address this issue, we propose UaRA: Uncertainty-aware Rank Allocation for Parameter-Efficient Fine-Tuning of Diffusion Large Language Models, which dynamically allocates low-rank capacity according to the evolving adaptation demand across noise levels. UaRA first calibrates gradient-weight sensitivity using the known loss-scaling coefficient, removing noise-induced scale bias from parameter-importance estimation. It then models the importance of each rank component as a function of the noise level using second-order Legendre bases fitted by online recursive least squares, enabling the active-rank distribution to adapt continuously throughout diffusion training. Extensive experiments demonstrate that UaRA consistently outperforms strong PEFT baselines, validating the effectiveness of uncertainty-aware and noise-conditioned rank allocation for dLLMs.