Are Learned LoRA Rank Allocations Reliable? A Controlled Audit on Llama-2-7B

Most learning-based and heuristic LoRA rank-allocation methods rest on an uncontrolled comparison: in the coupled-seed protocol, each candidate is evaluated on its random initialization, so allocation and seed are inseparable in one number. We propose a seed-decoupled protocol: one master seed derives three independent streams fixing scoring-head initialization, LoRA initialization, and data order/dropout; each candidate is compared pairwise with a shared equal-budget uniform_r8 reference. Within a master seed only the allocation varies, structurally removing the shared component that accounts for 94.1% of run-to-run variance. Cost: one extra reference run per master seed; gain: 4.7–6.6× tighter comparison error at equal runs. On the same panel and budget, the two protocols disagree: the coupled one puts all four search methods ∼1 point above uniform rank 8; under ours, the advantage disappears (the sole equal-budget search output retested: $-0.63$ pp, Holm $p=0.578$). The divergence comes from the training protocol, not the search methods. Fixing the parameter budget and rank multiset while permuting only internal positions shifts accuracy by 4.3 points, four times any method difference this field seeks to detect, and the search output we retest ranks below its own three hand permutations: reported gains thus fall within the variance the literature leaves uncontrolled. Our protocol lets a new rank-allocation method be audited before it claims superiority over uniform rank; fair comparison needs at least three conditions: equal parameter budget, fixed rank multiset, shared reference. Without all three, a comparison may measure placement alone.