Gauge Symmetries Impose Limits on Global Parameter Recovery for LoRA and Attention

Low-Rank Adaptation (LoRA) updates and Multi-Head Attention (MHA) heads have inherent symmetries, so many weight arrays describe the same update or attention computation. In this work, we ask how many deterministic continuous candidate rules are needed to recover parameters up to these symmetries. Each rule takes a population distribution as input and returns a point in the (compact) quotient space and we require that, for every target, at least one rule returns a nearby point. We show that for even embedding dimension $d$ and even rank $r$, sufficiently accurate LoRA recovery requires at least $r(d-r)/4+r/2$ candidates in the low rank-regime. So for example, at $d=4096$ and $r=64$, this gives that $64544$ continuous candidate rules are required for recovery. For even $d$ and $h$ attention heads with equal even key/value dimension $r$, we show a corresponding lower bound of $hr(d-r)/2+(h+1)(r/2-1)+1$. Thus stable, uniform, population-level proper recovery requires lists of size $\Omega (rd)$ for LoRA and $\Omega (hrd)$ for MHA. Our approach for the proof is to turn the success domains of the candidate rules into a topologically simple cover and then argue about its size through nonzero products in cohomology, which is measured by the cup-length. We develop a cup-length addition principle for fiber bundles admitting a global section under a freeness condition, and show that stronger hypotheses yield an exact formula. We do an explicit computation using tools from algebraic topology to identify the surviving classes for the quotient parameter space of LoRA, and symmetrized products handle head permutations for MHA. Together, these necessary list sizes establish an architecture-level lower bound to stable parameter reconstruction for both LoRA and MHA.