OPSA: Orthogonal Phase-Scattering Adapter for Parameter-Efficient Fine-Tuning on Long-Tailed Data

Parameter-efficient fine-tuning on a long-tailed dataset fails in a specific, measurable way. Gradients from majority classes reinforce one direction of the weight change over and over, that direction becomes the principal singular vector, and the rest are overridden, so minority-class behaviour is forgotten. We quantify this with the spectral imbalance $\mathcal{I}(\Delta W)$, the ratio of the largest singular value of the update to the mean singular value, and show that $\mathcal{I}$ alone upper-bounds how much of the update the weakest direction can retain. We then introduce the Orthogonal Phase-Scattering Adapter (OPSA), which represents every bottleneck activation as a quadrature pair. A normalized token is projected to a semantic amplitude, and its phase is generated from that amplitude by a fixed ninety-degree rotation, so the two streams are exactly orthogonal at every token and every step, with no penalty term and no extra parameters. The two streams then gate each other across the phase basin, which curves an otherwise flat response surface and applies a gain that falls as amplitude grows. We prove this contracts $\mathcal{I}$ by exactly the ratio of the dominant gain to the spectrum-weighted mean gain. Because descent carries momentum across the basin floor and up the far wall, momentum-aware phase clipping resets that momentum on contact, which bounds the amplitude the state reaches. On a long-tailed DermaMNIST benchmark with a ViT-B/16 backbone, OPSA attains the best macro F1 against six parameter-efficient adapters, at a spectral imbalance of $2.96$ on the scale $[1,r]$ of Proposition 1, against $4.90$–$7.97$ for every baseline — the lowest value in the comparison and a factor of $2.6$ below plain LoRA. On long-tailed 20 Newsgroups with a frozen DeBERTa-v3-base it attains the best tail and macro F1 against four adapters, leading the strongest baseline on tail F1 by $10.7$ points.