Reducing The Adaptation Gap Through Reachable Fisher Geometry

Parameter-efficient fine-tuning (PEFT) determines not only how many parameters are trained but which local directions a model can move, so similar adapters can affect subgroup losses differently. The relevant curvature matrices are infeasible to form at adapter scale, so practice uses scalar summaries, such as the Fisher trace. What can it reveal, and what does it discard? We answer through the *reachable Fisher*: each subgroup's full-model Fisher pulled back through the adapter Jacobian, under likelihood losses the Gauss–Newton curvature the adapter can reach, whose trace is computable from score-gradient norms without forming the matrix. Under matched subgroup gradients, if the reachable-Fisher difference is positive definite on model-changing directions by a margin exceeding the Hessian–Fisher defect, every sufficiently small nonzero update along them increases the signed gap. The restricted operator norm of the Hessian difference instead determines worst-case quadratic change; a matrix-free Frobenius discrepancy upper-bounds its reachable-Fisher component, with defect and estimation errors kept separate. A trace cannot certify definiteness, nor do matched traces control magnitude: equality rules out a positive-definite difference yet permits the largest operator mismatch that trace allows. LoRA's factorization also precludes unrestricted definiteness in raw coordinates. Across 306 single-seed models, higher trace accompanies greater subgroup difficulty in 75.7% of 1,218 eligible evaluations, and trace matching during training narrows the best–worst subgroup gap in all 30 dataset–encoder–adapter combinations. Yet a held-out audit exposes what it leaves uncontrolled: the observed-label operator discrepancy falls in 23 of these 30, the primary unbiased squared-Frobenius statistic in only 16. The trace is therefore a scalable diagnostic and training heuristic, not a certificate of local gap behavior or matrix alignment.