When Fisher Information Mispredicts Prior Sensitivity: Model Misspecification, Not Noise

Physics-based inverse models resolve what the measurements cannot determine by adding a prior, so part of every recovered quantity comes from the prior rather than from the object measured. Fisher information is the standard tool for predicting how large that part is. We show that the prediction fails, characterise when, and give the mechanism. For a maximum a posteriori fit whose Fisher matrix has a near-null direction $v$, the omitted curvature term multiplies every parameter's prior sensitivity by a single scalar, $1-v^{\top }Cv/\lambda$, in which the parameter index cancels. The sign of that scalar is the correlation between the fit residual and the model's curvature along $v$. That correlation vanishes for noise independent of the model, which is what the standard justification assumes. On 4,608 clinical MRI exams it does not vanish, and two nulls built from the cohort's own residuals reject both halves of the assumption. The Fisher prediction is then too large by 29–38%, exceeds the measured sensitivity in $92.8\%$ to $98.8\%$ of exams, and loses the ordering across exams for $T_1$. Because the error occupies one eigendirection, it can be repaired without computing the curvature: shifting a single eigenvalue recovers 92–99% of the full-Hessian correction at two gradient evaluations, whatever the parameter count. Both the failure and the repair reproduce on public diffusion MRI from another laboratory and on outdoor photovoltaic modules, and both vanish on transit photometry, where the residual is genuinely noise. The same substitution underlies Laplace approximations, influence functions and Fisher-based continual learning. Measured on a neural network, with weight decay as the prior, Fisher over-predicts how far the weights follow a shifted prior by $2.0$ to $3.4$ times across seven widths, and the exact curvature recovers it to $0.5\%$. On the clinical images, self-supervised networks differing only in random seed disagree $2$ to $3.2\times$ more along the unconstrained direction than across it. What the measurements cannot determine, the training run decides.