Learning Where to Adapt: Target-Aware Parameter-Efficient Fine-Tuning for Neural Operators

Parameter-efficient fine-tuning (PEFT) adapts pretrained neural operators to new physical regimes with a small trainable budget, but performance depends on where that budget is placed. We show that neural-operator PEFT is governed by two distinct constraints: accessibility, whether the trainable parameterization can reach useful target updates, and capacity, how much of those updates it can represent. Across controlled physical shifts, useful updates concentrate in different architectural parameter groups—the spectral kernels of a Fourier Neural Operator and the branch network of a DeepONet. Once capacity is adequate, restricting adaptation away from these updates imposes an error floor that additional rank cannot remove. We turn this observation into a cheap target-aware placement rule: a short multi-batch warm-up on target training data, scored by groupwise functional and task-loss improvement, identifies useful parameter groups and detects diffuse regimes where concentrated placement is unwarranted. On FNO, placing a standard adapter where the probe points reaches $0.93$–$1.07\times$ the full-fine-tuning error using 3–12% of the trainable degrees of freedom and outperforms the tested fixed-placement and adaptive-rank PEFT baselines at matched budget on all six shifts. On DeepONet, the probe identifies branch-localized adaptation while assigning low concentration to diffuse regimes. Increasing PEFT capacity cannot compensate for useful target updates that its trainable parameterization cannot access.