No Free Soup for PINNs: Just Keep the Best

Averaging the weights of a pool of fine-tuned models into a model soup yields ensemble-level accuracy at the cost of a single inference, a practice well established in vision and language. For Scientific Machine Learning (SciML) and its techniques, such as Physics-Informed Neural Networks (PINNs), this is especially attractive: the result stays a single continuous, differentiable surrogate. We present the first systematic benchmark of weight averaging for PINNs for the forward problem, spanning several PDE families in both soft- and hard-constrained form and a range of widths, in double precision throughout. The odd activations standard in PINNs induce a sign symmetry that permutation matching leaves intact. We therefore enlarge the matching group to $(\mathbb{Z}/2\mathbb{Z}{)}^d⋊S_d$, an extension we call Sign Re-Basin. We evaluate it across training regimes, from independently trained ingredients to models that share most of their optimisation trajectory. The output-space ensemble, exact by superposition on a linear PDE, serves as the control that isolates parameter-space from function-space averaging. Using a reference-free selection criterion, the greedy recipe almost never accepts a second ingredient and never a third, and using the reference solution in its acceptance test barely changes this. When a soup does improve on the selected ingredient, the gain is within selection noise, i.e. the gap between the model selected by a reference-free criterion and the best model in the pool. The ensemble stays accurate but loses the main advantage of a PINN over classical mesh-based solvers, a single continuous and differentiable representation, multiplying the query cost by $k$ for a variance reduction that model selection already provides. Given a pool of PINNs, just keep the best model.