What Drives the Prediction? How Neural Models Rely on Physical Information

Physical quantities such as energy and momentum can be decoded accurately from a neural model even when its predictions respond differently to a change in available signals. We study this gap between physical representation and predictive reliance. Matched Invariant Interchange (MII) compares activation exchanges selected for their correlation with a physical quantity against random exchanges of the same size. A complementary regression task varies the availability of a target-correlated hint during training and removes its informativeness at evaluation. Our contributions are: i) Decodability–coupling dissociation: across five dynamics systems and five exchange sizes per system, high decodability coexists with a maximum absolute MII gap of 0.0068. ii) Distribution-dependent reliance: across five invariant-regression systems, informative-hint and decorrelated-hint training both yield decoding $R^2=0.99$–$1.00$, yet mean absolute output–target correlation after hint permutation is 0.09 versus approximately 1.00. Decorrelated-hint training also increases sensitivity to invariant information relative to hint information. iii) Output-local coupling: in pretrained Poseidon-B, the decoder MII gap is 0.893 for the spatial integral of one velocity component, $\int u$, and 0.027 for kinetic energy. The comparisons distinguish readable physical information, training-dependent signal reliance, and sensitivity arising near the output. They support evaluating physical representations through matched interventions and behavior after a predictive training signal changes.