Can Score-Based Diffusion Models Discover Physical Constraints?

Scientific data often satisfy physical constraints that are unknown a priori. In this work, we study whether a score-based diffusion model trained on scientific data can encode geometric information that may be used for constraint localization. To do so, we first show that an appropriately-scaled score function provides a projection-like displacement toward the data-support manifold; and that we can leverage this projection-like behavior to reveal geometric information about constraints that were never explicitly provided during training. Motivated by these observations, we propose a new algorithm, SCoREMaP (Score-based Constraint Residual Extraction and Manifold Projection), to localize constraints from a trained score field. Theoretically, we show that for sufficiently accurate trained score fields, the resulting constraint region converges to the manifold in Hausdorff distance. We also show that we can propagate uncertainty over the score-model parameters to obtain pointwise epistemic uncertainty for the extracted constraint information. Empirically, we show that SCoREMaP localizes physical constraints more accurately than data-only baselines across a variety of geometric, molecular, and PDE-based tasks. These results show that diffusion models implicitly learn geometric information about physical constraints.