acceptodds
Withdrawn from ICLR 2027

Can Score-Based Diffusion Models Discover Physical Constraints?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Scientific data often satisfy physical constraints that are unknown a priori. In this work, we study whether a score-based diffusion model trained on scientific data can encode geometric information that may be used for constraint localization. To do so, we first show that an appropriately-scaled score function provides a projection-like displacement toward the data-support manifold; and that we can leverage this projection-like behavior to reveal geometric information about constraints that were never explicitly provided during training. Motivated by these observations, we propose a new algorithm, SCoREMaP (Score-based Constraint Residual Extraction and Manifold Projection), to localize constraints from a trained score field. Theoretically, we show that for sufficiently accurate trained score fields, the resulting constraint region converges to the manifold in Hausdorff distance. We also show that we can propagate uncertainty over the score-model parameters to obtain pointwise epistemic uncertainty for the extracted constraint information. Empirically, we show that SCoREMaP localizes physical constraints more accurately than data-only baselines across a variety of geometric, molecular, and PDE-based tasks. These results show that diffusion models implicitly learn geometric information about physical constraints.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
discovery science, learning physics from data, score-based diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gpEoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses a trained score-based diffusion model to locate the low-dimensional constraint set (manifold) that the training data lies on. Under the manifold hypothesis, prior work shows that a score perturbation approaches the projection onto this manifold The authors build a new method, which they call SCoREMaP, based on this. This method also allows for uncertainty computations as well. The main contributions are theoretical support for the SCoREMaP algorithm, the ability to use a Laplace approximation to get uncertainty maps in the ambient space, and experiments that demonstrate it beating unlearned baselines.

Critical strengths

The method is simple and computationally cheap with a pretrained score model as the primary primitive. The most expensive piece of the method is the score evaluation. Because it operates directly on the learned score field without requiring explicit physical equations, it is potentially applicable across a broad range of scientific domains.

The experiments demonstrate substantial improvements over all tested unlearned baselines. These gains hold across multiple problems and random seeds, suggesting that the learned score field captures constraint geometry more effectively than conventional methods. On molecular tasks, the method also remains competitive with baselines that explicitly incorporate rotational symmetry.

Critical weaknesses

First, the theoretical novelty of the method seems limited. The central residual used follows naturally from the connection between denoising and manifold approximation in the work of Permenter and Yuan (2024). The main theorem is mainly an application of a triangle inequality argument under stringent assumptions. Therefore, I don't know that this work adds much to the theoretical literature around score-based models or manifold learning.

The title is misleading, and the framing is overstated. The method does not appear to extract physical laws or governing equations, and rather is just another manifold learning algorithm. Therefore, the contribution of the work to the literature is much less practical than if it were more tailored and focused on the recovery of specific physical laws.

Questions for authors

Why don't the authors compare to other learned baselines? I feel that they should actually position this work in relation to the broadest possible literature. Could the authors compare SCoREMaP against learned alternatives to establish whether its gains arise specifically from the score-based formulation rather than from learning a flexible representation of the data geometry?

Could the authors comment more on the theoretical novelty - what in particular was developed here that is new and will have broader impacts?

Reviewer_E3mPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks whether a diffusion model trained on samples from a constraint manifold M can localize M. The method "SCoREMaP" evaluates the Tweedie displacement σ_t² s_\theta(y, t) of a trained score model at probe points at one small noise level (σ_t = 0.05), uses its squared norm F_{\theta,t}(y) as a surrogate for dist(y, M)^2, and thresholds it to obtain a tube around M. The theory imports \aht{M}_t = M + O(σ_t²) from Huang et al. (2026), assuming the learned projection is ε-accurate in sup-norm on the tube. A last-layer Laplace approximation adds a pointwise uncertainty U_t. Experiments cover 23 noise-free synthetic tasks (sphere products, Stiefel manifolds, molecular bond constraints, 1-D Burgers trajectories) and report a pooled IoU of 0.87 against at most 0.29 for untrained nonparametric baselines, parity with O(3)-invariant baselines on the molecular tasks, and a change of −11% to +43% in Burgers rollout error when the projection is inserted into a SINDy rollout.

Critical strengths

The method allows to read out potential manifolds from the model evaluations. F_{\theta,t} follows dist(y, M)^2 over several orders of magnitude on all 23 tasks (Fig. 2), and on sphere-product and Stiefel tasks it beats every nonparametric estimator per task. This matters because any trained score model can be queried this way at no training cost. 2. The authors are honest about their shortcomings. Training curves, full σ_t × \tau sweeps (Figs. 8–11) and the rollout degradation at low viscosity (Fig. 12) are shown. This matters because most of the weaknesses below can be checked from the paper itself.

Critical weaknesses

The main problem is the framing of the idea: why would we have train and evaluate a model if the model is trained on the data so we could also just analyze the data itself? By definition of the manifold hypothesis, the model can't do better than the training data itself. The main point of generative models is to interpolate on a given manifold to generate new samples. While this implies that the model learned the manifold and can reconstruct it, as a function approximator it can't really be better than the original data. In that case I can prevent the whole round trip through the nodel training and instead directly evaluate the data. On a more abstract level, understanding your data is a crucial component of machine learning. The author's approach would assume that the data is taken as a black box. A simple question like "How is the data generated?" or asking a domain expert what constraints are enacted on the data would obviate a lot of the extra work through training a model. A particular example are the rich geometric priors that flowed into models like AlphaFold or how diffusion models train on the score while keeping the entire diffusion machinery around it in place (which is a very strong generative prior). In both cases people though long and hard to understand the underlying data and embedding significant priors. Similarly to train the diffusion model one would have to generate a sufficient amount of data which could then be analyzed by the domain expert. I'd argue that it doesn't take too much work to plot multiple data axes against each other. This would be significantly faster than training a whole diffusion model and subsequently rerunning the sampling/inference again.

Questions for authors
  1. See the weaknesses.
  2. How would the manifold information be used downstream on a new taks? Currently my understanding is that for each task a new diffusion model has to be trained. This necessitates generation the entire training data, training the model and rerunning the sampling/inference again for that particular task?
Reviewer_Gir3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SCoREMaP, a theoretically justified algorithm for localizing unknown physical constraint manifolds using trained score-based diffusion models, without requiring explicit knowledge of the underlying governing equations. Building on the projection-like behavior of score fields, the authors construct an approximate manifold projector and introduce a surrogate metric, (Fθ,t(y)F_{\theta,t}(y)), for estimating the squared distance from a point to the underlying manifold. They also apply parametric epistemic uncertainty quantification (UQ) to provide uncertainty estimates. The proposed method is evaluated on various tasks, including synthetic geometric manifolds, molecular geometry, and PDE simulations.

Critical strengths
  1. Simplicity and intuitive motivation. The proposed algorithm builds on a simple observation from existing score-based diffusion literature. The method is intuitive and theoretically justified under its stated assumptions. Moreover, it does not require explicit knowledge of the underlying physical constraints, making it potentially applicable to a broad range of scientific problems.

  2. Strong performance over baselines. Table 1 demonstrates substantial improvements over traditional data-only baselines across geometric and matrix manifold tasks. Figure 2 also shows strong correlations between (Fθ,t(y)F_{\theta,t}(y)) and the true squared distance to the manifold, suggesting that the proposed metric can serve as a useful surrogate for constraint residual estimation.

  3. Diverse experimental settings. The authors evaluate the proposed method across several types of constraints, including geometric, molecular, and PDE-based constraints. They further demonstrate its potential utility in uncertainty quantification and downstream numerical simulation, providing evidence beyond basic manifold localization.

Overall, this paper builds on a simple yet useful intuition and proposes an effective approach to extracting geometric information from trained score-based diffusion models.

Critical weaknesses
  1. Limited improvements in Table 2. Although SCoREMaP substantially outperforms the baselines in Table 1, its advantage becomes much smaller in Table 2 when compared against O(3)-invariant baselines. In particular, it performs worse than O(3)-invariant KDE on the Fixed-Angle Chain task. Further analysis is needed to explain why the proposed method provides limited improvements in these settings and what factors determine its advantage over existing methods.

  2. Insufficient evaluation of the surrogate metric. Figure 2 mainly reports Pearson correlations between (F_{\theta,t}(y)) and the true squared distance to the manifold. However, high correlation does not necessarily imply accurate distance estimation. Quantitative error metrics, such as RMSE or relative error, as well as calibration analysis, would provide stronger evidence that (F_{\theta,t}(y)) is a reliable surrogate. An analysis of cases with large estimation errors would also be valuable.

  3. Limited improvements and unexplained failures in PDE simulations. Figures 3 and 12 show relatively modest improvements over vanilla SINDy, with substantial performance degradation under low-viscosity conditions. For example, the error increases by 43.3% at (\nu=0.03). The authors briefly attribute this to the difficulty of solving Burgers' equation at low viscosity, but do not investigate why the additional manifold projection worsens the results. Further analysis of these failure cases would strengthen the paper.

  4. Limited analysis of manifold complexity. The proposed method performs particularly well on geometric and matrix manifold tasks, while its advantages are less consistent in molecular geometry and PDE applications. This raises questions about how performance depends on manifold complexity. Experiments varying curvature, intrinsic dimension, codimension, or sampling density would help characterize the conditions under which the proposed method succeeds or fails.

  5. Lack of detailed failure-case analysis for FarFPR. Although Table 1 reports low FarFPR overall, it would be useful to investigate the remaining false positives. For example, the authors could visualize their locations relative to the true manifold, examine how false-positive rates vary with distance, and analyze whether these points correspond to large surrogate estimation errors or high epistemic uncertainty. Such analysis would help explain the limitations of the learned score field.

  6. Comparison with physics-informed methods. Although the proposed method intentionally assumes no explicit knowledge of the governing physics, comparisons with physics-informed or trained alternatives could help clarify its practical strengths and limitations. For example, PINNs or FNO-based approaches could serve as reference points in the PDE simulation task, while acknowledging their different assumptions and objectives.

  7. Gap between constraint localization and physical law discovery. The paper demonstrates that score-based diffusion models can approximate the distance to an underlying constraint manifold. However, it does not show how the recovered manifold or surrogate residual can be translated into explicit, interpretable governing equations. This leaves a gap between the demonstrated constraint localization capability and the broader goal of physical law discovery. Furthermore, the selection of localization thresholds relies on oracle-based validation, and it is unclear how these thresholds should be chosen when the underlying manifold is truly unknown.

Overall, my concerns primarily relate to experimental evaluation and analysis. While the paper presents results across diverse tasks, the limited analysis of failure cases and performance differences makes it difficult to understand when and why the proposed method works well, as well as its practical limitations. I have concerns about whether the current empirical comparisons sufficiently establish the advantages of the proposed approach.

I am open to increasing my score if the authors adequately address the concerns raised above.

Questions for authors

See weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.