acceptodds
Withdrawn from ICLR 2027

Enhancing Diffusion Weather Model's Physical Consistency with Test-time Guidance

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion-based data-driven weather models now surpass numerical weather prediction in forecast skill, but the meteorology community still does not fully trust their output, because the predictions can violate basic physical principles. Violations such as hydrostatic imbalance and dry-air mass drift not only make a single forecast step look questionable, they also compound over the autoregressive rollouts that operational forecasting relies on. Physics-informed training or fine-tuning can reduce these violations, but the computational cost can be prohibitive. Recent advances in test-time steering offer another opportunity to enforce physical principles, by guiding a frozen diffusion model during sampling. Existing methods such as Manifold Preserving Guided Diffusion, however, measure the violation on a coarse estimate of the final state, which limits how much they can correct. We therefore propose Intermediate Gradient Guidance (IGG), a training-free, plug-and-play method that measures the violation on the intermediate state itself and steers each sampling step of a frozen diffusion model toward states that better satisfy a target physical constraint. Our experiments show that IGG improves physical consistency by up to 72.4%, with no loss in forecast skill and negligible extra computation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
diffusion model, test-time guidance, physics-informed machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hBjuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The main goal of this paper is to improve the physical consistency of pretrained weather forecasting models without retraining or substantial additional inference cost. Authors proposes Intermediate Gradient Guidance (IGG), a training-free method for improving the physical consistency during sampling. To improve the constraint satisfaction of sampling, IGG calculate the normalized gradient for the correction based on the noisy state rather than the clean state. The authors verify the method on GenCast and Atlas with hydrostatic-balance and dry-air-mass objectives. The result show improvements in physical-residual agreement with low computational overhead.

Critical strengths
  • The method demonstrates improvements across two forecasting backbones, GenCast and Atlas
  • Authors provide abundant ablation study to empirically support the proposed method.
Critical weaknesses

1. Insufficient justification of the method.

While the experiments support the method’s empirical effectiveness, a logical gap remains between reducing constraint violations in a noisy state and improving constraint satisfaction in the final output. Since intermediate-state residuals may reflect sampling noise rather than the constraint violation in the clean output, the resulting guidance direction may be inaccurate for the final objective. The paper therefore needs a clearer conceptual or theoretical justification for this connection

2. Marginal novelty.

The proposed method is simple and closely follows MPGD, with the main modifications being the gradient evaluation point and correction scaling. These changes appear incremental, and their combination does not clearly establish a substantial methodological advance beyond the existing guidance framework. Moreover, insufficient justification on the proposed approach further limits the contribution of this work.

Questions for authors

1. Justification for the guidance based on noisy-state

The justification for evaluating physical residuals on noisy intermediate states is insufficient. These states are not yet clean forecasts, so their constraint violations may not be the correct update for the overall final output. To fully support the proposed approach, the paper needs a clearer theoretical or conceptual link demonstrating why noisy-state gradient corrections provide an effective direction for constraint satisfaction of final output, rather than relying solely on empirical performance

2. Comparison with clean-state based correction.

It is recommended to provide an analysis of the relationship between noisy-state and clean-estimate corrections, $g_{\text{noisy}} = \nabla_{z} L(z) $ and $g_{\text{clean}} = \nabla_{z} L(\hat x) $. Measuring their cosine similarity and norm ratio across sampling stages would clarify how their directions and magnitudes differ. Applying both directions with the same correction weight and compare their results would further help to demonstrate the noisy-state correction as a useful surrogate of the clean-state correction.

3. Empirical support in image generation

The proposed method is presented as a general guidance approach rather than a domain-specific weather forecasting technique. However, the experiments are limited to weather models and atmospheric constraints. An additional image-generation experiment comparing IGG with clean-estimate guidance would help to demonstrate the benefits of the proposed approach, correction using noisy-state.

Reviewer_izCpOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an intermediate gradient guidance method (IGG) that does not require training and is plug-and-play, improving the physical consistency of the generative weather predictor during testing. Different from the previous manifold-preserving guidance (MPGD), IGG evaluates the physical residuals at the intermediate state of the sampler, scales the correction values based on the updated norm matching ratio of the sampler itself, thus reducing multiple network backpropagations, and can be run on the GenCast and Atlas networks. Under the conditions of hydrostatic balance and dry air mass constraints, with negligible computational overhead, the ERA5 reference residuals are significantly reduced, and the performance is also good with a moderate reduction in numerical accuracy.

Critical strengths
  1. Evaluating the intermediate state rather than a clean estimate - this is a simple and clear change that avoids reliance on the assumption of the posterior mean and does not require differentiation through a denoiser.
  2. Scales the correction values based on the updated norm matching ratio of the sampler itself, thus reducing multiple network backpropagations.
  3. No need for re-training or architecture changes, and it can improve physical rationality at a lower cost.
Critical weaknesses
  1. The physical consistency was transformed into residuals like those of ERA5, with constraints being simple and without measurement cross-influences.
  2. By rephrasing the same physical quantity, the method could be reversed completely. The author did not provide the corresponding selection rules.
  3. λ\lambda was adjusted on a small development set, and more extensive sensitivity analyses were limited.
  4. No references were filled in a lot of (?) places.
  5. Sec 4.5 should have written Appendix 12, but in fact it was Table 12.
  6. Tables 9 and 10 are identical. The positions where the scales of reproducibility displayed are incorrect.
  7. The MMPS expenses are inconsistent in Section 4.1 and Appendix G.
Questions for authors
  1. Section 5 states that “we do not have a rule for making this choice in advance”. Please answer: (a) Is there a criterion that does not rely on the results of the test set? (b) If the “operator selection” is also included in the selection of the dev set, will it make the small dev set now more prone to overfitting? (c) Recalculating based on native operators, do the abstract of 72.4% and the conclusion “IGG is superior to MPGD”still hold?
  2. Could the authors clarify whether physical consistency throughout the paper should be read as agreement with ERA5's evolving residual structure rather than satisfaction of the discretised governing equations? Since the same σlandσM\sigma_l and \sigma_M (calibrated on March 2022) enter both the guidance objective and DERA5D_{ERA5}, would the Table 1 ranking persist if the scales were recalibrated on the test period, or if a different analysis product used as a verifier?
  3. Section 5 lists cross-constraint effects and multi-objective weighting as open. Is there any preliminary evidence that hydrostatic guidance does not degrade dry air mass drift? Concretely, when two residuals are summed, how is λ\lambda chosen under norm matching, and how are conflicting gradient directions handled? A single initialisation two-constraint ablation reporting DERA5D_{ERA5} for both the guided and the unguided constraint would substantially strengthen the plug-and-play claim.
  4. How sensitive is the selection of λ\lambda by IGG to different seasons/regions and different initial conditions? Could you please share the brief sensitivity scan results on the retained months?
  5. Please fix the layout errors.
Reviewer_jGZ8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies that generative weather emulators (especially pre-trained forecasting models) violate physical constraints that the data they where trained on carries. This is a critical concern in the meteorology community as they historically relied on numerical methods which force these physical biases and are expected to be maintained in an emulator. The paper propose to add a constraints-aware guidance at generation, to avoid re-training while improving the physical consistency of the models, potentially helping the forecast (ideally, not worsening it at least). For that, they select several posterior sampling algorithms and build their own on top of this literature to benefit both from efficiency and reliable conditioning.

Critical strengths

Care about efficiency

  • They propose to revisit MPGD to keep light computation cost of the conditioning algorithm while improving conditioning.
  • They add a new perspective to guidance which is interesting for practical settings. Mixed/lower precision inference makes it faster, at the cost of bigger error. They identify that, when speed is critical in practice, its defect (of speed tricks) can be corrected by guidance.

Interesting question

The scientific community heavily rely on physical properties and assumptions for traditional numerical methods. And they care a lot about how this is respected in the generated distribution of deep learning based emulators. This is a nice attempt to bridge the gap between properties learned from data and constraints at sampling time.

Complete analysis of their method

The method is interpreted (each pieces) and ablated to better understand what each pieces bring. Although the comparison to other guidance algorithms is light, the study of the proposed one is complete. Some interpretation of different stages might miss however.

Critical weaknesses

As the review is asked to flag main weaknesses first, I will insist on the three main weaknesses I identified here. Referring to the Questions section for more concerns and interrogations + suggestions. To summarize, I do think this paper suffer from heavy LLM-based re-writing (the authors state that they used LLMs for writing) and is not easier to understand. If they used it to polish the text and make it clearer, it is not a success. The text is dense, hard to follow, and mined by a lot of ";" separators making sentences huge and not always coherent. (I will try to flag a few below). Moreover, I do think they overlook a bit the main point they claim at the beginning. They could insist more on the positive impact of physical consistency and discussion about why doing that (and why not) is a bit poor. And a lot of notations are complex for nothing and too far from usual naming in the field. Use the clear parallel between physical constraints and likelihood, observation of the field, this naturally lead to posterior sampling as the constraint is an observation of the state, ... This is not clear and notations are too different in this paper.

Metrics

Please, when evaluating a probabilistic model, especially in weather were a collection of usually reported metrics exist, report them. Point-wise fit is not sufficient to evaluate a generative model. And CRPS alone is light. At least report Skill, CRPS, Spread (or spread-skill ratio).

Jumping on the previous point, I'm worried about adding guidance towards physical constraints at sampling. This is half a question I ask to the authors and a discussion I would love to see in the paper. When a prior is learned from data, let's call it p(x)p(x), it should learn physical properties if optimal. In an hypothetical setup where we can split the physical properties ϕ\phi from the rest as p(x)∝p(x∣ϕ)p(ϕ)p(x) \propto p(x\mid \phi) p(\phi), then conditioning the learned prior on the same physical properties will not target the right distribution as it could double the constraint p(ϕ)p(\phi). Math are not rigorous, but it gives the intuition. It would be similar to using a temperature on the likelihood when conditioning, this is not targeting the right posterior but a tempered version. If it doubles the constraint, then the distribution might be under-dispersed. The prior here does not learn perfectly the constraints as you highlight it, but it learns part of them. How should you account for that? Should you? Especially, previous work cared about physical constraints (you mentioned WIND) such as Appa [1] which reports in appendix some physical check compared to how they are respected in the training distribution (ERA5). And they show that the consistency of the prior is quite reasonable.

I also do think that all your tables are hard to understand/easily interpret. Are they similar to score-cards? If yes, it should be better to include a format that is usual in the field. And the captions are not clear not self-explanatory.

[1] Appa: Bending Weather Dynamics with Latent Diffusion Models for Global Data Assimilation

Other methods + comparison + motivation

I do think more discussion should be made about your guidance algorithm and how it motivates from others. Why dropping the tweedie covariance (MMPS) is not a problem? Is is the case in some setups only? What do you lose/gain? What are the assumptions this implies? How can we interpret the steps, especially the call to the likelihood with a noisy state. I also do think that IGG deserves a proper benchmark/evaluation. This is the problem when proposing an algorithm and asking a specific question at the same time. It could be really valuable to the community that you report how the method scores (time, memory, ...) against others in broader settings (I refer to Appendix of MMPS paper for an idea of what could be interesting to include in you appendix as well).

What the baseline consists in is not clear. Is it ERA5, unguided models (yes, if I understood). This is again not clear. And, as mentioned, not simple to read in the results either. Can you show how the physical properties are respected in the data itself? Are they pure or loose constraints? This is missing as well. Furthermore, the authors mention several time that the forecast performances are not changed despite the guidance. But it is not clear what is reported. From the figures and tables, it looks like only metrics on physical properties are reported and not on the forecasted states themselves (so basically, as presented, it seems that only posterior predictive checks are reported, i.e. how more physical are the states after guidance). Although showing that guidance behaves as expected is important, I do think that the main focus should be on the effect it has on the forecast. If more physical states does not imply better forecast or better rollout, ... then what is the added value?

A discussion about the need of physical bias in the samples is also missing. Recent work tried to bypass completely physics-driven biases (via simulated data, ...) and learn directly from observations (e.g. [2]). Their claim is that our knowledge of the world is limited, and biases we put in our models do not represent the complex reality, it remains assumptions. So do we really need to have hard physical constraints? Compared to smooth tendencies for some biases?

[2] AIFS-DOP: End-to-End Medium-Range Weather Prediction from Observations Alone with Machine Learning

Phrasing/reading comprehension + missing the main point

I do think a lot of this point is covered in the rest of the review.

Questions for authors

I lean towards a weak rejection for now. This can change depending on the rebuttal. I do think the idea of the paper is sound but it needs sharper analysis. I asked a lot of questions and I will be happy to re-consider my score if the following are addressed or discussed although I do think there are a lot of shadow area that needs to be addressed for now.

Questions

  1. It is hard to compare to other methods as very little are doing exactly that, hence I understand to choice of the authors to compare at the guidance level. But how do you think this would compare to PINN trained with a loss aware of these physical properties? Shouldn't you mention this part of the literature?

  2. As you use guidance for physical constraints, can you discuss on how this would combine with conditioning on observations (assimilation task)? Practitioners are solving a lot of inverse problems on a daily basis (reanalysis, filtering, operational forecast from observations, ...). It is important to show/discuss/give insights on how compatible it is with observations. As this is a posterior sampling problem, this is compatible since constraints can just be treated as observations. But as many posterior algo you presented are optimization algorithm of a constraint, do you think that the methods might trade-off part of the physical constraints for more observation fit? And vice-versa? How is it a problem, or not?

  3. As already mentioned in the weaknesses, how valid is your algorithm? The observation operator (or the constraint) is defined in clean-space xx as p(y∣x)p(y|x). Usually, methods like MMPS or DPS bridge from the clean to the noisy space by approximating p(y∣xt)p(y\mid x_t) using Gaussian or Dirac delta assumptions of p(x∣xt)p(x\mid x_t). This allows to cleanly express a likelihood on the noisy state. From what you describe, you directly apply the clean likelihood to the noisy state which is theoretically not motivated. Can you elaborate on why, why would it be valid, or which assumption (potentially reasonable) this rely on and arrive to IGG? Quick intuition: if your likelihood is a Navier-Stokes simulator and that you call it on a noisy guess of a fluid state, I'm convinced that this will not work as the pde solver might never terminate/converge. By definition, likelihoods p(y∣midx)p(y|mid x) are not made for noisy states.

  4. How do you pick/select λ\lambda? One of the main advantage of MMPS is to be parameters-free. The guidance scale (or step size) is directly given by a kalman gain based update. Can you comment on that? Or discuss.

  5. In some settings, results are worse with physical constraint (e.g. CRPS in table 5). Do you know why? Do you have intuitions?

Remarks

  1. line 53 there is a "," in the middle of the sentence which should not be present (after across).

  2. Beware that previous work [1] built on top of SDA and also tried to make the posterior sampling more efficient by stopping the gradient at the tweedie estimate. This is not completely what you do here, but it is worth mentioning (closer to MPGD I think) here.

  3. Why using zz for the noisy state? And this is not consistent with the figure 2 where you use xtx_t. Stick to classical notations and stay consistent.

  4. Algorithm 1 is really not clear. Again, with notations cleaning and better upstream definition of each components, this could be easier to understand.

  5. You mention twice FA-APF but never cite it [2]. Please double check that this does not happen with other methods I might have missed.

  6. Please include MMPS in all your comparisons (Table 1). Why removing it to some?

[1] A Generative Framework for Probabilistic, Spatiotemporally Coherent Downscaling of Climate Simulation

[2] Training-Free Bayesian Filtering with Generative Emulators

Suggestion

The figure 3 has really little added value for me. You could replace it with samples showing a more faithful following of the GT if it is the case, or just a comparison between constrained and unconstrained samples. Or you could use the space for something else.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.