acceptodds
Under review as a conference paper at ICLR 2027

PhysLensDiff: Valid-by-Construction Lensing Emulation, and Why Physics Penalties Fail

Abstract

Simulators obey their equations but are slow; the generative models that emulate them are fast but usually carry no guarantee that their samples obey those equations, especially far from the training data. We study the two standard fixes on strong gravitational lensing, where validity can be checked exactly. The soft fix adds the lens equation to the training loss as a penalty. It lowers the violation, but by darkening the images: we prove that the penalty always rewards darkening, and observe that the samples lose their content and stop following the requested lens, in lensing and in divergence-free flows. Reweighting or normalising the penalty does not help. The hard fix avoids this. PhysLensDiff generates the source, the lens and the dark-matter substructure, then ray-traces them through a differentiable lens layer, so every sample is exactly valid for any requested lens, at about 0.1% of generation time. Unconstrained image generators, even when given the full lens, remain far from valid out of distribution. Our with/without-substructure pairs train dark-matter classifiers that are more accurate on held-out simulated systems than the best alternative from image generators.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
strong gravitational lensing, physics-constrained generative models, diffusion models, differentiable simulators, dark matter substructure, scientific emulators, hard and soft constraints, physics-informed diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_b9pmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper shows that if generative models for physics simulation still follow the rules of physics beyond the training data. There are two constraints: soft and hard constraints. This paper demonstrates that soft constraints will most likely make the image darker instead of improving the quality. The authors propose a method PhysLensDiff by using hard constraints instead; the results are more consisttant and the generated image results are always match with phsycial lens with negligible cost.

Critical strengths

S1. Clear message with a proof. Proposition 1 explains why the soft penalty fails (it rewards darkening), and the experiments show the same: at λ=100\lambda=100 out-of-distribution samples keep only about a fifth of their energy, and the shear response drops from +0.44 to +0.04 (§4, Table 2).

S2. Fair comparisons. The baselines get the same budget and architecture, and even the DDPM with the full ray map still gets 12.5× worse out of distribution (Table 1). So the failure is not just missing information or small model.

Critical weaknesses

W1. Title is broader than the experiments. The title says "Why Physics Penalties Fail", but only one kind of penalty is tested, a penalty in the training loss. Using the physics during sampling (like DPS) is not tested, which authors also say in §6. So either add one experiment with sampling-time guidance, or make the title more specific, e.g. "training-loss penalties".

W2. Only one simulated dataset, not public. All lensing results use Model A, an unpublished simulated dataset (§4, §6). I understand public simulators may not give the needed intermediate values, but it is hard for others to check or reproduce the results. Also the FlowLensing baseline was trained 15k of the published 70k updates.

Questions for authors
  1. Did you try sampling-time guidance like DPS, even on small scale? Would you expect the same darkening?
  2. Will Model A or the code be released?
  3. Would FlowLensing do better if trained closer to the 70k updates?
  4. The soft penalty is only tested on the DDPM, does the same darkening effect happen on the other generative models, such as the flow matching model.
Reviewer_5znoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes PhysLensDiff, a diffusion-based method for joint source-lens emulation in strong lensing systems. Two diffusion models are trained on datasets of simulated sources and lenses, respectively, then samples are fed to a simple ray-tracer to produce lensed images. The method is compared to a couple of prior works and a soft penalty.

Critical strengths

None

Critical weaknesses
  1. The motivation behind the proposed method itself is unclear. The authors make a dataset of simple simulated systems, then train a diffusion model on it, with the claimed intention of generating more data for downstream tasks. However, the training data already represent samples from very simple analytic priors to begin with and are, by the authors' own admission in appendix I, not even realistic to the target task. At best the method can replicate the training data, and it is not even clear that their proposed pipeline is more efficient in terms of hardware-matched compute, as the simulator was timed on one CPU core.
  2. The paper is extremely difficult to read. I do not typically believe that writing style warrants paper rejection, but I am very familiar with the strong lensing problem, and it took me a very long time to decode the content of this draft. For the machine learning community at large, especially for the typical attendees of this conference, I am confident that this paper would be borderline undecipherable and provide little to no value.
Questions for authors

None

Reviewer_u96ROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper claims to propose a new strong lensing emulator. However, careful reading of the paper reveals that the main contribution of the paper is to introduce a differentiable ray-tracing software (which seems to be build on the existing Lenstronomy) with hard-coded source, substructure, macro-model, and pixelization choices, represented by generative models.

The only use of generative modelling in the paper is to represent the population-level distribution of sources and substructure fields. The authors train 2 generative models (diffusion models) on simple source images and on substructure fields, and uses a simple parametric model to represent the main lens. However, the accuracy of the trained generative models themselves is not clearly assessed, and seems to be poor given the text in Appendix I.

Critical strengths

Proposing a strong lens simulation package with integrated generative models for including the effect of a library of substructure physics and is an interesting and novel idea (but would need to be documented much better in terms the physical parameters spanned by the models).

Same goes for a library of population-level source priors that could be easily sampled (but this would need to be extended from parametric sources only to real galaxy images, simulated galaxies, and varying resolutions, in a way that a user could easily select which pop-level prior is of interest)

Critical weaknesses

The paper claims 4 main contributions, however, I see multiple problems with all of them:

(i) the claim that the new simulator is fast: this is not benched properly against SOTA simulators. (ii) for contribution 2, it’s not clear at all what is meant by the text in the introduction, but it’s also not clear that it’s useful, since other codes actually solve the lens equation analytically for arbitrary pixellated sources and non-parametric lenses at arbitrary resolution for any combination. It’s customary to check against established codes when proposing new ones. Moreover, there already exists many 2-sample tests or coverage tests to check the validity of emulated samples when training emulators (e.g. PQMass, 2402.04355), and looking at modelling residuals is not a new contribution.
(iii) Showing that soft optimization has pathological failure modes: this is not surprising nor novel, especially in strong lensing where this specific failure mode is known, just like the fact that it is known that since the lens equation is non-linear, it has perfectly degenerate solutions (e.g. a circular source with no lens mass) (iv) this was already possible to do with analytical simulations that include substructure fields, it’s not clear what the new contribution is here. Training a classifier to distinguish simulated strongly lensed systems with or without subhalos is something that was also already done in astrophysics.

Given those, I do not think the paper would have sufficiently substantiated contributions and demonstrated usefulness to be published in a tailored domain journal like JOSS or an astrophysics publication. But here my main and most fundamental criticism is that this submission has no demonstrated interest to the general ICML community beyond the limited domain application of strong lensing.

Below I describe the weaknesses I see with the work in its current form in more details:

Weaknesses:

  • Limited interest to to the general ICML community, as the only demonstrated application is strong lensing, a very niche field in astrophysics.

  • Time improvement claimed of 0.1% of generation time, but not compared to actual specific simulation from a specific existing simulator, and it’s not explained what hardware or configurations are used (CPU? GPU? Which? Batched?). Some of those numbers are hidden in appendices and are only compared with FlowLensing, which is by very far not the most commonly used SOTA lensing code. The Introduction cites Lenstronomy as a ‘slow code’ and other codes built on it as raytracing being slow, but actually modern GPU-based simulators (e.g written in PyTouch or JAX) demonstrated to be much faster, e.g. SLSim or Caustics, and the paper doesn’t benchmark against them.

  • Insufficient review of the literature (omission of important other codes like PyAutoLens, insufficient review for Strong lensing with ML that misses or miss-represent many works).

  • Accuracy on lensed image generation or forward model gradients is not demonstrated when compared to those baseline codes, which any new raytracing code would need to demonstrate, even if it’s a traditional solver.

  • Scientific interest of the emulator is not demonstrated clearly when compared to other existing software.

  • I have a hard time seeing how the maths derived is relevant for anything more than strong lensing (this links back to weakness 1), yet all results derived are already established in the field of strong lensing (e.g. prop 1).

  • The writing is very poor. Many sentences are very convoluted and make very little sense. For example: ‘whose unit scale is measured by regressing shipped deflections on solved ones’ and ‘and the gap is invisible exactly where emulators are most useful: away from the training data.’ And ‘trade a little exactness for that convenience’ and ‘the residual alone can be satisfied by degenerating the sample’ and ‘it lies on a linear manifold recovered to 0.08% RMS’. Also this sentence is very obscure for a general reader: ‘an exactly solved residual with a convergence certificate, anchored on real data between a matched-strength wrong lens and an image that ignores the requested lens’

  • Modeling choices are hard-baked in the framework, and are hard to understand from a scientific perspective. There was no justification provided for why these choices are sufficient or appropriate for the analysis of what data (which telescope?):

    • In addition to there being no physical size details about source FOV assumptions, the source prior and pixellation choice (64x64 pixels) are not appropriate for most modern data. This is insufficient for most modern strong lensing observations, especially for the scientific purpose of substructure analysis. It’s known (e.g. Stacy et al. 2025 and Nightingale and Dye, 2015) that insufficient source resolution will bias substructure analysis. Also, if a collection of or a single Sersic is assumed for the source model, then the source can be described much more accurately with a sum of analytic profiles which the ray tracer can just directly evaluate with no need to interpolate on a grid. The chosen framework just introduces the unnecessary problems and systematics associated with a pixellated parametrization for no gain. In addition, it is hardly justified to train a diffusion model that then requires 100 diffusion steps at sampling time for each source generated when the source profile parameters can just be sampled from a prior and the brightness evaluated directly for every simulation.

    • Subhalo fields are 128x128. This is hardly justified for the source resolution chosen. The substructure appendix is very hard to understand. It gives no context for non-expert about what the quoted numbers mean, and the phrasing is very obtuse (e.g. there is clearly a lack of units here and I’m not sure what ‘real axions’ are : Fitting a log–log slope to the radial power spectrum of the substructure residual over k = 3–25 gives−2.27,−1.92 and−1.83 for real axion, CDM and WDM systems, against−3.99,−3.53 and −3.78 for ours. ) Overall to me this sounds like the authors should have concluded they did a poor job of training a generative model over their substructure maps, but no quality assessment of this is made anywhere. Also, there are no details about FOV generated to avoid sub halo fields adding global convergences, how the field of view is handled, what profiles are assumed, range of masses for WDM, etc… Figure 7 lacks any of these very important scientific details that would allow practitioners to understand what is being done.

    • Also, for modern lens modelling, 3-parameters macro-models are clearly insufficient. Modern modelling tools use at minimum EPL profiles +external shear + multiples (which are essential to include in case of substructure modelling to avoid biases O'Riordan and Vegetti, 2024)

Questions for authors
  • Can you provide clear quality metrics and coverage measure about your trained source and substructure maps?
  • The proposed simulation framework would be tailored to what data? What is the expected sensitivity of this data, e.g. to substructure or to other source science, and how does your framework ensures it models at or beyond its requirements?
  • Can you benchmark agains other existing GPU-accelerated codes like SLSim or Caustics?

Comments on the paper

2 comments

Dear Reviewers,

Thank you for your thoughtful reviews. The authors have provided detailed responses to the concerns raised.

I would appreciate it if you could revisit the author responses and clarify whether your main concerns have been adequately addressed, particularly which issues remain critical to your overall assessment.

Given the differences in the initial recommendations, I also encourage you to consider and discuss one another's key arguments to help clarify the remaining disagreements and develop a more consistent understanding of the paper's merits and limitations.

Please feel free to update your ratings if your assessments have changed following the discussion.

Thank you for your continued contributions.

AuthorsOpenReview ↗

Revised PDF uploaded

We have uploaded a revised PDF. It is rewritten for clarity (shorter abstract, terms defined at first use) and includes the experiments described in our responses: FlowLensing at its full published budget, the penalty on a flow-matching model, guidance at sampling time, the comparison with GPU ray tracers, and a parametric source mode. The code that generated the dataset is now public and is cited in the revised version. Regarding the reference raised by Reviewer u96R, we now cite both Stacey et al. (2024) and Stacey et al. (2025). We thank the reviewers and the area chair for their time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.