acceptodds
Under review as a conference paper at ICLR 2027

Measurement-Aligned Diffusion: Closing the Loop Between Generative Models and Catalog Measurements

Abstract

Using a generative model as a scientific instrument requires more than plausible images: catalog controls must be meaningful, behaviorally faithful, and testable against real observations. We use an eight-dimensional catalog vector—redshift, multiband fluxes, ellipticity, effective radius, and S\'ersic index—as an interpretable interface to a conditional diffusion model, aligned by a frozen, non-adaptive measurement proxy. On the test set of real images, the proxy has mean and degrades in the extreme S\'ersic tail. With that proxy fixed, six one-coordinate counterfactual paths track the requested axis with Spearman correlation at least , and leave-group ablations raise the error on the removed axes, most strongly for shape and scale. On matched catalog conditions, alignment lowers the closed-loop error of this same proxy by when sampling from noise and by in image-to-image sampling, relative to an unaligned diffusion baseline. Accepted samples then retrieve real galaxies. In leave-one-out rare-object search over catalog images, Recall@500 is , compared with for catalog-space nearest neighbors. Screening cutouts with prototypes, two blinded experts mark of their judgments on pairs drawn from the top-50 neighbors as visually similar. Closed-loop scores measure consistency with the fixed proxy; retrieval and expert judgments test the outputs on real survey images. The same interface can then query future wide-field surveys by rendering a stated catalog hypothesis as a prototype and matching it to real cutouts.

Primary area:
generative models
Keywords:
conditional diffusion, galaxy morphology, catalog measurements, frozen proxy alignment, counterfactual evaluation, image retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FzJUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a conditional diffusion framework for galaxy image generation and retrieval, using a frozen measurement network MM to guide joint fine-tuning. The generated images serve as prototypes to retrieve real survey images. The authors evaluate measurement consistency, control ablations, counterfactuals, rare-object retrieval, and expert reviews.

Critical strengths
  • Generative modeling for scientific data requiring strict measurement control is a well-motivated and meaningful goal.

  • The paper includes multiple evaluation angles (proxy accuracy, controls, retrieval, and expert review) and acknowledges several limitations.

Critical weaknesses
  • The alignment loss and main improvements are evaluated using the exact same frozen proxy (MM) used to train the generator, risking exploitation of proxy errors without independent validation.

  • The rare-object retrieval and "leave-one-out" setup are unclear (e.g., how candidates, queries, and training overlaps are handled), making the Catalog-NN Recall results hard to interpret.

  • The experiments lack a matched baseline (joint fine-tuning with λalign=0\lambda_{\mathrm{align}}=0), failing to prove whether the performance gains come from measurement alignment or simply extra joint training.

Questions for authors
  1. Do the alignment gains hold when evaluated using measurements completely independent of the training proxy MM?
  2. Can you precisely define the query, gallery, and exclusion rules in the leave-one-out rare-object retrieval experiment?
  3. How does the model compare against an otherwise identical baseline trained without the alignment loss?
Reviewer_Lxn3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a new framework to train conditional generative models and apply it to DESI cutouts of galaxy images, conditioned on galaxy caglatog properties. To ensure the conditional generator does not ignore or drop its conditioning labels, the authors propose to first train a diffusion model followed by a SwinIR super-resolution module, which are then fine-tuned using a frozen measurement proxy network that provides a differentiable consistency loss. The paper compares the fine-tuned model against the un-aligned baseline, and evaluate their method through ablation, a thorough counterfactual analysis, and leave-one-out rare object retrieval on real catalog data and finally a blinded expert review.

Critical strengths
  • The paper tackles an important challenge of generative modern generative modelling: the gap between visual realism (typically evaluated with FID or log Likelihood metrics) and distributional fidelity of the entire learnt distribution which is essential for many scientific applications of machine learning.
  • The design choice was interesting and has a demonstrated impact on the quality of the generated samples. Freezing the measurement network for the fine-tuning phase of training is, as explained in the paper, necessary to prevent co-adaptation between the generator and the evaluator.
  • The counterfactual sweep study is very well designed and is excellent in terms of the interpretability it allows to gain out of the conditioning that was learnt. There was clearly a lot of care given to designing evaluations that would be insightful and informative.
Critical weaknesses
  • The problem is framed in an extremely narrow domain silo, and therefore I doubt that the results will be of convincing interest to the broader ICLR community. The paper evaluates exclusively on a single astrophysical task without demonstrating transfer or scalability of the proposed method to any other scientific field or other more general conditional generation computer vision benchmarks (medical imaging MRI/CT scans, climate modelling or weather downscaling benchmarks, physics-informed problems, or standard computer vision attribute control on data sets like COCO or ImageNet). Even within astrophysics, there is no demonstration that the proposed framework performs well on any other conditional generative task (spectra, time series, images of clusters, etc..), or that it would work on data of a different format, for example that the performances would be maintained if it were trained on data with different resolution, different PSFs or instrumental artifacts, or different conditioning vector. There was also no attempt at testing the one-shot or transfer generalizability of the model, which was disappointing. Demonstrating the flexibility of the proposed framework beyond one narrow task is essential in an era of multi-instrument and multi-resolution heterogeneity in astrophysics, especially if the claim is to build a generative model acting as a general scientific instrument.
  • Conversely, the baseline comparison is very weak. The authors only benchmarked their fine-tuned conditional model against their own unaligned model. There were no comparison against existing state-of-the-art conditional generative models used either in general machine learning or in the specific context of scientific ML. There is no quantitative comparison to other frameworks established in astrophysics like specialized GAN frameworks, or more general methods like ControlNet or T2I-Adapter, or other classifier-free guidance tuning variants, and there is also no comparison with overlapping lines of research in interpretable machine learning like Concept Bottleneck Generative Models and C-VCE or Attribute-Disentangled Diffusion Models.
  • The Related Work section does not provide a literature review of how recent conditional generative models are used and perform in other applications, either scientifically or for generic generative modelling. It does refer to core foundational work in the field, and provides a good map of the recent contributions in generative models for survey astrophysics, but it misses the entire body of recent literature on general alignment and control for diffusion models, architecture-level control or general attribute disentanglement papers. It also misses the opportunity to draw parallel with other scientific disciplines that face similar challenges as astrophysics when it comes to generative modelling.

Together, the previous points leave the contribution of the paper disconnected from the generative modelling field. There isn't sufficient context and comparisons to determine whether the proposed frozen proxy alignment is just a trick for galaxy data, or if it is fundamentally a superior architectural pattern for conditional modelling generally.

  • I also find that there is a strong disconnect between the claims and motivations in the introduction and what has actually been demonstrated. The overall framing of the paper is the criticism of conditional generative models usually being evaluated on the quality of individual samples as opposed to the true coverage of the high-dimensional conditional distribution learnt. This seems to imply that the goal is to establish rigorous validation of the coverage of the entire learnt distribution. However, the actual validation is empirical and boils down to nearest-neighbour similarity retrieval and small scale human expert study, even if the validation framework is used to retrieve real samples. This method of evaluating the conditional generative model has no guarantee to detect or correct overconfidence or underdispersion in the learnt conditional, to be able to learn small modes, or to avoid repetitive generation, as long as the generated samples trick the proxy into finding a similar sample in the catalog. Closed-loop MAE using the same proxy the model was aligned against and nearest-neighbour retrieval does not test distributional calibration or diversity, which is the problem the introduction is framed as aiming to solve.
  • The paper does not present any coverage test of their conditional model, however, many real images of the conditional are readily available. It would have definitely added to the credibility of the claims.
  • The paper is very hard to read and understand. The language is overly engineered and dense, with a writing style that obscures the flow between ideas. The manuscript relies on hyper-compressed phrasing and dense noun-adjunct chains that hinders readability, and makes parsing the methodology an exhausting experience. Reading it, as a human, I felt that the text was not intended for me.
Questions for authors
  • You have demonstrated that the proposed alignment loss works for the specific architecture proposed, but can you provide quantitative evidence that it is superior to or competitive with existing SOTA conditional models on your task?
  • Can you provide quantitative evidence that your method is competitive with or superior to other SOTA methods on other conditional generative task of similar nature in astrophysics and in other scientific fields?
  • Can you provide a proper calibration or coverage test of your trained, fine-tuned model? Statistical tests like TARP or MIRA would provide a measure of the calibration of the entire conditional distribution learnt.
  • Minor point, but if you could label the modules G, S and M in Fig 1 it would make the readability much better and referring to it would be much easier.
Reviewer_EUV8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I am not sure what this paper wants to present. Looks like they train a Conditional Diffusion models on galaxy image properties, but there is hardly anything meets the ICLR standard.

Critical strengths

This paper trains a diffusion model on galaxy image properties like redshift, ellipticity, etc.

Critical weaknesses

This paper seems like be written by AI. The results are presented badly, and it is really hard to follow. There is nothing more novel than training a conditional diffusion model on redshift, ellipticity and so on, which are things that very early works have explored in the age of GAN.

Questions for authors

Do you use AI to write this paper? It is quite obvious, and the writing is hard to follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.