acceptodds
Under review as a conference paper at ICLR 2027

Projected Energy Matching for Generative 3D Priors

Abstract

Transport-based generative models, which learn a time-dependent vector field that moves noise to data, have become a dominant paradigm. However, these models typically do not explicitly encode the data distribution. Energy-based models (EBMs) instead represent the data distribution explicitly through a scalar energy landscape, which *Energy Matching* learns by combining transport learning with contrastive refinement. Its transport objective, however, fits energy gradients to stochastic targets, whose variance degrades the training signal at scale. We introduce *Projected Energy Matching*, which learns this landscape through a more stable route: we first train a time-independent transport teacher, then freeze it and fit the negative energy gradient to its predicted velocities. This projection replaces noisy transport targets with deterministic supervision, while contrastive refinement shapes the landscape near the data manifold. On CIFAR-10, gradient-noise analysis reveals a cleaner training signal, accompanied by faster convergence than Energy Matching at matched, teacher-free training budgets. In the latent space of CT volumes, our method enables 3D CT generation and achieves better FID scores than flow models. The learned scalar potential serves as a zero-shot prior for the ill-posed inverse problem of sparse-view cone-beam CT reconstruction. By making explicit energy landscapes practical at volumetric scale, this work opens a path to wider adoption of energy-based formulations, bringing their flexibility to high-dimensional generation and inverse problems.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
generative modeling, energy-based models, inverse problems, 3D medical imaging, sparse-view CBCT reconstruction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_SHRTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors propose Projected Energy Matching (PEM). It has 3 stages for training a time-independent scalar energy potential. Stage 1 trains an autonomous flow teacher with minibatch OT couplings. Stage 2 freezes the teacher and projects the teacher field onto the conservative subspace. Stage 3 adds the Energy Matching contrastive loss with Langevin negatives to keep the projection term active.

On CIFAR-10, they show faster FID convergence than EM. In 3D CT, PEM reaches a slightly lower FID than OT Flow Matching and Rectified Flow on same backbone

Critical strengths
  • EM bottleneck is the high variance of stochastic transport targets. The authors identify this, and through PEM, replace stochastic targets with deterministic supervision from a pretrained velocity model. This frozen teacher is easy to implement
  • Explicit scalar energy has several downstream advantages, compared to only a transport vector field. The same unconditional model is used for generation, distribution-shift scoring, and incorporation into an inverse-problem objective without task-specific retraining.
  • The authors also propose negative-sample caching during gradient accumulation to make the contrastive phase practical in 3D.
  • PEM shows better FID on both CIFAR-10 and 3D CT
Critical weaknesses
  • Table 3 Phase 1+2 significantly increases FID. Is Phase 2 projection needed in that case?
  • The authors mention Params in Table 1, but a fair comparison would be the inference and training time (teacher + projection + refinement). I think the authors omit the teacher params while reporting
  • Eq. 1 trains the teacher on OT-paired interpolants from a memory bank M. Algorithms 1 and 2, however, sample x_data and x_noise independently (lines 5–7 / 3–5) and form x_t from random pairs, while the comment says “interpolate along optimal transport paths.” The projection is therefore evaluated on a different x_t distribution than the one the teacher was trained on, and the teacher is queried off its training support. Sec. 2.2 states that the projection is weighted by “the density of interpolants along the transport paths,” and Sec. 2.2 calls OT couplings “crucial.” The authors must clarify which coupling is actually used in Phases 2 and 3 and whether the claimed projection interpretation holds.
  • Can authors show absolute number of FID from Fig 2?
  • In experiments, standard diffusion and flow solvers for this task are missing (DPS, DiffusionMBIR, plug-and-play denoisers, and ICTM, which the paper itself cites).
  • The conclusion claims “improved seed stability” on CIFAR-10, which is not shown in the main text.
Questions for authors

I think for me, Table 3's Phase 1+2 is the main point of contention for me. The authors should clarify this, and address other points in weakness section

1 comment

We thank the reviewer for the positive assessment, and for recognising the variance of stochastic transport targets as the bottleneck of Energy Matching and the downstream value of an explicit scalar energy in 3D.

W1. Training phases. We apologise for the ambiguous labelling. We have updated Table 3 (now Table 4) with a description of each phase:

PhaseLearned modelParamsPEM CT-RATE FID ↓
1. Time-independent flow teacher (Phase 1)velocity field, no scalar energy10.3M41.90
2. Projection (Phase 1 + 2)scalar potential, no data basin10.7M62.60
3. Contrastive (Phase 1 + 2 + 3)energy-based model with data basin10.7M40.87

Only the final phase yields an energy-based model. Phase 2, the projection, is the transport phase of Energy Matching: it yields a conservative flow without a basin at the data, so its samples follow the transport funnel without settling (FID 62.60). It cannot be omitted, because contrastive refinement draws its Langevin negatives from noise and relies on this funnel to bring them toward the data. Energy Matching likewise initialises its contrastive phase from its transport phase. Contrastive refinement then forms the basin, and the complete energy model matches its unconstrained teacher (40.87 vs. 41.90) while providing an explicit energy.

W2. Compute and parameters. Table 1 compares sample quality at comparable model size. We make no claims about training time or inference speed relative to flow models, where these excel. Params reports the model used at inference: PEM samples with ϕθ\phi_\theta alone (10.7M), while the teacher (10.3M) is used only during training. Training an energy model requires double-backward differentiation and is more costly per step than training a flow model. This is the price of an explicit energy (scalar scoring, an explicit reconstruction objective, stable sampling).

W3. Couplings. The teacher (Phase 1) is trained on OT-coupled pairs, which give mainly straight, non-crossing paths. This is why OT couplings are crucial for the teacher. In Phases 2 and 3, the student matches the teacher's predicted velocity vteacher(xt)v_{\text{teacher}}(x_t), a target that depends on xtx_t alone, so no pairing is needed. Data and noise are paired independently only to choose query points, which spreads them beyond the OT paths, so the potential is also shaped where Langevin chains travel. The teacher is not queried off its support, both couplings share the same data and noise marginals, so xtx_t covers the same space and is only weighted differently. The projection therefore still returns the closest gradient field to the teacher, assuming the teacher generalises to these query points. We have corrected the algorithm comment and stated this in Sec. 2.2.

W4. Absolute FID. Absolute FIDs behind Fig. 2(b) (CIFAR-10, batch size 32, 10K samples), mean over 3 seeds with min–max in brackets:

Training stepEMPEM
2,054184.5 [113–309]96.1 [85–108]
5,957111.3 [86–131]75.9 [67–93]
17,28188.8 [86–90]56.1 [46–70]
28,12084.4 [66–117]46.5 [46–47]
50,12967.7 [61–75]41.1 [39–42]
70,30059.2 [51–69]49.8 [40–55]
145,00062.3 [52–81]38.0 [36–42]

PEM has lower mean FID at every checkpoint and reaches 56 after 17K steps, a level the EM mean does not reach within 145K steps. These values serve the controlled training-signal comparison only and do not represent the FID attainable under full training. This table is now in the Appendix.

W5. Reconstruction baselines. The sparse-view experiment is a controlled ablation of the prior, not a benchmark of reconstruction pipelines. The energy and flow priors share backbone, training data, latent space and forward operator, so the comparison isolates the prior up to solver details, which we give in Appendix G. Following the suggestion, we have extended it with DPS, a second solver for the same flow prior (10 volumes):

MethodPSNR ↑SSIM ↑LPIPS ↓
OT-Flow Matching DPS (new)29.29 ± 0.730.8204 ± 0.03230.2009 ± 0.0377
OT-Flow Matching MAP31.92 ± 1.220.8881 ± 0.03030.0960 ± 0.0244
PEM MAP (ours)32.55 ± 1.300.9161 ± 0.02520.0621 ± 0.0154

PEM MAP outperforms the flow prior under both solvers. Benchmarking a higher-resolution 3D PEM prior against complete pipelines, including DiffusionMBIR, plug-and-play denoisers and ICTM, is the subject of follow-up work.

W6. Seed stability. Pooling all replicate seeds on CIFAR-10 (3 seeds × 9 strata), the seed standard deviation of log FID is 0.127 for PEM and 0.376 for EM, i.e. about 3× smaller for PEM (95% HDI [1.7, 4.5]). This is now reported in the main text.

Q1. Please see W1. Phase 2 is the transport phase required for contrastive refinement, and the final model matches its teacher while providing an explicit energy.

Reviewer_gESQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PEM (projected energy matching), a three-stage framework for learning a time-independent scalar energy model. It first trains an autonomous flow teacher, projects the frozen teacher field onto the gradient of a scalar potential, and then refines the potential using a contrastive energy matching objective. On CIFAR-10, PEM produces a stronger gradient signal and faster fid convergence than energy matching when teacher pretrained is excluded. In the latent space of CT-RATE datasets, the learned potential is evaluated for 3D generation, distribution-shift detection, extended-horizon sampling, and sparese-view CBCT reconstruction.

Critical strengths
  1. This paper proposes a simple and practical route to explicit 3D energy models. Projecting a frozen velocity teacher onto a conservative field is conceptually straightforward, and the paper demonstrates the feasibility of energy matching-style training in a volumetric latent space. The same unconditional potential is used for generation, distribution-shift scoring, extended-horizon sampling, and sparse-view reconstruction without task-specific retraining.

  2. The training-signal analysis is carefully designed on the CIFAR-10 dataset. The comparison uses matched runs with the same architecture, initialization, data order, optimizer, and seed , together with a statistical analysis. This provides strong evidence that deterministic teacher targets yield a stonger gradient singal than stochastic interpolant targets.

Critical weaknesses
  1. The experimental evidence does not fully support the claimed benefit of the energy formulation, because the improvement in generation quality cannot be clearly attributed to it. In Table 3, it shows that an FID of 41.90 can be achieved using only the first-stage teacher model, this performance already surpasses that of the previously reported OT Flow Matching and Rectifield Flow baseline models. Subsequently, the projection operation caused the FID to deteriorate to 62.60, whilst contrastive refinement restored it to 40.87. Consequently, the final PEM model showed only a slight improvement over its own teacher model, while the projection stage itself significantly reduced the quality of the generated images. It therefore remains unclear how much of the final performance can be attributed specifically to the projected energy formulation. These differences are reported on 1,000 samples without multi-seed uncertainty. Moreover, the strongest FID result uses MedNeXt without a corresponding MedNeXt teacher or flow baseline. Consequently, the reported numerical advantage over flow models cannot currently be attributed specifically to the projected energy formulation.

  2. End-to-end efficiency and downstream benefits based on energy consumption remain unclear. The CIFAR-10 comparison did not explain the pretraining costs associated with the flow-based teacher. Consequently, while this comparison suggests that fitting the outputs of a deterministic teacher is easier than learning from random targets, it does not prove that the complete teacher-projection-refinement pipeline is more economical than energy matching.

  3. Evidence that explicit energy provides benefits to the teacher model that it could not otherwise obtain is also rather limited. Compared to the OT-Flow MAP baseline, CBCT achieved an improvement of only approximately 0.6 dB across 10 slices, with a standard deviation of approximately 1.3 dB, and no paired uncertainty analysis was conducted. These results demonstrate the feasibility of the method but have not yet established its significant advantages over flow or diffusion based prior models.

Questions for authors
  1. What is the total computational cost of PEM when teacher pretraining, projection, and contrastive refinement are all included? How does this compare with Energy Matching in wall-clock time, memory, and total optimization steps?

  2. What capabilities does the explicit energy provide for OOD detection and reconstruction that cannot be obtained from the teacher or a standard flow prior? Can the authors include stronger flow- and diffusion-based inverse-problem baselines?

  3. Table 3 shows that the Phase-1 teacher already reaches FID 41.90, close to the final PEM result of 40.87. How do the authors separate the contribution of the energy formulation from that of the teacher? How does a MedNeXt teacher compare with PEM using MedNeXt?

3 comments

We thank the reviewer for the detailed review and for recognising that the projection step is simple and practical. Below, we address the points raised and hope to clarify the remaining ambiguities.

W1(a) Energy interface. We thank the reviewer for the opportunity to reiterate the main benefit of the energy formulation, it directly exposes a scalar energy function that can be evaluated, used for diagnostics and combined with constraints as a prior. For example, in our sparse-view CBCT experiment, this prior is combined with measurement consistency to improve conditional reconstruction, increasing PSNR by 0.63 dB and reducing LPIPS by 35% relative to the flow prior. Our aim is to retain the quality of flow models while providing this explicit interface, not necessarily to outperform the teacher in unconditional generation. The flow teacher directly predicts a velocity field, whereas PEM provides a scalar potential whose gradient defines its field.

The teacher (Phase 1, FID 41.90) is a reference without an energy interface. Projection (Phase 2, FID 62.60) is an intermediate conservative transport stage that guides Langevin negatives from noise toward the data, and contrastive refinement (Phase 3) forms basins near the data, recovering teacher-level quality (FID 40.87) with an explicit energy. We have clarified these roles in Table 3 (now Table 4).

W1(b) FID evaluation. FID uses 2D slices along all three planes of the 1,000 generated and 1,000 held-out volumes, i.e. 320,000 slices per set, with the same protocol for all methods. Given the cost of generating volumes at this scale, we report point estimates.

W1cc Backbone. Our claim of competitive quality rests on the common U-Net backbone (FID 40.87 vs. 42.94 and 42.39). MedNeXt has no timestep conditioning, so the time-dependent baselines cannot use it without modification, whereas the time-independent PEM can (FID 35.87). We have made this distinction explicit.

W2. Total compute. We conducted an additional experiment comparing PEM and Energy Matching at matched compute budgets, using the same MedNeXt backbone and potential head on the same hardware. The comparison includes teacher pretraining.

CIFAR-10EMPEM
Teacher training–3.5 h
Transport phase (EM) / projection (PEM)45.3 h45.0 h
Contrastive refinement57.2 h51.9 h
Total102.5 h100.4 h
Best FID-50K14.1710.23

With the same backbone and including teacher pretraining, PEM achieves 28% lower FID than Energy Matching at comparable total compute.

Memory and optimization steps. Both methods peak at about 33 GB during contrastive refinement. The frozen teacher adds only 0.3–0.5 GB during projection, and teacher training (9.1 GB) runs separately. The teacher adds about 145K first-order steps, projection/transport (about 145K steps) and refinement (2,292 steps, each with 500 Langevin steps) are identical for both methods. The table reports time to the best checkpoint.

W3. Reconstruction. The evaluation uses 10 full 3D volumes, with the reported 1.3 dB standard deviation reflecting variation between volumes. Compared with the OT-Flow Matching MAP baseline, PEM improves PSNR by 0.63 ± 0.24 dB and SSIM by 0.028 ± 0.014, while reducing LPIPS by 0.034 ± 0.018 (mean ± standard deviation of paired differences). All three metrics improve on every volume, with Wilcoxon p = 0.00195 for each. With DPS (Chung et al., ICLR 2023) on the same flow prior, the gap widens to 3.3 dB PSNR (32.55 vs. 29.29 dB, see Q2).

Q1. Please see W2 above.

Q2 The explicit energy offers three capabilities the teacher or a flow prior lacks: (1) direct scoring with a single forward pass of ϕθ\phi_\theta, whereas a flow likelihood requires ODE integration with a divergence estimate (AUROC above 0.9 at 200 of 600 projections, Fig. 5); (2) one reconstruction objective, so MAP and Langevin posterior sampling with null-space repulsion use the same prior without a time schedule (Table 6); and (3) stable sampling over extended MCMC chains (Fig. 8).

Regarding baselines, the sparse-view experiment is a controlled ablation of the prior (see W3), which we have extended with DPS as a second solver for the same flow prior. PEM MAP outperforms the flow prior under both solvers on all metrics (DPS: 29.29 dB, SSIM 0.820, LPIPS 0.201; MAP: 31.92 dB, 0.888, 0.096; PEM: 32.55 dB, 0.916, 0.062, see Table 2).

Q3. Please see W1(a) and Q2. The 3D U-Net ablation shows that PEM retains teacher-level sample quality, with FID 40.87 versus 41.90 for the teacher, while providing a scalar energy for scoring and reconstruction.

For MedNeXt, our matched teacher–student comparison is on CIFAR-10, where the teacher achieves FID 8.45 and PEM achieves 10.23. The benefit of PEM lies in its explicit energy interface for scoring and reconstruction, it is not expected to improve on its teacher’s unconditional FID.

Thank you for the detailed rebuttal and additional experiments. The authors provided the comparison including teacher pretraining (W2) and the paired per-volume analysis for CBCT (W3) address my concerns on these points. For W1, I think it remains only partially resolved. I still think the MedNeXt result needs a matched teacher on CT-RATE, and the FID gaps in Table 4 would be more convincing with some uncertainty estimate.

Overall, this rebuttal is positive to me, i am inclined to raise my score from weak reject to weak accept.

We are glad our additional experiments resolved many initial concerns and thank the reviewer for continuing the discussion and for the opportunity to address the remaining questions.

Q1. Scope of the backbone experiments. The CT-RATE MedNeXt experiment demonstrated that PEM can natively use a different backbone without timestep conditioning. We thank the reviewer for highlighting that a controlled quality comparison would require the same MedNeXt backbone for PEM, the teacher and the other baselines. We have therefore removed this row from Table 1 and clarified the wording. Our quality comparisons use matched backbones: a shared U-Net on CT-RATE (PEM 40.87 vs. teacher 41.90) and a shared MedNeXt on CIFAR-10 (PEM 10.23 vs. teacher 8.45).

Q2. Interpreting FID differences. Each FID is computed on 320,000 slices from 1,000 generated and 1,000 held-out volumes, using an identical protocol across methods. We appreciate the reviewer’s concern. Given the computational cost of evaluating 3D generative models, we report point estimates. We interpret these results as supporting comparable generative quality and acknowledge that small FID differences should be interpreted cautiously.

We appreciate the reviewer’s active engagement in the discussion and the opportunity to provide these clarifications.

Reviewer_z3oLOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper introduces a method called PEM, that distills a pretrained flow teacher into a time-independent energy function by conservative projection and contrastive refinement. This method is evaluated by an analysis of training dynamics on CIFAR-10, and also includes applications to 3D CT generation, out-of-distribution detection, and sparse-view CT reconstruction.

Critical strengths
  1. Practical extension of energy-based generative modeling to 3D medical data. The manuscript demonstrates the feasibility of training an explicit energy-based prior on volumetric CT data. It addresses the computational challenges associated with high-dimensional energy models.

  2. Versatility of the learned energy prior. The same pretrained energy function supports unconditional generation, out-of-distribution detection, and sparse-view CT reconstruction without task-specific retraining.

Critical weaknesses
  1. Limited methodological novelty and insufficient experimental validation.

The manuscript does not adequately distinguish its contributions from closely related work. Energy Matching (Balcerak et al., 2025) already combines time-independent energy functions, transport learning, and contrastive refinement, while Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo (Thornton et al., 2025) explores energy distillation from pretrained generative models. The proposed method appears to largely combine existing techniques without sufficiently demonstrating their necessity. Moreover, the original Energy Matching is missing as a 3D baseline, and the improvement over OT Flow Matching MAP in sparse-view reconstruction is low (0.63 dB PSNR, although the improvement in LPIPS is more pronounced, the evaluation relies on only 10 volumes, with no paired test or significance reporting). Stronger baselines, ablations, and end-to-end computational comparisons are needed to justify the proposed framework.

  1. Incomplete literature review and limited comparisons for 3D medical reconstruction.

Despite its emphasis on 3D medical reconstruction, the manuscript overlooks several closely related studies, including From Sparse X-rays to 3D CT (Zhang et al., 2026), Cross-Distribution Diffusion Priors-Driven Iterative Reconstruction for Sparse-View CT (Li et al., 2026), and When are Diffusion Priors Helpful in Sparse Reconstruction? (Cheung et al., 2025). These works also investigate pretrained generative priors, physics-based reconstruction, and discuss the effectiveness in sparse-view CT. The manuscript neither sufficiently discusses these approaches nor provides representative comparisons.

References

  • Zhang, Z., Hiller, M., Ehinger, K. A., and Drummond, T. (2026). From Sparse X-rays to 3D CT: Training-Free Reconstruction with Diffusion Priors. MICCAI 2026 Workshops and Challenges.
  • Li, H., et al. (2026). Cross-Distribution Diffusion Priors-Driven Iterative Reconstruction for Sparse-View CT. IEEE Transactions on Medical Imaging, 45(7), 3878–3894.
  • Cheung, M. Y., et al. (2025). When are Diffusion Priors Helpful in Sparse Reconstruction? A Study with Sparse-view CT. IEEE International Symposium on Biomedical Imaging (ISBI).
Questions for authors

Q1. Could the authors clarify the fundamental methodological differences between PEM and existing approaches, particularly Energy Matching (Balcerak et al., 2025) and energy distillation (Thornton et al., 2025)? What specific benefits arise from the proposed projection strategy beyond combining these existing techniques?

Q2. Could the authors include the original Energy Matching method as a baseline in the 3D experiments? Could the authors provide controlled ablations isolating the contribution of the projection stage, along with end-to-end computational comparisons that include teacher pretraining?

Q3. How does PEM compare with existing diffusion-prior-based sparse-view CT reconstruction methods, particularly Zhang et al. (2026) and Li et al. (2026)? Could the authors also clarify the advantages of the proposed energy-based formulation and provide representative comparisons under matched experimental settings?

3 comments

We thank the reviewer for the constructive review, and for recognising the practical extension of explicit energy-based modelling to 3D medical data and the versatility of the learned prior.

W1(a) Contributions. We thank the reviewer for the opportunity to clarify our contributions.

(i) To our knowledge, PEM is the first method to project an autonomous flow model, trained with first-order gradients, onto a time-independent conservative model, whose training requires second-order (double-backward) gradients. This decouples learning the transport, cheap and stable for an unconstrained network, from learning the explicit energy, and we hope the reviewer will recognise it as the key methodological contribution.

(ii) In 20 matched CIFAR-10 run pairs differing only in this target, the median ∥G∥2\|G\|^2 is approximately 43×43\times larger and PEM reaches lower FID in fewer steps (Fig. 2).

(iii) The result is, to our knowledge, the first energy-based model of 3D medical volumes that is a single, time-independent network defining the distribution explicitly. This enables one unconditional potential to serve generation, distribution-shift scoring and sparse-view CBCT reconstruction.

W1(b) Relation to prior work. Energy Matching (Balcerak et al., 2025) learns transport from scratch against stochastic targets x1−x0x_1 - x_0. PEM replaces these with the deterministic teacher prediction, removing the dominant variance term. Thornton et al. (2025) learn time-dependent energy models, while PEM learns one time-independent landscape that is sampled by Langevin dynamics, scored in a single forward pass and used directly as a reconstruction objective.

W1(c) Energy Matching in 3D. Including teacher pretraining, PEM reaches a better FID than Energy Matching on CIFAR-10 at slightly lower total cost (10.23 vs. 14.17; 100.4 h vs. 102.5 h). The obstacle to Energy Matching in 3D is its training signal. Its gradient signal is about 43× weaker and its signal-to-noise ratio lower at every batch size (Fig. 2a), so it needs larger batches to average out the noise, while 3D memory limits the batch size.

W1(d) Reconstruction. The sparse-view experiment is a controlled ablation of the prior, not a benchmark of reconstruction pipelines: the energy and flow priors share backbone, training data, latent space and forward operator, so the comparison isolates the prior up to solver details, which we give in Appendix G. Against the flow MAP, PEM improves every metric on all 10 volumes. PSNR by +0.63±0.24+0.63 \pm 0.24 dB, SSIM by +0.028±0.014+0.028 \pm 0.014 and LPIPS by −0.034±0.018-0.034 \pm 0.018 (a 35% reduction), with paired Wilcoxon p=0.00195p = 0.00195 for each metric (the smallest attainable for n=10n = 10). We have also added DPS with the same flow prior (see Q3).

W2. Literature. We thank the reviewer for these references, which we have included in the paper. As noted in W1, our sparse-view experiment is a controlled ablation of the prior. These works build complete reconstruction pipelines around different time-dependent diffusion priors, training data and resolutions, so comparing against them would conflate the prior with all of these factors. Within the ablation, we have added DPS as a second solver for the same flow prior.

Q1. Please see W1(a).

Q2. At volumetric scale, memory limits the batch size, and Energy Matching’s lower gradient signal-to-noise ratio (Fig. 2a) means it would need larger batches than PEM to train stably, which is why we did not train it in 3D. The contribution of the projection is isolated in the controlled CIFAR-10 study, where it is the only difference between arms, and we now include teacher pretraining: PEM reaches FID 10.23 versus 14.17 at 100.4 h versus 102.5 h (see W1). In 3D the projection is the transport phase of Energy Matching and cannot be removed, as contrastive refinement relies on its transport funnel to draw Langevin negatives from noise.

Q3. As noted in W1 and W2, the sparse-view experiment is a controlled ablation of the prior. We have extended it with DPS, a second solver for the same flow prior (10 volumes). PEM MAP outperforms the flow prior under both solvers (PSNR 32.55 vs. 31.92 dB for flow MAP and 29.29 dB for flow DPS). Zhang et al. (2026) and Li et al. (2026) are complete pipelines with different priors, training data and resolutions. Benchmarking a higher-resolution 3D PEM prior against such pipelines is the subject of follow-up work.

The energy formulation adds three advantages: the prior is a scalar that enters the reconstruction objective directly (Eq. map) without a time schedule. The same objective supports both MAP and Langevin posterior sampling with null-space repulsion, which improves the ensemble-mean PSNR from 30.02 to 30.22 dB (Table 6) and the same potential scores distribution shift in a single evaluation, retaining AUROC above 0.9 at 200 of 600 projections (Fig. 5).

Thank you to the authors for the detailed response. The clarification that PEM first learns unconstrained transport and then projects it onto a conservative energy field, avoiding direct learning of energy gradients from stochastic targets makes sense to me. This provides a more convincing methodological contribution.

We thank the reviewer for their active engagement, recognition of PEM’s methodological contribution, and revised assessment.

Reviewer_YjmwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Projected Energy Matching (PEM) learns a time-independent scalar energy model using a pretrained autonomous transport teacher. It first trains an unconstrained velocity model, then fits the negative gradient of a scalar potential to the frozen teacher’s predictions, and finally adds contrastive refinement while retaining teacher supervision. The paper studies optimization on CIFAR-10 and evaluates a latent-space CT prior for unconditional generation, distribution-shift detection, and sparse-view CBCT reconstruction. With a common U-Net backbone, PEM reports FID 40.87 versus 42.94 and 42.39 for two flow baselines. On ten reconstruction volumes, it improves PSNR from 31.92 to 32.55 dB relative to the reported OT-flow baseline.

Critical strengths

The teacher-supervised objective offers a plausible way to improve the training signal of an energy model that requires double-backward differentiation. The paired CIFAR-10 experiments control architecture, initialization, data ordering, and optimization while changing the regression target (Section 3.1). The gradient-noise and convergence results provide useful evidence for a student-training benefit.

The same learned scalar potential is evaluated for generation, distribution-shift scoring, and reconstruction without task-specific retraining. The common-backbone generation comparison and sparse-view reconstruction results demonstrate the potential usefulness of an explicit volumetric energy beyond unconditional sampling.

Critical weaknesses
  1. The probabilistic interpretation and sampling conventions require clarification. Equation (2) and Algorithm 1 use the drift (−D∇ϕ)(-D\nabla\phi), with noise scaled as (2Δt)(\sqrt{2\Delta t}). Under the usual Langevin convention, this targets a density proportional to (exp⁡(−Dϕ))(\exp(-D\phi)), whereas Section 2.3 defines the model density as (exp⁡(−ϕ))(\exp(-\phi)), and the reconstruction/posterior objectives use the unscaled potential. These conventions describe different temperatures unless the scaling is absorbed consistently elsewhere. The claim that projection preserves the transport objective also needs qualification: it holds under an appropriate conditional-mean teacher and a common interpolation distribution, but Equation (1) uses an OT memory bank while Algorithms 1–2 show separately sampled data and noise without an explicit OT-pairing step.

  2. The experiments establish a student-stage optimization improvement more clearly than an overall computational advantage. The CIFAR-10 comparison explicitly excludes teacher pretraining, and no teacher-inclusive wall-clock or compute comparison is provided. The CT ablation further shows FID worsening from 41.90 for the teacher to 62.60 after projection, then recovering to 40.87 after contrastive refinement (Table 3). This makes the cost–quality contribution of each stage important to quantify. The stronger MedNeXt result lacks a corresponding flow baseline with that backbone, so its advantage cannot be attributed solely to PEM.

Questions for authors
  1. Is the intended model energy (ϕ)(\phi) or (Dϕ)(D\phi)? Are generation and reconstruction using the same prior temperature?

  2. Are the teacher and student trained on the same OT-coupled interpolation distribution?

  3. How is “OT-Flow Matching MAP” in Table 2 defined and implemented?

5 comments

We thank the reviewer for recognising the controlled CIFAR-10 study and the value of the explicit energy prior, and for the opportunity to clarify the remaining ambiguities.

W1(a) Energy scaling. The intended energy is Eθ=NϕθE_\theta=N\phi_\theta, with N=DN=D.
Using NN consistently throughout the paper would make this convention clearer and we have updated the notation accordingly.
The factor NN is absorbed into the MAP prior weight, and generation and reconstruction use the same prior distribution, pθ∝exp⁡(−Nϕθ)p_\theta\propto\exp(-N\phi_\theta).

W1(b) OT coupling. The teacher is trained on OT-coupled pairs, which give mainly straight, non-crossing paths. The student matches the teacher's predicted velocity vteacher(xt)v_{\text{teacher}}(x_t), a target that depends on xtx_t alone, so no pairing is needed. Data and noise are paired independently only to choose where the teacher is queried. Independent pairing spreads these query points beyond the OT paths, so the potential is also shaped where Langevin chains from noise and reconstructions from FBP initialisations travel. This assumes that the teacher generalises from OT-coupled to independently paired query points, which cover the same space but are weighted differently. We have now stated this assumption in the text and qualified the equivalence claim, which holds for a teacher that has learned the transport exactly.

W2(a) Total compute. We thank the reviewer for the opportunity to clarify the computational comparison. We conducted an additional experiment comparing PEM and Energy Matching with matched compute budgets, using the same MedNeXt backbone and potential head on the same hardware. The comparison includes teacher pretraining.

CIFAR-10EMPEM
Teacher training–3.5 h
Transport phase (EM) / projection (PEM)45.3 h45.0 h
Contrastive refinement57.2 h51.9 h
Total102.5 h100.4 h
Best FID-50K14.1710.23

The teacher accounts for approximately 3% of the total training cost. With the same backbone and including teacher pretraining, PEM achieves 28% lower FID than Energy Matching at comparable total compute.

W2(b) Training phases. Phase 1 is a flow teacher, not an energy-based model, and does not provide the explicit energy interface we seek. Its FID of 41.90 is reported as a reference for the teacher’s sample quality.

Phase 2 projects the teacher’s transport field into a scalar potential (FID 62.60), providing the initialization that guides Langevin negatives from noise toward the data. This is an intermediate transport-training stage. Phase 3 then shapes low-energy basins near the data through contrastive refinement. The final model reaches FID 40.87, recovering teacher-level sample quality while providing an explicit energy for scoring and reconstruction. We thank the reviewer for pointing out this ambiguity and have specified the role of each training phase in Table 3 (now Table 4).

PhaseLearned modelParamsPEM CT-RATE FID ↓
1. Time-independent flow teacher (Phase 1)velocity field, no scalar energy10.3M41.90
2. Projection (Phase 1 + 2)scalar potential, no data basin10.7M62.60
3. Contrastive (Phase 1 + 2 + 3)energy-based model with data basin10.7M40.87

W2cc Backbone. Our aim is to achieve the quality of flow models, not necessarily exceed it, while providing an explicit energy interface. To our knowledge, our work enables this combination for the first time at the scale of real-world 3D data, such as medical scans.

The CIFAR-10 comparison with Energy Matching uses the same MedNeXt backbone in both arms, isolating the effect of the training method. Our controlled 3D comparison with flow models uses the common U-Net backbone. The MedNeXt result in Table 1 demonstrates that PEM can directly use backbones without timestep embeddings, whereas the time-dependent flow baselines require adding time conditioning.

Q1. Please see W1(a).

Q2. Please see W1(b).

Q3. The flow baseline regularises latent reconstruction with the pretrained OT Flow Matching model fFMf_{\text{FM}}, using the gradient

g(z)=∇z∑i=1s∥ymeas,i−Ai(D(z))∥22+λFM(z−z^t),g(\mathbf{z}) = \nabla_{\mathbf{z}} \sum_{i=1}^{s} \big\Vert \mathbf{y}_{\text{meas},i} - A_i\big(\mathcal{D}(\mathbf{z})\big)\big\Vert_2^2 + \lambda_{\text{FM}}(\mathbf{z} - \hat{\mathbf{z}}_t),

with zt=tz+(1−t)ξ\mathbf{z}_t=t\mathbf{z}+(1-t)\boldsymbol{\xi}, z^t=zt+(1−t)fFM(zt,t)\hat{\mathbf{z}}_t=\mathbf{z}_t+(1-t)f_{\text{FM}}(\mathbf{z}_t,t), ξ∼N(0,I)\boldsymbol{\xi}\sim\mathcal{N}(0,I) and t∈(0,1)t\in(0,1); z^t\hat{\mathbf{z}}_t is detached, so no gradient flows through the flow model. PEM shares the data term, forward operator, decoder, initialisation and step budget, replacing λFM(z−z^t)\lambda_{\mathrm{FM}}(\mathbf{z}-\hat{\mathbf{z}}_t) with the weighted gradient of its explicit energy. We have added this description to the appendix.

Thank you for the clarifications and the additional CIFAR-10 experiment. I appreciate that the flow baselines share the same MAISI VAE. Two questions remain. Beyond slice-based FID, is there quantitative evidence of anatomical consistency across the generated 3D volumes? For sparse-view CT reconstruction, how does PEM compare with a 3D diffusion prior using the same MAISI VAE and reconstruction protocol?

We are glad that our clarifications and additional experiments have addressed many of the reviewer’s concerns. We welcome the continued discussion and thank the reviewer for the opportunity to address the remaining questions.

Q1. Anatomical consistency. The reviewer raises a valid point. We have conducted an additional experiment on 3D volumes to provide quantitative evidence beyond slice-based FID. Using CADS [Xu et al., 2026, Nature Communications], we segmented the lungs and heart in 1,000 generated volumes per model and 1,000 real CT-RATE scans, and compared organ volumes and lung-to-heart ratios. These measurements assess volumetric organ sizes and relative proportions. Values are median [25th–75th percentile].

ModelLung (mL)Heart (mL)Lung/heart
GT (real raw)4050 [3215–5194]453 [371–556]9.1 [6.3–12.5]
OT-Flow Matching4013 [3164–4931]480 [390–596]8.6 [6.1–11.3]
Rectified Flow3782 [2976–4626]466 [372–598]8.2 [5.8–11.0]
UNet PEM (ours)3651 [2977–4425]494 [392–608]7.6 [5.6–9.9]
MedNeXt PEM (ours)4484 [3738–5139]473 [405–559]9.3 [7.7–11.4]

For all models, the median lung volume, heart volume and lung-to-heart ratio fall within the interquartile range of the real scans, and the interquartile ranges overlap substantially with the real distribution. This indicates that the generated volumes have realistic organ sizes and proportions. This analysis is now included in Appendix C and serves as supporting evidence of volumetric anatomical plausibility.

Xu, M., et al. CADS: a comprehensive anatomical dataset and segmentation for whole-body anatomy in computed tomography. Nature Communications, 2026.

Q2. Non-flow-based priors. We thank the reviewer for the opportunity to contextualize the baseline selection. Our choice of OT-flow was motivated by a paper on noisy linear inverse problems [Pokle et al., 2024, TMLR], where the proposed flow-based method outperformed the evaluated diffusion-based methods in most settings. Furthermore, our OT-flow baseline in Table 2 uses the same MAISI VAE, forward operator, initialisation and step budget as PEM. A diffusion-based prior would likely require method-specific calibration and a potentially larger sampling budget, which would complicate a matched comparison. We therefore chose OT-flow to maintain a controlled comparison under a shared protocol. We hope this clarifies the baseline selection.

Pokle, A., et al. Training-free Linear Image Inverses via Flows. Transactions on Machine Learning Research, 2024.

We thank the reviewer for the constructive discussion, which has strengthened the paper. We hope the additional evidence and clarifications resolve the remaining questions and would be grateful if the reviewer would consider updating their initial score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.