- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this work, the authors propose Projected Energy Matching (PEM). It has 3 stages for training a time-independent scalar energy potential. Stage 1 trains an autonomous flow teacher with minibatch OT couplings. Stage 2 freezes the teacher and projects the teacher field onto the conservative subspace. Stage 3 adds the Energy Matching contrastive loss with Langevin negatives to keep the projection term active.
On CIFAR-10, they show faster FID convergence than EM. In 3D CT, PEM reaches a slightly lower FID than OT Flow Matching and Rectified Flow on same backbone
Critical strengths
- EM bottleneck is the high variance of stochastic transport targets. The authors identify this, and through PEM, replace stochastic targets with deterministic supervision from a pretrained velocity model. This frozen teacher is easy to implement
- Explicit scalar energy has several downstream advantages, compared to only a transport vector field. The same unconditional model is used for generation, distribution-shift scoring, and incorporation into an inverse-problem objective without task-specific retraining.
- The authors also propose negative-sample caching during gradient accumulation to make the contrastive phase practical in 3D.
- PEM shows better FID on both CIFAR-10 and 3D CT
Critical weaknesses
- Table 3 Phase 1+2 significantly increases FID. Is Phase 2 projection needed in that case?
- The authors mention Params in Table 1, but a fair comparison would be the inference and training time (teacher + projection + refinement). I think the authors omit the teacher params while reporting
- Eq. 1 trains the teacher on OT-paired interpolants from a memory bank M. Algorithms 1 and 2, however, sample x_data and x_noise independently (lines 5–7 / 3–5) and form x_t from random pairs, while the comment says “interpolate along optimal transport paths.” The projection is therefore evaluated on a different x_t distribution than the one the teacher was trained on, and the teacher is queried off its training support. Sec. 2.2 states that the projection is weighted by “the density of interpolants along the transport paths,” and Sec. 2.2 calls OT couplings “crucial.” The authors must clarify which coupling is actually used in Phases 2 and 3 and whether the claimed projection interpretation holds.
- Can authors show absolute number of FID from Fig 2?
- In experiments, standard diffusion and flow solvers for this task are missing (DPS, DiffusionMBIR, plug-and-play denoisers, and ICTM, which the paper itself cites).
- The conclusion claims “improved seed stability” on CIFAR-10, which is not shown in the main text.
Questions for authors
I think for me, Table 3's Phase 1+2 is the main point of contention for me. The authors should clarify this, and address other points in weakness section
We thank the reviewer for the positive assessment, and for recognising the variance of stochastic transport targets as the bottleneck of Energy Matching and the downstream value of an explicit scalar energy in 3D.
W1. Training phases. We apologise for the ambiguous labelling. We have updated Table 3 (now Table 4) with a description of each phase:
Only the final phase yields an energy-based model. Phase 2, the projection, is the transport phase of Energy Matching: it yields a conservative flow without a basin at the data, so its samples follow the transport funnel without settling (FID 62.60). It cannot be omitted, because contrastive refinement draws its Langevin negatives from noise and relies on this funnel to bring them toward the data. Energy Matching likewise initialises its contrastive phase from its transport phase. Contrastive refinement then forms the basin, and the complete energy model matches its unconstrained teacher (40.87 vs. 41.90) while providing an explicit energy.
W2. Compute and parameters. Table 1 compares sample quality at comparable model size. We make no claims about training time or inference speed relative to flow models, where these excel. Params reports the model used at inference: PEM samples with ϕθ alone (10.7M), while the teacher (10.3M) is used only during training. Training an energy model requires double-backward differentiation and is more costly per step than training a flow model. This is the price of an explicit energy (scalar scoring, an explicit reconstruction objective, stable sampling).
W3. Couplings. The teacher (Phase 1) is trained on OT-coupled pairs, which give mainly straight, non-crossing paths. This is why OT couplings are crucial for the teacher. In Phases 2 and 3, the student matches the teacher's predicted velocity vteacher(xt), a target that depends on xt alone, so no pairing is needed. Data and noise are paired independently only to choose query points, which spreads them beyond the OT paths, so the potential is also shaped where Langevin chains travel. The teacher is not queried off its support, both couplings share the same data and noise marginals, so xt covers the same space and is only weighted differently. The projection therefore still returns the closest gradient field to the teacher, assuming the teacher generalises to these query points. We have corrected the algorithm comment and stated this in Sec. 2.2.
W4. Absolute FID. Absolute FIDs behind Fig. 2(b) (CIFAR-10, batch size 32, 10K samples), mean over 3 seeds with min–max in brackets:
PEM has lower mean FID at every checkpoint and reaches 56 after 17K steps, a level the EM mean does not reach within 145K steps. These values serve the controlled training-signal comparison only and do not represent the FID attainable under full training. This table is now in the Appendix.
W5. Reconstruction baselines. The sparse-view experiment is a controlled ablation of the prior, not a benchmark of reconstruction pipelines. The energy and flow priors share backbone, training data, latent space and forward operator, so the comparison isolates the prior up to solver details, which we give in Appendix G. Following the suggestion, we have extended it with DPS, a second solver for the same flow prior (10 volumes):
PEM MAP outperforms the flow prior under both solvers. Benchmarking a higher-resolution 3D PEM prior against complete pipelines, including DiffusionMBIR, plug-and-play denoisers and ICTM, is the subject of follow-up work.
W6. Seed stability. Pooling all replicate seeds on CIFAR-10 (3 seeds × 9 strata), the seed standard deviation of log FID is 0.127 for PEM and 0.376 for EM, i.e. about 3× smaller for PEM (95% HDI [1.7, 4.5]). This is now reported in the main text.
Q1. Please see W1. Phase 2 is the transport phase required for contrastive refinement, and the final model matches its teacher while providing an explicit energy.