- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a graph-based local sampling framework for noise-contrastive estimation. The work aims to address geometric shortcuts that can reveal the positive sample in a locally generated candidate pool without learning the target energy function. The authors first formulate a multi-way classification task over jointly sampled candidates. Then they use symmetric generation graphs to construct sampling kernels whose proposal terms cancel, yielding an energy-only objective. They further introduce GORI-CE, a cyclic Gaussian-bridge sampler, and establish Fisher consistency under stated assumptions and a connection to contrastive divergence. Experiments cover synthetic density estimation, five real-data density-estimation benchmarks, and latent-space image generation on SVHN, CIFAR-10, and CelebA.
Critical strengths
- The geometric shortcut identified in local multi-negative NCE is an interesting problem. Using graphs to design the candidate pool is an intuitive way to address it.
- The paper has a strong theoretical focus, with detailed derivations supporting the proposed sampling framework.
Critical weaknesses
- It is unclear whether the gains come from the cyclic structure or simply from sampling negatives at different distances. A multi-scale Gaussian baseline with a similar distance distribution and the appropriate NCE correction would help answer this question.
- The paper lacks ablation studies on the number of negative samples, the sampling distance, and the number of anchors in cyclic anchoring. The authors should vary these settings to show how each affects performance.
- The image experiments are limited to SVHN, CIFAR-10, and CelebA, all at low resolutions. Results on a larger and more diverse dataset, such as ImageNet at a suitable resolution, would make the case for scalability and generalization more convincing.
- The experimental evaluation is still limited in terms of datasets and metrics. The authors should provide a broader assessment of generation quality, sample diversity, and computational efficiency.
Questions for authors
See weaknesses. If the authors can adequately address these concerns, I would consider raising my score.
6 comments
W3: ImageNet
The image experiments are limited to SVHN, CIFAR-10, and CelebA, all at low resolutions. Results on a larger and more diverse dataset, such as ImageNet at a suitable resolution, would make the case for scalability and generalization more convincing.
- On additional dataset(ImageNet): Our original experiments on SVHN, CIFAR-10, and CelebA64 follow the established evaluation settings of latent-space EBMs [1,2,3,4], where quantitative results are typically reported at or resolution. Higher-resolution studies in this line often involve additional components, such as diffusion-amortized MCMC [4]. Moreover, existing EBM studies commonly evaluate on ImageNet32 [5–8], while ImageNet64 is a widely used benchmark for unconditional generative modeling [9–12]. We therefore extend our evaluation to ImageNet64, retaining the resolution of CelebA64 while substantially increasing dataset scale and diversity (1.28M training images, 1,000 classes).
- Since our primary focus is on EBM training objectives rather than generator architectures, we use a publicly available ImageNet-pretrained autoencoder DC-AE [13] as the frozen first stage and train only the latent EBM prior. All methods share the same autoencoder, sampling procedure (GMM-based SIR followed by MALA), and evaluation protocol (FID, precision, and recall), allowing us to isolate the effect of the EBM training objective.
- All Local NCE methods achieve competitive FID scores of approximately 85–86, comparable to CD-10, while DSM achieves the lowest FID (84.0). In terms of precision and recall, GORI-CE achieves higher recall than DSM (0.216 vs. 0.188) while maintaining comparable precision. These results demonstrate that our Local NCE framework remains competitive on a substantially larger and more diverse dataset, supporting its applicability beyond the original image benchmarks. We have added these results to Appendix E.4 of the revised paper.
W4: Sample quality and cost
The authors should provide a broader assessment of generation quality, sample diversity, and computational efficiency.
- On sample quality and cost: We additionally evaluate sample quality and diversity across the Local NCE methods and CD/DSM baselines using the improved precision and recall metrics [14] (Appendix E.3, Figure 10 of the revised paper). We also compare wall-clock time per training update (Appendix E.3, Figure 11 of the revised paper).
- The results reveal dataset-dependent quality–diversity trade-offs. In particular, GORI-CE achieves one of the highest precision values on CelebA64 and remains competitive on SVHN and CIFAR-10, although no method consistently dominates both metrics. Local NCE methods require only 2.0–2.3 ms per update, substantially less than DSM (3.2 ms) and CD-1/5/10 (4.1/7.8/12.3 ms).
- Together, these results highlight the practical value of the Local NCE framework, which supports diverse sampling constructions while maintaining competitive generation quality and low computational cost.
Reference
[1] Tobias Schröder, Zijing Ou, Jen Lim, Yingzhen Li, Sebastian Vollmer, and Andrew Duncan. Energy discrepancies: a score-independent loss for energy-based models. Advances in Neural Information Processing Systems, 36:45300–45338, 2023.
[2] Pang et al. Learning Latent Space Energy-Based Prior Model. NeurIPS 2020.
[3] Xiao and Han. Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model. NeurIPS 2022.
[4] Yu et al. Learning Energy-Based Prior Model with Diffusion-Amortized MCMC. NeurIPS 2023.
[5] Du and Mordatch. Implicit Generation and Modeling with Energy-Based Models. NeurIPS 2019.
[6] Du et al. Improved Contrastive Divergence Training of Energy-Based Models. ICML 2021.
[7] Lee et al. Guiding Energy-based Models via Contrastive Latent Variables. ICLR 2023.
[8] Zhu et al. Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood. ICLR 2024.
[9] Chrabaszcz et al. A Downsampled Variant of ImageNet as an Alternative to the CIFAR Datasets. arXiv:1707.08819, 2017.
[10] Child. Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images. ICLR 2021.
[11] Nichol and Dhariwal. Improved Denoising Diffusion Probabilistic Models. ICML 2021.
[12] Kingma et al. Variational Diffusion Models. NeurIPS 2021.
[13] Chen et al. Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models. ICLR 2025.
[14] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems, 32, 2019.
Thanks for the response. I appreciate the authors' efforts to address my concerns. However, I still have three concerns:
- ImageNet results: Figure 12 provides visual results, but I could not find a table with the exact FID, precision, and recall values.
- Image resolution: The evaluation is still limited to 64×64 or lower, which is insufficient for modern image generation tasks.
- Generation quality: The reported FID scores are much worse than those of existing strong generative models, raising concerns about practical effectiveness. Overall, I appreciate the additional efforts, and I will maintain the score.
We thank the reviewer for their thoughtful comments and constructive feedback. We address each point below, with the corresponding changes incorporated into the revised manuscript and highlighted in blue.
W1: Multiscale comparison
W2: Ablation on RMS v, negative counts J, and anchor count K
On RMS v: We perform an ablation study on the RMS v∈0.1,0.25,0.5,0.75,1.0 across five density-estimation benchmarks and three image datasets. Please see Section 6.2, Figure 3; Appendix E.2, Figure 7 of the revised paper. While the methods perform comparably at smaller distances, GORI-CE exhibits substantially less performance degradation as v increases, demonstrating greater robustness to the sampling scale.
FID vs. RMS distance v (CelebA 64; mean ± SD over 3 prior seeds; FID against the test set. Lower is better.)
On Negative count J: We chose the default J=19 to use a negative-sample budget comparable to the M=16 samples employed in the image generation experiments of ED [1]. We additionally conducted an ablation with J∈3,19,49,79,99 on HEPMASS and MINIBOONE in Appendix E.2, Figure 8 in the revised version. Both Anchor and GORI-CE improve as J increases, but GORI-CE exhibits substantially larger gains, with a widening performance gap over Anchor.
Test log-likelihood vs. number of negatives J (HEPMASS and MINIBOONE; mean ± SD over seeds 0–2; J+1 candidates per pool. Higher is better.)
On anchor count K: We vary K∈1,2,5,10,20 in cyclic anchoring on HEPMASS and MINIBOONE, fixing J=19 and v=1 (Appendix E.2, Figure 9 of the revised paper). Increasing K consistently improves test log-likelihood, but even with K=20, cyclic anchoring remains substantially worse than GORI-CE. This suggests that increasing the number of anchors alone cannot fully reproduce the benefits of cyclic sampling.
Test log-likelihood vs. Number of anchors K (HEPMASS and MINIBOONE, mean ± SD over seeds 0–2; cyclic anchoring with K anchors × 20/K nodes each; K=1 is Anchor (ED), K=2 is CA 2×10)