acceptodds
Withdrawn from ICLR 2027

Mode Collapse Is Cheap to Detect: A Ground-Truth-Free Pre-Flight Check for Neural Samplers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural samplers are trained against an unnormalised target with no samples from , so a practitioner cannot tell whether an expensive run has silently dropped part of the target. The usual diagnostics are expectations under the model and are therefore confined to its support: we exhibit a sampler with self-normalised that misses of the mass. Splitting the missing mass into lost support and misallocated weight shows the two established diagnostic families have disjoint blind spots – ESS is blind to the first, and normaliser comparison is provably blind to the second at any budget, returning while up to of the mass is misallocated. We then argue that detecting missing mass is easier than sampling it – detection needs one point per missed basin plus local curvature, while correcting the sampler generally costs substantially more – and turn this into a pre-flight check using only . On exactly solvable targets the check reaches -level accuracy at a few percent of training cost on the harder benchmarks, while a tuned SMC reference requires – of training cost, and it applies unchanged to a controlled-SDE sampler with no tractable density. A self-diagnostic flags configurations whose estimate should not be believed, but cannot certify a run: a mode with attraction probability below escapes the estimator and the diagnostic together. We map the resulting boundary on LJ-13.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
mode collapse, neural samplers, sampling from unnormalised densities, diagnostics without ground truth

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nncpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates mode collapse detection in neural samplers without access to ground-truth samples. The authors analyze the limitations of conventional diagnostics, including ESS and normalizing-constant comparisons, and propose an auditing method combining multi-start mode search, Laplace mass estimation, and empirical basin-occupancy comparisons. Theoretical results characterize the limitations of these diagnostics and the proposed estimator. Experiments on synthetic multimodal targets and a controlled-SDE sampler demonstrate promising accuracy and computational efficiency, while also highlighting the method's limitations on more challenging energy landscapes.

Critical strengths
  1. The paper addresses an interesting and important problem: detecting mode collapse in neural samplers without access to ground-truth samples, where conventional diagnostics may provide misleading assessments. Developing reliable and computationally efficient diagnostics in this setting is practically valuable and deserves further investigation.
Critical weaknesses
  1. The proposed method relies heavily on established techniques, as acknowledged by the authors in Section 3. Similar procedures have also been explored in existing Laplace-mixture approximation methods, such as [1] (Sec. 3.5). The main additional step is to compare the estimated target basin weights with empirical occupancy frequencies from the trained sampler, resulting in a standard total variation distance between two discrete distributions. While this provides a potentially useful application to sampler auditing, the construction appears relatively straightforward and does not exploit any specific properties of neural samplers. Consequently, the paper does not sufficiently establish a substantial methodological advance beyond adapting existing mode-based approximation techniques to a new evaluation task.
  2. Although the paper presents eight propositions, most of the theoretical results characterize the limitations of existing diagnostics or the failure modes of the proposed approach, rather than establishing new positive guarantees for the estimator itself. In particular, Propositions 1–4 largely follow from standard importance-sampling identities, while Propositions 5–7 demonstrate the impossibility of finite-query certification and the potential failures of incomplete mode enumeration. Proposition 8 provides a conditional bound on local-mass approximation errors, but does not establish an overall estimation guarantee accounting for mode discovery and sample assignment. More importantly, despite the central claim that mode collapse is cheap to detect, the paper provides no theoretical characterization of the computational complexity required to achieve a prescribed estimation accuracy. Consequently, the theoretical analysis is informative but offers limited support for the claimed accuracy and computational advantages of the proposed auditing method.

[1] Huang Y. Sampling via Gaussian Mixture Approximations[J]. arXiv preprint arXiv:2509.25232, 2025.

Questions for authors
  1. Most theoretical results characterize diagnostic limitations rather than establish positive guarantees for the proposed estimator. Under what assumptions can the authors guarantee its estimation accuracy, particularly when mode enumeration is incomplete?
  2. Given that discovering important modes and computing their Hessians can be expensive, what justifies the central claim that mode collapse is cheap to detect beyond the specific benchmarks considered? How does this advantage depend on the complexity of the target energy landscape?
Reviewer_HHJZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to assess mode coverage of neural samplers when only an unnormalised target density and its derivatives are available. It defines missing mass as the total variation distance between target and sampler probabilities over a basin partition, and separates support loss from weight misallocation. It characterises failure cases of importance-weight ESS and normaliser comparisons, and empirically examines the limited power of KSD on separated multimodal targets.

The proposed audit combines multistart optimisation, Hessian-based Laplace mass estimates, and Mahalanobis assignment of sampler draws. Search-growth and singleton-mass statistics provide a heuristic self-diagnostic. Experiments cover Gaussian mixtures, Many-Well variants, normalising flows, and a controlled-SDE sampler, with SMC and parallel-tempering comparisons. Additional experiments examine finite-sample resolution, local approximation bias, and limitations on LJ-13.

Critical strengths
  1. The paper addresses a consequential evaluation problem with a sampler-independent interface. A sampler can appear healthy under diagnostics evaluated on its own draws while failing to represent important target regions. The proposed method uses target derivatives and sampler draws without requiring a tractable sampler density. The controlled-SDE experiments demonstrate why this interface matters, and the support/weight decomposition helps explain what different diagnostics can reveal.

  2. The empirical analysis carefully exposes both successes and failure mechanisms. The use of analytically or quadrature-computable target weights makes the synthetic evaluation interpretable. Appendix D adds a direct basin-weight competitor, avoiding exclusive reliance on a normaliser-gap baseline that is structurally insensitive to weight misallocation. The analyses of partition agreement, reference-sample resolution, heterogeneous local-mass bias, and LJ-13 substantially improve the credibility of the reported findings. I particularly appreciate the explicit disclosure that a cleared diagnostic does not certify accuracy.

Critical weaknesses
  1. The central detection-versus-correction claim is not established. In Table 6, the Laplace mixture produced by the audit already substantially improves mode coverage without additional target evaluations. Successful demonstrations remain concentrated on small synthetic landscapes, while LJ-13 provides no usable multimodal demonstration. This leaves the practical contribution beyond classical mode enumeration and Laplace approximation insufficiently established.

  2. The headline cost understates the budget required by the recommended procedure. At the advertised 2.7% budget, the self-diagnostic clears only 1/5 Many-Well runs and 0/5 rotated Many-Well runs (Table 13). Moreover, cleared runs can still have errors up to 0.1101 on held-out targets (Table 14). The cost and reliability of obtaining an actionable audit result therefore remain unclear.

Questions for authors
  1. Clarification of the cost claim: Could you report the cost of the actual recommended procedure—starting from a specified budget and increasing it according to the self-diagnostic—together with the fraction of runs that remain flagged at the maximum budget? Please reconcile these results with the 2.7% headline and Table 13. Wall-clock accounting, including any repeated search and sampler-draw generation, would make this comparison more informative.

  2. Clarification of diagnostic reliability: For a prespecified accuracy tolerance, such as (|\widehat M-M|\leq 0.01), what fraction of held-out runs are cleared, and what fraction of cleared runs violate that tolerance? Table 14 reports worst errors but does not fully characterise this tradeoff. How should a practitioner interpret a cleared numerical estimate when local-mass bias remains unchecked?

  3. Additional evidence that would affect my assessment: Is there an end-to-end example with genuinely non-ellipsoidal or otherwise challenging multimodal basins where the audit accurately identifies missing mass at low cost, while the Laplace mixture returned by the same audit remains an inadequate sampler? Such evidence would help establish the practical distinction between inexpensive auditing and inexpensive sampling that motivates the paper.

Reviewer_7HBuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a diagnostic for evaluating whether a neural sampler misses modes or assigns incorrect probability mass across them. The method combines multi-start optimization to discover modes, Laplace approximations of basin masses, and empirical occupancies estimated from sampler draws. It also analyzes limitations of existing diagnostics and introduces a self-diagnostic for incomplete mode discovery. Experiments demonstrate accurate estimates on several synthetic targets and investigate limitations on LJ13.

Critical strengths

The problem is well motivated. Evaluating mode coverage and relative mode weights is important for neural samplers, particularly when reference samples from the target are unavailable. A diagnostic that identifies these failures using the target energy and sampler outputs would be practically useful.

Critical weaknesses
  1. Reliability depends strongly on mode discovery and basin geometry. The method requires discovering modes that collectively account for the relevant target mass. Multi-start optimization does not ensure this on complex landscapes, and an important undiscovered basin can escape both the estimator and its self-diagnostic.

    The local-mass calculation in Eq. (3) further relies on a quadratic approximation around each discovered mode, requiring a positive-definite Hessian of the negative log density after appropriate treatment of symmetry-induced zero directions. I recognize that the estimator does not require a globally accurate Gaussian-mixture approximation: Proposition 8 explains that a common multiplicative bias in basin masses cancels upon normalization. Nevertheless, accurate relative masses across basins with different geometries remain a substantive requirement. The heterogeneous Many-Well experiments partly address this concern, but also show residual errors despite complete mode discovery. The self-diagnostic does not assess this local-mass bias.

  2. Scalability and usefulness on complex landscapes remain insufficiently established. Appendix O evaluates dimensions from 16 to 128 while holding the number of Many-Well modes fixed at 32. This measures the cost of increasing ambient dimension without increasing the number of wells or substantially changing their geometry. It therefore provides limited evidence about problems in which dimension, landscape complexity, and mode-discovery difficulty increase together. The authors acknowledge the growing Hessian cost, but the proposed computational remedies are not implemented.

    The LJ13 results are my strongest concern. Appendix P does not demonstrate successful end-to-end collapse detection in a useful multimodal regime. At higher temperatures, enumeration misses substantial harmonic weight; at sufficiently low temperatures, the resolved harmonic reference is effectively dominated by one minimum. These limitations are acknowledged, but they leave the diagnostic’s usefulness on a complex physical landscape unestablished. The successful demonstrations remain concentrated on synthetic targets with comparatively favorable structure.

  3. The relationship to prior work on mode blindness deserves clearer discussion. [1] analyze the insensitivity of Fisher divergence to relative weights of well-separated components, including exact blindness on disconnected supports. [2] emphasize evaluating recovery of relative mode importance as separation and dimension increase. [3] address mode blindness in energy-based model training. Explaining the connection between these settings and the proposed audit would sharpen the paper’s positioning.

  4. I also found the presentation difficult to follow in places. A more direct explanation of the target quantity, the estimator’s steps, and the assumptions needed for each step would improve accessibility. These presentation and literature suggestions are secondary to the technical concerns underlying my recommendation.


[1] Mingtian Zhang, Oscar Key, Peter Hayes, David Barber, Brooks Paige, and François-Xavier Briol. Towards Healing the Blindness of Score Matching. arXiv:2209.07396, 2022.

[2] Louis Grenioux, Maxence Noble, and Marylou Gabrié. Improving the Evaluation of Samplers on Multi-modal Targets. ICLR Workshop on Frontiers in Probabilistic Inference: Learning meets Sampling, 2025.

[3] RuiKang OuYang, Louis Grenioux, and José Miguel Hernández-Lobato. A Diffusive Classification Loss for Learning Energy-based Generative Models. ICML, 2026.

Questions for authors
  1. Given that the self-diagnostic cannot assess local-mass bias, what information available without ground truth could help a practitioner judge whether relative Laplace masses are reliable on a new target?

  2. What do the existing experiments or analysis establish about applicability when the number of relevant modes and the heterogeneity of their geometries increase, beyond the fixed-32-mode dimension sweep? Clarifying the supported target class would help assess the method’s practical scope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.