- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ENFR (Evidence Networks with flow-matched references) combines flow matching with evidence networks to estimate the absolute evidence for likelihood-based and simulation-based inference techniques. Estimating the absolute evidence can address model misspecification and goodness-of-fit assessments. The paper proposes ENFR for single models (SingleENFR) and multiple models (MultiENFR). It also introduces a loss function to jointly estimate the Bayesian evidence and Bayesian factors for multiple models (MultiENFR).
Critical strengths
- Overall, the work is interesting and a valuable contribution: the estimation of the absolute value of evidence has been largely underexplored in the literature, to my knowledge.
- The work contains a wide range of tasks, building up from simple tasks with analytical solutions to complex neuroscience and cosmological problems.
Critical weaknesses
Figures
- Figure 1: It would be good to adjust the colors and style of the first figure, since the colors strongly resemble what AI models typically generate (although I am not implying that the figure is directly LLM-generated).
- Figures 3 and 4: Please add plot titles to the figures to increase clarity. For instance, for Figure 4, titles such as "Hodgkin-Huxley" and "Galaxy assembly bias" would make it visually clearer.
Language
- Line 157: "We choose flow matching for this task". Do you mean throughout all the tasks/experiments in the paper?
Experiments
- The baseline comparison with Flow matching and CTSM variants has not been shown in all experiments (e.g., no CTSM-SB + FlowRef in Figure 3). Please add these baseline comparisons or explain why you did not compare your approaches (SingleENFR/MultiENFR) to them.
- A calibration test does not seem to me to be enough to validate accurate absolute evidence. Could you explain why you used this as the main conclusion that your method (MultiENFR) works on the Hodgkin-Huxley and Galaxy assembly tasks? Are there other ways to evaluate whether your method works on those tasks?
Reproducibility
- Please share the codebase via an anonymous link so that the code and results can be verified.
Minor points
- Line 045: "What we ultimately care about in science, however, is assessing the relative fits of models when confronted with data". It would be better to tone down this statement, since it is not the (only) ultimate goal of science.
- There are some issues with the references in this paper. Could you double-check whether you have the same issues on your side?
- Line 042: (e.g. von Wietersheim-Kramsta et al., 2025; Thomsen et al., 2026, …): These two citations are displayed differently from the others (not underlined/in blue).
- Line 098: (Masnadi-Shirazi & Vasconcelos, 2008): the link to Masnadi-Shirazi is not clickable
- Line 119: "While Rhodes et al. (2020)": "While" is a clickable link.
- Line 1090: Adam Kingma & Ba (2014): Should be "(Adam; Kingma & Ba, 2024)"
- Typos
- Line 320: damped(β...): no space between "damped" and "("
Overall, I am open to considering a weak accept if the authors share the codebase to reproduce and confirm the results, and if they can provide a justifiable explanation of why the calibration test is sufficient to validate accurate absolute evidence.
Questions for authors
-
Out of curiosity, do you have an intuition for how your approach would work with other generative models (rather than flow matching)?
-
In the limitations section, Line 475, one mentions: "Encouragingly, the evidence-network correction is most effective precisely in the small data regime (Figure 2b)". However, in the figure caption, the simulation budget is fixed at 5 × (Figure 2a) or varies between and , which does not seem to be a "small data regime". Could you clarify where this comment comes from?
3 comments
Thank you for your reply and for sharing the code. I could verify that the reported numbers are consistent with the code. Overall, I think this work presents an interesting and fresh idea for estimating the absolute evidence, which, to my knowledge, has been largely underexplored so far and deserves a place in the literature.
A small point of doubt came up regarding the code. I was wondering why the Flow, CTSM-VP, and CTSM-SB+FlowRef use a default 30-epoch cap in the GMM sweeps (MAX_EPOCHS="${GMM_DIM_MAX_EPOCHS:-30}"), while ENFR uses a default 100-epoch cap (EN_MAX_EPOCHS="${GMM_DIM_EN_MAX_EPOCHS:-100}"). Since the method uses early stopping, I would think that increasing the cap to a much higher value (especially for MAX_EPOCHS) and saving the best models would be a better practice, since this cap might have led to results that have not converged.
I change my score to a weak accept.
Response to Reviewer 3
We would like to thank the reviewer for explicitly checking our code and ensuring the reproducibility of our results.
We acknowledge the reviewer’s excellent point concerning the maximum epoch cap in our GMM experiments. We have rerun the flows and evidence networks for the GMM dimensionality scaling experiment, ensuring that every model is early stopped. We can report the following results with the newly trained flows (showing mean standard deviation of ):
| d | Flow (original) | SingleENFR (original) | Improvement (original) | Flow (updated) | SingleENFR (updated) | Improvement (updated) |
|---|---|---|---|---|---|---|
| 64 | 11.93 ± 0.65 | 7.05 ± 0.67 | 40.9% | 11.64 ± 1.19 | 6.94 ± 0.70 | 40.4% |
| 128 | 18.00 ± 2.32 | 10.33 ± 1.08 | 42.6% | 15.92 ± 1.59 | 9.39 ± 0.21 | 41.0% |
| 256 | 25.82 ± 0.54 | 11.86 ± 0.62 | 54.1% | 23.24 ± 0.43 | 11.19 ± 0.60 | 51.9% |
| 512 | 30.84 ± 0.65 | 28.20 ± 2.28 | 8.6% | 29.81 ± 0.12 | 24.31 ± 4.41 | 18.5% |
| 1024 | 34.17 ± 0.23 | 32.65 ± 2.28 | 4.4% | 32.28 ± 0.48 | 30.08 ± 1.55 | 6.8% |
While the flow-only results slightly improve with longer training, the ENFR improvement remains consistent within a few percentage points up to 256 dimensions. At higher dimensions the improvements are slightly larger than in the original experiments. We can confirm that all CTSM-VP runs did in fact early stop.
We commit to including the updated GMM runs in the manuscript along with new CTSM-SB + FlowRef results using the updated flows by camera-ready, if accepted.
Response to Reviewer 3
We thank the referee for their detailed feedback and for highlighting the value of our method for the simulation-based inference program.
W1: Colors in Figure 1
We have changed the colors to a palette that is also colorblind friendly.
W2: Plot titles
We have added plot titles to Figures 3 and 4.
W3: Flow matching language
Yes, in every application in this work we train flow matching. We have clarified the language in the main body.
W4: CTSM in other experiments
We do show CTSM-SB + FlowRef for the other likelihood-based experiments in the corresponding tables 5 and 7 in the appendix. We are not showing them in the same plot for readability. We are now explicitly referring to those tables in the figure caption. We are not showing CTSM-VP as its performance has been clearly worse than CTSM-SB + FlowRef already in the Gaussian example and is uncompetitive.
W5: Calibration test does not validate absolute evidences
We agree with the referee on this point. However, in the absence of a likelihood, there is no independent estimate to compare our simulation-based method to. In order to validate the absolute evidences of ENFR, we ran the likelihood-based benchmarks. We would argue that the simulation-based examples are not benchmarks of the absolute evidence, but research applications for which there is no known ground truth. As the absolute evidences cannot be tested, we instead test the calibration of the resulting evidence ratios.
W6: Publishing code
Please find the anonymized code at this repository: https://anonymous.4open.science/r/ENFR-F041/
W7 "minor points"
We have relaxed the statement about the ultimate goal of science and we corrected the two typos. In general, we could not reproduce the same problems with the pdf file in a standard Firefox browser.
Q1: Other generative models
Our approach is not restricted to flow matching. The essential requirements for the reference model are that we can sample from it and evaluate its density. Consequently, other generative models satisfying these requirements could be used, provided that they yield sufficient overlap with the target distribution.
Q2: Small data regime
For the GMM scaling tests, the smallest data set we have tested and shown in Figure 2b is 5×104 simulations. We have softened the language on calling this the small data regime.