- Rating
- 1
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies estimating a critical threshold, the value of a control variable at which a latent system changes state, from sparse data. The case study is the measles herd-immunity threshold: the MMR coverage at which the population-average effective reproduction number falls to one.
The authors propose a “qualification-first” framework: fix the target quantity, profile each method’s native estimand, and use a known-truth simulation to assign each method a role (target-responsive, calibrated, native diagnostic, or abstaining). Methods are fused only if at least two are calibrated for the same target. Three methods are tested: a hierarchical Bayesian model (HBM), change-point detection (CPD), and a biologically informed neural network (BINN).
In a simulation with three true thresholds and three replicates each, the HBM tracks the truth with positive bias, while CPD’s breakpoint moves the opposite way. On 10,153 US county-years (128 with cases), the HBM gives and a threshold of 0.991, CPD returns a non-significant breakpoint at 0.992, and the BINN abstains. No fusion is performed.
Critical strengths
- Critical-threshold estimation is an important problem, especially when simulating and calibrating epidemic models against observations. The core caution is correct and well illustrated. Similar numbers from different methods are not evidence they estimate the same quantity.
- Honest reporting. The paper refuses to fuse, lets the BINN abstain, and flags its own estimate as extrapolative.
Critical weaknesses
The problem is never made concrete, and it is smaller than the framing suggests. The measles threshold is the closed-form herd-immunity relation with fixed at 0.97, so the task reduces to estimating a mean from county-year counts. The “known-truth” generator (App. B) is a static regression with no transmission dynamics. The main text never states this, so a reader cannot tell how the threshold enters the model or what CPD and the BINN add. The framework itself (fix the target, state each estimand, validate by simulation, abstain when unidentified) is standard identifiability and simulation-based calibration practice. The contribution is a vocabulary for it plus one case study that yields no estimate.
Questions for authors
Clarifications
- Please state in the main text that the threshold is and the task is estimating . What does the framework add beyond standard identifiability checks?
- Why does the validation grid stop at 0.97 when the estimate is 0.991?
- Why is this paper suited to ICLR? The only learning component abstains.
Suggestions
- Replace Fig. 1 with a diagram showing the data, the latent , and where each method reads the threshold.
- Validate against a dynamic transmission simulator with importation.