- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
NeuronDiscover combines a mechanistic simulator, a learned world model, adaptive experiment selection, and a revisable hypothesis graph. It chooses interventions and measurements to distinguish biological mechanisms from simulator or sensor errors, then updates claims using independent evidence. Also, evaluation covers simulated brain-fluid transport and neuronal recordings from held-out donors
Critical strengths
- This paper addressed an important scientific inference problem -- explicitly separating mechanism uncertainty from simulator discrepancy helps prevent model errors from being mistaken for biological explanations. Also, its prospective testing, scoped claims, and preserved counterexamples make the discovery process more auditable.
- The paper demonstrates that there are substantial gains with substantial experimental controls. The method reports 4 resolved relations per world, compared with 3.4 for the matched external agent, while reducing false support from 8% to 5%. The paired improvement is +0.60 [95% CI: 0.41-0.79] (statistically significant) across 32 independent sources. Also, the component ablations and deterministic controls help establish where these gains arise (Tables 2-4).
Critical weaknesses
- The paper's claimed certification savings rely on an insufficiently justified calibration assumption. For example, equation 50 uses reference samples from each program’s outcome distribution, whereas Table 23 assumes that a single shared reference batch can calibrate all programs. Sharing event definitions does not establish that reuse is valid, and this matters because the directly demonstrated comparison favors direct reference certification: 8/8 programs certified versus 5/8, using the same 65,536 reference executions (Tables 21-22).
- Evidence for broader mechanistic discovery remains limited. Transport discovery operates within a supplied mechanistic ontology and executable model family. The real-recording tasks recover input resistance, firing gain, and adaptation, without establishing underlying biological mechanisms. These are useful demonstrations, but leave generalization beyond the designed hypothesis families uncertain.
Questions for authors
- What sampling or transfer argument makes the shared calibration batch in Table 23 valid for every program-specific discrepancy bound? Please account for all reference executions and explain whether the direct-certification baseline could reuse the same evidence.
- In omitted-relation tasks, what is withheld from the ontology and executable mechanism templates, versus merely omitted from the initial graph? Please provide a representative recovered relation and explain why the matched deterministic controller cannot generate it.
- Can you please provide a working link to your code?