- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to distill neural likelihood models into compact, interpretable, and computationally efficient symbolic expressions. The proposed method adds a parameter-recovery term to the symbolic regression objective to favor expressions that preserve downstream parameter-estimation performance, then further screens candidates through full posterior inference. The authors evaluate the recovery-directed objective on the diffusion decision model, compare the computational costs of symbolic, neural, and analytical likelihoods, and demonstrate hierarchical modeling on real behavioral data.
Critical strengths
1.Accurately approximating neural likelihood values does not necessarily preserve the structure needed for parameter estimation, and the paper addresses this mismatch directly. Adding the recovery-directed term increases the proportion of expressions passing the recovery criteria for all four parameters from 1.2% to 19.5%. The authors also report results using independent search runs as the unit of replication, avoiding treating correlated expressions from the same run as independent observations. This is the paper’s most convincing contribution.
2.Expression 2 contains only 25 nodes, and its structure is related to the exponent in a known first-passage-time density. The runtime comparisons include compiled implementations of the exact likelihood and LAN, as well as gradient evaluations, rather than merely comparing a simple formula against an unoptimized network. These comparisons provide convincing evidence of practically meaningful computational gains.
Critical weaknesses
My main concern is whether the symbolic expressions preserve reliable parameter inference. High recovery correlations can mask systematic bias, and the SBC results concern normalized expressions rather than the unnormalized functions used in the main experiments. Reporting bias, RMSE, and interval coverage for the actual inference procedure would help establish whether the speed–accuracy trade-off is acceptable.
Questions for authors
-
Could you report parameter bias, RMSE, and interval coverage for the unnormalized expressions used in the main experiments? High recovery correlations alone may not indicate accurate estimates or uncertainty.
-
Equation (6) evaluates each parameter while fixing the others at their true values. How well does this proxy predict recovery when all parameters are estimated jointly, especially when parameters interact or compensate for one another?
3.The abstract states that candidate selection jointly considers complexity, parameter recovery, and distributional distance, whereas Section 2.3 describes ranking primarily by the weakest parameter-recovery correlation. Please clarify how these criteria enter the selection process.
3 comments
Thank you for your response. It has addressed most of my concerns, and I will maintain my positive rating.
Follow up response on parameter inference
We really appreciate your quick response and positive confirmation. Here is the response to your main concern.
My main concern is whether the symbolic expressions preserve reliable parameter inference. High recovery correlations can mask systematic bias, and the SBC results concern normalized expressions rather than the unnormalized functions used in the main experiments. Reporting bias, RMSE, and interval coverage for the actual inference procedure would help establish whether the speed–accuracy trade-off is acceptable.
Could you report parameter bias, RMSE, and interval coverage for the unnormalized expressions used in the main experiments? High recovery correlations alone may not indicate accurate estimates or uncertainty.
Your concern about the posterior inference on unnormalized expression alone is fair. The unnormalized expression defines a generalized posterior , whose posterior mean/median is a consistent estimate of the true only if is maximized at (Bissiri et al., 2016).
However, due to the missing θ-dependent normalizer in , the reported uncertainty metrics obtained directly from MCMC have no calibration guarantee. For uncertainty quantification, we therefore recommend the normalized likelihood , whose posterior is a standard Bayesian posterior under the approximate model. That is the reason why the SBC results in Appendix H.2 are based on normalized expression. We now state explicitly in the manuscript when the unnormalized expression can be used directly and when normalization and additional calibration are needed in the METHOD section.
"Figure 1 summarizes the pipeline,...Finally, the top-ranked expression is used in MCMC sampling. Since is a generalized posterior (Bissiri et al., 2016), its mean serves as a consistent point estimate. Yet its credible regions require additional calibration, such as normalizing the expression by ."
In Section 5 LIMITATIONS AND FUTURE DISCUSSION, we also describe the future improvements:
" First, the discovered expressions are unnormalized densities. A further credible region calibration is needed for uncertainty quantification (Syring & Martin, 2019). "
To respond to your question directly, we fit the 90 held-out subjects (1000 trials for each) with each of the top five expressions. The HSSM-LAN serves as a reference. All fits used NUTS with 4 chains of 500 warm-up and 500 sampling iterations. Bias and RMSE are those of the posterior mean derived from against the true value, while the 95% credible interval coverage rate is reported by normalized expressions with credible intervals calibrated by a bootstrap-chosen learning rate (Syring & Martin, 2019). Specifically, RMSE is given relative to the parameter range to unify the parameter unit.
| Likelihood | 95% CI coverage ((v / a/ z / t) | ||
|---|---|---|---|
| HSSM LAN | +0.018 / +0.051 / −0.008 / +0.002 | 0.038 / 0.118 / 0.063 / 0.015 | 76% / 76% / 70% / 53% |
| Top-five expressions, mean | 38% / 27% / 47% / 32% |
Admittedly, the table shows that interval coverage remains low even for the normalized and calibrated expressions: on average 27–47% across parameters, against the nominal 95%. The HSSM LAN also falls short of the nominal (53–76%). So, part of the gap is common to approximate likelihoods, but the expressions are clearly worse. It is worth noting, however, that the posterior means of the unnormalized expressions (highlighted in blue) show little bias on average. Whether the speed–accuracy trade-off is acceptable, therefore, depends on what the analysis needs: it is very attractive for point estimates, but may not be economical when calibrated credible intervals are.
Thank you for your thoughtful comments. We hope our response addresses your concerns and are happy to discuss any further questions.
Clarification to pipeline details
We sincerely appreciate your constructive comments, which have helped us substantially improve the manuscript and rethink directions for further improvement. We are grateful for your recognition of the novelty, rigor, and empirical contributions of our pipeline design. Each weakness and question is addressed point by point below, grouped by topic. Revised content from the manuscript is quoted in our responses.
Regarding the one-at-a-time recovery proxy.
Response: We agree. Because R(f) perturbs one parameter at a time, it misses trade-offs that arise in joint estimation, and at large λ the search can exploit this; the proxy is therefore not always aligned with joint recovery under NUTS. We now compute the recovery proxy R(f) for all 3,371 candidates (achieved before stage 3) across five different λ values and compare it with NUTS r for joint sampling on the 60 test subjects (columns 3-4).
We found that Correlation between R(f) and NUTS recovery performance doesn't always positive. We report spearman correlation rate within each λ, since λ shifts R(f) itself (see mean R(f) column).
We include R(f) in the recovery-directed objective because it is cheap enough to evaluate for every candidate during the search, whereas NUTS is not. However, since R(f) is only an imperfect proxy, we never select expressions on R(f). Instead, we include stage 3 to screen and rank expressions by NUTS recovery. Because λ balances likelihood approximation against parameter recovery, it should be kept moderate and swept over several values.
We have added this analysis to Appendix A.2 VALIDITY OF THE RECOVERY PROXY in the manuscript.
Regarding the selection criteria.
Response: We apologize for the confusion. Section 2.3 is more rigorous, as only the recovery metric is used when ranking expressions in F∗. However, expression selection runs through the whole pipeline, not only Stage 3. Therefore, the abstract compressed it into one sentence that reads as a single joint score. The three criteria enter in sequence:
Distributional distance and complexity act mainly as filters. They admit qualified expressions into F∗, which are then ranked and checked for calibration in Stages 3–4. We have updated the corresponding sentence in the abstract: