- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies mechanism recovery in symbolic regression for systems biology and argues that trajectory fidelity alone may not adequately reflect the ability to recover the underlying mechanisms. The authors introduce the SysBio-SR benchmark and further propose KineSelect, which jointly considers trajectory fitting and mechanism identification. Experimental results show that KineSelect generally outperforms the compared methods in both trajectory prediction and reaction recovery.
Critical strengths
-
The benchmark focuses on an important but often underappreciated issue: predictive accuracy is not equivalent to mechanistic correctness.
-
The experimental evaluation is broad and includes informative failure-case analyses.
Critical weaknesses
-
The novelty and methodological positioning of KineSelect are not sufficiently established with respect to recent structure-aware symbolic regression methods. Recent works such as RSSP and StruSR (see the next comment for specific references) have explicitly investigated how physics-informed and structural information can be incorporated into symbolic search. Although these methods target settings that are not identical to biological reaction-mechanism recovery, they are conceptually closely related to the central motivation of KineSelect: constraining symbolic search with scientifically meaningful structural information rather than relying solely on numerical fitting.
-
The related-work coverage and methodological positioning appear incomplete, particularly with respect to recent structure-aware and physics-informed symbolic regression methods. The manuscript discusses sparse regression, evolutionary symbolic regression, neural symbolic approaches, and reaction-structured methods, but largely positions KineSelect only against conventional equation-discovery or reaction-structured baselines. Recent works such as Symbolic Regression with Physics-Informed Residual and Structural Sensitivity Pruning and StruSR: Structure-Aware Symbolic Regression with Physics-Informed Taylor Guidance explicitly investigate how structural and physical information can be incorporated into symbolic search rather than relying solely on numerical fitting. RSSP evaluates individual symbolic substructures using residual and structural sensitivities and uses these signals to guide evolutionary operators and pruning, while StruSR extracts Taylor-based structural priors from a trained surrogate and uses structural attribution to guide symbolic evolution. Although these methods address somewhat different problem settings from biological ODE mechanism recovery, they are conceptually close to the central motivation of KineSelect: constraining symbolic search with scientifically meaningful structural information instead of treating trajectory fit as the sole criterion. The absence of any discussion makes it difficult to assess which aspects of KineSelect are genuinely specific to biological mechanism recovery and which belong to the broader emerging paradigm of structure-aware, physics-informed symbolic regression. I therefore recommend that the authors substantially expand the Related Work section to discuss these methods and provide a mechanism-level comparison, clarifying differences in task definition, prior information, search representation, structural guidance, and evaluation objectives. If a direct empirical comparison is technically infeasible because RSSP and StruSR target different symbolic-discovery settings, this should be stated explicitly and justified; otherwise, including them or suitably adapted variants as baselines would considerably strengthen the experimental positioning.
-
The reference reaction annotations appear to be reconstructed from ODE expressions rather than taken directly as independent biological ground truth. This raises an important question for me: does the “reaction mechanism ground truth” in SysBio-SR come directly from the original SBML reaction networks, or is it inferred from the algebraic form of the ODEs? This distinction is important because an algebraic decomposition of an ODE is not necessarily equivalent to an independently specified biological mechanism.
-
The manuscript would benefit from further improvement in its overall organization and argumentation. The paper simultaneously emphasizes benchmark construction, the gap between trajectory fidelity and mechanism recovery, and the design of KineSelect, but the logical relationship among these three parts is not sufficiently tight. The correspondence between the problems introduced in the Introduction and the three components of KineSelect is not fully clear. In addition, the Related Work section remains somewhat method-listing-oriented and lacks a clear problem-driven progression. The coverage of related work is also incomplete, and I recommend incorporating the recent studies mentioned above.
Questions for authors
1.How do the authors position KineSelect relative to recent structure-aware and physics-informed symbolic regression methods such as RSSP and StruSR? In particular, which components of KineSelect are specific to biological reaction-mechanism recovery, and which belong to the more general paradigm of structure-guided symbolic search?
- The search procedure is stochastic, but the main experiments appear to use only a single base seed, and the noise experiments appear to use only one noise realization per noise level. Could the authors provide results over multiple random seeds and multiple noise realizations? This would help assess whether the reported OOD and reaction-F1 improvements are stable.
2 comments
Response to Reviewer 5E79 — Experimental Robustness and Organization
4. Stability across search seeds and noise seeds (Q2)
We have aggregated four clean-data runs, including the original paper's results, and expanded the noise evaluation to three noise seeds.
Results across search seeds with clean training data. The new experiments use search seeds 53, 25, and 81, with the same designated clean training trajectory in each run and OOD evaluation. The table summarizes four runs. We have added these results in Appendix C.6 (Tables 13-15; lines 1836).
| Method and selector | OOD R²-score ↑ | Reaction F1 ↑ |
|---|---|---|
| SISR / Alpha | 0.440 ± 0.015 | 0.387 ± 0.008 |
| SISR / FPMS | 0.386 ± 0.037 | 0.430 ± 0.006 |
| KineSelect / FPMS | 0.531 ± 0.025 | 0.454 ± 0.007 |
Across all four runs, KineSelect/FPMS achieves higher OOD R²-score and reaction F1 than both SISR/Alpha and SISR/FPMS, consistent with our original findings.
Robustness across noise seeds. We evaluate KineSelect with its default FPMS selector, SISR with its original Alpha selector, and SISR with FPMS for a shared-selector comparison. For each noise level, we fit and evaluate models using different noise seeds. We have added the results in Appendix C.4.1 (Tables 10-11).
OOD R²-score ↑ (columns indicate noise levels)
| Method and selector | 0.001 | 0.01 | 0.05 | 0.1 |
|---|---|---|---|---|
| SISR / Alpha | 0.436 ± 0.016 | 0.426 ± 0.018 | 0.417 ± 0.006 | 0.376 ± 0.031 |
| SISR / FPMS | 0.369 ± 0.024 | 0.352 ± 0.042 | 0.340 ± 0.054 | 0.278 ± 0.061 |
| KineSelect / FPMS | 0.532 ± 0.001 | 0.491 ± 0.008 | 0.419 ± 0.004 | 0.363 ± 0.028 |
Reaction F1 ↑ (columns indicate noise levels)
| Method and selector | 0.001 | 0.01 | 0.05 | 0.1 |
|---|---|---|---|---|
| SISR / Alpha | 0.388 ± 0.015 | 0.386 ± 0.010 | 0.362 ± 0.012 | 0.318 ± 0.001 |
| SISR / FPMS | 0.420 ± 0.014 | 0.397 ± 0.012 | 0.350 ± 0.016 | 0.306 ± 0.028 |
| KineSelect / FPMS | 0.454 ± 0.001 | 0.423 ± 0.004 | 0.354 ± 0.013 | 0.309 ± 0.004 |
The results across noise seeds retain KineSelect/FPMS's advantage over SISR/Alpha on both metrics at 0.1% and 1% noise. Under shared FPMS, KineSelect has higher mean OOD R²-score than SISR at all four noise levels, while their reaction F1 means are close at 5% and 10% noise.
5. Connecting the benchmark, empirical findings, and method design (W4)
SysBio-SR combines trajectory accuracy and reaction-signature agreement in a common evaluation through standardized trajectories and reference reaction annotations. The empirical analysis shows the gap between these measures, motivating a methodological question: how can reaction priors and model selection improve recovery of reaction structure while maintaining trajectory prediction accuracy? KineSelect explores this question through RSS, KFI, and FPMS, which address candidate structure, rate forms, and final model selection. We evaluate the resulting models using both trajectory prediction accuracy and reaction-signature agreement.
We have clarified this connection in the Introduction and added a discussion of structural and physical priors in Section 2.3 (lines 150).
Response to Reviewer 5E79 — Methodology and Annotations
Thank you for recognizing our focus on the gap between trajectory prediction accuracy and mechanistic agreement, and the value of the benchmark experiments and failure-case analyses.
1. Methodological contributions of KineSelect (W1, Q1)
KineSelect connects reaction structure search with explicit regulatory roles, kinetic form identification conditional on reaction structure, and model selection that balances fit and parsimony. These designs give search candidates explicit reaction semantics and preserve each reaction's shared contribution to different state equations during fitting.
The three components address reaction connectivity, rate forms, and final model selection, respectively:
Evolutionary search, regression fitting, and balancing fit against complexity are general methodological ideas. Our specific designs for biological reaction dynamics concern species roles, coupling at the reaction level, kinetic-family identification conditional on structure, and a dedicated measure of kinetic complexity. Section 5 details these components.
2. Positioning relative to RSSP and StruSR (W1, W2, Q1)
In their primary partial differential equation (PDE) settings, RSSP and StruSR seek analytical solutions to known governing equations. Their physics-informed neural networks (PINNs) provide Taylor guidance, and subtree sensitivities guide search over expression trees. KineSelect identifies unknown ordinary differential equation (ODE) vector fields from trajectories, using reaction roles and MA, MM, and Hill families as priors. It searches reaction structures and kinetic forms while sharing each reaction's rate across the affected state equations.
Both works also report general SR experiments. RSSP allows data-fitting loss and surrogate-derived structural guidance without a known PDE. Such surrogate-derived priors describe local function behavior. KineSelect instead encodes reaction-level structure through species roles, stoichiometric coupling, and admissible MA, MM, and Hill kinetics.
We have not evaluated RSSP or StruSR on SysBio-SR. Their primary PDE formulations recover analytical solutions to known equations, whereas our task identifies unknown ODE vector fields from trajectories.
We have added both works and clarified their relationship to KineSelect in Section 2.3 (lines 150).
3. Origin and interpretation of the reference reaction annotations (W3)
We construct SysBio-SR's reference reaction annotations from the standardized ODEs of the BioModels source models. These ODEs define the dynamics that generate the benchmark trajectories, so we use their structure as the reference for equation discovery. We encode consumption/production directions, species dependencies, and kinetic forms in the same state representation used by the evaluated methods. This enables structural checks beyond trajectory accuracy.
The annotations provide ODE-derived structural references, not independent biological ground truth for the original SBML reaction network.
As detailed in §3.3 (lines 218) and Appendix A.5, we verify the annotations through symbolic and numerical reconstruction and manual consistency review.