- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
PhDN combines symbolic regression with a trainable network of explicit mathematical operations, allowing a discovered formula to be refined and expanded without restarting symbolic search
Critical strengths
It presents a unified, interpretable framework for reusing symbolic intermediate expressions, refining coefficients, and adding correction terms without restarting symbolic search a problem with competing methods.
Critical weaknesses
it is fundamentally not clear to me that the PhDN framework cannot be ablated away. One could simply refine an ordinary symbolic computation graph using the same parameterization, analytic derivatives, optimization budget, and validation rules, without introducing the PhDN framework. I would like to see that comparison.
It is not clear to me how much of the heavy lifting is being done by the priors.
Please add results on noise on the state.
Questions for authors
None
7 comments
Response to Weakness 2 — Contribution of Physical Priors
We agree that informative structural knowledge can substantially improve symbolic recovery, particularly in the dynamical-system experiments. Our staged and prior-level ablations are designed to distinguish this effect from subsequent coefficient refinement and dictionary augmentation.
Performance without case-specific structural priors. The 27 Feynman benchmarks do not supply target-specific physical expressions to PhDN. Stage-I symbolic regression discovers candidate skeletons from a shared operator grammar, followed by PhDN compilation and refinement. These experiments demonstrate that the framework does not require a prescribed target equation, although the operator grammar itself remains an inductive bias.
Contribution of prior quality. In the generator experiments, G1–G3 provide progressively more informative initial-expression dictionaries, but these enter Stage I only as soft, unprotected SR initializations. They may be modified or discarded during symbolic evolution. The supplied expressions are not directly enforced as governing equations. At N_tr = 1000, the following ID vector-field RMSEs illustrate the distinct effects of symbolic initialization, fixed-support refinement, and augmentation:
| Prior | Stage-I SR | PhDN-Fixed | PhDN-Full |
|---|---|---|---|
| G1 | 2.56e-2 | 2.56e-2 | 2.55e-3 |
| G2 | 1.37e-1 | 1.37e-1 | 2.96e-4 |
| G3 | 4.72e-5 | 8.33e-10 | 7.45e-15 |
The strongest G3 prior supplies a substantially more adequate structural skeleton, allowing fixed-support refinement to achieve very high accuracy. Indeed, extended fixed-support optimization can reach accuracy comparable to PhDN-Full under G3, so we do not attribute this near-machine-precision result solely to augmentation.
In contrast, under G2, fixed-support refinement leaves the ID/OOD vector-field RMSEs essentially unchanged at 1.37e-1 / 6.50e-1, whereas PhDN-Full reduces them to 2.96e-4 / 1.15e-2. This demonstrates that branch-local dictionary augmentation can substantially reduce residual errors even when the selected symbolic skeleton remains imperfect, although the resulting correction is more complex and does not imply exact mechanistic recovery.
Interpretation. Physical priors perform substantial work when they supply an accurate symbolic structure, and we explicitly acknowledge this dependence. PhDN's complementary contribution is to compile, continuously refine, and, when necessary, augment the selected symbolic skeleton. The prior-level ablations therefore separate the benefits of informative initialization from the additional modeling flexibility provided by the explicit-dictionary architecture.
Response to Weakness 3-Robustness to State-Observation Noise
Thank you for this suggestion. We have added a state-observation-noise robustness experiment on both the generator and soft-saturated Lorenz–96 systems, complementing our existing derivative-label-noise evaluation.
We consider relative Gaussian state noise of 0%, 0.1%, 0.5%, and 1%, added to training and validation states while keeping derivative targets and held-out test data clean. We use and report the mean clean-test vector-field RMSE over three independent rounds. To examine robustness without introducing stronger prescribed physical priors, we focus on the weakest setting.
State-observation-noise results (lower is better):
| System | Noise | SR-+NLS | MLP | PhDN- |
|---|---|---|---|---|
| Generator | 0% | 0.05267 | 0.001658 | 0.01798 |
| 0.1% | 0.05252 | 0.009232 | 0.006327 | |
| 0.5% | 0.05338 | 0.05644 | 0.02886 | |
| 1% | 0.05648 | 0.12295 | 0.04418 | |
| Lorenz–96 | 0% | 3.26e-10 | 0.00674 | 2.15e-12 |
| 0.1% | 4.25e-8 | 0.01350 | 9.08e-5 | |
| 0.5% | 4.25e-8 | 0.03429 | 0.00434 | |
| 1% | 0.00904 | 0.05680 | 0.01272 |
Discussion. PhDN- achieves lower mean RMSE than MLP at all nonzero noise levels in both systems. It also improves upon SR-+NLS in the generator case, whereas the nearly exact SR skeleton remains more accurate under nonzero noise in Lorenz–96. This contrast suggests that PhDN's flexible augmentation can enhance robustness to noisy state observations when the initial symbolic structure is imperfect, while offering limited additional benefit when that structure is already highly accurate.
Manuscript revision: We have incorporated these findings into the main-text subsection Robustness to Noise and added the experimental results and discussion in Appendix Robustness to State-Observation Noise.
Response to Weakness 1 — PhDN versus Ordinary Symbolic Computation Graphs
We thank the reviewer for raising this fundamental question about whether PhDN provides a meaningful contribution beyond refining an ordinary symbolic computation graph. We agree that an ordinary DAG equipped with exactly the same operators, coefficient parameterization, augmented support, analytic Jacobians, optimization budget, and validation rules would define an equivalent learning problem. We therefore do not claim an intrinsic numerical optimization advantage over such an exactly matched implementation.
The architectural contribution lies in how an existing symbolic expression is parameterized and systematically extended. PhDN organizes symbolic computations into reusable intermediate states connected by SINDy-style multivariate explicit-dictionary branches. Each branch takes the form
b = W * Phi(h), wherePhi(h)may contain inherited symbolic operators, physical relationships, and additional approximation functions. Crucially, PhDN turns an accepted symbolic skeleton into a nested, explicitly parameterized family of symbolic corrections by attaching zero-initialized dictionary support to its existing computational branches. This preserves the initial symbolic mapping while introducing additional continuous optimization directions at intermediate stages, without changing the compiled state topology or restarting discrete symbolic search. The resulting corrections remain explicit and symbolically inspectable.Our additional ablations distinguish this structural mechanism from ordinary coefficient refinement and additional optimization. Independent nonlinear least square (NLS) refitting of the original SR expressions achieves accuracy comparable to PhDN-Fixed, confirming that fixed-support improvements primarily arise from continuous parameter refinement rather than an intrinsic advantage of DAG compilation. In the controlled dual-input CSTR experiment (Sec. 4.4), SR-Refined, PhDN-Fixed, and PhDN-Fixed-Extended achieve nearly identical accuracy, indicating that additional optimization on the original symbolic support cannot adequately resolve the imposed structural mismatch. PhDN-OutputOnly improves the predictions through output-level correction, whereas PhDN-Full achieves 4.32-fold lower OOD vector-field RMSE and 14.5-fold lower mean trajectory RMSE by releasing internal dictionary support. The final model retains 22 active coefficient positions, including seven augmentation-only positions, with its explicit symbolic expression reported in the Appendix.
We also compare against renewed symbolic evolution. Starting from the same imperfect symbolic skeleton, SR-Extended performs further discrete structural search followed by nonlinear coefficient refinement. Across three seeds, it achieves a mean trajectory RMSE of 1.55e-3, compared with 9.04e-4 for PhDN-Full, while requiring approximately fourfold training time (886 vs. 220 s). It demonstrates that continuous exploration of PhDN's internal correction space can offer a more favorable accuracy–runtime trade-off than renewed symbolic search under the evaluated settings.
Thus, PhDN's contribution is not that its DAG intrinsically outperforms an identical computation graph. Rather, it provides a systematic explicit-dictionary parameterization that transforms an ordinary symbolic expression into a continuously refinable family of internal symbolic corrections. The controlled experiment illustrates the practical benefit of this representation: structural mismatch can be addressed through continuous dictionary-support expansion, retaining explicit symbolic readout and potentially avoiding the computational cost of renewed discrete structural search.