acceptodds
Under review as a conference paper at ICLR 2027

PhDN: A Physics-embedded Dictionary Network for Data-Driven Discovery of Physical Relationships

Abstract

Discovering physical relationships from data requires flexible approximation, symbolic interpretability, and incorporation of physical priors. Black-box neural networks offer strong approximation but limited access to physical structure, while interpretable dictionary and symbolic methods remain sensitive to predefined features, operator grammars, or discrete structural choices. We propose the Physics-embedded Dictionary Network (PhDN), a compositional explicit-dictionary architecture that organizes SINDy-style local dictionaries as a multilayer differentiable DAG with reusable intermediate states. Local dictionaries combine symbolic expressions, problem-specific operators, and approximation bases, so symbolic structure can be compiled, refined, and augmented within PhDN. PhDN can exactly represent finite symbolic expressions built from its admissible operators, while increasing polynomial or neural dictionaries provide universal approximation on compact domains. When informative structure is unavailable, SR supplies a skeleton for PhDN compilation; partial physical knowledge can enter as soft structural priors. Additional dictionary bases enable local correction without renewed symbolic search. Experiments on Feynman relationships and two dynamical systems show favorable ID/OOD accuracy. Staged ablations show fixed-support gains mainly reflect parameter refinement, while added bases address structural mismatch. Informative priors improve recovery and rollout robustness under derivative-label noise.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Physics-embedded Dictionary Network, scientific machine learning, interpretable machine learning, SINDy, symbolic regression, system identification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jZ7kOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

PhDN combines symbolic regression with a trainable network of explicit mathematical operations, allowing a discovered formula to be refined and expanded without restarting symbolic search

Critical strengths

It presents a unified, interpretable framework for reusing symbolic intermediate expressions, refining coefficients, and adding correction terms without restarting symbolic search a problem with competing methods.

Critical weaknesses

it is fundamentally not clear to me that the PhDN framework cannot be ablated away. One could simply refine an ordinary symbolic computation graph using the same parameterization, analytic derivatives, optimization budget, and validation rules, without introducing the PhDN framework. I would like to see that comparison.

It is not clear to me how much of the heavy lifting is being done by the priors.

Please add results on noise on the state.

Questions for authors

None

7 comments
AuthorsOpenReview ↗

Response to Weakness 1 — PhDN versus Ordinary Symbolic Computation Graphs

We thank the reviewer for raising this fundamental question about whether PhDN provides a meaningful contribution beyond refining an ordinary symbolic computation graph. We agree that an ordinary DAG equipped with exactly the same operators, coefficient parameterization, augmented support, analytic Jacobians, optimization budget, and validation rules would define an equivalent learning problem. We therefore do not claim an intrinsic numerical optimization advantage over such an exactly matched implementation.

The architectural contribution lies in how an existing symbolic expression is parameterized and systematically extended. PhDN organizes symbolic computations into reusable intermediate states connected by SINDy-style multivariate explicit-dictionary branches. Each branch takes the form b = W * Phi(h), where Phi(h) may contain inherited symbolic operators, physical relationships, and additional approximation functions. Crucially, PhDN turns an accepted symbolic skeleton into a nested, explicitly parameterized family of symbolic corrections by attaching zero-initialized dictionary support to its existing computational branches. This preserves the initial symbolic mapping while introducing additional continuous optimization directions at intermediate stages, without changing the compiled state topology or restarting discrete symbolic search. The resulting corrections remain explicit and symbolically inspectable.

Our additional ablations distinguish this structural mechanism from ordinary coefficient refinement and additional optimization. Independent nonlinear least square (NLS) refitting of the original SR expressions achieves accuracy comparable to PhDN-Fixed, confirming that fixed-support improvements primarily arise from continuous parameter refinement rather than an intrinsic advantage of DAG compilation. In the controlled dual-input CSTR experiment (Sec. 4.4), SR-Refined, PhDN-Fixed, and PhDN-Fixed-Extended achieve nearly identical accuracy, indicating that additional optimization on the original symbolic support cannot adequately resolve the imposed structural mismatch. PhDN-OutputOnly improves the predictions through output-level correction, whereas PhDN-Full achieves 4.32-fold lower OOD vector-field RMSE and 14.5-fold lower mean trajectory RMSE by releasing internal dictionary support. The final model retains 22 active coefficient positions, including seven augmentation-only positions, with its explicit symbolic expression reported in the Appendix.

We also compare against renewed symbolic evolution. Starting from the same imperfect symbolic skeleton, SR-Extended performs further discrete structural search followed by nonlinear coefficient refinement. Across three seeds, it achieves a mean trajectory RMSE of 1.55e-3, compared with 9.04e-4 for PhDN-Full, while requiring approximately fourfold training time (886 vs. 220 s). It demonstrates that continuous exploration of PhDN's internal correction space can offer a more favorable accuracy–runtime trade-off than renewed symbolic search under the evaluated settings.

Thus, PhDN's contribution is not that its DAG intrinsically outperforms an identical computation graph. Rather, it provides a systematic explicit-dictionary parameterization that transforms an ordinary symbolic expression into a continuously refinable family of internal symbolic corrections. The controlled experiment illustrates the practical benefit of this representation: structural mismatch can be addressed through continuous dictionary-support expansion, retaining explicit symbolic readout and potentially avoiding the computational cost of renewed discrete structural search.

AuthorsOpenReview ↗

Response to Weakness 2 — Contribution of Physical Priors

We agree that informative structural knowledge can substantially improve symbolic recovery, particularly in the dynamical-system experiments. Our staged and prior-level ablations are designed to distinguish this effect from subsequent coefficient refinement and dictionary augmentation.

Performance without case-specific structural priors. The 27 Feynman benchmarks do not supply target-specific physical expressions to PhDN. Stage-I symbolic regression discovers candidate skeletons from a shared operator grammar, followed by PhDN compilation and refinement. These experiments demonstrate that the framework does not require a prescribed target equation, although the operator grammar itself remains an inductive bias.

Contribution of prior quality. In the generator experiments, G1–G3 provide progressively more informative initial-expression dictionaries, but these enter Stage I only as soft, unprotected SR initializations. They may be modified or discarded during symbolic evolution. The supplied expressions are not directly enforced as governing equations. At N_tr = 1000, the following ID vector-field RMSEs illustrate the distinct effects of symbolic initialization, fixed-support refinement, and augmentation:

PriorStage-I SRPhDN-FixedPhDN-Full
G12.56e-22.56e-22.55e-3
G21.37e-11.37e-12.96e-4
G34.72e-58.33e-107.45e-15

The strongest G3 prior supplies a substantially more adequate structural skeleton, allowing fixed-support refinement to achieve very high accuracy. Indeed, extended fixed-support optimization can reach accuracy comparable to PhDN-Full under G3, so we do not attribute this near-machine-precision result solely to augmentation.

In contrast, under G2, fixed-support refinement leaves the ID/OOD vector-field RMSEs essentially unchanged at 1.37e-1 / 6.50e-1, whereas PhDN-Full reduces them to 2.96e-4 / 1.15e-2. This demonstrates that branch-local dictionary augmentation can substantially reduce residual errors even when the selected symbolic skeleton remains imperfect, although the resulting correction is more complex and does not imply exact mechanistic recovery.

Interpretation. Physical priors perform substantial work when they supply an accurate symbolic structure, and we explicitly acknowledge this dependence. PhDN's complementary contribution is to compile, continuously refine, and, when necessary, augment the selected symbolic skeleton. The prior-level ablations therefore separate the benefits of informative initialization from the additional modeling flexibility provided by the explicit-dictionary architecture.

AuthorsOpenReview ↗

Response to Weakness 3-Robustness to State-Observation Noise

Thank you for this suggestion. We have added a state-observation-noise robustness experiment on both the generator and soft-saturated Lorenz–96 systems, complementing our existing derivative-label-noise evaluation.

We consider relative Gaussian state noise of 0%, 0.1%, 0.5%, and 1%, added to training and validation states while keeping derivative targets and held-out test data clean. We use Ntrain=1000N_{\mathrm{train}}=1000 and report the mean clean-test vector-field RMSE over three independent rounds. To examine robustness without introducing stronger prescribed physical priors, we focus on the weakest G1G_1 setting.

State-observation-noise results (lower is better):

SystemNoiseSR-G1G_1+NLSMLPPhDN-G1G_1
Generator0%0.052670.0016580.01798
0.1%0.052520.0092320.006327
0.5%0.053380.056440.02886
1%0.056480.122950.04418
Lorenz–960%3.26e-100.006742.15e-12
0.1%4.25e-80.013509.08e-5
0.5%4.25e-80.034290.00434
1%0.009040.056800.01272

Discussion. PhDN-G1G_1 achieves lower mean RMSE than MLP at all nonzero noise levels in both systems. It also improves upon SR-G1G_1+NLS in the generator case, whereas the nearly exact SR skeleton remains more accurate under nonzero noise in Lorenz–96. This contrast suggests that PhDN's flexible augmentation can enhance robustness to noisy state observations when the initial symbolic structure is imperfect, while offering limited additional benefit when that structure is already highly accurate.

Manuscript revision: We have incorporated these findings into the main-text subsection Robustness to Noise and added the experimental results and discussion in Appendix Robustness to State-Observation Noise.

Reviewer_1eegOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PhDN, which stacks SINDy-style dictionaries into a multilayer DAG so that an expression found by PySR can be compiled into the network and refined with nonlinear least squares. Extra polynomial or tanh basis terms are added at zero and released later to correct structural errors, and the method is tested on 27 Feynman equations, a generator model and a soft-saturated Lorenz-96 system.

Critical strengths
  1. Adding the extra basis terms inside the branches with zero initial weight is a simple and natural idea. It lets the authors fix an imperfect SR expression without running the symbolic search again.

  2. The design of staged ablation from SR to PhDN-Fixed to full PhDN is sound. The authors also report failure modes where augmentation makes OOD error worse or rollouts fail.

Critical weaknesses
  1. The paper also never shows a final pruned model, so I can’t tell if the results are still interpretable. There is no ablation comparing basis terms added only at the output (basically SR plus a residual model) with basis terms added at all branches, so the value of the multilayer DAG is not shown.

  2. The best dynamical results depend on priors that almost give the answer (G3 already contains the true P_e and V_t forms), while MLP, EQL and KAN get no priors. With the weaker G1 prior PhDN is roughly on par with MLP, and on Lorenz-96 the rollout error stays around 1e-2 even when the vector-field error is 1e-12, which suggests the saturation law was not really recovered.

  3. Stage III gets another round of NLS iterations on top of Stage II, so it is not clear whether the gain comes from the added basis terms or just from more optimization. For example, under G3 the generator error drops from 8.33e-10 to 7.45e-15 even though the skeleton already seems structurally correct.

Questions for authors
  1. Does the big gain from fixed-support refinement on Feynman come from numerical precision? The cases that jump from about 1e-8 to 1e-15 (I.6.2, II.6.15a, II.11.27) are exactly the ones with non-integer constants, which looks like float32 constants from PySR’s default setting, so a simple float64 refit of the SR constants might give the same result without PhDN.

  2. If you run Stage II with the same iteration budget as Stage III, does PhDN-Fixed reach the same error as full PhDN?

  3. Can you show the final pruned expressions for a few cases (e.g., I.6.2b and Lorenz-96 G2), including how many augmentation terms survive? Can you also report the OOD vector-field RMSE, which you generated but did not include in Tables 15 and 16?

10 comments
AuthorsOpenReview ↗

Weakness 1 — Interpretability of the Final Pruned Model and the Value of Branch-Level Augmentation

We thank the reviewer for highlighting the need to demonstrate the interpretability of the final pruned model and to isolate the contribution of branch-level augmentation. To address both concerns, we added a controlled dual-input CSTR experiment in Sec. 4.4, starting from a prescribed imperfect symbolic skeleton. All methods begin with the same initial expression:

MethodWhat it isolates
SR-RefinedDirect NLS refinement of the original symbolic expression
PhDN-FixedCoefficient refinement on the compiled, fixed symbolic support
PhDN-Fixed-ExtendedWhether additional optimization budget suffices without new support
PhDN-OutputOnlyCorrection using output-level augmentation alone
PhDN-FullInternal branch-level dictionary augmentation and joint refinement
SR-ExtendedRenewed symbolic search and coefficient refinement from the same initial skeleton

SR-Refined and PhDN-Fixed achieve nearly identical accuracy, confirming that the symbolic structure can be faithfully compiled and refined within the PhDN DAG. Extending the fixed-support optimization budget gives no meaningful further improvement. Output-level augmentation helps, but the results favor internal augmentation:

CSTR metricPhDN-OutputOnlyPhDN-FullImprovement
OOD vector-field RMSE7.86e-31.82e-34.32× lower
Mean free-rollout RMSE1.31e-29.04e-414.5× lower

Further SR search from the same skeleton achieves a mean trajectory RMSE of 1.55e-3 over three seeds, versus 9.04e-4 for PhDN-Full, despite approximately fourfold training time.

The final PhDN-Full retains 22 active coefficient positions, including seven augmentation-only positions. Its explicit symbolic expression and fitted coefficients appear in Appendix “CSTR Case: Symbolic-Structure Correction.” This demonstrates interpretable correction of an imperfect symbolic prior without renewed SR search, not exact recovery of the underlying physical law.

AuthorsOpenReview ↗

Weakness 2 — Physical Priors and Lorenz–96 Rollout Accuracy

We agree that informative physical priors can substantially improve prediction and generalization. Structured architectures such as EQL and KAN can facilitate the incorporation or interpretation of mathematical relationships. PhDN provides an explicit-dictionary DAG into which selected symbolic expressions can be compiled and refined. Our comparisons respect the methods' native architectures, including the use of relevant mathematical operators in EQL-Div.

Importantly, the generator and Lorenz–96 G1–G3 settings do not impose completed governing expressions on PhDN. Incomplete expressions instead enter as soft, unprotected Stage-I SR initializations, which may be modified or discarded during symbolic search. Only the selected SR structures are subsequently compiled into PhDN. These experiments therefore examine the use of physical knowledge without enforcing prescribed equations.

Regarding Lorenz–96, near-machine-precision vector-field accuracy does not guarantee equally accurate OOD rollouts or globally exact mechanistic recovery. Conversely, non-negligible rollout errors do not necessarily imply substantial structural misidentification. Our additional analysis reveals that the learned symbolic structure can be locally equivalent to the true saturated dynamics, while certain OOD initial conditions exhibit strong finite-time amplification of small perturbations. Specifically, the elevated rollout error is concentrated at an alternating-symmetry initial condition, where the true dynamics exhibit strong finite-time amplification of transverse perturbations. This sensitivity can magnify the effects of residual structural and numerical discrepancies in the learned model, helping explain the substantial rollout error despite near-machine-precision vector-field accuracy. In contrast, simulation trajectories from all other initial points maintain a high level of performance. See Appendix “Lorenz–96 Prior and Method Settings” → “Local symbolic equivalence and rollout sensitivity”, and Question 3 below for the quantitative example.

AuthorsOpenReview ↗

Weakness 3 and Question 2 — Matched Optimization Budget for PhDN-Fixed

The outcome depends on the adequacy of the initial symbolic skeleton. With a sufficiently accurate Stage-I structure, extended fixed-support refinement can reach performance comparable to full PhDN. For generator G3 at N_tr = 1000:

MetricExtended PhDN-Fixed-G3PhDN-Full-G3
ID vector-field RMSE5.44e-157.45e-15
Mean trajectory NRMSE1.78e-162.07e-16

Thus, the apparent Stage-III gain for this strong-prior case should not be attributed solely to augmentation. Under weaker G1/G2 generator priors, fixed-support refinement produces more limited improvements, whereas releasing additional dictionary support can substantially reduce residual error. The controlled CSTR experiment in Sec. 4.4 further shows that matching the fixed-support optimization budget does not reproduce the gains from internal augmentation. We clarified this distinction in Sec. 4.3.1 and the corresponding Appendix analysis: parameter refinement and support expansion provide complementary mechanisms.

Reviewer_ok9jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents PhDN, a network of explicit dictionaries that reuses intermediate expressions. It compiles a symbolic model into a differentiable network, refines its coefficients, and releases additional dictionary terms for correction. Representation results and experiments on Feynman functions and two dynamical systems support the approach.

Critical strengths
  1. The proposed method provides a way to preserve an initial symbolic model while adding trainable local corrections. Its shared intermediate states support reuse of explicit expressions.
  2. The staged ablations are informative. Dictionary expansion improves both vector-field and rollout errors in all generator settings, providing clear evidence that the method offers more than coefficient refinement alone.
Critical weaknesses
  1. The comparisons do not fully establish the benefit of the proposed method representation over direct expression refitting or compiled KAN refinement and expansion. Starting these methods from the same symbolic models would better isolate this contribution.
  2. Interpretability after learning is not sufficiently demonstrated. Final expressions and complexity measures are especially important in cases where expansion produces large gains, to show whether the corrections remain compact and physically meaningful.
Questions for authors

What final expressions and complexities result from the weak-prior generator experiments? How do they compare with matched symbolic refinement?

3 comments
AuthorsOpenReview ↗

Weakness 1 — Benefits Beyond Direct Symbolic Refitting

We thank the reviewer for the suggestion. To better isolate the benefit of PhDN's representation beyond direct symbolic refitting, we added a controlled dual-input CSTR symbolic-structure correction experiment in Sec. 4.4, with all methods starting from the same prescribed imperfect symbolic skeleton.

MethodComparison role
SR-RefinedNLS coefficient refinement of the initial expression
PhDN-FixedSame inherited support, refined in the PhDN DAG
PhDN-Fixed-ExtendedSame support with a larger optimization budget
PhDN-OutputOnlyAugmentation restricted to the output layer
PhDN-FullInternal dictionary augmentation
SR-ExtendedRenewed SR search from the same skeleton, followed by coefficient refinement

SR-Refined, PhDN-Fixed, and PhDN-Fixed-Extended achieve nearly identical accuracy, showing that fixed-expression coefficient refinement cannot adequately resolve this structural mismatch under the tested budgets. Output-only augmentation improves accuracy, but PhDN-Full reduces OOD vector-field RMSE from 7.86e-3 to 1.82e-3 (4.32×) and mean trajectory RMSE from 1.31e-2 to 9.04e-4 (14.5×) relative to PhDN-OutputOnly. Further SR search yields mean trajectory RMSE 1.55e-3 over three seeds, versus 9.04e-4 for PhDN-Full, despite approximately fourfold training time; individual SR runs can still perform better.

The final PhDN-Full retains 22 active coefficient positions, including seven augmentation-only positions. Its explicit expression and fitted coefficients are in Appendix “CSTR Case: Symbolic-Structure Correction.” These comparisons demonstrate the practical advantage of joint internal refinement and branch-level augmentation over the tested fixed-support and frozen-backbone output-only correction strategies.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 1 — Final Symbolic Structures and Complexity under Weak Priors

Thank you for highlighting the importance of final symbolic interpretability and complexity, particularly when dictionary augmentation yields substantial improvements. We have added a detailed analysis in the Appendix, under "Physical-Prior and Method Settings" → "Final symbolic structures and sparsification trade-off," immediately following "Fixed-support and augmentation ablation."

Original weak-prior generator results. At N_tr = 1000, the G2 Stage-I SR model identifies the characteristic x3*sin(x1) and sin(2*x1) interactions, but its AVR expression lacks the nonlinear terminal-voltage magnitude. Matched PhDN-Fixed refinement leaves the ID/OOD vector-field RMSEs essentially unchanged, whereas PhDN-Full substantially improves both metrics through additional dictionary support.

Supplementary sparsification. To examine the accuracy–complexity trade-off, we further performed progressive post-training sparsification with support-restricted NLS refitting. The coefficient-pruning threshold was increased from 1e-6 to 1e-1, and a candidate was accepted only if its validation RMSE did not exceed 1.01 times that of the original PhDN checkpoint. OOD test errors were not used for model selection.

The resulting complexities and errors at N_tr = 1000 (one experimental run) are:

ModelActive positionsAugmentation-only positionsID VF RMSEOOD VF RMSE
SR-G12302.56e-21.35e-1
PhDN-Full-G1 (original)1361132.55e-33.41e-1
PhDN-G1 (additional sparsification)87648.00e-49.84e-2
SR-G21901.37e-16.50e-1
PhDN-Fixed-G21901.37e-16.50e-1
PhDN-Full-G2 (original)112932.96e-41.15e-2
PhDN-G2 (additional sparsification)79601.82e-42.36e-2

Final symbolic expressions. The revised Appendix provides the factored symbolic readout of the additionally sparsified G2 model, including all four state equations and their intermediate expressions. For example, its first three equations are:


dx1/dt = 376.99112*x2

dx2/dt = -0.71429*x2 - 0.20408*x3*sin(x1)

+ 0.06673*sin(p1) + 0.01236*cos(x1) + 0.04308

dx3/dt = -0.39286*x3 + 0.12500*x4 + 0.26786*cos(x1)

p1 = 2.03563*x1 - 0.07499 - 0.03079*x1^2

These expressions retain recognizable electromechanical and flux-decay relationships. The fourth (AVR) equation remains a more elaborate but explicit composition of polynomial, multiplicative, and trigonometric operations, rather than an exact recovery of the terminal-voltage magnitude.

Interpretability and limitations. The original G2 PhDN reduces OOD vector-field RMSE from 6.50e-1 to 1.15e-2, at the cost of expanding the active support from 19 to 112 positions. Supplementary sparsification reduces this support to 79 positions while maintaining substantial improvement over Stage-I SR, although its OOD RMSE increases to 2.36e-2. The G1 results illustrate a different outcome, with both model complexity and OOD error decreasing after sparsification.

These findings demonstrate that PhDN's explicit dictionary representation allows the learned intermediate relationships and augmentation corrections to be inspected symbolically and further sparsified under a prescribed validation-accuracy constraint. Symbolic inspectability is retained, but compactness and exact mechanistic recovery are not guaranteed.

Dear reviewer ok9j,

Thanks very much for your additional feedback! Have we addressed all your questions and concerns? If not, can you please let us know, and we will work on additional studies. And if yes, we'd be very grateful if you could consider updating the score.

Comments on the paper

1 comment

Dear authors and reviewers,

The author-reviewer discussion period closes on November 18 (AoE). Please engage/continue to engage during this period so that we have our issues discussed thoroughly.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.