acceptodds
Under review as a conference paper at ICLR 2027

From Noisy Neural Black Boxes to Concise Physical Formulas: Structure-Constrained Exact Formula Recovery

Abstract

Neural networks can fit noisy physical observations accurately but rarely expose the compact laws governing the underlying system. We present a structure-constrained neural-to-symbolic framework for recovering verifiable formulas from neural black-box surrogates. The method combines bounded search over physically motivated expression families, exactification of continuous coefficients into sparse symbolic constants, and noisy-data cross-fitting with structural and stability checks. This separates denoised structure proposal from discrete identification and validation on the original observations. We evaluate 13 development formulas and 20 held-out formulas over five seeds and Gaussian noise levels from 0% to 50%. On held-out formulas, exact-equivalence recovery decreases from 57.9% without noise to 19.2% at 50% noise; the development rate decreases from 69.2% to 61.5%. Removing exactification reduces recovery to 0% at both 20% and 50% noise, while removing raw-data refit cross-fitting reduces the 50%-noise development rate from 66.7% to 28.3%. Compared with PySR, TPSR, uDSR, Operon, and DSR, our method achieves the highest recovery rate at every nonzero noise level in the evaluated comparison. These results show that numerical agreement alone is insufficient for scientific formula recovery: exact constants, structural validity, and noise-aware validation are essential.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
symbolic regression, scientific machine learning, physical law discovery, neural surrogates, symbolic distillation, equation discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HExNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an hybrid framework that recovers concise physical formulas from noisy observations via a structure‑constrained search pipeline and a designed exactification module. Ablation studies demonstrate the effectiveness and advantages over baselines.

Critical strengths
  1. The work develops from clear research motivation. The exactification module well addresses the challenge of achieving exact symbolic equivalence, which is somtimes ignored in conventional symbolic‑regression works.
  2. The authors honestly discuss failure modes and the boundaries of their approach, which improves the overall credibility of the presented results.
Critical weaknesses
  1. Although the paper emphasizes the importance of exact symbolic recovery, there is no aggregate statistical analysis over failure cases of both the proposed method and competing baselines. It remains unclear whether the failure cases mainly stem from fundamentally incorrect candidate structures, or just from the mistaken projection or slight error from the exactification step. As acknowledged in Appendix J.3, exactification can increase numerical error, which seems to be a negative effect that remains unquantified.
  2. It seems that the main experiments utilize the realized injected noise mean-square σreal2\sigma^2_{\text{real}} from simulation metadata for computing the excess-risk score. I would like to confirm whether this oracle quantity is available only to the proposed method and is never provided to competing baselines. If so, the positive gain contributed by this setting needs to be quantified. The authors are recommended to clarify by testing the full pipeline without access to σreal2\sigma^2_{\text{real}} (e.g., estimating the noise purely from data) and repeat the comparison against baselines.
Questions for authors
  1. Are exactly identical judging criteria (exact recovery=True) applied to both the proposed method and baselines? Appendix I.1 illustrates cases where numerically close constants are marked as failures under the exact‑recovery criteria. While the competing baselines may be designed to minimize the predictive loss rather than perform exact symbolic recovery, evaluating baselines using strict symbolic equivalence without applying the same exactification module creates an asymmetric comparison.
  2. Are all ablation experiments in this paper exclusively conducted on the development set, without evaluations on the held‑out unseen formula partition? It would be valuable to understand how individual components of this pipeline contribute to performance on previously‑unseen formulas.
Reviewer_KGizOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Neural-to-Symbolic framework for recovering concise physical formulas from a neural network "black box" trained on noisy data. The core idea is as follows: first, a fully connected neural network is used to fit noisy physical observation data. Then, a structure-constrained search is conducted within a family of bounded expressions driven by physical motivations. Subsequently, the continuous regression coefficients are converted into sparse symbolic constants through a refinement step. Finally, cross-fitting verification is performed on the original noisy data to screen candidate formulas and prevent overfitting. Experiments on 20 retained formulas demonstrate that this method outperforms various symbolic regression baselines such as PySR and TPSR under different noise levels. Ablation studies further validate the crucial roles of the refinement and cross-fitting components.

Critical strengths
  1. The insight presented in this article, "Low NMSE ≠ Correct Formula," is profound and strikes at some of the shortcomings within the symbolic regression community.
  2. The paper adheres to writing standards and is sincere. Appendix J specifically lists and categorizes failure cases, and such practices should be encouraged.
  3. The paper explicitly states that "the method should not be interpreted as a universal symbolic solver", which is uncommon in writing.
Critical weaknesses
  1. Appendix H "Additional implementation details" only has a title and no content
  2. The recovery rate at 0% noise shows two different values. The abstract, introduction, section 4.2, and conclusion all state 57.9%, but section 4.4 and Figure 4 state 56.3% ("61.5% versus 56.3% for ours") 3.The number of test data in Section 4.2 and Section 4.3 seems to be different. It is inferred that Section 4.2 uses 65 test cases while Section 4.3 uses 60. Is this correct?
Questions for authors

What are the training details of surrogate neural networks? How much training data is used? Surrogate neural networks are sometimes difficult to train, which can affect the accuracy of subsequent processing. Moreover, the training of surrogate neural networks often requires a large amount of training data, especially for high-dimensional mathematical expressions.

Reviewer_oRSVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors train neural networks on noisy data generated from 33 (13 dev / 20 test) selected equations. They propose a pipeline to go from these observations to the exact underlying equations (e.g., 0.4999 sin(x) + 0.0003x would not count if the answer was 0.5 sin(x)). Their pipeline works, roughly, by training a NN on the data, performing a very restricted SR on the NN outputs, “exactifying” the terms in the resulting equations, returning to the original data to refit candidates, and finally selecting among them based on simplicity and structure. The experiments use 5 seeds and seven noise levels. The results find that for exact recovery on the test set PySR is similar at 0% noise but that their method outperforms all others with noise present. Ablations examine exactification, the refitting methods, and smoothing, while comparisons include five existing symbolic-regression systems.

Critical strengths
  1. Recovering exact equations is a well-motivated goal. Anyone who has used SR is familiar with having to pick out the form they expected from wacky similar ones.
  2. The ablation with and without NN surrogates on noisy data is interesting though I have some reservations. If the result (that this NN step helps with noise present) holds up, that would be a cool finding.
Critical weaknesses
  1. I cannot tell what is responsible for their method’s observed advantage and suspect that it is due to experimental design issues.
    1. Their SR step searches a restricted set of equation structures but they don’t determine if this restriction or something else is where gains are coming from.
    2. Furthermore, why have this weird restricted SR at all? Why not plug the existing SR methods (e.g., PySR, Sindy, Operon) into this pipeline. Why not give all the methods the same opportunity to benefit from the proposed workflow?
    3. This workflow might be useful but the experiments don’t really establish that. Realistically, it seems strange to suggest a method that has such a limited search space.
  2. The dataset and experiments are not described clearly enough to for me to fully evaluate the work.
    1. How were the 33 equations selected, where did they come from, and what are their input domains? I could not find the full list anywhere in the paper, only an appendix section pointing to the not included supplementary materials.
    2. The dev/test split is supposed to check if the restrictions imposed on the SR step extend beyond the equations used to design it but without information on how they were selected there is no way to know. If these were selected in the same manner, then the results may just be a result of the restriction in search space more than anything else.
    3. There are also some weird things about the percentages reported as appear in my reading notes.

Below are some general comments from my reading. They are not critical weakness but I have included them for reference.

  • Broken citation in Section 2, line 93.
  • 13 dev / 20 test is a pretty small dataset. How were these equations selected?
    • The split is totally meaningless if they were selected in the same manner by the researchers because then anything used to develop the restricted SR approach on the dev set will obviously transfer to the test set.
  • “elementary transcendental atoms” - I get what is being said but please use normal words for this.
  • The 33 equations used for the dataset are not provided and the method of selection is not described. This is a critical piece of information. The paper says information is in the supplementary materials but that was not included in the submission.
  • p. 7, lines 372–373 reports 75% and 66.7% but with a denominator of 13 x 5 = 65, these are not possible. What is going on here? Happens again at p. 9, lines 466–467.
  • Overall the writing has some clarity issues.
Questions for authors

My main concerns regard the experimental design (particularly the oddly restricted SR step) and the small dataset along with its construction details. I think there is potential for some interesting findings here, but with a fairly different set of experiments.

  1. How were the equations selected and what are they? What are the input domains they are evaluated on and are they all even representable with the selected operators / equation classes (across all the methods tested). What was known before restricting the SR step of the proposed method?
  2. Did the baselines receive only the raw inputs or the same NN outputs as the proposed SR step? Is there a comparison inserting other SR baselines into the overall pipeline? In particular, it would be interesting to know if this NN step is generally useful across methods.
  3. What is going on with the trial counts and reported accuracies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.