acceptodds
Under review as a conference paper at ICLR 2027

Deep Divide-and-Reduce in Symbolic Regression

Abstract

Symbolic regression (SR) aims to discover underlying mathematical expressions from data while preserving interpretability. Most existing learning-based SR methods primarily optimize expressions from observations without explicitly exploiting their structural mathematical properties. AI Feynman introduced a complementary paradigm that leverages such properties to recursively decompose complex expressions, but its decomposition criteria cover only restricted structural forms and its treatment of nested composition can require brute-force search over candidate sub-expressions. Building on this paradigm, we propose Deep Divide-and-Reduce in Symbolic Regression (DDRSR), a mathematically grounded framework that systematically generalizes expression decomposition and variable reduction. DDRSR extends translational symmetry to coefficient- and exponent-interfered forms, enables variable separation under overlapping variables and additive constant offsets, and generalizes the identification of nested compositional structures. We further characterize an intrinsic non-identifiability limitation of decomposition when no effective variable separation is induced. Experiments across multiple symbolic regression algorithms and benchmark datasets show that DDRSR identifies a broader range of decomposable structures than AI Feynman and overall improves downstream regression accuracy and exact-expression recovery.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Symbolic Regression, Expression Decomposition

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8msjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper extends the divide-and-conquer idea of AI Feynman and proposes a more comprehensive framework for identifying reducible substructures in a complex data-generating function, thereby facilitating easier downstream SR tasks. The irreducible substructures are then fed to existing SR methods for symbolic search, and the final expression is derived by combining the symbolic expression of each irreducible substructure.

Critical strengths
  1. The paper describes a comprehensive list of potential reducible structures of the data-generating function ff.
  2. The empirical study covers a comprehensive list of datasets.
Critical weaknesses
  1. The authors mentioned DDRSR can be paired with "different downstream SR methods," but their evaluations in Section 4.1 only focused on quite dated SR methods published in 2021 to 2023. In a 2026 paper, readers will expect to see much more recent methods. Furthermore, the authors hand-pick baseline SR methods in subsections of 4, rather than reporting the effect of DDRSR on all 5 baseline SR methods. For instance, Table 1 only considers 3/5 SR baselines, Table 3 only considers 1/5 SR baselines, etc. More importantly, Table 4 shows that the downstream solvers receive only 40 samples in (0,1)n(0,1)^n, which explains the negative baseline R2R^2 at 6+ variables in Table 7 and inflates the apparent benefit of decomposition, and Table 1 shows DDRSR uses 2–7$\times$ the runtime of the baseline with no compute-matched comparison.
  2. The evaluated setting is not symbolic regression from data. Every criterion queries ff at chosen points: slices with variable groups fixed to constants, two-point rectangles for the constant-offset recovery, and finite differences at step 10−510^{-5} (Table 4: 20 query points ×\times 10 constant settings). Appendix C concedes that the method needs "an active sampling environment, such as a directly queryable physical field". AI Feynman handles this with a neural surrogate, which is never used here, so neither the actual AI Feynman system nor a data-only DDRSR is ever run, and the abstract and introduction present the results as benchmark gains without this caveat.
  3. The uniform noise added to the data is insignificant compared to the signal: the largest amplitude, 10−510^{-5} on inputs in (0,1)(0,1), equals the finite-difference step in Table 4 and is three to four orders of magnitude below the noise levels used in SRBench. A proper signal-to-noise ratio control is needed. See Ye & Li (arXiv:2410.13681; ICML 2025) for instance.
  4. The paper needs a significant rewrite so its presentation is readable to human audiences.
    • Notation issues: f(x1,x2,…)f(x_1, x_2, \ldots) seems to imply f(⋅)f(\cdot) is infinitely dimensional, but it is actually nn-dimensional. More critically, Section 3.2.1 uses SX1SX_1 without defining SS and XiX_i. The operation of SXiSX_i is also very confusing, as it implies matrix multiplication, but XiX_i's are not matrices and are definitely of arbitrary dimension, yet SS is not differentiated, so SXiSX_i makes no sense. As far as I can tell, SXi∈Rn1SX_i \in \mathbb{R}^{n_1} seems to be a subvector of XX with arbitrarily selected dimensions n1<nn_1 < n. If so, why not use XS1X_{\mathcal{S}_1}, where S1⊂[n]\mathcal{S}_1 \subset [n]?
    • The paper also did not mention what datasets were used in the empirical study. Figures 2(c)-(d) suggest there are multiple datasets used in the empirical study, but none are discussed or cited in the main paper. The figures also suggest a set of datasets called "Others", which Table 5 in Appendix F never labels.
    • Section 4.2 mentioned that only the "best-performing baseline SR methods" are reported in the comprehensive SR experiments, but how is performance measured? You seem to have results for all 5 SR methods. Why not just report all of them like in Figures 2(e)-(f)? Same thing for Section 4.4. Why not report the runtime for all SR methods? Furthermore, the Appendices list the CPU and GPU used for the empirical study, but do not report which methods run on CPU and which run on GPU, so the reported runtime is not useful.
    • The list of writing issues just doesn't stop.
  5. The "Relative Separation Score" is also oddly defined in Section 4.1, where all mid-field methods are assigned a score of 0.5, leaving nothing to differentiate them. Furthermore, the best method is given a score of 1, but there is no telling how many decomposition patterns it misses. It is also defined relative to the competing method, so DDRSR, whose rule set is a superset of AI Feynman's, scores 1 by construction, and the "maximum number of correct decomposition patterns" is never operationalized.
Questions for authors

See weaknesses above. Also,

  1. Section 4.3 states that the custom complex-test datasets are excluded from top-down expression separation and are used solely for bottom-up variable composition. What's the justification?
  2. Can DDRSR run from a fixed dataset, e.g., through a neural surrogate as AI Feynman does? Please report recovery rates under that setting.
  3. Does the advantage survive noise amplitudes of 10−310^{-3} to 10−110^{-1}?
  4. Why disable AI Feynman 2.0's candidate search for hh, given that it is AI Feynman's mechanism for the forms you claim it cannot handle? Please run the published AI Feynman 2.0.
  5. Which 23 expressions fail to decompose at zero noise, and were any accepted decompositions wrong?
  6. The decomposition criteria are necessary conditions checked at 10 random points with a relative threshold of 0.01 (Appendix B), and their false-positive rate is never measured. Section 4.5 nonetheless asserts that false positives are "unlikely", and the claim that DDRSR "is generally expected to perform no worse than direct symbolic regression" rests on it.
Reviewer_PhwoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a deep decomposition and dimensionality reduction framework for symbolic regression (DDRSR), aiming to improve the performance of subsequent symbolic regression algorithms by decomposing complex target expressions into multiple simple subproblems. Based on AlFeynman, DDRSR extends the recognition rules for translation symmetry, variable separability, and nested composition structures, enabling it to handle variable combinations containing constant coefficients and power exponents, expression separation with overlapping variables, and structures with additive constant offsets. In addition, the paper proves through theoretical analysis that, in the absence of effective variable separation, some expression decompositions are unidentifiable. The authors conduct experiments on multiple symbolic regression benchmark datasets and self-built expressions, and combine DDRSR with symbolic regression algorithms such as PySR, GP-GOMEA, and E2E. Experimental results show that DDRSR can identify a broader range of mathematical decomposition structures than AlFeynman, and improves regression accuracy and exact expression recovery rate on some complex and high-dimensional expressions.

Critical strengths

Symbolic regression needs to find the target expression in a huge expression search space. For target functions with complex structures, the combinatorial explosion of the search space may make it difficult for existing algorithms to find the correct expression within limited computational resources and time. Therefore, before applying symbolic regression algorithms, identifying and decomposing the structure of the target function based on observed data to reduce the complexity of subsequent search is a research direction of important value. The paper systematically extends multiple structural decomposition mechanisms of AlFeynman, and especially proposes broader mathematical criteria for the separation of expressions with overlapping variables, variable combinations containing constant coefficients and power exponents, and multiplicative separation with constant offsets. These extensions can identify more expression structures that are difficult for traditional methods to handle.

Critical weaknesses
  1. Exact recovery rate decreases on some tasks, and there is a lack of failure cause analysis. Although DDRSR improves regression accuracy and exact recovery rate overall, Appendix Table 8 shows that DDRSR+PySR has a lower exact recovery rate than original PySR on datasets such as Keijzer, Nguyen, and Livermore. In addition, Table 13 also contains individual expressions with decreased recovery rate. However, some results in the two tables are inconsistent, and the authors need to verify them first. The paper lacks an analysis of the reasons for performance degradation. The degradation may stem from incorrect decomposition, that is, the algorithm accepts a decomposition structure that is not strictly equivalent to the true target function; it may also stem from failure in searching subexpressions after correct decomposition. The authors are suggested to verify the experimental results, report the incorrect decomposition rate, and combine specific failure cases to distinguish structural recognition errors from downstream search failures, so as to explain the reasons for the decrease in recovery rate.

  2. Lack of analysis of decomposition benefits and performance reliability. As a heuristic preprocessing method, DDRSR does not need to guarantee improved performance for any target expression, but the paper should further analyze the conditions and limitations of its performance improvement, and try to establish corresponding worst-case bounds or performance guarantees. For example, under certain assumptions, can the relationship between search complexity or exact recovery probability before and after decomposition be derived, sufficient conditions for beneficial decomposition be given, a performance lower bound for exact recovery rate be provided, or a worst-case bound for the magnitude of performance degradation be given? If it is difficult to establish theoretical guarantees, at least the frequency and magnitude of performance degradation should be quantified experimentally. In addition, the fallback mechanism proposed in the paper mainly targets cases where an effective decomposition structure cannot be identified. It cannot guarantee that an accepted decomposition is correct, nor can it avoid performance degradation caused by failure in searching factor expressions after correct decomposition. Therefore, the authors are suggested to further study the relationship between mathematical decomposition and the final exact recovery rate, and consider introducing adaptive decomposition decisions or performance protection mechanisms to improve the reliability of the method.

  3. Limited applicability under practical application conditions. The core decomposition rules of DDRSR rely on mathematical relationships among multiple variables, so it is difficult for them to provide obvious advantages for univariate expressions or multivariate expressions that cannot be effectively decomposed. In addition, multiple structural criteria require obtaining target function values at specific input positions and estimating derivatives through numerical differentiation. However, practical symbolic regression tasks usually only have fixed input-output observation data and cannot arbitrarily query the unknown target function. At the same time, structure detection is sensitive to noise and numerical errors; the paper’s experiments also show that as noise increases, the number of expressions that fail to be decomposed increases significantly. These limitations make the practical applicability of DDRSR depend on relatively strict data conditions and target function structures. Therefore, clarifying the applicable scope and performance guarantees of the method is particularly important. The authors are suggested to further analyze the effectiveness of the algorithm under different conditions, and consider designing performance protection mechanisms to reduce the risk of performance degradation in inapplicable scenarios. Although some limitations stem from the mathematical decomposition method itself, excessive applicability conditions may also weaken the practical value of the method as a general symbolic regression preprocessing framework.

Questions for authors

See Weaknesses.

1 comment

Thanks to the authors for their detailed and careful rebuttal. Some of my concerns have been addressed, and the quality of the paper has improved. After re-examining the paper, I believe my score is already reasonably appropriate, so I plan to keep my score.

Reviewer_4bV1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper generalizes and builds upon on the decomposition and variable reduction methods of AIFeynman. The authors illustrate and prove seven decomposition methods. Subsequently, the authors apply these methods within a pipeline of 3 downstream symbolic regression algorithms.

Critical strengths
  • The paper systematically investigates, proposes and proves novel mathematical decomposition methods for formulas using the properties of given operators.

  • The method achieves better separation/decomposition than AIFeynman.

  • The paper is well structured, the evaluation datasets are diverse and the evaluated methods include state of the art downstream algorithms like PySR.

Critical weaknesses
  • Practical application: There is not method section that explains how the decomposition is realized in practice. The method section only lists the decompositions themselves, but it remains unclear to me how they are applied in practice and how the subresults are fed into downstream algorithms and then composed back. For example, which result is taken from a downstream regressor? Is it the one achieving the best score?

  • Evaluation: The authors name two methods, NGGP and MetaSymNet, but do not apply them. In the results section, I only find E2E, GPGO and PySR. I am also missing a rationale why exactly these methods were selected.

  • Metrics: It is not explicitly stated what the R2 score reported in the results section is based on. Is it the fit to the train samples for the best returned equation from the downstream algorithm? Or is it R2 on an in-domain test set? The community has concluded that a pure fit based evaluation is not an adequate measure, especially when measured in the training domain, and has therefore suggested to use extrapolation measures (e.g. R2 or Acc0.1 on OOD samples, as illustrated in SRBench++ or LLM-SRBench).

  • Comparison: In section 4.4, the authors show that DDRSR needs roughly 3x more runtime than the baseline method, because it splits the problem into many sub-problems and runs sub-searches on partial problems. Evolutionary methods like PySR and GP-GOMEA have been shown to steadily improve their recovery and fit performance when given more search time. Therefore, a fair comparison against these methods would have to give all methods the same wall clock time, e.g. by letting the baseline methods run ~3 times longer.

  • Noise Robustness: My biggest concern is section 4.5 reporting on robustness against noise. One main reason why AIFeynman does not see the same broad acceptance among practitioners as, e.g., PySR is its fragility under noise. The fragility of decomposition-based preprocessing has been known in the community since quite some time. Being open about the struggles of the method in this respect and showing potential remedies or ways to handle it would be very valuable for the community. The paper seems to deliberately hide this weakness from the reader. The tested noise levels (Table 2) are very small (10−710^{-7} to 10−510^{-5}), compared to noise levels that are applied on public benchmarks (10−310^{-3}, 10−210^{-2}, 10−110^{-1}) as SRBench or SRBench++. The proposed decomposition-based method is theoretically sound, but struggles in practice under noise. Since practitioners are almost always confronted with noisy setups and changing distributions, it would be a real benefit for the community to show the limits of the method regarding this aspect and potential ways to improve on rather than trying to hide it. The results in Table 3 are showing R2 scores under noise which feel a bit cherry-picked. Why not transparently also report symbolic recovery rate and R2 under noise transparently here?

  • A minor point: It would be helpful to explain the notation using SS as in SXSX and SCSC in the Methods section.

Questions for authors

A future paper of the paper should address:

  • How is the robustness against realistic levels of noise?
  • How is the performance on extrapolation measures?
  • How does the method perform in comparison with other methods when all are given an equal amount of runtime?
1 comment

We have reduced the length of the manuscript to meet the 10-page limit. Specifically, we have consolidated the original Sections 3.3.1–3.3.4 into a single subsection entitled “Nested Separation of Logarithmic and Exponential Functions.” Additionally, we have moved the tables from the runtime and noise experiments to the appendix, while retaining appropriate references to these tables in the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.