acceptodds
Under review as a conference paper at ICLR 2027

Disentangling Invariant Structures from Parameter Variations for Parameterized Equation Discovery

Abstract

While Symbolic Regression (SR) excels at discovering mathematical expressions from data, it traditionally treats numerical coefficients as fixed, recovering isolated mathematical instances. However, many scientific applications require Parameterized Equation Discovery (PED): identifying an invariant symbolic skeleton alongside instance-specific parameters across multiple parameter-varying environments. A fundamental challenge in PED is parameter-induced structural ambiguity, where parameter variations can drastically alter the data distribution of a single skeleton or cause distinct skeletons to become observationally aliased. Existing multi-environment SR methods rely heavily on heuristic search and post-hoc aggregate fitness evaluation, lacking a data-driven prior to separate structural invariants from parameter variations. In this work, we propose DisPV (Disentangling invariant structures from Parameter Variations), the first pre-training framework specifically designed for PED. DisPV designs a dual-stream architecture that explicitly disentangles universal mathematical structures from environment-specific constants at the representation level. Through a cross-environment contrastive learning strategy, the skeleton stream isolates invariant structural features, while the parallel constant stream predicts instance-specific parameters. By learning structural invariance directly from parameter-varying data representations, DisPV provides a highly accurate structural prior. Extensive experiments on synthetic and physics-inspired benchmarks demonstrate that DisPV, coupled with genetic programming refinement, significantly outperforms state-of-the-art search-based baselines with an average .

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Parameterized Equation Discovery, Symbolic Regression, Invariant Representation Learning, Pretrained Models, AI for Science

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hBPVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DisPV is a paper on Parameterized Equation Discovery (PED): addressing "parameter-induced structural ambiguity"—where the same symbolic skeleton exhibits vastly different distributions under different parameters, or even becomes observationally aliased with an incorrect structure—it proposes the first pre-training framework specifically designed for PED. At its core is a dual-stream architecture: after encoding multi-environment data with a shared SetTransformer, the skeleton stream extracts environment-invariant symbolic structures via cross-environment contrastive learning, while the parameter stream in parallel regresses the instance-specific constants, explicitly disentangling the "universal skeleton" from the "instance parameters" at the representation level, followed by lightweight genetic programming refinement. Its main contributions are: formalizing the structural ambiguity challenge of PED, proposing this dual-stream pre-training framework, and achieving a state-of-the-art R2=0.900R2=0.900 on synthetic and physics-inspired benchmarks, while being over 10× faster at inference than search-based baselines.

Critical strengths

The paper provides a clear formulation of PED’s central challenge, parameter-induced structural ambiguity, distinguishing between distributional shift and structural aliasing through an illustrative underdamped oscillator example. It also highlights a key limitation of existing methods, including PySR, PhySO, and MSR, which enforce cross-environment consistency through optimization objectives but lack data-driven structural priors. Building on this insight, the authors introduce a pre-training framework specifically designed for PED, using a dual-stream architecture to explicitly disentangle structural skeletons from parameters at the representation level.

Critical weaknesses
  1. The paper reports the maximum R² across 10 independent runs (best-of-N), which systematically overestimates the reliability of the method and masks its variance. It is recommended to report the median / mean ± standard deviation together with the success rate.
  2. The manuscript contains many typos, such as hyperapameter, candidatas, Montle Carlo, parameteried etc.
Questions for authors
  1. The manuscript states that “Each experiment is repeated over 10 independent runs and the maximum Rˉ2\bar{R}^2 are reported.” We wonder why only the maximum values are presented. Could the mean and variance also be provided?
Reviewer_6FyGOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DisPV, a pre-training framework for Parameterized Equation Discovery (PED), which aims to recover a shared equation structure from multiple datasets with varying parameters. To address Distribution Shift and Structural Aliasing, DisPV employs a dual-stream architecture to separate shared structures from environment-specific parameters, together with cross-environment contrastive learning and genetic programming (GP) refinement. Experiments on synthetic and physics-inspired benchmarks demonstrate improved equation-fitting performance over existing methods.

Critical strengths

S1. Important problem with clear and insightful motivation. The paper identifies two key challenges in PED: Distribution Shift, where the same equation structure produces different observations under varying parameters, and Structural Aliasing, where different structures produce similar observations.

S2. Intuitive and well-motivated method design. The dual-stream architecture explicitly separates shared equation structures from environment-specific parameters, with each component addressing a clear challenge.

S3.Solid experimental results. DisPV achieves better equation-fitting performance across several benchmarks. Ablation studies further support the effectiveness of cross-environment aggregation, contrastive learning, and GP refinement.

S4. Clear presentation. The paper is well organized and easy to follow.

Critical weaknesses

W1. Structural consistency does not demonstrate correct structure recovery. DisPV aggregates structural features across environments and decodes a single shared equation. Therefore, the 100% Structural Consistency reported in Fig. 3 is largely guaranteed by the model design, rather than demonstrating successful recovery of the ground-truth structure. Even if the model predicts the same incorrect equation for all environments, it would still achieve 100% consistency. Thus, this experiment does not directly support the claim that DisPV resolves Structural Aliasing.

W2. OOD results do not sufficiently support the claim of structural invariance. Table 1 shows that DisPV outperforms PySR in the in-distribution setting (58.3% vs. 42.7%), but its recovery rate drops to 41.6% under OOD parameter ranges, slightly below PySR (44.4%). These results demonstrate an advantage in-distribution, but do not establish superior robustness to unseen parameter ranges. Therefore, the claim that DisPV learns genuine structural invariance rather than overfitting to the training parameter distribution appears overstated.

Questions for authors

Q1: A natural alternative is to first recover candidate equations for each environment using traditional symbolic regression, and then identify their shared mathematical structure through symbolic reasoning or an LLM. What is the key advantage of DisPV over this two-stage approach? Intuitively, why should DisPV perform better?

2 comments

I thank the authors for their rebuttal, which has addressed most of my concerns. However, given my limited expertise in this area, I do not feel sufficiently confident to make a further judgment. Therefore, I will maintain my original score.

Thank you for your time and for carefully reading our rebuttal. We appreciate that our responses have addressed most of your concerns. We fully respect your decision to maintain your original score. We have updated the manuscript to incorporate the experimental results on LLM-based refinement and conventional symbolic regression methods for PED into the main body of the paper. Should you have any further questions or suggestions, we would be happy to provide additional clarifications.

Reviewer_FZrFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies parameterized equation discovery (PED), where several datasets share one symbolic skeleton but have different, unobserved parameters. It proposes DisPV, a pre-trained model with a skeleton stream (trained with a cross-environment contrastive loss) and a constant stream, whose outputs seed a GP search. On benchmark equations, DisPV with GP refinement reports higher mean accuracy and exact recovery than PySR and other PED baselines.

Critical strengths
  • Using a pre-trained structural prior for multi-dataset SR is a reasonable idea, since existing PED methods search from scratch for every task.

  • The problem is well motivated. The oscillator example and the consistency analysis clearly show why fitting each dataset separately can lead to the wrong skeleton.

Critical weaknesses
  • It is unclear which part of the method drives the improvement. Joint selection across datasets is also used by the baselines and the GP refiner, the contrastive loss shows a small and mixed effect in the ablation, and baselines that would separate these factors are missing.

  • The evaluation is limited. The benchmark is small, simple, and noiseless, results are reported as best-of-10 without per-equation breakdowns, possible overlap between training and test skeletons is not discussed, and the settings where PED should help most (noisy or small data, unseen environments) are not tested.

  • Important implementation and training details are missing, which makes the results hard to reproduce. Some claims, such as guaranteed disentanglement and the 10× speedup measured without GP, go beyond the evidence.

Questions for authors
  1. I am curious how much of the improvement comes from the proposed design itself versus from seeding GP with a neural prior in general. Have the authors tried seeding the same GP with candidates from a standard pre-trained SR model (e.g., NeSymReS run on each dataset), or a simple pipeline that runs PySR on each dataset and selects the candidate with the best mean R² across datasets? It would also be interesting to see whether DisPV candidates help when used to seed PySR, since the GP alone appears fairly weak (R² 0.47 in Table 2).

  2. My understanding is that the negatives in the contrastive loss are the other skeletons in a random batch, which are mostly easy to distinguish. I was wondering whether the authors considered harder negatives that target aliasing, such as nested variants with one term removed. It would also help to know how equivalent skeletons written in different forms are handled, and whether there is direct evidence that the contrastive loss helps on aliased cases.

  3. Could the authors clarify whether the benchmark skeletons were excluded from the 5M training set, for example by checking symbolic equivalence after replacing constants with placeholders?

  4. A few reporting details were unclear to me. What do the error bars in Fig. 4 represent, given that the best of 10 runs is reported? Could the authors also report mean or median results and a per-equation table, clarify which test set is used in Table 2 and Fig. 8 (the counts exceed 36), explain how the 42.7% recovery for PySR is computed, and report the total runtime including the GP stage?

  5. I would expect PED to be most useful when each dataset is small or noisy. Have the authors evaluated the method under noise, with fewer points per dataset, with different numbers of datasets at inference, on held-out environments, or on extrapolation? Results on an existing PED benchmark such as the Class SR benchmark would also make the comparison easier to place.

  6. Could the authors share how MSR, SEQL, and HEQL were adapted and tuned for this setting, given their low scores? I was also wondering why MvSR is not included, and why the PySR operator set excludes pow while the GP includes it.

  7. The skeletons in Fig. 6 seem structurally quite different from each other. It would be informative to see aliased or nested pairs, and the same plot for a baseline encoder. In Fig. 7, the true parameter similarities appear to lie between 0.92 and 1, while parameters sampled from [−2, 2] would usually give some negative cosine similarities. Were the parameters transformed? A quantitative correlation between the matrices would also be helpful.

  8. Could the authors report the model size, key hyperparameters (k, h, λ₁, λ₂, τ), optimizer and training schedule, and the data generator details (operators, maximum number of variables, input range, points per dataset)? It would also help to clarify whether “5M equations” refers to skeletons or skeleton–parameter pairs.

Comments on the paper

1 comment

Dear reviewers,

Thank you very much for your time and effort in reviewing this submission. The authors have now submitted their rebuttal, and the discussion period is nearing its end. We would greatly appreciate it if you could take some time to carefully review the authors’ responses and share your assessment of whether your concerns have been adequately addressed. Thank you again for your valuable time and contributions to the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.