acceptodds
Under review as a conference paper at ICLR 2027

Self-Regeneration As a Search Signal For Diffusion Symbolic Regression

Abstract

Symbolic regression seeks explicit mathematical expressions that describe relationships in numerical observations, but the vast discrete expression space makes it difficult to find high-quality solutions. Pretrained diffusion generators learn expression priors and directly produce candidates, while their continuous reverse process provides an interface for inference-time refinement before discrete decoding. Yet black-box fitness evaluates only completed expressions and does not directly indicate how to steer this process. We observe that time-aligned deviations between an original generation trajectory and its self-regeneration trajectories contain reusable directional information that can guide subsequent denoising toward useful symbolic transitions. Based on this insight, we introduce SelfRegeneration Controlled Diffusion Evolution (SR-CDE), which constructs such deviations when the target trajectory is unknown and assigns suitable donors to individual recipient contexts. SR-CDE injects the selected deviations during reverse diffusion and uses black-box fitness on the original regression task to select and evolve improved trajectory states. It thereby turns temporal information from past generations into trajectory states that can be reused and iteratively refined during inference-time search, while keeping the pretrained generator frozen. Across a 73-task symbolic regression benchmark, SR-CDE achieves competitive overall performance against leading methods. Controlled experiments on Hard-18 further show that full trajectory deviations consistently outperform endpoint-only transfer and random directions matched in summary statistics, confirming the joint importance of temporal application and self-regeneration-specific directional information.

Primary area:
interpretability and explainable AI
Keywords:
Symbolic Regression, Diffusion Model, Evolutionary Search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_MzHDOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Self Regeneration Controlled Diffusion Evolution, an inference-time search method for diffusion based symbolic regression that extracts trajectory-level perturbation directions by comparing original diffusion trajectories with self regenerated trajectories conditioned on candidate expressions. These perturbations are injected into the following denoising steps, while a population based evolutionary search procedure uses black box fitness evaluations and recipient specific donor selection to iteratively improve generated expressions without retraining the diffusion model. The method is evaluated on 73 symbolic regression benchmarks and 18 tasks comparing trajectory level guidance against endpoint optimization and randomized perturbation strategies.

Critical strengths
  1. Against the baselines compared against, the proposed method achieves convincing empirical results outperforming several established approaches.
  2. This paper includes useful ablation experiments comparing full trajectory level guidance against endpoint only optimization and randomized perturbations. This is helpful and important.
  3. This method works with black box fitness evaluation and does not require differentiating through a symbolic expression evaluator. This is appealing for practical use cases and potentially applicable to other discrete optimization problems.
Critical weaknesses
  1. Although the paper includes several symbolic regression and endpoint-search baselines, it lacks comparisons against alternative diffusion-based evolutionary search methods and simpler trajectory-level perturbation strategies. This is a critical limitation.
  2. SR-CDE requires repeated self-regeneration and multiple controlled denoising rollouts, resulting in considerably higher computational costs than the endpoint-search baselines. Without comparisons under matched computational budgets, it is difficult to determine whether the reported performance improvements arise from the proposed search mechanism or the additional computation.
  3. The proposed method combines several established ideas, including diffusion-based inference-time guidance, black-box optimization, and evolutionary search. This paper is very incremental with respect to the prior work.
Questions for authors
  1. Could the authors include comparisons against simpler trajectory-level perturbation strategies, such as differences between independently generated trajectories without self-regeneration, or diffusion-based evolutionary mutation approaches? These comparisons would help establish whether self-regeneration provides meaningful advantages over alternative methods for constructing search directions.
  2. There would additionally be immense benefit for the authors to provide additional ablations isolating the contributions of recipient conditioned donor selection, Pareto based matching, and population based evolution, for example how the proposed approach perform with random donor selection or simpler trajectory similarity based matching.
  3. Provided the higher computational cost of SR-CDE compared with the endpoint-search baselines, how would the reported performance improvements change under matched computational budgets?
Reviewer_jJXtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces Self-Regeneration Controlled Diffusion Evolution (SR-CDE) for symbolic regression with a frozen conditional diffusion generator. Given a target-validated anchor expression, SR-CDE evaluates it on the observed inputs, conditions the generator on the resulting synthetic observations, and runs DDIM from independent noises to obtain self-regeneration trajectories. The time-aligned deviations between the anchor trajectory and these regenerated trajectories form structured proposal directions. SR-CDE archives these deviations, assigns donors to recipient contexts, injects the selected deviations at DDIM steps with signed scales, and uses black-box target fitness to select and evolve improved trajectory states. On 73 SRBench tasks, it reports mean R² 0.996984, P10 0.993003, average rank 1.8973, and 64/73 strict recoveries at R² > 0.999. On a controlled Hard-18 subset, it reports +0.076987 mean R² over initialization and better performance than endpoint-only and random controls.

Critical strengths
  1. Novel and well-motivated idea. Using the generator’s own self-regeneration behavior as a source of trajectory-level search directions is clever. The time-aligned deviation preserves positional, channel-wise, and cross-token structure, which endpoint differences discard.

  2. Thoughtful problem formulation. The analysis of hard decoding, expression regions, and basins explains why endpoint fitness is a weak search signal and why interventions should be applied during denoising. This framing is a genuine conceptual contribution.

  3. Strong main results. The 73-task results are competitive: mean R² 0.996984, P10 0.993003, rank 1.8973, and 64/73 strict recoveries, better than E2E, DiffuSR, MOD-SR, and GenSR. The Hard-18 experiment shows 18/18 tasks improved and 14/18 wins.

  4. Good controlled ablations. The comparison with endpoint-only transfer (0.958702), timestep-RMS Gaussian (0.953313), and covariance-matched controls (0.978121) is the right way to test the core hypothesis. The fact that trajectory deviations (0.988849) outperform covariance matching is meaningful evidence that the temporal structure matters.

  5. Frozen generator, no retraining. The method is a plug-in inference-time search procedure, which is practical and avoids the cost of retraining a diffusion model.

  6. Clear algorithmic description. The overall pipeline and the donor–recipient assignment are understandable, and the paper includes pseudocode-level details in the appendix.

Critical weaknesses
  1. The PDF is not ready for review: numerous unresolved references. The main text contains many ?, ??, and ??? placeholders, including citations and cross-references (e.g., “DiffuSR and MOD-SR instantiate this form of generation for symbolic regression (??)” in the introduction, “Genetic programming ... (?)” in related work, “Equation ??” in Section 3, and “Table ??” in Section 5). This is a serious presentation defect that makes the paper difficult to review and suggests the submission was not proofread. All references must be resolved.

  2. Benchmark saturation. Initialization already exceeds R² = 0.99 on 55 of 73 tasks. The mean R² differences among methods are therefore tiny, and the benchmark has limited ability to discriminate. Hard-18 is a post-hoc subset defined by initialization performance; although it is frozen before search, it is small and selected on the same benchmark. A harder, pre-registered benchmark would be more convincing.

  3. Limited baselines and non-iso-compute comparison. The paper compares only with four neural SR methods (E2E, DiffuSR, MOD-SR, GenSR). It does not compare with strong GP/search baselines such as PySR, Operon, GP-GOMEA, DSR/uDSR, or recent LLM-based SR. Since SR-CDE is a search procedure, these are natural baselines. The runtime is 203.01 s vs. 39.51–60.79 s for the endpoint methods, so the comparison is not iso-compute; a budget-matched comparison is needed.

  4. No error bars or statistical tests. The main results and the Hard-18 gains are reported as point estimates. The per-task table shows several tasks where covariance matching beats trajectory deviation (e.g., Livermore-3: 0.997027 vs. 0.983929; Keijzer-11: 0.980713 vs. 0.961549; Jin-5: 0.999858 vs. 0.999529). The abstract’s claim that full trajectory deviations “consistently outperform” the controls is therefore too strong. The authors should report win/loss counts and confidence intervals, and revise the claim to “on average” or “on 14/18 tasks.”

  5. The cost accounting excludes important components. The reported 203 s runtime excludes self-regenerations and replay; the Hard-18 study uses 3,489 controlled rollouts. Donor assignment also requires evaluating donors on the target task. The paper should report the total cost including self-regeneration, donor evaluation, and replay, and compare at equal total budget.

  6. Hyperparameter sensitivity is underexplored. The method has many choices: number of self-regenerations, donor assignment, guidance-start index, signed scales, population size, and expansion allocation. The paper reports some ablations, but not the sensitivity to these choices. A method with this many knobs needs a clear default setting and a sensitivity analysis.

  7. Only one diffusion backbone. The method is evaluated with a single generator. Its generality across diffusion SR models (and across generator quality) is not tested. A second backbone, or a synthetic generator with controlled quality, would strengthen the claims.

  8. The baselines do not test whether the gains are specific to diffusion. The baselines are all neural generative SR methods, and the diffusion-based comparisons share a similar backbone family, so the experiments do not separate the effect of trajectory-level deviations from the effect of the diffusion generator. A non-diffusion learned search baseline would be informative—for example, a neural-guided dynamic symbolic network such as DySymNet (Li et al., ICML 2024), run at a matched target-evaluation budget. This does not require re-running the full 73-task suite; the Hard-18 subset plus a few easy tasks would be enough to test whether SR-CDE's advantage persists against a different learned search mechanism. The related work should also position the method relative to this line.

Questions for authors
  1. Please resolve all unresolved references before resubmission. This is essential for a fair review.
  2. How many independent runs/seeds were used for the main results and Hard-18? Please report mean ± std and per-task win/loss counts for the controls.
  3. On how many of the 18 tasks does trajectory deviation beat covariance matching? The per-task table suggests several losses; how does this affect the "consistently outperform" claim?
  4. Is the 73-task benchmark saturated? Can you add harder tasks or report results on a pre-registered subset with initial R² < 0.99?
  5. Why are GP/search baselines (PySR, Operon, GP-GOMEA, DSR) excluded? Can you compare at equal wall-clock or equal target-evaluation budget?
  6. What is the total cost including self-regeneration, donor evaluation, and replay? How does it compare at iso-budget?
  7. How sensitive are the results to the number of self-regenerations, guidance start, scale set, and donor assignment?
  8. Does the method work with a different diffusion backbone?
  9. Will the code be released?
  10. Can you add a non-diffusion learned search baseline (e.g., DySymNet) on the Hard-18 subset at a matched target-evaluation budget, to test whether the gains are specific to the diffusion backbone?
Reviewer_ohEROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method that searches for symbolic expressions by intervening in the intermediate trajectories of a frozen conditional diffusion generator. Given a candidate expression, it evaluates expression on the observed inputs, regenerates expressions under the resulting synthetic observations, and subtracts regenerated trajectories from the candidate's anchor trajectory to construct time-aligned proposal directions. It selects donors for fixed recipient contexts using trajectory compatibility and fitness criteria, then tests signed scales during DDIM denoising and retains improvements according to black-box regression fitness after constant fitting. On 73 noiseless benchmark tasks, it compared with DiffuSR and GenSR. Controlled experiments on 18 tasks not solved at initialization favor trajectory-level interventions over endpoint-only search, although the specific-direction advantage over covariance-matched random controls is sensitive to one task and the manuscript leaves important reproducibility and robustness questions unresolved.

Critical strengths
  1. The headline accuracy result is substantial on the reported benchmark.

  2. The method can use discrete black-box expression feedback without training a new differentiable expression head. Signed multi-scale controls are judged after decoding and constant optimization, while the condition encoder, denoiser, decoder, and schedule stay frozen.

Critical weaknesses
  1. The evidence for self-regeneration-specific superiority is dominated by one task. From Table 7, full trajectory control beats covariance-matched control on six tasks, ties seven, and loses five, but excluding Livermore-11 reverses the ordering. Livermore-11 alone contributes a lot. This undermine language such as “consistently outperform”. Also there is no reported repeated-run for the central search results, so not statistically reliable. Strong symbolic-search baseline families are missing, such as TPSR and PySR.

  2. Search effort is not controlled well enough to attribute the endpoint advantage. Its full-benchmark runtime is approximately 3 times DE's and 5 times GWO's. A trajectory rollout is much more expensive than decoding an endpoint. Cost should be controlled. The direction ablation also does not isolate temporal alignment or self-conditioning causally. Missing controls include time-permuted real deviation, cross-token-shuffled deviation, and differences between two target-conditioned trajectories without candidate self-conditioning.

  3. The submitted document is not review-ready in several verifiable places. The pdf contains unresolved citations, and the method and results repeatedly refer to “Figure ??” and “Table ??”. Writing details are also not rigorous, e.g., the actual frozen backbone and evaluation protocol aren't specified.

Questions for authors
  1. Does the advantage over covariance-matched directions survive repeated runs and task-level paired analysis after accounting for Livermore-11's influence?

  2. What happens when real deviations are time-permuted or generated without the candidate-induced self-condition?

  3. Which exact backbone/checkpoint and training data were used, and which reported values are evaluated on observations unseen by conditioning, fitting, and search selection?

  4. How many outputs are symbolically equivalent to the true formula, and how do results change with noise, extrapolation, and a common complexity budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.