acceptodds
Under review as a conference paper at ICLR 2027

From Score to Diagnosis: Search-Independent Post-Diagnosis for Symbolic Regression

Abstract

Symbolic regression (SR) solvers employ diverse search mechanisms, yet most rely on the same low-information scalar fitness. Such scores provide little guidance on why a skeleton fails or how it should be revised. Recent methods develop new methods to address this bottleneck. Their enhanced feedback mechanisms are tightly coupled to their search processes and are difficult to reuse across diverse SR solvers. Instead, we ask whether a shared feedback mechanism can further improve diverse solvers facing the same bottleneck. To this end, we propose PostDiag, a search-independent feedback mechanism, which can be shared to different SR solvers. PostDiag introduces Patch-wise Feature Profiling (PFP) to capture local behavior through patch-wise features of level, trend, and roughness. Based on PFP, PostDiag quantifies Skeleton Expressiveness Risk by comparing the fitted candidate with the observations, and Skeleton Stability Risk by measuring variability of behavior after bootstrap refitting. The Diagnostic Risk Plane (DRP) combines two risks to determine whether a skeleton should be preserved, repaired, simplified, or rewritten, while the patch-wise discrepancies provide evidence for the intervention. An LLM editor then performs the diagnosis-guided intervention, and the validated revision is returned to the original solver through a lightweight adapter. Experiments across multiple LLM-based and GP-based SR solvers show that PostDiag consistently improves accuracy and accelerates convergence.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Symbolic Regression, Scientific Discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CFBYOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PostDiag, an external diagnostic module for symbolic regression. Its central idea is to move beyond using a single error metric such as MSE as feedback. Instead, it analyzes the local behavior of candidate expressions and diagnoses them along two dimensions, namely expressiveness and stability, to determine whether a candidate should be preserved, repaired, simplified, or rewritten. An LLM is then used to modify the symbolic structure accordingly.

Critical strengths
  1. Moving from scalar fitness to actionable candidate-level diagnosis is a practically meaningful direction. Scalar fitness functions are effective for ranking candidate expressions, but provide very limited information about where a candidate fails, why it fails, or what kind of structural modification should be made. Moving from purely scalar evaluation toward actionable candidate-level diagnosis is therefore a useful research direction.

  2. The overall framework is reasonably modular, and the experimental evaluation covers a relatively broad range of solvers and settings.

Critical weaknesses
  1. The paper presents structured diagnosis as one of its main methodological contributions. However, it does not discuss several highly relevant works, including Symbolic Regression with Physics-Informed Residual and Structural Sensitivity Pruning (RSSP, IEEE TEVC 2026), StruSR: Structure-Aware Symbolic Regression with Physics-Informed Taylor Guidance (arXiv 2025), and An Interpretable Approach to PDE Solution Discovery via Structural Experience Distillation (arXiv 2026). In particular, as one of the earliest works exploring this direction, RSSP already performs fine-grained evaluation of candidate substructures through residual sensitivity and structural sensitivity, and uses such information to guide pruning, mutation, and crossover. The authors should cite and systematically compare against RSSP, while also discussing the relationship between PostDiag and the other closely related structure-aware methods. Such a comparison is necessary to appropriately narrow the paper’s claims that existing structured feedback is generally tightly coupled to specific search processes, as well as its novelty claims regarding solver-independent diagnosis. The absence of these comparisons makes the current related-work discussion incomplete and makes it difficult to determine the actual boundary of the paper’s contribution.

  2. Limited methodological novelty is one of the main concerns. The underlying problem itself was not first identified by this paper. Moreover, the Level, Trend, and Roughness features used in PFP essentially correspond to a local mean, a local linear trend, and a local nonlinear residual, respectively. These are fairly standard local statistical or local regression descriptors. The paper does not sufficiently demonstrate that this particular combination offers a clear advantage over simpler alternatives such as residual maps, local gradients, or curvature-based diagnostics.

  3. The current experiments do not sufficiently establish that the mechanisms emphasized by the paper, namely PFP, DRP, the four quadrant formulation, and search independent diagnosis, are individually necessary, correct, or substantively novel. As a result, the paper has not yet convincingly shown that its claimed core diagnostic mechanism, rather than other components of the overall pipeline, is responsible for the observed improvements.

  4. Interpreting high expressiveness risk directly as “underfitting” and high stability risk as “overfitting,” and then mapping these cases to repair, simplification, or rewriting, is intuitive but lacks sufficient theoretical justification. High instability may also result from parameter non-identifiability, optimization failure, or insufficient data, rather than structural redundancy. The current method does not adequately distinguish among these possibilities.

  5. High Bias does not necessarily imply insufficient skeleton expressiveness, and high Var does not necessarily imply overfitting or redundancy. The labels “Underfit,” “Overfit,” and “Mismatched” are therefore better understood as heuristic semantic interpretations of the two risk measures rather than validated structural diagnoses.

  6. The paper also has several noticeable writing issues. The Introduction tends to conflate two distinct issues, namely solver specific diagnostic feedback and the limitations of scalar fitness, thereby overstating the gap between prior work and the proposed method. The term “search independent” is also too strong, since PostDiag still affects subsequent search trajectories through candidate replacement and memory updates. In addition, the methodology currently reads more like a collection of modules than a mechanism driven design. The experimental section should be reorganized around clearly stated claims and the corresponding evidence required to support them. Finally, several conclusions should be stated more cautiously to avoid implying a level of general applicability that exceeds the scope of the current empirical validation.

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer CFBY 1/4

We thank the reviewer for recognizing the practical value of actionable diagnosis and the framework's modularity. We address the relationship to prior work, the evidence for each component, and the interpretation of the diagnostic risks below.


📝 W1 / W6 Relationship to Structure-Aware SR and Cross-Solver Compatibility

Thank you for highlighting these relevant works. We will cite RSSP (Gong Y, IEEE TEVC 2026), StruSR (Gong Y, arXiv 2025), and SED-MCTS (Gong Y, arXiv 2026) and clarify their relationship to PostDiag. SED-MCTS's arXiv preprint was posted on October 8, 2026, after the September 25 submission deadline; we will discuss it as well.

1. The distinction concerns what is diagnosed and how the evidence guides intervention.

  • RSSP and StruSR assess individual subtrees through masking, measuring their contributions to residual and Taylor-alignment objectives. These sensitivities guide crossover and mutation, with RSSP additionally performing sensitivity-guided pruning.
  • SED-MCTS uses counterfactual subtree contributions to accumulate grammar-edge experience and maintain a structural archive, guiding subsequent expression construction and editing.
  • PostDiag diagnoses a fitted skeleton's behavioral adequacy and refitting stability. PFP compares predicted and observed level, trend, and roughness across input-space patches, while bootstrap refitting measures how these behaviors change with the fitting samples. DRP combines these signals to prioritize preservation, repair, simplification, or rewriting, and the evidence map specifies the local behaviors to address.

Thus, inspired by the clinical process of examination, diagnosis, and intervention, PostDiag introduces a dedicated diagnostic pipeline for symbolic regression, comprising patch-wise behavioral evidence examination, risk diagnosis and localization, and evidence-guided intervention to provide richer feedback for different solvers. Since PostDiag's diagnosis is independent of the underlying search process, it can be integrated with GP-based, LLM-based, and agent-based methods through lightweight adapters. This integration improves performance across diverse solvers, enabling them to achieve performance comparable to or even better than higher-budget searches with fewer search resources.

2. Feasible integration with these methods.

For RSSP and StruSR, PostDiag can examine candidates before population selection, propose edits within the host's expression grammar, and return validated candidates for native evaluation. Their sensitivity-guided operators remain active. For SED-MCTS, PostDiag can examine completed expressions and submit revised candidates for reevaluation. Each revision should receive its own valid grammar derivation and attribution before entering the archive or updating structural experience.

3. Direct performance comparison on LSR-Synth.

We compare RSSP, PySR, and PySR + PostDiag on the same LSR-Synth tasks and data splits, with accuracy aggregated over three random seeds. We adapt the RSSP to the non-PDE setting by evaluating candidate fit against the observed responses and estimating the structural reference using only the training data. Both PySR configurations use 65 GP iterations. PySR + PostDiag uses Qwen3.5-27B for editing.

As shown in Table R3.1, adding PostDiag to PySR improves both in-domain and out-of-domain accuracy. PySR + PostDiag achieves the highest accuracy on both metrics among the three methods.

Table R3.1. Comparison with RSSP on LSR-Synth.

MethodID [email protected] (%) ↑OOD [email protected] (%) ↑Mean runtime (s) ↓
RSSP (adapted to LSR-Synth)19.623.3107.2
PySR23.827.18.8
PySR + PostDiag28.731.010.5
AuthorsOpenReview ↗

Author Response to Reviewer CFBY 2/4


📝 W2 / W6 Motivation, Novelty, and the Choice of PFP Features

Thank you for the feedback. We clarify the connection between our motivation and the resulting design.

1. The Introduction presents a connected motivation. Scalar fitness provides limited diagnostic information, as prior work has recognized. Several methods address this by designing richer feedback together with a new search procedure. We investigate a complementary question: can one diagnostic plug-in improve different solvers while preserving their distinctive search mechanisms? Since different solvers employ distinct internal search mechanisms, diagnosis must rely solely on candidate skeletons and observations, with lightweight adapter interfaces enabling integration across solvers. This motivates extracting richer information about the candidate's behavior. Inspired by the clinical workflow of examination → diagnosis → intervention, we use PFP to examine local behavioral patterns, DRP to assess discrepancy and refitting stability, and an LLM editor to execute the prescribed interventions. We will clarify this process more explicitly in the Introduction and Method sections of the revised manuscript.

2. Why level, trend, and roughness? These descriptors address three complementary questions:

  • Level: is the local output magnitude correct? Derivative-based features alone cannot detect an additive offset.
  • Trend: does the candidate vary in the right directions and at the right rates? The local least-squares slope provides a patch-level estimate of the gradient.
  • Roughness: how much behavior remains unexplained by the local affine fit? It summarizes nonlinear variation without estimating a full Hessian.

This gives an ordered description of constant behavior, linear variation, and remaining nonlinear variation, estimated from groups of observations. Curvature is a useful higher-order alternative for the third role; it does not replace level or trend. Our contribution is the use of these patch profiles for a shared diagnostic mechanism, jointly comparing fitted behavior with observations and across bootstrap refits.

3. Relationship to the suggested alternatives. Figure 2b of the submitted text already compares PFP-based feedback with residual-based NMSE feedback through NMSE-only and NMSE+DRP. A gradient estimated by the same local linear fit coincides with our trend feature, making an additional comparison unnecessary. We add an ablation study (Table R3.2) replacing roughness with local curvature while retaining level and trend, to assess whether additional second-order information improves editing performance. Replacing Roughness with Curvature neither improves performance nor causes a noticeable decline. However, using Curvature alone leads to a substantial performance drop. We believe this is because Level and Trend are more direct and informative features, enabling more accurate DRP localization and evidence maps.

Table R3.2. Replacing roughness with local curvature.

SolverPatch featuresID [email protected] (%) ↑ID NMSE ↓
LLM-SR + PostDiagLevel + Trend + Roughness31.02.12E-04
LLM-SR + PostDiagLevel + Trend + Curvature29.53.28E-04
LLM-SR + PostDiagCurvature only26.47.53E-04
PySR + PostDiagLevel + Trend + Roughness28.71.34E-03
PySR + PostDiagLevel + Trend + Curvature27.62.06E-03
PySR + PostDiagCurvature only25.32.41E-03

4. Meaning of “search-independent.” The term distinguishes the diagnostic core from feedback that requires a particular search procedure. With the candidate, data, fitting routine, and random draws fixed, PostDiag computes the same report regardless of which solver supplied the candidate. The adapter returns accepted revisions to the solvers, allowing them to improve subsequent search. Independence concerns how diagnosis is computed; this intended downstream influence is the purpose of integration. We will clarify the scope and definition of this claim in the revised manuscript.

AuthorsOpenReview ↗

Author Response to Reviewer CFBY 3/4

📝 W3 Why the Diagnostic Components Are Needed

Thank you for raising this concern. We explain the design rationale, the need for search-independent diagnosis, and the contribution of each diagnostic component.

1. A connected examination–diagnosis–intervention workflow. Inspired by clinical practice, we designed three stages with distinct, connected roles:

  • PFP performs the examination. NMSE aggregates pointwise errors into a single global score, providing limited information about where and how a candidate's behavior differs from the observations. PFP instead examines local level, trend, and roughness across input-space patches and their stability across bootstrap refits.
  • DRP provides diagnostic localization. It organizes behavioral discrepancy and refitting instability into a two-dimensional risk plane. Their combination distinguishes four risk patterns and assigns an explicit priority: preservation, repair, simplification, or rewriting.
  • The LLM editor performs the intervention. It uses this priority and the patch-wise evidence to revise the skeleton; the revision must pass the audit-error check before returning to the solver.

The stages therefore form a connected progression: behavioral evidence → intervention priority → validated revision.

2. Why diagnosis should be search-independent. Our objective is to equip existing SR methods with richer diagnostic feedback while preserving their distinctive search mechanisms. These methods use different internal states and candidate-generation operators, but share a common interface: a candidate skeleton and observed data. We therefore compute diagnostics through this interface without relying on solver-specific search statistics, and use lightweight adapters to integrate PostDiag with different solvers. This allows the same examination–diagnosis–intervention pipeline to serve multiple solvers. Table 1 of the submitted text demonstrates this reusability across seven LLM-based and GP-based methods.

3. Ablations test the contribution of the components. Within each solver, all variants use the same native search budget, LLM editor, and validation procedure. NMSE-only supplies scalar error; NMSE+DRP builds risk guidance from pointwise errors; Evidence-only supplies PFP evidence without DRP guidance; Full PostDiag combines PFP and DRP. Var-only routing and Bias-only routing are ablations of the two risk axes. In these variants, the evidence map contains only Var or Bias information, and risk diagnosis relies solely on the corresponding axis. For example, under Bias-only routing, both low-discrepancy/low-instability and low-discrepancy/high-instability cases are assigned to Preserve, while both high-discrepancy/low-instability and high-discrepancy/high-instability cases are assigned to Repair. As shown in Table R3.3, Full PostDiag achieves the highest in-distribution accuracy across all three solvers. The comparisons with NMSE + DRP and Evidence-only support combining local PFP evidence with DRP guidance for editing. Full PostDiag also consistently outperforms both single-axis variants, supporting the use of both behavioral discrepancy and instability across refits in diagnosis.

Table R3.3. Component ablation on LSR-Synth. ID [email protected] (%), Qwen3.5-27B.

Feedback variantLLM-SR ↑SR-Scientist ↑PySR ↑
NMSE-only22.526.124.0
NMSE + DRP24.828.225.3
Evidence-only27.131.525.1
Var-only routing23.025.824.3
Bias-only routing25.128.727.4
Full PostDiag31.035.728.7
Reviewer_G7S6OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PostDiag, which diagnoses candidate expressions from symbolic regression solvers by local bias and variance and maps each diagnosis to an LLM edit operation. On LLM-SRBench, PostDiag improves the accuracy and convergence of both LLM-based and GP-based solvers.

Critical strengths

Turning bias and variance diagnostics into explicit edit instructions gives the LLM more targeted feedback than a single error score. This idea could make LLM-guided search more efficient.

Critical weaknesses
  1. KNN becomes unreliable in high-dimensional spaces. This is acceptable for LLM-SRBench, which focuses on low-dimensional problems, but the authors should acknowledge this limitation and discuss a potential solution, such as using interpretable representation learning to reduce the dimensionality before KNN [1].
  2. The variance measure only captures prediction changes after refitting the coefficients of a fixed skeleton. An overfitted expression without refittable coefficients therefore has zero variance and cannot be detected by PostDiag.
  3. The paper normalizes each input dimension but does not state whether the target is normalized. Bias and variance are sensitive to the target scale, so the authors need to clarify this detail.
  4. The paper does not report how often each diagnosis occurs during the search and the acceptance rate of the corresponding edits. It is thus unclear whether all four diagnoses contribute in practice.
  5. The paper does not show that each mapped operation is the best choice. For example, for a candidate with low bias and high variance, the authors should try all four operations to check whether Simplify is indeed the best choice.
  6. In the ablation study, all variants share the same gate, but each variant produces different candidate expressions as the search proceeds, which may trigger different numbers of LLM edits. The search budget may therefore differ across variants.
  7. The LLM does not know which input region each patch corresponds to. Adding the anchor coordinates of each patch to the prompt could provide more informative feedback for editing.
  8. The paper describes the NMSE-based gate but does not give the values of its lower and upper NMSE thresholds. The authors should report these values.
  9. The NMSE+DRP variant infers skeleton adequacy and stability from NMSE-based signals, but the paper does not give a formula for deriving bias and variance from NMSE. The authors should clarify this computation.

[1] H. Zhang et al., "Contrastive Symbolic Regression: Aligned Representations, Adaptive Prediction, and Diverse Ensembles," International Conference on Machine Learning, 2026.

Questions for authors
  1. How does PostDiag handle overfitted expressions that have no refittable coefficients?
  2. Is the target normalized before computing bias and variance?
  3. How often does each diagnosis occur during the search, and what is the acceptance rate of each edit?
  4. For a candidate with low bias and high variance, does Simplify outperform the other three operations?
  5. Is the number of LLM edits the same across ablation variants?
  6. Would adding the anchor coordinates of each patch to the prompt help the LLM locate the input region to edit?
  7. What are the lower and upper NMSE thresholds of the NMSE-based gate?
  8. How are bias and variance derived from NMSE in the NMSE+DRP variant?
7 comments
AuthorsOpenReview ↗

Author Response to Reviewer G7S6 1/4

We thank the reviewer for the positive assessment and constructive suggestions. We clarify the implementation details and propose targeted controls below.


📝 W1 Neighborhood Construction in Higher Dimensions

Thank you for this helpful suggestion. We address it in three parts.

1. Guidance for high-dimensional inputs. We will discuss the limitations of kNN in high-dimensional spaces in the revised manuscript. The dimensionality reduction approach suggested by the reviewer is a feasible solution, and we will also include a discussion of this approach in the revision.

2. Another simple alternative patch constructor. Inspired by the reviewer’s suggestion, we introduced random projection tree blocks for high-dimensional spaces: project observations onto a random direction, split at the median, repeat within each group, and use the resulting small groups as patches. Multiple trees provide overlapping coverage. Each split uses a one-dimensional ordering, avoiding full-space nearest-neighbor ranking. This choice is motivated by RP-tree methods, which can exploit low intrinsic dimensional structure within high-dimensional inputs.

3. Sensitivity to patch construction. To assess PostDiag's sensitivity to patch construction, we conducted additional experiments comparing kNN and random projection trees on LSR-Synth, keeping all other components unchanged and matching the number and approximate size of patches. The two methods achieve comparable performance, suggesting that PostDiag is relatively insensitive to the choice of patch construction method. Thus, patch construction can be adapted to the geometry of the input space while preserving the same diagnostic mechanism. As shown in Table R2.1, when random projection trees replace the default Anchor-kNN patch constructor, PostDiag still improves in-domain accuracy and reduces NMSE for both LLM-SR and PySR relative to their respective base solvers.

Table R2.1. Patch-construction comparison on LSR-Synth. (Qwen3.5-27B, original native budgets and three paired seeds)

SolverPatch constructorID [email protected] (%) ↑NMSE ↓
LLM-SR—22.01.45E-03
LLM-SR + PostDiagAnchor-kNN31.02.12E-04
LLM-SR + PostDiagRandom projection trees28.93.16E-04
PySR—23.82.67E-03
PySR + PostDiagAnchor-kNN28.71.34E-03
PySR + PostDiagRandom projection trees28.29.57E-04

📝 W2 / Q1 Expressions Without Refittable Coefficients

Thank you for identifying this edge case. We clarify the current behavior and examine a targeted extension.

1. How the current pipeline handles it. The reviewer is correct: without refittable coefficients, stability risk is zero, which can miss an overfitting diagnosis. However, coefficient fitting and risk evaluation use different data subsets. An expression that overfits the fitting observations can still exhibit substantial discrepancies on the audit set. For such candidate entering diagnosis, low stability risk and high expressiveness risk lead to Repair, and the LLM can use the evidence map to correct the expression despite the missed overfitting label.

2. A Feasible Solution. To address the reviewer's concern, we propose a simple modification. For candidate skeletons without explicit coefficients, we manually introduce coefficient placeholders before structural diagnosis. For example, x2+sin⁡xx^2+\sin x becomes ax2+bsin⁡x+ca x^2+b\sin x+c, enabling the evaluation of coefficient sensitivity. Even if the ground-truth equation is x2+sin⁡xx^2+\sin x , the placeholders can recover the original expression by fitting a=b=1a=b=1 and c=0c=0. We believe this provides a potentially feasible solution to the edge case raised by the reviewer. To this end, we conducted additional experiments to evaluate the effectiveness of this minor modification.

As shown in Table R2.2, adding coefficient placeholders had a modest effect on overall performance. In a three-seed experiment using Qwen3.5-27B, candidates without refittable coefficients were rare among those that passed the diagnostic gate after deduplication (0/4,119 for LLM-SR, 13/1,736 for SR-Scientist, and 3/4,296 for PySR). Adding coefficient placeholders nevertheless enables us to assess stability under refitting in this edge case.

Table R2.2. Coefficient placeholders for coefficient-free candidates. Qwen3.5-27B, same budgets in the submitted text and three seeds.

SolverTreatment of coefficient-free candidatesID [email protected] (%) ↑NMSE ↓
LLM-SR + PostDiagNo added coefficient placeholders31.02.12E-04
LLM-SR + PostDiagAdd coefficient placeholders31.52.29E-04
PySR + PostDiagNo added coefficient placeholders28.71.34E-03
PySR + PostDiagAdd coefficient placeholders28.41.22E-03
AuthorsOpenReview ↗

Author Response to Reviewer G7S6 2/4

📝 W3 / Q2 Target Normalization

We apologize for not stating this clearly. The target is normalized before diagnostic evaluation. PFP compares observations and candidate predictions on the same normalized output scale, including the bootstrap-refitted predictions. Consequently, the diagnostic scores are not computed in the raw target units. We will explicitly describe the normalization in the revised implementation details.


📝 W4 / Q3 Diagnosis Frequencies and Edit Acceptance

Thank you for this helpful suggestion. We selected three different base solvers (LLM-SR、PySR、SR-Scientist) and report the frequency of each diagnostic outcome and the acceptance rate of its suggested modifications after integrating PostDiag (PD) in Table R2.3. The counts are based on editor invocations during search and cover all 129 tasks for each solver. Accepted edits, unchanged returns, and rejected edits are mutually exclusive, with invalid proposals counted as rejected. Each percentage is calculated independently from its integer numerator and denominator and rounded half-up to one decimal place; displayed percentages may sum to 99.9% or 100.1%. We report unchanged outputs separately.

As shown in Table R2.3, all four diagnostic states occurred during search with each solver. High discrepancy with low instability accounted for about half of each solver’s diagnoses, and most proposed Repair edits in this state passed audit validation. The state associated with Simplify was the least frequent for all three solvers. Note: A high frequency of Unchanged outcomes is expected for Low discrepancy / low instability-type diagnoses, as the editor is instructed to Preserve/minimal edit these candidates.

Table R2.3. Diagnosis and intervention statistics.

Solver and DRP stateOperationCountFrequency (%)Accepted edits (%)Unchanged (%)Rejected (%)
LLM-SR+PD — Low discrepancy / low instabilityPreserve/minimal edit137527.325.451.023.6
LLM-SR+PD — High discrepancy / low instabilityRepair250249.863.08.228.8
LLM-SR+PD — Low discrepancy / high instabilitySimplify4288.566.411.921.7
LLM-SR+PD — High discrepancy / high instabilityRewrite72414.455.25.439.4
SR-Scientist+PD — Low discrepancy / low instabilityPreserve/minimal edit106426.435.949.414.7
SR-Scientist+PD — High discrepancy / low instabilityRepair202150.163.07.329.7
SR-Scientist+PD — Low discrepancy / high instabilitySimplify3919.771.910.517.6
SR-Scientist+PD — High discrepancy / high instabilityRewrite55813.864.54.131.4
PySR+PD — Low discrepancy / low instabilityPreserve/minimal edit121226.336.550.513.0
PySR+PD — High discrepancy / low instabilityRepair225048.964.48.327.3
PySR+PD — Low discrepancy / high instabilitySimplify49710.862.07.230.8
PySR+PD — High discrepancy / high instabilityRewrite64414.055.96.237.9
AuthorsOpenReview ↗

Author Response to Reviewer G7S6 3/4

📝 W5 / Q4 Whether the Mapped Operation Is Appropriate

Thank you for the suggestion. The LLM editor uses both the assigned operation and the evidence map. Even when the assigned operation is inappropriate, the editor may still find a useful revision based on the original evidence. To examine the effects of each, we compare four operations on the same set of candidates using either their original evidence maps or randomly substituted maps.

Given the time and computational constraints of the rebuttal, we focused on the most frequent diagnostic category in Table R2.3: high discrepancy and low instability. For the comparison in Table R2.4, we sampled 500 candidates from this category. For each candidate, we paired each of the four operations with the candidate’s original evidence map and with a single randomly substituted map shared across all four operations. We generated one edit proposal per combination. All conditions used the same coefficient fitting and audit acceptance procedures. The prompt included the operation and evidence map and omitted the numerical DRP coordinates. On an independent evaluation set, we computed ΔNMSE as the NMSE after editing minus the NMSE before editing. As shown in Table R2.4, Repair achieves the lowest mean ΔNMSE under both evidence conditions, demonstrating that matched diagnosis and intervention effectively improve performance. Repair further reduces NMSE when provided with the candidate's own evidence map.

Table R2.4. Four operations × two evidence conditions for 500 High Discrepancy / Low Instability candidates.

OperationEvidence mapMean eval ΔNMSE
Preserve/minimal editOriginal-1.239E-04
Repair (matched)Original-3.461E-01
SimplifyOriginal+1.382E-02
RewriteOriginal+3.628E-02
Preserve/minimal editRandom+4.203E-03
Repair (matched)Random-6.239E-02
SimplifyRandom+8.334E-02
RewriteRandom+1.219E-01

ΔNMSE as the NMSE after editing minus the NMSE before editing.


📝 W6 / Q5 Editing Calls Across Ablations

Thank you for raising this point. All ablation variants use the same gate for triggering edits. However, edits can change which candidate expressions the solver generates later in the search, so the actual number of editing calls varies across variants.

Table R2.5 reports the average numbers of native LLM calls and editing calls per run for each ablation variant. NMSE-only made the fewest editing calls for SR-Scientist. Compared with Evidence-only, Full PostDiag averaged 13.0 vs. 9.4 editing calls for LLM-SR, 10.4 vs. 10.2 for SR-Scientist, and 11.9 vs. 14.3 for PySR. Thus, in this experiment, full PostDiag achieves the best performance despite not having the largest computational budget.

Table R2.5. Mean LLM calls by feedback variant.

SolverFeedback variantMean native LLM callsEditor calls
LLM-SRNMSE-only100.012.7
LLM-SRNMSE + DRP100.013.8
LLM-SREvidence-only100.09.4
LLM-SRFull PostDiag100.013.0
SR-ScientistNMSE-only191.77.1
SR-ScientistNMSE + DRP215.811.8
SR-ScientistEvidence-only237.910.2
SR-ScientistFull PostDiag223.110.4
PySRNMSE-only-12.2
PySRNMSE + DRP-15.7
PySREvidence-only-14.3
PySRFull PostDiag-11.9

📝 W7 / Q6 Adding Patch Coordinates

We apologize for the confusion caused by the abbreviated prompt template. In our implementation, each patch's coordinate ranges are encoded in its patch ID and supplied to the editor as part of the evidence map. For a three-dimensional input, an example is as below. The editor therefore receives the patch's input-space location. Figure 4 uses the generic heading “Patch ID” without showing its range-valued contents; we will add an explicit example in the revision.

Patch ID: {x1: [0.20, 0.40], x2: [-1.00, -0.50], x3: [2.00, 2.50]}
Reviewer_7DAXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that the overreliance on scalar fitness values as a limitation of existing SR algorithms. The paper thus proposes PostDiag, a diagnosis method that can augment existing symbolic regression (SR) algorithms, e.g., LLM-based and GP-based algorithms. PostDiag takes a candidate equation, does a patch-wise profiling of the local mean, local linear slope and roughness at each patch. These are used to calculate 2 metrics for diagnosis and the 2 metrics serve as axis on a 2D plot, with the thresholds of expressiveness risk (0.25) and stability risk (0.15). The resultant 4 quadrants from the threshold is the 'diagnosis' and an LLM (independent from the one used in the chosen LLM-based and GP-based algorithm, if any) is used to modify the candidate equation based on the diagnosis. The modified candidate is returned if the performance is better, else the original candidate is returned. The paper evaluates PostDiag on LLM-based algorithms (direct LLM, SGA, LLM-SR, LaSR, SR-Scientist) and GP-based algorithms (PySR, gplearn) and finds that performance improves across the algorithms.

Critical strengths
  1. The paper proposes a non-invasive method that can easily be added to most SR-algorithms. This makes it relevant to a broad range of SR literature.

  2. The paper is written in a clear and organized manner.

Critical weaknesses
  1. The paper omits a major part of SR literature on semantic symbolic regression which tackles the issue of scalar fitness and has dealt with vectors instead of scalar fitness for years. In fact, the paper actually cites some semantic SR works, but did not engage with the non-scalar aspect of the works. The work should explain and justify why the PFP and the 2 DRP axes are preferable to established semantic representations and semantic error signals in semantic SR literature.

  2. There is a lack of discussion and reproducible process of the selection of multiple numbers. For instance, how are the nonnegative weights in "skeleton expressiveness risk" of (0.4, 0.4, 0.2) chosen? What does it mean to say "In practice, we assign a smaller weight to roughness because it is more sensitive to local fluctuations in finite patches.", what does "smaller" mean exactly?

  3. The empirical evidence is weakened because no error bars are provided, it is hard to tell if the errors bars are overlapping. No significance tests are conducted as well. Importantly, the paper admits only three seeds are run. From my knowledge, the SR algorithms are not stable, with each of the 3 seeds showing potentially drastically varying results, thus more seeds should be required to provide stronger evidence of the claims.

  4. Given the motivation provided by the paper, there are no comparison with SR algorithms that use non-scalar fitness. The paper should include some of these methods in the comparison, such as some instances of semantic GP SR.

Questions for authors
  1. Please address the weaknesses above.

  2. In DRP, how are the threshold values of 0.25 and 0.15 selected? What exactly are the preliminary experiments on a small held-out subset of the tasks? Can the paper provide information on the process?

  3. The improvements PostDiag provides come at the cost of additional compute resources. Can the paper give a comment on what it would be like if the original methods are given the same amount of compute resource (e.g., runtime, or CPU/GPU usage).

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer 7DAX 1/3

We thank the reviewer for this valuable suggestion. We will include additional background and discussion on Semantic SR in the revised manuscript. Below, we clarify the relationship and key differences between our work and Semantic SR in several points.

1. Relationship to semantic SR. Thank you for highlighting this connection. We distinguish PostDiag from th semantic GP methods in two respects.

  • Behavioral representation and diagnosis. Geometric Semantic GP and Semantic Backpropagation represent semantics through pointwise outputs or desired intermediate outputs, and use it to construct offspring through semantic search operators. PostDiag constructs input-space patches and profiles local level, trend, and roughness, independently of the solver's search operators. It uses discrepancies from observations and variability across bootstrap refits to position each candidate on the Diagnostic Risk Plane, guiding targeted preservation, repair, simplification, or rewriting.
  • Integration across solvers. The cited methods use semantics to design GP crossover and mutation operators. PostDiag provides a shared interface requiring only a candidate skeleton and observed data. The same diagnostic and editing pipeline connects to LLM-based and GP-based solvers through lightweight adapters, while retaining their native search mechanisms. Its diagnosis therefore does not require solver-specific crossover, mutation, or generation rules.

2. Why PFP and DRP. Compared with the cited semantic GP methods, PostDiag adds explicit diagnosis to behavioral guidance: what differs, how stable it is, and which intervention is appropriate. This follows the clinical sequence of examination, diagnosis, and treatment.

(1) PFP provides interpretable local evidence. Semantic GP uses pointwise output relationships to guide offspring construction. PFP summarizes behavior within input-space patches through level, trend, and roughness, identifying both where discrepancies occur and whether they involve local offsets, trends, or deviations from the local linear trend. This gives the editor explicit behavioral properties to address.

(2) DRP distinguishes problems requiring different interventions. Similar fitting errors can conceal different risks. DRP combines behavioral discrepancies with instability across bootstrap refits: stable mismatch motivates repair; low discrepancy with high instability motivates simplification; high discrepancy and instability motivate rewriting; low risks motivate preservation. This provides an explicit basis for choosing how to intervene.

(3) The intervention mechanism is shared across solvers. The cited methods encode semantic guidance in GP crossover and mutation operators. PostDiag supplies a risk-selected operation and patch-wise evidence to a shared LLM editor, enabling the same diagnostic intervention across GP-based and LLM-based solvers through lightweight adapters. Revisions are accepted only after an audit-error check.

3. Integrating PostDiag into Semantic GP. In fact, PostDiag, as a post-diagnosis plugin, is compatible with Semantic GP. As a GP-based solver, Semantic GP can be easily integrated with PostDiag. We compare each semantic GP solver with its PostDiag-augmented counterpart (Table R1.1). Such integration not only provides richer diagnostic feedback across different solvers but also preserves their inherent strengths, which is one of the core ideas of our work. Both solvers already use semantic feedback to guide their search. As shown in Table R1.1, adding PostDiag improves both ID and OOD [email protected] and reduces NMSE for both solvers.

Table R1.1. The comparison between Semantic GP and Semantic GP+PostDiag on LSR-Synth tasks

MethodID [email protected] (%) ↑OOD [email protected] (%) ↑NMSE ↓
Geometric semantic GP4.13.63.52E-02
Geometric semantic GP + PostDiag6.24.71.88E-02
GP with semantic backpropagation5.44.13.45E-02
GP with semantic backpropagation + PostDiag7.85.22.70E-02
AuthorsOpenReview ↗

Author Response to Reviewer 7DAX 2/3

📝 W2 / Q2 Reproducible Selection of Weights and Thresholds

We thank the reviewer for this suggestion. Below, we clarify the rationale behind our hyperparameter choices and present additional ablation studies to evaluate performance under different hyperparameter settings.

1. Feature weights. The three feature weights are adjustable hyperparameters. Roughness captures behavior remaining after removing a patch's overall level and linear trend. On small patches, this residual component can be disproportionately affected by outliers and observation noise. We therefore give roughness a smaller weight, setting (ωμ,ωβ,ωρ)=(0.4,0.4,0.2)(\omega_\mu,\omega_\beta,\omega_\rho)=(0.4,0.4,0.2). This retains higher-order behavioral information while reducing its influence relative to level and trend. The selected weights remain fixed across solvers and domains.

2. Threshold Selection. We generated 200 synthetic skeleton–data pairs (which are different from those in Section 4.3) using the same generation procedure in Section 4.3. By examining their Skeleton Expressiveness Risk and Skeleton Stability Risk values, we selected the thresholds (τB,τV)=(0.25,0.15)(\tau_B,\tau_V)=(0.25,0.15), which were then fixed for all subsequent experiments. We apologize for not clearly explaining this calibration procedure and will clarify the rationale for threshold selection in the revised manuscript.

3. Additional Ablation Studies on Weights and Thresholds. We conducted additional experiments to assess the effects of feature weights and thresholds on performance, varying each independently while keeping the rest of the pipeline unchanged. As shown in Tables R1.2 and R1.3, the default feature weights and DRP thresholds perform best on average across LLM-SR and PySR in their respective comparisons. We therefore retain these settings as shared defaults for both solvers.

Table R1.2. Sensitivity to feature weights. DRP thresholds are fixed at (0.25, 0.15). Results are reported as ID [email protected] (%).

Level weightTrend weightRoughness weightLLM-SR + PDPySR + PD
0.400.400.2031.028.7
0.330.330.3329.528.9
0.450.450.1030.725.1

Table R1.3. Sensitivity to DRP thresholds. Feature weights are fixed at (0.40, 0.40, 0.20). Results are reported as ID [email protected] (%).

Expressiveness thresholdStability thresholdLLM-SR + PDPySR + PD
0.250.1531.028.7
0.200.1526.626.1
0.300.1528.929.2
0.250.1027.929.5
0.250.2027.430.0

📝 W3 Statistical Reliability Across Random Seeds

Thank you for emphasizing run-to-run variability. We use ten paired seeds for Direct LLM, LLM-SR, SR-Scientist, and PySR, each with and without PostDiag. These cover LLM search, agentic search, and GP search. We report the mean and sample standard deviation across seeds and will add them to the corresponding comparisons in the final manuscript. The ten-seed evaluation in Table R1.4 shows clear gains in mean ID [email protected] with PostDiag across all four solvers. The standard deviations across seeds are comparable in magnitude with and without PostDiag.

Table R1.4. Ten-seed evaluation on all 129 LSR-Synth tasks. Results are reported as ID [email protected] (%).

MethodMean (3 seeds)Mean (10 seeds)SD (10 seeds)
Direct LLM2.62.871.42
Direct LLM + PostDiag4.45.271.86
LLM-SR22.022.402.44
LLM-SR + PostDiag31.030.232.71
SR-Scientist25.627.523.41
SR-Scientist + PostDiag35.734.344.14
PySR23.824.653.48
PySR + PostDiag28.728.912.56
AuthorsOpenReview ↗

Author Response to Reviewer 7DAX 3/3

📝 Q3 Performance Under Matched Computational Budgets

We thank the reviewer for this question.

1. Comparison with higher-budget base solvers. Appendix H.3 specifically examines whether allocating substantially more search computation to the original solver reproduces PostDiag's gains. The High setting increases LLM-SR from 100 to 1,000 LLM calls, LaSR from 20 to 40 iterations, SR-Scientist from 10 to 40 iterations, and PySR from 65 to 130 iterations. All reported runtimes include PostDiag's diagnosis, refitting, and editing costs.

Even against these higher-budget baselines, low-budget LaSR+PostDiag achieves 21.4% accuracy in 595.4 s, versus 15.5% in 1118.4 s for high-budget LaSR. PySR+PostDiag achieves 28.7% in 10.5 s, versus 28.4% in 15.5 s for high-budget PySR. LLM-SR+PostDiag and SR-Scientist+PostDiag also approach their high-budget baselines' accuracy in substantially less time (Table R1.5). These comparisons show that additional native search does not consistently reproduce the benefit of diagnostic feedback, providing evidence against attributing all gains simply to more computation.

2. Comparison at a common wall-clock cutoff. Within each solver, both variants use identical hardware and serving concurrency and share the same per-task wall-clock cutoff. Timing includes search, diagnosis, and editing. We save the best expression after each round and score the last one saved by the cutoff. A round already in progress may finish after the cutoff. As shown in Table R1.6, PostDiag improves both in-domain and out-of-domain accuracy for LLM-SR and PySR at the same per-task cutoff as their respective baselines.

Table R1.5. Budget-efficiency comparison. Qwen3.5-27B is used for LLM inference. Budget specifies native base-solver search; all additional PostDiag computation is included in Time.

SolverBudgetID [email protected] (%) ↑NMSE ↓Time (s)
LLM-SRLow: 100 LLM calls22.01.45E-03167.5
LLM-SR + PostDiagLow: 100 LLM calls31.02.12E-04191.7
LLM-SRHigh: 1,000 LLM calls34.13.65E-041603.4
LaSRLow: 20 outer iterations13.41.06E-02559.2
LaSR + PostDiagLow: 20 outer iterations21.48.62E-04595.4
LaSRHigh: 40 outer iterations15.58.13E-031118.4
SR-ScientistLow: 10 outer iterations, ≤25 turns each25.61.12E-03648.0
SR-Scientist + PostDiagLow: 10 outer iterations, ≤25 turns each35.71.92E-04702.2
SR-ScientistHigh: 40 outer iterations, ≤25 turns each37.72.23E-042547.1
PySRLow: 65 iterations23.82.67E-038.8
PySR + PostDiagLow: 65 iterations28.71.34E-0310.5
PySRHigh: 130 iterations28.41.76E-0315.5

Table R1.6. Performance under a common per-task wall-clock cutoff.

MethodWall-clock cutoff (s)ID [email protected] (%) ↑OOD [email protected] (%) ↑
LLM-SR20023.021.7
LLM-SR + PostDiag20031.827.9
PySR1224.826.4
PySR + PostDiag1228.429.2

Dear Reviewer,

Thank you again for your time and thoughtful feedback. We have added experiments and provided detailed responses to address the points you raised. With the discussion period coming to a close, we would be grateful if you could take another look when your schedule allows. Please let us know if any concerns remain or if further clarification would be helpful.

We appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.