acceptodds
Under review as a conference paper at ICLR 2027

Matching Numerical Supervision to the Output Interface in Symbolic Regression

Abstract

Neural symbolic regressors pass coefficient placeholders or predicted constants to numerical solvers. We show that this output interface can reverse conclusions about numerical supervision. In scale-augmented Flash-ANSR continuation, role targets favor placeholder requests, whereas literal targets favor constant requests. Independent confirmation on 256 newly generated families and three training seeds yields interactions of and percentage points at and , with both multiplicity-adjusted intervals excluding zero. Exploratory paired comparisons show that the interaction varies with continuation exposure. Balanced 50/50 continuation provides a shared-checkpoint training response: it improves the weaker pairings by and points, reaching observed coverage within points of the respective specialists. Controlled interventions distinguish boundary supervision, numerical context, and optimizer initialization. Fixed-history probes locate immediate format steering at coefficient entry; fixed-pool replays show that predicted coefficients carry useful initialization information across candidate expressions. Together, these findings identify the generation–refinement interface as a design axis for numerical supervision and support balanced exposure when both output capabilities are required.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
symbolic regression, numerical supervision, output interfaces, numerical optimization, multi-format training

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_GVKsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how numerical supervision interacts with the requested output format in neural symbolic regression. Starting from pretrained Flash-ANSR checkpoints, it compares continuation training with literal numerical targets and role-based targets, and evaluates each under placeholder and constant output requests. A confirmatory evaluation on 256 newly generated expression families and three training seeds finds an interaction between training targets and output requests at two predictive-accuracy thresholds. The paper also studies a balanced 50/50 training mixture and conducts diagnostic interventions on numerical-token supervision, decoder context, and optimizer initialization. Its main contribution is an empirical study of the generation–refinement interface and a training recipe for retaining both output capabilities.

Critical strengths
  1. A credible empirical interaction with confirmation on newly generated families. The study evaluates both target formats under both output requests, rather than drawing conclusions from a single interface. Confirmation on newly generated families, multiple training seeds, paired comparisons, and multiplicity-adjusted intervals strengthens the evidence that the relative performance of the two continuation-training schemes depends on the requested interface within this setting.

  2. Informative diagnostic experiments and reporting of inconclusive results. The distinction between boundary supervision, value-byte supervision, numerical conditioning history, and optimizer initialization is useful. In particular, the boundary/value-byte controls and fixed-parent initialization experiments help identify different ways numerical information enters the pipeline. The paper also reports unresolved structural results, nonfinite outcomes, and inconclusive public-data comparisons, which makes its empirical limitations more assessable.

Critical weaknesses
  1. Limited explanatory insight beyond training-format specialization. Literal and Roles jointly change instructions, target representations, and numerical supervision, leaving their individual contributions unclear. The diagnostic experiments identify effects on generation and initialization, yet insufficiently explain how these effects produce the ranking reversal. The scientific insight beyond matching training formats to output requests remains underdeveloped. The dense presentation and scattered qualifications make it difficult to trace how the diagnostic experiments support the central argument.

  2. Unclear practical value of native dual-interface support. The practical need for native dual-interface support and its benefits over matched single-interface workflows or simple output conversion remain unclear. Mixed-request search offers no confirmed advantage over repeated constant requests, while the supervision–interface interaction remains inconclusive on public datasets. These limitations weaken the motivation for adopting the proposed approach.

Questions for authors
  1. Which results distinguish numerical-supervision effects from specialization to training formats?

  2. How do the diagnostic findings explain why scale-augmented continuation changes the supervision–interface interaction?

  3. What do the common-fitting controls establish about the solver’s contribution to the ranking reversal?

  4. Which practical scenarios benefit from native dual-interface support over matched single-interface workflows or simple output conversion?

  5. How do the inconclusive public-data results affect the scope of your conclusions and recommendation for balanced training?

  6. Some experiments lack comparisons with relevant methods; could these be provided?

Reviewer_VfXcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Pre-trained symbolic regression networks learn coefficient using placeholders or embeded tokens (literal constants). This paper investigates based on Flash-ANSR, a pre-trained model that supports both coefficient representations. The major finding is that when you fine-tune a symbolic regression model using placeholder or literal constants, you should match it in inference. The results compare different combination of coefficient settings in multiple seedings.

Critical strengths

This paper investigated coefficient settings in tons of experiments using multiple datasets.

Critical weaknesses

This paper is poorly written. Many terms (for example, "role targets", "literal targets") are not explained. The logic is hard to follow.

This paper's major finding seems trivial. Fine-tuning using placeholder or literal constants naturally makes inference using matching ones better.

Questions for authors
  1. Is the paper's main contribution just finding that inference should match fine-tuning in constant representation?

  2. What are the scenarios that both representations of constants are needed and balanced fine-tuning has a clear advantage?

  3. Do you have any innovations in fine-tuning except the constant representation setting and if so, any evidence that they bring significant improvement?

Reviewer_oK6cOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Sorry for the wrong review post. This paper examines the interaction between numerical supervision and the output format requested at inference time in neural symbolic regression. It compares Literal targets, which directly specify numerical values, with Role targets, which represent free coefficients using placeholders, and shows that performance rankings after numerical optimization reverse depending on the requested format. It also proposes continued training with a mixture of both formats and conducts interventions on the loss for numerical prediction, generation histories containing numerical values, and optimization initializers.

Critical strengths

Crossing training formats with inference-time output formats and comparing performance after numerical optimization has some empirical value. The paper confirms the ranking reversal within the studied setting using independently generated synthetic data and multiple training seeds. Failures, computational costs, and additional control experiments are reported in relatively extensive detail, providing information for assessing the results and their limitations.

Critical weaknesses

The benefits of matching training and inference formats, and of supporting both formats through mixed training, are readily anticipated, leaving the conceptual contribution of the central finding insufficiently clear. Kamienny et al. (NeurIPS 2022) already investigated the benefits of training with numerical values and initializing optimization with predicted coefficients, so the paper needs to clarify what additional design insights it provides.

A more important concern is that the main synthetic evaluation uses the same rules for generating coefficients as training, so the benefits of predicted initializers may be explained by adaptation to those rules. Even when expression structures are unseen, knowledge of coefficient signs, typical values, and distributional shapes remains useful; structural separation alone therefore does not address this concern. To determine how strongly the usefulness of predicted initializers depends on the shared rules, evaluation should change the distributions used to generate coefficients in the base expressions.

Questions for authors
  1. What is the central insight beyond prior work?

Although the persistence of the ranking reversal after numerical optimization is not self-evident, what makes it a new finding beyond the effects of matching training and inference formats? Please clarify what this paper adds to the benefits of numerical supervision and predicted initializers established in prior work. Does this insight yield concrete design guidelines that apply beyond the configurations studied here?

  1. Do the benefits of predicted coefficients persist when the coefficient distributions of the base expressions change at evaluation time?

With the trained model held fixed, do predicted initializers still improve performance when evaluation uses different rules for generating coefficients in the base expressions? Separately from rescaling input and output coordinates, please evaluate changes in coefficient signs, typical magnitudes, and distributional shapes. Within each evaluation condition, please compare initialization strategies using the same candidate pool and matched optimization budgets to establish how strongly the benefit depends on agreement with the rules used during training.

  1. To what extent do prompts and the different uses of numerical information explain the remaining performance differences?

Since format compliance is already high in Literal/F, simple noncompliance appears insufficient to explain the performance gap under F requests. Could you compare prompts under common output constraints, or retain predicted coefficients as initializers while sequentially replacing them with placeholders in the history used for subsequent generation? Given that correct numerical values remain in the history supplied through teacher forcing even when the numerical loss is removed, please clarify which conclusions each intervention supports regarding instructions, prediction loss, numerical history, and initialization.

1 comment
AuthorsOpenReview ↗

Response to oK6c (Review Mismatch)

Thank you for reviewing for ICLR 2027. We think this review may have been posted to the wrong submission by mistake.

Your review discusses governing-equation identifiability using a consistency map T, the null space of a physics operator, a neural-network tangent space, and an uncertainty-ratio elbow rule for selecting a constraint weight lambda (citing Miao et al., 2011; He et al., 2024; Bona-Pellissier et al., 2022). None of these methods or weight-selection experiments are in Submission 16331.

Our paper studies symbolic regression, specifically how Literal, Roles, and Balanced numerical supervision interact with template versus constant output requests, scale augmentation, and solver initialization.

Could you please check your assigned papers and update the review for Submission 16331 when you have a chance? We have also uploaded our revised paper and are happy to answer any questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.