acceptodds
Under review as a conference paper at ICLR 2027

BiTempo: Adaptive Two-Timescale Latent Inference with Language Models for Scientific Equation Discovery

Abstract

Scientific equation discovery requires searching over symbolic structures whose quality is revealed only after their constants are fitted to data. Language-model-based methods typically carry feedback forward as example programs or textual memory, without explicitly separating persistent structural preferences from local corrections in their search state. We propose BiTempo, a test-time latent inference framework that accumulates numerical feedback in factorized global and local distributions over generation preferences. The global block encodes variables, operators, and composition, while the local block encodes refinement directions informed by training residuals. Sampled latents condition program generation and compatible-parent retrieval; each evaluated equation contributes evidence for distribution updates, with the local block adapting faster than the global one. A lightweight utility surrogate ranks latent proposals before language-model calls, without fine-tuning the language model. On the four LSR-Synth domains of LLM-SRBench, BiTempo with 400 candidates improves domain-averaged Acc\@0.01 over the strongest baseline from to with Llama-3.1-8B and from to with Qwen3-4B. On measured stress–strain data, it reduces ID and OOD NMSE by and relative to the strongest baseline. Ablations show that removing distribution updates or collapsing the two timescales lowers ID accuracy from to and , respectively.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Scientific equation discovery, Language models, Test-time inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HmovOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BITEMPO, a test-time latent inference framework for scientific equation discovery, aiming to improve how LLM-based symbolic regression methods leverage historical numerical feedback during iterative search. Specifically, the method constructs a latent search space consisting of global structural preferences and local refinement preferences. By maintaining two probability distributions that are updated at different frequencies, the framework guides structural exploration and local refinement separately. In addition, the authors introduce a lightweight utility surrogate based on ridge regression to select promising latent proposals before invoking the language model.

Critical strengths
  1. The paper addresses a practically meaningful research problem. Effectively leveraging historical numerical feedback to guide subsequent equation generation remains an important challenge in LLM-based scientific equation discovery.
  2. BITEMPO adapts its search strategy through test-time updates of latent distributions without modifying the language model parameters. This makes it potentially suitable for integration into existing LLM-based symbolic regression frameworks as a separate search controller.
Critical weaknesses
  1. The methodological novelty remains somewhat limited. Although BITEMPO employs explicit probability distributions to represent search preferences, the paper does not sufficiently distinguish this approach from existing adaptive search methods. A comparison with a conventional Cross-Entropy Method (CEM) operating in the same search preference space would help clarify the benefits of the proposed framework.
  2. The effectiveness of the utility surrogate is not sufficiently evaluated. The authors employ a ridge-regression-based utility surrogate to select promising generation preferences. However, the paper does not directly evaluate the predictive accuracy of this surrogate, for example, by reporting the correlation between predicted and observed utilities of latent proposals.
  3. Performance does not consistently improve with a larger candidate budget. In Table 1, under the Llama-3.1-8B setting, [email protected] on the Material subset decreases from 68.00% with BITEMPO-200 to 64.00% with BITEMPO-400.
  4. The effectiveness of the two-timescale design is not sufficiently supported by the ablation experiments. Although the paper compares BITEMPO against a Single-Timescale Updates variant, the existing experiments do not conclusively demonstrate the necessity of using different update frequencies for global and local preferences. In particular, the exact update interval used in the Single-Timescale baseline is not clearly specified. Additional experiments with different global/local update intervals, such as (8, 8), (16, 16), (32, 32), and the reversed (8, 32) configuration, would help isolate the effect of timescale separation.
Questions for authors
  1. Given that the 40-dimensional latent space consists of manually predefined generation preferences, how does BITEMPO fundamentally differ from existing adaptive search methods? What specific advantages does the proposed latent distribution representation provide?
  2. How accurately does the ridge-regression surrogate predict the actual utility of latent proposals? Have the authors evaluated the correlation between predicted and observed utilities?
  3. What exact update interval is used for the Single-Timescale baseline? Have the authors evaluated alternative update schedules, such as (8, 8), (16, 16), (32, 32), and the reversed (8, 32) configuration? How can the reported improvement be attributed specifically to timescale separation rather than update frequency?
  4. In Table 1, why does increasing the candidate budget from 200 to 400 correspond to a reduction in [email protected] on the Material subset? Can the authors analyze whether this result is due to stochastic search behavior or the final candidate selection mechanism?
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer Hmov (1/3)

Dear Reviewer Hmov:

Thank you for reviewing our work. We respond to each point below.

Preference space and CEM controls (W1, Q1)

BiTempo lets local preferences respond quickly to feedback while keeping the global search distribution fixed between updates. BiTempo keeps separate distributions for structural and refinement preferences. It updates them at different intervals using the same candidate evaluations. The CEM control updates all coordinates together. BiTempo can update the local distribution while keeping the global distribution fixed. Measured residuals guide the local updates.

We use the same sampled latent to guide generation and choose compatible parents. Previously evaluated candidates provide reference programs. Their preferences and scores also guide future proposals. CEM fits the distribution, and a diagonal Gaussian represents its state. The surrogate ranks latent proposals before generation, and retrieval supplies previously evaluated programs. BiTempo connects these steps through the preference state. Candidate feedback updates its global and local distributions at different rates, and each sampled latent guides both generation and retrieval. We have revised the manuscript to clarify these methodological contributions in lines 186–190 and Appendix H.1 (lines 1245–1259).

We define the meaning of each preference coordinate by hand. The 40-dimensional space gives us explicit controls over generation. Its 30-dimensional global block represents variable, operator, interaction, and complexity preferences. Its 10-dimensional local block represents residual responses and edit preferences. Candidate feedback changes the means and scales of their distributions during search. These changes shift the sampling probabilities, adjust the range of exploration, and control how quickly structural and refinement preferences respond. We added this explanation of the preference coordinates to Appendix H.1 (lines 1259–1274).

BiTempo has higher ID and OOD accuracy than Standard CEM and Joint-CEM in the same preference space. We use the same generation interface, numerical evaluation code, candidate budget, and final selection rules for all methods. Standard CEM fits the distribution to elites from the current generation. Joint-CEM keeps the other mechanisms and updates all coordinates every 8 feedback observations.

Table R9: CEM comparison in the same preference space (60 ID tasks, 32 OOD tasks).

MethodID [email protected] (%)OOD [email protected] (%)
Standard CEM31.6753.13
Joint-CEM28.3353.13
BiTempo45.0059.38

BiTempo has the highest ID/OOD accuracy. Relative to Standard CEM, ID/OOD accuracy increases by 13.33/6.25 percentage points, respectively. Relative to Joint-CEM, the increases are 16.67/6.25 percentage points, respectively. We added this CEM comparison as Table 12 in Appendix H.1 (lines 1279–1295).

Update schedules and counts (W4, Q3)

BiTempo has higher OOD accuracy than the synchronous control when both use the same number of block updates. The global block controls structural preferences. The local block controls residual and edit preferences. The global parameters stay fixed between global updates. Each round can still sample a different latent and choose different parents, so equation structures can change. We revised lines 57–67 of the manuscript to clarify this distinction.

We set K_g=32 and K_l=8 to update local preferences more often than global preferences. This lets local preferences respond faster to residual and edit feedback. Both blocks use the same candidate history. We count each interval in latent feedback observations. Once we have enough samples and valid elites, each global period has four opportunities for a local update. We added this explanation of the update intervals to lines 246–249 of the manuscript.

Table R1 compares the default (32,8), synchronous (16,16), and reversed (8,32) schedules, along with (13,13), which matches the total number of block updates.

Table R1: Effect of update schedules on OOD performance.

ConditionOOD [email protected] (%)Mean OOD NMSE
BiTempo (32,8)59.388.93019
Synchronous blocks (16,16)56.251136.98
Reversed blocks (8,32)56.25472.833
Count-matched synchronous blocks (13,13)53.130.35714

We added this schedule comparison as Table 13 in Appendix H.2 (lines 1297–1316).

U_g and U_l count updates to the global and local distributions. E counts events that update at least one block. Updating both blocks at once counts as one event and two block updates.

AuthorsOpenReview ↗

Author Response to Reviewer Hmov (2/3)

Table R2: Actual update counts.

ConditionU_gU_lUpdate events ETotal block updates U_g+U_lCoordinate-update volume 30U_g+10U_l
BiTempo (32,8)6242430420
Count-matched synchronous blocks (13,13)15151530600

We added the actual update counts as Table 14 in Appendix H.2 (lines 1317–1332).

The default schedule has the highest OOD accuracy. It updates the local block more often while using the same total number of block updates as the matched control. The matched control has lower mean NMSE. The two metrics therefore favor different schedules.

Surrogate accuracy and component ablations (W2, Q2)

The surrogate predicts Qwen utility better than the median baseline, and removing its ranking in a separate ablation lowers ID and OOD accuracy.

Prediction evaluation. Mean Pearson r is 0.7255, mean Spearman ρ is 0.5871, and MAE is 45.07% lower than the historical-median baseline. The ridge surrogate predicts candidate training utility from preference features computed from each latent. We paired each saved prediction with the utility measured after generation. We restored the prediction scale using only the history available when the prediction was made.

Table R7: Historical online prediction accuracy.

Model / budgetPrediction pairsPearson rSpearman ρSurrogate MAEHistorical-median MAE
Qwen / 4008,2250.72550.58710.57431.0455

We added this prediction evaluation as Table 16 in Appendix H.5 (lines 1379–1391).

Component ablations. The ablations show that removing any one of the three components lowers mean ID accuracy by about 4 to 6 percentage points and mean OOD accuracy by about 7 to 9 percentage points.

We remove each component separately and keep the other generation steps, numerical fitting, update schedules, and final selection rules unchanged:

  • Without surrogate ranking: We stop using surrogate scores to rank latents before generation. We keep the proposal-pool size and sampling rules. We described the ranking ablation in Appendix H.4 (lines 1355–1357).
  • Without latent-compatible parent retrieval: We stop using latent compatibility to reorder parent candidates and choose parents by quality. We keep the archive rules and the maximum number of parents. We added the parent-selection setting in Appendix H.4 (lines 1357–1359).
  • Without residual guidance: We stop using measured residuals to adjust the update targets for local residual coordinates. We keep these coordinates and their elite-based updates, along with the prompt channel, local update schedule, and edit coordinates. We specified the residual-guidance removal in Appendix H.4 (lines 1359–1361).

Table R3: Component ablations (60 ID tasks, 32 OOD tasks). Values are mean ± sample SD across three runs (%).

ConditionID [email protected] (%)OOD [email protected] (%)
Full BiTempo40.56 ± 4.1955.21 ± 3.61
Without surrogate ranking36.11 ± 0.9646.88 ± 3.13
Without parent retrieval34.44 ± 0.9645.83 ± 1.80
Without residual guidance35.56 ± 2.5547.92 ± 1.80

Relative to the full method, mean ID accuracy drops by 4.45, 6.12, and 5.00 percentage points without surrogate ranking, parent retrieval, and residual guidance, respectively. Without surrogate ranking, the three runs succeed on 15, 16, and 14 of the 32 OOD tasks. Without parent retrieval, they succeed on 15, 15, and 14. Without residual guidance, they succeed on 15, 15, and 16. Mean OOD accuracy is 8.33, 9.38, and 7.29 percentage points below the full method. We added these ablation results as Table 15 in Appendix H.4 (lines 1363–1378).

Removing pre-generation ranking reduces mean ID Acc by 4.45 percentage points and mean OOD Acc by 8.33 percentage points. Table R7 measures how well the predictions match actual utility. Table R3 measures how removing each component changes search accuracy.

AuthorsOpenReview ↗

Author Response to Reviewer Hmov (3/3)

Material budget effects and final selection (W3, Q4)

A larger budget improves average accuracy in the repeated runs, but the shortlist can drop successful candidates. Material [email protected] fell from 68.00% to 64.00% in the original table, a loss of 1 successful task. In two supplementary runs under the same inference environment, mean ID Acc rose from 56.00% to 66.00%, and both seeds improved with the larger budget. We added these budget-comparison results in Appendix H.8 (lines 1459–1463).

The shortlist can exclude a successful candidate that is still in the full pool. We reran final selection at the 200/400 cutoffs of the same search trajectories. The 200-slot cutoff here is a prefix of the 400-candidate trajectories, so its accuracy (60.00%) differs from the separate 200-candidate runs above (56.00%). We compared the candidate pool, shortlist, and final output. We used external labels only to evaluate the results after search. We revised lines 292–294 of the manuscript to explain this final-selection analysis.

Table R8: Candidate success coverage and final selection on the same trajectories.

Evaluation splitCutoffCandidate-pool success coverage (%)Shortlist success coverage (%)Final Acc (%)
ID200 slots82.0068.0060.00
ID400 slots82.0066.0066.00
OOD200 slots92.0084.0076.00
OOD400 slots94.0082.0082.00

Along the same trajectories, ID and OOD each gained 4 successes and lost 1, for a net gain of 3. The full pool kept or increased its coverage of successful candidates, but the shortlist lost coverage. Adding candidates can therefore change which ones pass the final filtering step. We added these results as Table 20 in Appendix H.8 (lines 1464–1480) and included Figure 7(c), whose caption appears in lines 1500–1503.

In both cases that lost success, the earlier successful candidate remained in the pool but was missing from the shortlist. When we added it back, the same training-based selection rule chose it again. The shortlist step caused these losses. A larger budget can therefore improve average accuracy while the final selection loses success on individual tasks. We added this add-back diagnosis in Appendix H.8 (lines 1480–1485).

AuthorsOpenReview ↗

Thank you and we are looking forward to your feedback!

Dear Reviewer Hmov,

Thank you again for your valuable comments, which have greatly improved our paper! We have carefully addressed your concerns. As the discussion phase nears its end, we would be deeply grateful if you could kindly share your thoughts on our rebuttal and consider raising your score. We truly appreciate your time and effort in reviewing our work.

Best regards, Authors

Reviewer_zFDJOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BiTempo, an LLM-based framework for scientific equation discovery. It represents generation preferences using global and local latent variables and updates their sampling distributions at different frequencies based on numerical feedback. Sampled preferences guide both equation generation and parent retrieval, while a ridge-regression surrogate screens latent proposals before LLM generation. Evaluation includes LLM-SRBench, equation-recovery examples, and measured stress–strain data.

Critical strengths
  1. The problem is meaningful. Improving search efficiency through accumulated feedback and coordinating structural exploration with local refinement are important challenges in scientific equation discovery.
  2. The evaluation is broad. Experiments cover two LLM backbones, multiple scientific domains, ID/OOD evaluation, ablations, and real measurements.
Critical weaknesses
  1. Limited novelty. Much of the underlying approach is established: DE[1] updates search-operator probabilities from rewards; RESTART[2] combines short-term residual correction with long-term structural experience; and GenSR[3] samples latent variables, evaluates decoded equations, and updates the search distribution. Besides, Using accumulated search experience to guide subsequent proposals is well established. LaSR[4] evolves a concept library, STRIDE combines reflective feedback with semantic memory, A-SR[5] adapts role-selection policies using evaluation feedback, and PiT-PO[6] updates the generator through in-search reinforcement learning. Against this background, BiTempo’s explicit preference representation and two-timescale updates appear to be an incremental extension of existing feedback-guided search. The paper does not establish which specific limitation of these approaches requires the proposed separation or why it constitutes a substantive methodological advance.

  2. Slowly updating a global distribution does not establish continuous refinement of a stable equation structure. The motivation is to preserve a promising structural direction while making frequent local corrections. However, the algorithm resamples latent variables at each iteration and may retrieve different parents. An unchanged global distribution can produce different preferences, and even identical preferences can lead the LLM to generate structurally different equations. The method therefore stabilizes the distribution of generation preferences, but does not necessarily maintain a consistent structure for successive local refinements. Figure 2 mainly confirms the prescribed update frequencies; it does not establish reduced structural switching or more coherent refinement. The final-performance ablations likewise do not directly validate this central mechanism.

  3. The global–local partition is heuristic and overlooks dependencies between decisions. Global coordinates include operators, interactions, and complexity, whereas local coordinates include residual families and edit actions. These categories overlap: residual guidance can introduce new functional families and change the overall structure, while “explore” is not inherently a local operation. Moreover, the factorized distribution does not explicitly model which local correction is appropriate for a particular global structure. Independently recombining preferences from successful candidates may produce incompatible strategies. The paper provides insufficient justification for either the partition or the independence assumption.

  4. The reliability of latent updates is insufficiently established, and strategy effects are confounded with other factors. Candidate quality depends on generation preferences, retrieved parents, stochastic decoding, and constant optimization. The method do not sufficiently separates these effects. For example, the paper explicitly notes that identical compiled preference features can retrieve different parents. Consequently, better parents or favorable optimization outcomes may be attributed to better preferences.

  5. Baseline coverage is limited, and performance advantages are conditional. The baseline selection is insufficiently broad and omits several strong recent methods, including SR-Scientist[8], RESTART, and PiT-PO. Comparisons with experience-guided approaches are also incomplete: DE and LaSR are included, but closely related methods such as STRIDE[5] and A-SR are not evaluated. These omissions limit the evidence for both practical competitiveness and the added value of two-timescale adaptation. Even among the included baselines, improvements are inconsistent across metrics. With Qwen and 400 candidates, BiTempo has worse mean NMSE than DE in all four domains. In Materials, NMSE increases from 1.47×10−41.47\times10^{-4} to 1.29×10−11.29\times10^{-1}, while [email protected] decreases from 56% to 44%. These limitations raise concerns about the method’s absolute performance and competitiveness with recent approaches.

[1] Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs. ICML 2026 [2] Robust Equation Structure Learning with Adaptive Refinement. ICLR 2026 [3] GenSR: Symbolic regression based on equation generative space. ICLR 2026 [4] Symbolic Regression with a Learned Concept Library. NeurIPS 2024 [5] STRIDE: A Self-Reflective Agent Framework for Reliable Automatic Equation Discovery. [6] A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination. [7] LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization. KDD 2026. [8] SR-Scientist: Scientific Equation Discovery With Agentic AI. ICLR 2026.

Questions for authors

No

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer zFDJ(1/3)

Dear Reviewer zFDJ:

Thank you for reviewing our work. We respond to each point below.

Method design and update schedules (W1)

BiTempo lets local preferences respond quickly to feedback while keeping the global search distribution fixed between updates. BiTempo keeps separate distributions for structural and refinement preferences. It updates them at different intervals using the same candidate evaluations. The CEM control updates all coordinates together. BiTempo can update the local distribution while keeping the global distribution fixed. Measured residuals guide the local updates.

We use the same sampled latent to guide generation and choose compatible parents. Previously evaluated candidates provide reference programs. Their preferences and scores also guide future proposals. CEM fits the distribution, and a diagonal Gaussian represents its state. The surrogate ranks latent proposals before generation, and retrieval supplies previously evaluated programs. BiTempo connects these steps through the preference state. Candidate feedback updates its global and local distributions at different rates, and each sampled latent guides both generation and retrieval. We have revised the manuscript to clarify these methodological contributions in lines 186–190 and Appendix H.1 (lines 1245–1259).

Update intervals and update-count controls. The default schedule updates local preferences more often and has the highest OOD accuracy in this comparison. It also has higher OOD accuracy than the control with the same total number of block updates. The matched control has lower mean NMSE.

We set K_g=32 and K_l=8 to update local preferences more often than global preferences. This lets local preferences respond faster to residual and edit feedback. Both blocks use the same candidate history. We count each interval in latent feedback observations. Once we have enough samples and valid elites, each global period has four opportunities for a local update. We added this explanation of the update intervals to lines246–249 of the manuscript.

Table R1: Effect of update schedules on OOD performance.

ConditionOOD [email protected] (%)Mean OOD NMSE
BiTempo (32,8)59.388.93019
Synchronous blocks (16,16)56.251136.98
Reversed blocks (8,32)56.25472.833
Count-matched synchronous blocks (13,13)53.130.35714

We added this schedule comparison as Table 13 in Appendix H.2 (lines 1297–1316).

U_g and U_l count updates to the global and local distributions. E counts events that update at least one block. Updating both blocks at once counts as one event and two block updates.

Table R2: Actual update counts.

ConditionU_gU_lUpdate events ETotal block updates U_g+U_lCoordinate-update volume 30U_g+10U_l
BiTempo (32,8)6242430420
Count-matched synchronous blocks (13,13)15151530600

We added the actual update counts as Table 14 in Appendix H.2 (lines 1317–1332).

Global distributions and equation structure (W2)

Slow global updates keep the distribution fixed between updates, but generated equations can still change structure. The global parameters stay fixed between global updates. Each round can still sample a different latent and choose different parents, so equation structures can change. We revised lines 57–67 of the manuscript to clarify this distinction.

We analyzed parent–child records for four schedules using Llama-3.1-8B, a budget of 200, and seed 42. Each schedule covers the same 60 tasks and 11,940 generation records. We call the reference equations in the prompt the parents and the generated equation the child. We compare normalized expression ASTs to check whether the child keeps a parent's structure. We count effective refinement when the child keeps that structure and has lower training-time internal CV NMSE than the best parent in the prompt. We normalize variable and fitted-parameter names while keeping parameter sharing, numerical literals, and operators. We compute each rate per task and then average across tasks with equal weights.

For the default (32,8), synchronous (16,16), reversed (8,32), and count-matched (13,13) schedules, structure-retention rates are 2.79%, 1.64%, 2.28%, and 1.99%, and effective-refinement rates are 0.57%, 0.28%, 0.40%, and 0.43%, respectively. We calculate structure retention over valid parent–child relations that we can parse. We calculate effective refinement over all generation records with valid parents. The default has the highest rate for both measures. Structures change in more than 97% of cases in all four groups. Of the 68 effective-refinement records under the default, 61 have the same code as a parent. The refinement measure therefore includes improvements from fitting constants in repeated equations. We added this parent–child structure and refinement analysis in Appendix H.3 (lines 1335–1348).

AuthorsOpenReview ↗

Author Response to Reviewer zFDJ(2/3)

Block design and cross-block dependence (W3)

The two blocks use the same feedback, but their distributions do not model correlations between blocks. We divide the coordinates by what they control and how often they update. The 30-dimensional global block controls variable, operator, interaction, and complexity preferences. The 10-dimensional local block controls residual responses and edit preferences. Residual guidance can introduce new function families based on current numerical feedback, and explore can change equation structure. Both blocks help construct candidates.

We use q(z_g,z_l)=q_g(z_g)q_l(z_l) and update both distributions from the same candidate history and elite scores. Each score evaluates a combination of global and local preferences and guides both updates. A 40-dimensional diagonal covariance uses 40 scale parameters, compared with 820 independent entries for a full covariance. We therefore estimate fewer parameters within the evaluation budget. We added these explanations of block roles and distribution parameterization to Appendix H.1 (lines 1259–1274).

We keep the same coordinate groups and reverse the update intervals from (32,8) to (8,32). OOD Acc changes from 59.38% to 56.25%. We added this reversed-schedule comparison to Table 13 in Appendix H.2 (lines 1297–1316).

Preferences, parents, and candidate quality (W4)

The preferences, retrieved parents, generated equations, and fitted constants all affect candidate utility. A latent guides generation and helps select compatible parents. We measure candidate utility after generation and constant fitting. Different continuous latents can produce the same preference features but retrieve different parents. The updates therefore learn which combinations of latent preferences and retrieved parents produce useful candidates during the current search. We added this explanation of the joint interface to the manuscript in lines 1274–1279.

Component ablations. The ablations show that removing any one of the three components lowers mean ID accuracy by about 4 to 6 percentage points and mean OOD accuracy by about 7 to 9 percentage points.

We remove each component separately and keep the other generation steps, numerical fitting, update schedules, and final selection rules unchanged:

  • Without surrogate ranking: We stop using surrogate scores to rank latents before generation. We keep the proposal-pool size and sampling rules.
  • Without latent-compatible parent retrieval: We stop using latent compatibility to reorder parent candidates and choose parents by quality. We keep the archive rules and the maximum number of parents.
  • Without residual guidance: We stop using measured residuals to adjust the update targets for local residual coordinates. We keep these coordinates and their elite-based updates, along with the prompt channel, local update schedule, and edit coordinates.

Table R3: Component ablations (60 ID tasks, 32 OOD tasks). Values are mean ± sample SD across three runs (%).

ConditionID [email protected] (%)OOD [email protected] (%)
Full BiTempo40.56 ± 4.1955.21 ± 3.61
Without surrogate ranking36.11 ± 0.9646.88 ± 3.13
Without parent retrieval34.44 ± 0.9645.83 ± 1.80
Without residual guidance35.56 ± 2.5547.92 ± 1.80

Relative to the full method, mean ID accuracy drops by 4.45, 6.12, and 5.00 percentage points without surrogate ranking, parent retrieval, and residual guidance, respectively. Without surrogate ranking, the three runs succeed on 15, 16, and 14 of the 32 OOD tasks. Without parent retrieval, they succeed on 15, 15, and 14. Without residual guidance, they succeed on 15, 15, and 16. Mean OOD accuracy is 8.33, 9.38, and 7.29 percentage points below the full method. We added these settings and results as Table 15 in Appendix H.4 (lines 1355–1378).

Baselines, metrics, and run variability (W5)

BiTempo reports higher Qwen OOD accuracy than A-SR, but it uses different NMSE evaluation rules from DE.

Comparison with recent methods. With Qwen3-4B-Instruct-2507 and a budget of 400 candidates, the original BiTempo paper reports a four-scientific-domain average OOD [email protected] of 40.16%, compared with 37.57% for online A-SR, a difference of 2.59 percentage points (A-SR, Table 10).

With Llama-3.1-8B-Instruct and a budget of 400 candidates, the four-domain average [email protected] values are 52.81% for BiTempo (ID) and 48.30% for online A-SR (the average of training and validation performance) (A-SR, main table and Section 4.1). SR-Scientist, RESTART, and STRIDE use other backbone models.

Qwen-400's raw mean NMSE is higher than DE's reported value in all four domains. DE discards the worst 5% of prediction points. Our original NMSE uses all evaluation points (DE, Section 4.1). We added these comparisons, the backbone information, and the evaluation protocols to Appendix H.9 (lines 1506–1515).

AuthorsOpenReview ↗

Author Response to Reviewer zFDJ(3/3)

[email protected] and mean NMSE. Task accuracy counts successful tasks, while mean NMSE averages errors across tasks. Under [email protected], a task succeeds when at least 95% of evaluation points have relative error no greater than 1%. When the target's absolute value is no greater than 1e-12, absolute error must be no greater than 1e-12. A few tasks with large errors can raise mean NMSE. We compute NMSE as MSE/max(Var(y),1e-12). A small OOD target variance can make NMSE large even when absolute MSE is small. NMSE can exceed 1. We added this explanation of the metrics in Appendix H.6 (lines 1395–1409).

Task-level errors. Among the 129 tasks in the original paper's four scientific domains, 98 tasks (75.97%) have raw ID NMSE no greater than 1e-3. Median NMSEs for Biology, Chemistry, Material, and Physics are 1.32e-6, 1.89e-7, 1.91e-6, and 2.35e-4, respectively. The two largest-error tasks in each domain account for 96.48%, 85.91%, 95.61%, and 69.38% of the total NMSE, respectively. We added these task-level error statistics to Appendix H.6 (lines 1395–1409) and included Figure 7(a), whose caption appears in lines 1500–1503.

In Material, these two tasks raise the mean even though the median is low. [email protected] requires at least 95% of evaluation points to meet the relative-error threshold. A low fitting error therefore does not always make a task successful.

Of the 42 tasks that succeed on ID data, 39 also succeed on OOD data, giving a retention rate of 92.86%. We evaluated the same final equation on both datasets for each task. We added this ID/OOD retention analysis to Appendix H.6 (lines 1395–1409).

Run variability and errors. Both models have higher mean ID and OOD accuracy with the larger candidate budget.

Table R4: Llama and Qwen accuracy across three seeds. Values are mean ± sample SD (%).

ModelCandidate budgetID Acc (%)OOD Acc (%)
Llama20053.06 ± 1.2056.58 ± 1.20
Llama40059.77 ± 5.9661.53 ± 5.59
Qwen20026.28 ± 1.2931.23 ± 6.00
Qwen40031.56 ± 3.1137.92 ± 3.51

We added these multi-seed results as Table 17 in Appendix H.7 (lines 1418–1426) and included Figure 7(b), whose caption appears in lines 1500–1503.

Table R5: 95% confidence intervals for mean accuracy across three seeds (%).

We compute each interval as mean ± 4.30265 × SD/√3, using a two-sided Student's t interval with two degrees of freedom. The calculation uses the rounded means and sample SDs in Table R4.

ModelCandidate budgetID 95% CIOOD 95% CI
Llama200[50.08, 56.04][53.60, 59.56]
Llama400[44.96, 74.58][47.64, 75.42]
Qwen200[23.08, 29.48][16.33, 46.13]
Qwen400[23.83, 39.29][29.20, 46.64]

We added these confidence intervals as Table 18 in Appendix H.7 (lines 1427–1435).

Three-seed NMSE and tail sensitivity. A few tasks with large errors raise mean OOD NMSE in Biology, Material, and Physics. Removing these tasks lowers the means substantially. Physics results still vary across seeds after we remove the largest errors, so both effects contribute to its mean NMSE. We added this tail-sensitivity analysis in Appendix H.7 (lines 1435–1455).

Within each domain and seed, we remove the floor(0.05×number of tasks) tasks with the highest NMSE. Physics still has mean OOD NMSE of 2.3847e4 after this step, so large errors remain.

Table R6: Qwen-400 ID error distributions for three complete seeds.

DomainNumber of tasksRange of median ID NMSE across seedsTasks with NMSE≤1e-3 (%, mean±SD)Share of total NMSE from the two largest-error tasks (%, mean±SD)
Biology243.92e-07—2.9e-0684.72 ± 2.4197.57 ± 2.03
Chemistry361.8e-08—3.14e-0693.52 ± 8.9367.50 ± 6.55
Material251.82e-06—2.14e-0584.00 ± 0.0098.49 ± 0.87
Physics440.000502—0.34638.64 ± 20.8347.13 ± 31.95

We added the three-seed error distributions as Table 19 in Appendix H.7 (lines 1442–1455).

Most Biology, Material, and Chemistry tasks have low ID error across seeds. A few large errors still raise the mean, even when the median is low.

AuthorsOpenReview ↗

Thank you and we are looking forward to your feedback!

Dear Reviewer zFDJ,

Thank you for your careful reading and for sharing your perspective. We have carefully addressed your concerns. As the discussion phase nears its end, we would be deeply grateful if you could kindly share your thoughts on our rebuttal and consider raising your score. We truly appreciate your time and effort in reviewing our work.

Best regards,

Authors

Reviewer_1sM7OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BITEMPO, a test-time latent inference framework for scientific equation discovery. The authors divide generation preferences into global structural preferences and local refinement preferences, which are updated at different intervals based on numerical evaluation feedback. The method further leverages latent variables to guide LLM-based equation generation, parent retrieval, and surrogate-assisted candidate selection.Well-motivated approach. By accumulating search feedback across different equations through explicit latent distributions, the method provides a more structured representation of the search state than simply storing previously evaluated candidate programs.

Critical strengths

Well-motivated approach. By accumulating search feedback across different equations through explicit latent distributions, the method provides a more structured representation of the search state than simply storing previously evaluated candidate programs.

Strong results on several benchmarks. With Llama-3.1-8B, BITEMPO-400 improves the domain-averaged [email protected] from 25.79% to 52.81%, with particularly substantial gains in Chemistry and Biology.

Useful experimental analyses. The paper includes ablation studies on distribution updates, two-timescale adaptation, and global/local latent components, as well as analyses of latent evolution trajectories and experiments on real-world stress–strain data.

Critical weaknesses

Limited novelty and insufficient justification of the core design. The proposed framework primarily combines existing techniques, including CEM, diagonal Gaussian distributions, residual-guided refinement, surrogate modeling, and historical program retrieval. The global/local two-timescale adaptation appears to be the main methodological contribution. However, the choice of fixed update intervals of 32 and 8 is not sufficiently justified, either theoretically or empirically.

Incomplete ablation studies. Although the Single-Timescale ablation provides some evidence supporting the two-timescale design, it does not adequately control for the total number and frequency of distribution updates. Moreover, independent ablations of the surrogate model, latent-compatible parent retrieval, and residual guidance are missing, making it difficult to isolate the contribution of each component.

Inconsistent performance across evaluation metrics. Despite substantial improvements in [email protected], BITEMPO achieves worse mean NMSE than DE across all four domains with Qwen3-4B. Furthermore, improvements in in-distribution accuracy do not consistently translate into better out-of-distribution performance. These discrepancies raise concerns about the robustness and generalization benefits of the proposed method.

Insufficient statistical robustness analysis. The framework involves several stochastic components, including LLM generation, latent sampling, and numerical optimization. However, the main experiments lack sufficient multi-seed evaluations and confidence interval analyses, making it difficult to assess the stability and statistical reliability of the reported improvements.

Questions for authors

1, Does the two-timescale approach still provide a clear advantage over a single-timescale baseline when the total number of distribution updates is matched? How sensitive is the performance to different global and local update intervals?

2, Could the authors provide independent ablation studies removing the surrogate model, latent-compatible parent retrieval, and residual guidance to clarify their respective contributions?

3, Why do the substantial improvements in [email protected] fail to translate into better mean NMSE on Qwen? Could the authors provide task-level error distributions and results across multiple random seeds to better explain this discrepancy?

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer 1sM7 (1/3)

Dear Reviewer 1sM7:

Thank you for reviewing our work. We respond to each point below.

Relationship to CEM and prior methods (W1)

BiTempo lets local preferences respond quickly to feedback while keeping the global search distribution fixed between updates. BiTempo uses candidate scores to adjust its generation preferences within the evaluation budget. It updates global and local preferences at different rates to balance the search for new structures with the refinement of existing candidates.

BiTempo keeps separate distributions for structural and refinement preferences. It updates them at different intervals using the same candidate evaluations. The CEM control updates all coordinates together. BiTempo can update the local distribution while keeping the global distribution fixed. Measured residuals guide the local updates.

We use the same sampled latent to guide generation and choose compatible parents. Previously evaluated candidates provide reference programs. Their preferences and scores also guide future proposals. CEM fits the distribution, and a diagonal Gaussian represents its state. The surrogate ranks latent proposals before generation, and retrieval supplies previously evaluated programs. BiTempo connects these steps through the preference state. Candidate feedback updates its global and local distributions at different rates, and each sampled latent guides both generation and retrieval. We have revised the manuscript to clarify these methodological contributions in lines 186–190 and Appendix H.1 (lines 1245–1259).

Update intervals, structural preferences, and update counts (W1, W2, Q1)

BiTempo has higher OOD accuracy than the synchronous control when both use the same number of block updates. The global block controls structural preferences. The local block controls residual and edit preferences. The global parameters stay fixed between global updates. Each round can still sample a different latent and choose different parents, so equation structures can change. We revised lines 57–67 of the manuscript to clarify this distinction.

We set K_g=32 and K_l=8 to update local preferences more often than global preferences. This lets local preferences respond faster to residual and edit feedback. Both blocks use the same candidate history. We count each interval in latent feedback observations. Once we have enough samples and valid elites, each global period has four opportunities for a local update. We added this explanation of the update intervals to lines 246–249 of the manuscript.

Table R1 compares the default (32,8), synchronous (16,16), and reversed (8,32) schedules, along with (13,13), which matches the total number of block updates.

Table R1: Effect of update schedules on OOD performance.

ConditionOOD [email protected] (%)Mean OOD NMSE
BiTempo (32,8)59.388.93019
Synchronous blocks (16,16)56.251136.98
Reversed blocks (8,32)56.25472.833
Count-matched synchronous blocks (13,13)53.130.35714

We added this schedule comparison as Table 13 in Appendix H.2 (lines 1297–1316).

U_g and U_l count updates to the global and local distributions. E counts events that update at least one block. Updating both blocks at once counts as one event and two block updates.

Table R2: Actual update counts.

ConditionU_gU_lUpdate events ETotal block updates U_g+U_lCoordinate-update volume 30U_g+10U_l
BiTempo (32,8)6242430420
Count-matched synchronous blocks (13,13)15151530600

We added the actual update counts as Table 14 in Appendix H.2 (lines 1317–1332).

The default schedule has the highest OOD accuracy. It updates the local block more often while using the same total number of block updates as the matched control. The matched control has lower mean NMSE. The two metrics therefore favor different schedules.

AuthorsOpenReview ↗

Author Response to Reviewer 1sM7 (2/3)

Individual component contributions (W2, Q2)

Removing any one of the three components lowers mean ID accuracy by about 4 to 6 percentage points and mean OOD accuracy by about 7 to 9 percentage points.

We remove each component separately and keep the other generation steps, numerical fitting, update schedules, and final selection rules unchanged:

  • Without surrogate ranking: We stop using surrogate scores to rank latents before generation. We keep the proposal-pool size and sampling rules. We described the ranking ablation in Appendix H.4 (lines 1355–1357).
  • Without latent-compatible parent retrieval: We stop using latent compatibility to reorder parent candidates and choose parents by quality. We keep the archive rules and the maximum number of parents. We added the parent-selection setting in Appendix H.4 (lines 1357–1359).
  • Without residual guidance: We stop using measured residuals to adjust the update targets for local residual coordinates. We keep these coordinates and their elite-based updates, along with the prompt channel, local update schedule, and edit coordinates. We specified the residual-guidance removal in Appendix H.4 (lines 1359–1361).

Table R3: Component ablations (60 ID tasks, 32 OOD tasks). Values are mean ± sample SD across three runs (%).

ConditionID [email protected] (%)OOD [email protected] (%)
Full BiTempo40.56 ± 4.1955.21 ± 3.61
Without surrogate ranking36.11 ± 0.9646.88 ± 3.13
Without parent retrieval34.44 ± 0.9645.83 ± 1.80
Without residual guidance35.56 ± 2.5547.92 ± 1.80

Relative to the full method, mean ID accuracy drops by 4.45, 6.12, and 5.00 percentage points without surrogate ranking, parent retrieval, and residual guidance, respectively. Without surrogate ranking, the three runs succeed on 15, 16, and 14 of the 32 OOD tasks. Without parent retrieval, they succeed on 15, 15, and 14. Without residual guidance, they succeed on 15, 15, and 16. Mean OOD accuracy is 8.33, 9.38, and 7.29 percentage points below the full method. We added these ablation results as Table 15 in Appendix H.4 (lines 1363–1378).

Acc, NMSE, and ID/OOD performance (W3, Q3)

A few large errors can raise mean NMSE even when many tasks meet the accuracy threshold.

[email protected] and mean NMSE. Task accuracy counts successful tasks, while mean NMSE averages errors across tasks. Under [email protected], a task succeeds when at least 95% of evaluation points have relative error no greater than 1%. When the target's absolute value is no greater than 1e-12, absolute error must be no greater than 1e-12. A few tasks with large errors can raise mean NMSE. We compute NMSE as MSE/max(Var(y),1e-12). A small OOD target variance can make NMSE large even when absolute MSE is small. NMSE can exceed 1. We added this explanation of the metrics in Appendix H.6 (lines 1395–1409).

Task-level errors. Among the 129 tasks in the original paper's four scientific domains, 98 tasks (75.97%) have raw ID NMSE no greater than 1e-3. Median NMSEs for Biology, Chemistry, Material, and Physics are 1.32e-6, 1.89e-7, 1.91e-6, and 2.35e-4, respectively. The two largest-error tasks in each domain account for 96.48%, 85.91%, 95.61%, and 69.38% of the total NMSE, respectively. We added these task-level error statistics to Appendix H.6 (lines 1395–1409) and included Figure 7(a), whose caption appears in lines 1500–1503.

Of the 42 tasks that succeed on ID data, 39 also succeed on OOD data, giving a retention rate of 92.86%. We evaluated the same final equation on both datasets for each task. We added this ID/OOD retention analysis to Appendix H.6 (lines 1395–1409).

AuthorsOpenReview ↗

Author Response to Reviewer 1sM7 (3/3)

Run variability and statistical reliability (W4, Q3)

Both models have higher mean ID and OOD accuracy with the larger candidate budget.

Table R4: Llama and Qwen accuracy across three seeds. Values are mean ± sample SD (%).

ModelCandidate budgetID Acc (%)OOD Acc (%)
Llama20053.06 ± 1.2056.58 ± 1.20
Llama40059.77 ± 5.9661.53 ± 5.59
Qwen20026.28 ± 1.2931.23 ± 6.00
Qwen40031.56 ± 3.1137.92 ± 3.51

We added these multi-seed results as Table 17 in Appendix H.7 (lines 1418–1426) and included Figure 7(b), whose caption appears in lines 1500–1503.

Table R5: 95% confidence intervals for mean accuracy across three seeds (%).

We compute each interval as mean ± 4.30265 × SD/√3, using a two-sided Student's t interval with two degrees of freedom. The calculation uses the rounded means and sample SDs in Table R4.

ModelCandidate budgetID 95% CIOOD 95% CI
Llama200[50.08, 56.04][53.60, 59.56]
Llama400[44.96, 74.58][47.64, 75.42]
Qwen200[23.08, 29.48][16.33, 46.13]
Qwen400[23.83, 39.29][29.20, 46.64]

We added these confidence intervals as Table 18 in Appendix H.7 (lines 1427–1435).

Three-seed NMSE and tail sensitivity. A few tasks with large errors raise mean OOD NMSE in Biology, Material, and Physics. Removing these tasks lowers the means substantially. Physics results still vary across seeds after we remove the largest errors, so both effects contribute to its mean NMSE. We added this tail-sensitivity analysis in Appendix H.7 (lines 1435–1455).

Table R6: Qwen-400 ID error distributions for three complete seeds.

DomainNumber of tasksRange of median ID NMSE across seedsTasks with NMSE≤1e-3 (%, mean±SD)Share of total NMSE from the two largest-error tasks (%, mean±SD)
Biology243.92e-07—2.9e-0684.72 ± 2.4197.57 ± 2.03
Chemistry361.8e-08—3.14e-0693.52 ± 8.9367.50 ± 6.55
Material251.82e-06—2.14e-0584.00 ± 0.0098.49 ± 0.87
Physics440.000502—0.34638.64 ± 20.8347.13 ± 31.95

We added the three-seed error distributions as Table 19 in Appendix H.7 (lines 1442–1455).

Most Biology, Material, and Chemistry tasks have low ID error across seeds. A few large errors still raise the mean, even when the median is low. These effects vary by domain and seed.

AuthorsOpenReview ↗

Thank you and we are looking forward to your feedback!

Dear Reviewer 1sM7,

Thank you again for your valuable comments, which have greatly improved our paper! We have carefully addressed all your concerns. As the discussion phase nears its end, we would be deeply grateful if you could kindly share your thoughts on our rebuttal and consider raising your score.

We truly appreciate your time and effort in reviewing our work.

Best regards,

Authors

Comments on the paper

2 comments
Area_Chair_TUU1OpenReview ↗

Review response

Dear Reviewers, the authors have posted detailed responses and a revised PDF, and the author-reviewer discussion ends soon. Please read their rebuttal, post a brief reply on whether your concerns are addressed, and update your scores if your assessment has changed. Thank you! AC.

AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for the careful evaluation of our paper and for the constructive comments, which have substantially improved it. We have posted detailed responses to each reviewer and uploaded a revised PDF.

During the rebuttal, we strengthened the paper in the following aspects:

  • Methodological contribution: We clarified that BiTempo connects generation, parent retrieval, and surrogate ranking through one preference state whose global and local blocks are updated at different rates.
  • Two-timescale validation: We compared the default schedule with synchronous, reversed, and count-matched schedules. The default has the highest OOD accuracy, including against the control with the same total number of block updates (Tables 13–14).
  • Structural consistency: We clarified that slow global updates stabilize the distribution of preferences, and added a parent–child analysis in which the default schedule has the highest structure-retention and effective-refinement rates (Appendix H.3).
  • Contribution of each component: We added separate ablations of surrogate ranking, latent-compatible parent retrieval, and residual guidance over three runs.
  • Predictive accuracy of the surrogate: We evaluated the surrogate's online predictions against measured utilities and showed that it correlates well with them and outperforms the historical-median baseline (Table 16).
  • Metric consistency: We explained the relation between [email protected] and mean NMSE, and showed that mean NMSE is dominated by a few large-error tasks while most ID-successful tasks also succeed on OOD data (Appendix H.6).
  • Statistical robustness: We added three-seed results with 95% confidence intervals for both backbones and both budgets, together with a tail-sensitivity analysis of NMSE (Tables 17–19).
  • Scaling with candidate budget: We showed that a larger budget improves average accuracy in repeated runs, and traced the single lost task on Material to the final shortlist step (Table 20).
  • Competitiveness with recent methods: We added comparisons with an additional baseline under matched backbones and budgets, and documented the backbones and evaluation protocols used by other recent methods (Appendix H.9).

Overall, we believe the rebuttal and revision substantiate the methodological contribution, the two-timescale design, the role of each component, the consistency of the evaluation, and the statistical robustness of the results. We hope these additions further clarify the contribution and scope of the paper.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.