BiTempo: Adaptive Two-Timescale Latent Inference with Language Models for Scientific Equation Discovery

Scientific equation discovery requires searching over symbolic structures whose quality is revealed only after their constants are fitted to data. Language-model-based methods typically carry feedback forward as example programs or textual memory, without explicitly separating persistent structural preferences from local corrections in their search state. We propose BiTempo, a test-time latent inference framework that accumulates numerical feedback in factorized global and local distributions over generation preferences. The global block encodes variables, operators, and composition, while the local block encodes refinement directions informed by training residuals. Sampled latents condition program generation and compatible-parent retrieval; each evaluated equation contributes evidence for distribution updates, with the local block adapting faster than the global one. A lightweight utility surrogate ranks latent proposals before language-model calls, without fine-tuning the language model. On the four LSR-Synth domains of LLM-SRBench, BiTempo with 400 candidates improves domain-averaged Acc\@0.01 over the strongest baseline from $25.79\%$ to $52.81\%$ with Llama-3.1-8B and from $24.58\%$ to $34.08\%$ with Qwen3-4B. On measured stress–strain data, it reduces ID and OOD NMSE by $48.6\%$ and $61.0\%$ relative to the strongest baseline. Ablations show that removing distribution updates or collapsing the two timescales lowers ID accuracy from $45.00\%$ to $30.00\%$ and $33.33\%$, respectively.