- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RISR, which uses residual encoding to guide a frozen LLM in generating candidate equations, followed by structural correction and further refinement based on predicted post-fit utility. The authors evaluate accuracy and symbolic recovery on LSR-Synth, achieving competitive results.
Critical strengths
The method jointly encodes inputs, current predictions, and pointwise residuals to guide candidate generation, and selects corrections based on post-fit utility. Compared with relying solely on aggregate fitting errors, this design provides more detailed numerical information. The complete system achieves competitive ID/OOD numerical accuracy across multiple scientific domains, outperforming some baselines that use the same LLM backbone.
Critical weaknesses
- The necessity and independent contribution of the additional Residual Encoder have not been sufficiently validated. The current experiments do not adequately distinguish the effects of the dedicated Encoder, additional supervised training, and Structural Editor. Key comparisons with direct input of the same numerical information and LLM fine-tuning without the Encoder are missing.
- The experiments primarily report accuracy, with limited analysis of how individual components contribute to symbolic structure recovery. The presented case study also does not sufficiently illustrate the structural discovery process or its associated complexity cost.
Questions for authors
- The paper introduces an additional Residual Encoder to jointly process inputs, predictions, and residuals. However, the LLM itself can also model relationships among these inputs through its attention mechanism. What advantages does the dedicated numerical encoder offer over directly providing the same information to the LLM? Two additional ablations would be valuable: removing the Encoder and providing the same information as text to the frozen LLM; and removing the Encoder and fine-tuning the LLM using the same synthetic training data.
- LSR-Synth uses newly constructed scientific equations to reduce reliance on LLM memorization. However, RISR introduces additional supervised training on synthetic equations, which may bring some test equation structures back into the model’s familiar training distribution. To what extent are the training and test equations mathematically equivalent or structurally overlapping? Results after excluding closely related templates would be valuable.
- In Figure 3, the LLM Proposal already contains all the target structural terms, while the additional term could potentially be eliminated by setting its coefficient to zero. Thus, the Proposal may already satisfy the paper’s symbolic recovery criterion. More diagnostic case studies would be helpful, such as an initial Proposal that lacks key structural terms but subsequent steps successfully recover the correct structure. Please also provide the complete search trajectory from the Initial Seed to the final equation. Representative failure cases would help distinguish the actual contributions of the LLM Proposal, Structural Editor, and coefficient fitting.
- RESTART also uses residual information to guide structural corrections of equations. Please clarify the key differences between the two methods in residual representation, candidate generation, and refinement mechanisms. A direct comparison using a consistent LLM backbone, search budget, and evaluation protocol would also be valuable.