- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether learning a knowledge domain during pretraining leads to effective exploration of that domain during autonomous generation. Using a controlled mathematical environment over with an exact verifier, the authors examine the relationships among teacher-forced loss, verified search allocation, and unique discovery coverage. The experiments show that models trained without the target knowledge family fail to generate verified proposals from that family, while models trained with it exhibit substantial variation in search allocation despite similar training losses. The authors also find that increasing the sampling budget can improve unique coverage without resolving low search allocation. Based on these findings, the paper argues that search allocation should be evaluated separately from loss, validity, and unique yield.
Critical strengths
-
This paper demonstrates that models with similar training losses can exhibit substantially different search allocation behaviors. It also shows that increasing the sampling budget can improve unique coverage without resolving inefficient search allocation.
-
This paper uses a controlled mathematical environment with an exact verifier to investigate search behavior. It also conducts experiments across multiple training seeds, sampling budgets, and decoding configurations, providing empirical support for its main observations.
Critical weaknesses
-
Limited novelty compared with existing research. The main finding that low teacher-forced loss does not necessarily lead to effective generation or exploration is closely related to existing studies on the mismatch between likelihood and generation behavior [1, 2]. More recent work has also investigated diversity collapse in LLM-based scientific hypothesis search [3]. Although this paper provides a controlled mathematical environment to study these phenomena, its main conclusions largely reinforce existing observations. Furthermore, the proposed search allocation metrics are relatively straightforward measures of the proportion of verified proposals. It is therefore unclear what fundamentally new insights this paper provides beyond previous research. The authors should more clearly distinguish their contributions from these studies and explain the novel understanding gained from their experiments.
-
Limited experimental settings and unclear generalizability. The main experiments are conducted exclusively in a simplified mathematical environment over , using relatively small language models trained from scratch. The discovery task mainly involves generating and verifying algebraic identities within a predefined set of mathematical structures, which differs substantially from realistic scientific discovery tasks. Although this controlled setting is useful for isolating experimental factors, it remains unclear whether the findings generalize to modern pretrained LLMs or more complex discovery systems. This limitation is particularly important given the paper's broader motivation concerning LLM-based scientific discovery. Evaluating the main findings on larger models and more realistic discovery tasks would substantially strengthen the paper.
-
Insufficient analysis of the underlying causes of search allocation divergence. In Section 4.1 and Figure 3, the paper demonstrates substantial differences in search allocation among models with similar teacher-forced losses. However, the most severe example of structural proposal starvation is primarily observed in Seed 2. Although the authors conduct additional experiments with different decoding configurations and prompts, these experiments mainly demonstrate that the phenomenon persists for this particular checkpoint. They do not sufficiently explain why such a large difference emerges between independently trained models or establish how frequently this failure occurs. The discussion in Section 5 attributes the phenomenon to preferences for unary identity patterns, but the underlying training dynamics remain unclear. A more systematic investigation of the causes and prevalence of this behavior would strengthen the paper's conclusions.
[1] "The Curious Case of Neural Text Degeneration", Holtzman et al., ICLR 2020.
[2] "Neural Text Generation with Unlikelihood Training", Welleck et al., ICLR 2020.
[3] "Towards Diverse Scientific Hypothesis Search with Large Language Models", Wang et al., ICML 2026.
Questions for authors
-
Could the authors further clarify the novelty of this work compared with existing studies on the mismatch between likelihood and generation behavior, particularly regarding the new insights provided by the proposed experimental setting?
-
Could the authors further explain why severe search allocation divergence occurs in certain training seeds, particularly Seed 2, and provide additional evidence on how frequently this phenomenon occurs?