- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In order to train LLMs to generate research ideas that can better match human researchers' tastes, the authors collect real-world research idea ratings from LLMs and human PhD researchers and propose the TIGER framework to learn from the collected preferences. TIGER's reward design consists of eight components that guide the learning process towards researchers' tastes while try to ensure the research idea's quality. Win-rate estimates and other analyses show that the learned policy in general matches the researchers' preferences.
Critical strengths
- This work provides more data for studying "research taste personalization" with LLMs for the community. The scarcity of relevant data is one of the bottlenecks for studying LLM agents in the process of automatic scientific discovery.
- The reward design of TIGER covers a lot of aspects of research idea quality. It covers research ideas' general quality indicators and research taste. A carefully designed and well-justified reward would help the LLM agents a lot in adapting to autoresearch tasks.
Critical weaknesses
-
The distinctive benefit of learning an individual researcher’s tastes is not sufficiently established. TIGER largely combines existing reward modeling and policy optimization techniques, so its contribution depends on demonstrating a substantive benefit from researcher-taste-specific adaptation. However, I feel that the current experiments do not adequately distinguish this benefit from general improvements in idea quality. This distinction is particularly important for the human results. The cross-policy generation comparisons in Table 2(b) only involve LLM judges. The two human researchers each compare their own policy against GPT-4.1, without comparing it against the other researcher’s policy or a shared policy. If this compison Consequently, the evidence remains preliminary regarding whether the system captures individual human research tastes or produces ideas that are generally more appealing. Given the limited methodological novelty, this unresolved attribution substantially weakens the demonstrated contribution. Besides, if benchmark dataset is one of the major contribution, it would be better to collect research tastes from more human researchers, two might not be enough.
-
The evaluation does not sufficiently validate the specific reward design. The eight-term reward design in Eq. (1) is only partially examined. Table 3 removes the taste reward, removes all five quality penalties together, or removes neutral rewriting; format and length penalties remain in every variant. These experiments support the usefulness of the quality-penalty bundle, but do not establish which quality dimensions are necessary, whether some are redundant, or how sensitive the results are to their weighting relative to personal taste. If the componenets are redundant, maybe idea quality as one term should be enough?
Questions for authors
- What if we directly feed researchers' tastes to an LLM; will that achieve even better results? Describing a researcher's taste can be difficult, but I think we can ask LLMs to summarize it from preference data.
- What if we improve the granularity of the reward design's ablation study? The reward has many components, but we haven't clearly identified each component's contribution.
5 comments
We thank the authors for their detailed rebuttal. As we are not sure whether the reward design is necessary, I will keep my rating. In the next version, I would suggest that the authors verify whether removing the taste term from the reward still yields a similar human-LLM taste alignment percentage score as you showed in the rebuttal. If the alignment gets worse without the taste term, it can help to verify that learning research taste is necessary and beneficial, which will make this work more insightful.
We thank the reviewer for timely response and insightful feedback. We would like to provide results we just got and resolve the concern regarding the contribution of the taste reward. The researcher-specific RM supplies the personalized preference signal, while the five auxiliary penalties provide shared scientific-quality guidance.
We have updated Table 4, which evaluates a quality-only control that removes the taste reward while retaining the quality penalties scores. Under Terra’s judgments, full TIGER is preferred over this variant in 51 of 76 comparisons (67.1%). If we only have the taste reward and remove all penalties, it is 90.8%. This means learned taste reward and shared quality penalties both have contribution where taste reward is more important to reach human-LLM alignment.
We also revised our Eq.1 in Section 3.3 on Page 6 where the original form may be misleading. Our quality penalties are directly output by the LLM judge as a whole with our prompting the LLM for the requirement about faithfulness, novelty, etc. rather than 7 individual penalties term produced by the judge.
Together, the taste-removal ablation can conclude that human matched-policy comparisons support the value of learning researcher-specific preferences alongside shared scientific-quality guidance.
We thank the reviewer for their constructive feedback and recognition of our presentation and soundness. To address the reviewer's concerns, we have updated the manuscript with additional human cross-policy comparisons and clarified the roles of different reward components.
W1 Benefit of learning an individual researcher’s tastes. We would like to clarify that our primary contribution is a framework for learning and aligning LLM-generated research ideas with an individual researcher's taste, rather than constructing a large-scale benchmark of human research preferences. As discussed in Section 2, defining universally "good" research taste is inherently challenging, as different researchers may have different perspectives and preferences. Our goal is therefore to learn and adapt to these individual preferences rather than optimize solely for general idea quality.
To directly address the reviewer's concern, we have added human cross-policy comparisons to Table 3(b). Both PhD researchers evaluate the same 30 candidate pairs generated by their respective policies:
Both researchers prefer their individually aligned policies over the other researcher's policy. They also prefer their matched policies over the three LLM-aligned alternatives. These results provide direct evidence that TIGER captures researcher-specific preferences rather than merely improving general idea quality.
Regarding the number of human researchers, we agree that involving more researchers would strengthen the generalizability of our findings.
However, our current objective is to demonstrate the feasibility of learning an individual's research taste from offline preference annotations. The framework does not assume a shared preference across researchers: each researcher provides their own ratings, from which a corresponding reward model and ideation policy are trained. We will clarify this scope and consider broader human preference collection in future work.
Q1: What if we directly feed researchers' tastes to an LLM. We appreciate this suggestion, which provides an important baseline that was missing from our initial evaluation. We have added a comparison between TIGER-8B and the same Qwen3-8B backbone using preference-based prompting. Specifically, we provide Qwen3-8B with four preferred/less-preferred idea pairs from the researcher's training annotations and ask it to generate new ideas conditioned on these examples. Across 30 held-out research requests, the researcher prefers TIGER-8B in 52/60 judgments (86.7%), with both A/B presentation orders evaluated. These results suggest that learning a personalized policy through reward-guided optimization provides substantial benefits over directly conditioning the same backbone on preference examples. We have included the experimental details and results in Section 4.3 and Appendix G.5.
W2 Roles of the reward components We appreciate the reviewer's suggestion to examine the reward components at a finer granularity. Our reward design separates the researcher-specific taste reward from five scientific-quality penalties (faithfulness, novelty, feasibility, significance, and coherence) and two output-control penalties (format and length). The five quality dimensions are evaluated in a single call to a frozen quality judge and combined into a scalar reward for policy optimization.
In Section 4.3 and Table 4, we examine three major components of the framework: the taste reward, the auxiliary quality-feedback bundle, and neutral rewriting. Full TIGER is preferred over the corresponding ablated variants in 67.1%, 90.8%, and 59.2% of comparisons, respectively. These results demonstrate the complementary contributions of personalized preference learning, scientific-quality feedback, and neutral rewriting.
We agree finer-grained ablations and comparisons with a single overall quality score would further help establish the contribution of each component. Regarding the finer-grained of each penalty component in Eq1, due to the computing resources, we will later add in the results (currently we are running). We currently update our manuscript with the limitation section.