acceptodds
Under review as a conference paper at ICLR 2027

TIGER: Learning from Researcher Taste to Generate Tailored Research Ideas

Abstract

Large language models (LLMs) can generate research ideas, but their outputs often fail to match an individual researcher’s taste. We introduce Tailored Idea GEnerator for Research (TIGER), a framework that learns individual preferences to guide idea generation. Using 400 real-world research requests and 2,000 candidate ideas, we collect ratings from three LLM researchers and two PhD researchers. For each researcher, we train a separate Qwen3.8-27B reward model (RM) on their annotations, then fix its parameters and use its scores to optimize a Qwen3-8B ideation policy through reinforcement learning. Neutral rewriting and auxiliary quality checks guide optimization alongside the preference reward. In pairwise comparisons where each judge chooses between an idea generated by its corresponding TIGER-8B policy and one generated by GPT-4.1, the three LLM researchers select the TIGER-8B idea in 56.6%–68.4% of 76 comparisons per researcher. PhD 1 and PhD 2 select the TIGER-8B idea in 19 of 30 pairs (63.3%) and 17 of 30 pairs (56.7%), respectively. All six directed cross-policy comparisons likewise favor the judge’s matched TIGER-8B policy over an off-target policy, supporting the use of individual preference feedback for tailored research ideation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
research taste, personalized alignment, reward modeling, research idea generation, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sMXLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In order to train LLMs to generate research ideas that can better match human researchers' tastes, the authors collect real-world research idea ratings from LLMs and human PhD researchers and propose the TIGER framework to learn from the collected preferences. TIGER's reward design consists of eight components that guide the learning process towards researchers' tastes while try to ensure the research idea's quality. Win-rate estimates and other analyses show that the learned policy in general matches the researchers' preferences.

Critical strengths
  1. This work provides more data for studying "research taste personalization" with LLMs for the community. The scarcity of relevant data is one of the bottlenecks for studying LLM agents in the process of automatic scientific discovery.
  2. The reward design of TIGER covers a lot of aspects of research idea quality. It covers research ideas' general quality indicators and research taste. A carefully designed and well-justified reward would help the LLM agents a lot in adapting to autoresearch tasks.
Critical weaknesses
  1. The distinctive benefit of learning an individual researcher’s tastes is not sufficiently established. TIGER largely combines existing reward modeling and policy optimization techniques, so its contribution depends on demonstrating a substantive benefit from researcher-taste-specific adaptation. However, I feel that the current experiments do not adequately distinguish this benefit from general improvements in idea quality. This distinction is particularly important for the human results. The cross-policy generation comparisons in Table 2(b) only involve LLM judges. The two human researchers each compare their own policy against GPT-4.1, without comparing it against the other researcher’s policy or a shared policy. If this compison Consequently, the evidence remains preliminary regarding whether the system captures individual human research tastes or produces ideas that are generally more appealing. Given the limited methodological novelty, this unresolved attribution substantially weakens the demonstrated contribution. Besides, if benchmark dataset is one of the major contribution, it would be better to collect research tastes from more human researchers, two might not be enough.

  2. The evaluation does not sufficiently validate the specific reward design. The eight-term reward design in Eq. (1) is only partially examined. Table 3 removes the taste reward, removes all five quality penalties together, or removes neutral rewriting; format and length penalties remain in every variant. These experiments support the usefulness of the quality-penalty bundle, but do not establish which quality dimensions are necessary, whether some are redundant, or how sensitive the results are to their weighting relative to personal taste. If the componenets are redundant, maybe idea quality as one term should be enough?

Questions for authors
  1. What if we directly feed researchers' tastes to an LLM; will that achieve even better results? Describing a researcher's taste can be difficult, but I think we can ask LLMs to summarize it from preference data.
  2. What if we improve the granularity of the reward design's ablation study? The reward has many components, but we haven't clearly identified each component's contribution.
5 comments

We thank the reviewer for their constructive feedback and recognition of our presentation and soundness. To address the reviewer's concerns, we have updated the manuscript with additional human cross-policy comparisons and clarified the roles of different reward components.

W1 Benefit of learning an individual researcher’s tastes. We would like to clarify that our primary contribution is a framework for learning and aligning LLM-generated research ideas with an individual researcher's taste, rather than constructing a large-scale benchmark of human research preferences. As discussed in Section 2, defining universally "good" research taste is inherently challenging, as different researchers may have different perspectives and preferences. Our goal is therefore to learn and adapt to these individual preferences rather than optimize solely for general idea quality.

To directly address the reviewer's concern, we have added human cross-policy comparisons to Table 3(b). Both PhD researchers evaluate the same 30 candidate pairs generated by their respective policies:

Target researcherMatched policy selectedOther human's policy selected
PhD 166.7%33.3%
PhD 260%40%

Both researchers prefer their individually aligned policies over the other researcher's policy. They also prefer their matched policies over the three LLM-aligned alternatives. These results provide direct evidence that TIGER captures researcher-specific preferences rather than merely improving general idea quality.

Regarding the number of human researchers, we agree that involving more researchers would strengthen the generalizability of our findings.

However, our current objective is to demonstrate the feasibility of learning an individual's research taste from offline preference annotations. The framework does not assume a shared preference across researchers: each researcher provides their own ratings, from which a corresponding reward model and ideation policy are trained. We will clarify this scope and consider broader human preference collection in future work.

Q1: What if we directly feed researchers' tastes to an LLM. We appreciate this suggestion, which provides an important baseline that was missing from our initial evaluation. We have added a comparison between TIGER-8B and the same Qwen3-8B backbone using preference-based prompting. Specifically, we provide Qwen3-8B with four preferred/less-preferred idea pairs from the researcher's training annotations and ask it to generate new ideas conditioned on these examples. Across 30 held-out research requests, the researcher prefers TIGER-8B in 52/60 judgments (86.7%), with both A/B presentation orders evaluated. These results suggest that learning a personalized policy through reward-guided optimization provides substantial benefits over directly conditioning the same backbone on preference examples. We have included the experimental details and results in Section 4.3 and Appendix G.5.

W2 Roles of the reward components We appreciate the reviewer's suggestion to examine the reward components at a finer granularity. Our reward design separates the researcher-specific taste reward from five scientific-quality penalties (faithfulness, novelty, feasibility, significance, and coherence) and two output-control penalties (format and length). The five quality dimensions are evaluated in a single call to a frozen quality judge and combined into a scalar reward for policy optimization.

In Section 4.3 and Table 4, we examine three major components of the framework: the taste reward, the auxiliary quality-feedback bundle, and neutral rewriting. Full TIGER is preferred over the corresponding ablated variants in 67.1%, 90.8%, and 59.2% of comparisons, respectively. These results demonstrate the complementary contributions of personalized preference learning, scientific-quality feedback, and neutral rewriting.

We agree finer-grained ablations and comparisons with a single overall quality score would further help establish the contribution of each component. Regarding the finer-grained of each penalty component in Eq1, due to the computing resources, we will later add in the results (currently we are running). We currently update our manuscript with the limitation section.

We thank the authors for their detailed rebuttal. As we are not sure whether the reward design is necessary, I will keep my rating. In the next version, I would suggest that the authors verify whether removing the taste term from the reward still yields a similar human-LLM taste alignment percentage score as you showed in the rebuttal. If the alignment gets worse without the taste term, it can help to verify that learning research taste is necessary and beneficial, which will make this work more insightful.

We thank the reviewer for timely response and insightful feedback. We would like to provide results we just got and resolve the concern regarding the contribution of the taste reward. The researcher-specific RM supplies the personalized preference signal, while the five auxiliary penalties provide shared scientific-quality guidance.

We have updated Table 4, which evaluates a quality-only control that removes the taste reward while retaining the quality penalties scores. Under Terra’s judgments, full TIGER is preferred over this variant in 51 of 76 comparisons (67.1%). If we only have the taste reward and remove all penalties, it is 90.8%. This means learned taste reward and shared quality penalties both have contribution where taste reward is more important to reach human-LLM alignment.

We also revised our Eq.1 in Section 3.3 on Page 6 where the original form may be misleading. Our quality penalties are directly output by the LLM judge as a whole with our prompting the LLM for the requirement about faithfulness, novelty, etc. rather than 7 individual penalties term produced by the judge.

Together, the taste-removal ablation can conclude that human matched-policy comparisons support the value of learning researcher-specific preferences alongside shared scientific-quality guidance.

Reviewer_q1fmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TIGER, an framework designed to generate personalized research ideas that align with the individual preferences of researchers. The authors collected real-world research needs and also multiple candidate ideas, which were then annotated by three researchers specializing in LLMs and two doctoral researchers. For each individual researcher, TIGER trains a dedicated Qwen3-8B reward model to capture their specific preferences; subsequently, an Qwen3-8B-based idea-generation policy is fine-tuned using the GRPO. The framework also incorporates a mechanism and auxiliary quality penalties to prevent the reward model from being "gamed" (i.e., reward hacking) and to ensure the quality of the generated ideas. Experimental results based on pairwise preference evaluations demonstrate that the personalized TIGER-8B policy outperforms GPT-4.1, achieving a win rate of 56.6%–68.4% against LLM-domain researchers and 56.7%–63.3% against human researchers.

Critical strengths

Experimental results based on pairwise preference evaluations demonstrate that the personalized TIGER-8B policy outperforms GPT-4.1, achieving a win rate of 56.6%–68.4% against LLM-domain researchers and 56.7%–63.3% against human researchers.

Critical weaknesses

In my view, the paper has several notable shortcomings:

  1. It relies on human-annotated data, which is costly and prone to annotator bias.
  2. The methodological framework lacks any innovation.
  3. The Qwen3-8B model is relatively weak; if it fails to generate suitable ideas during the rollout phase, the entire batch will lack gradients, rendering learning impossible. The research could be strengthened by including the following experiments: measuring the variance of rewards within groups, calculating the proportion of sampled groups with an "advantage" value of zero, and investigating the impact of using a stronger base policy or a larger sample size.
Questions for authors

No

2 comments

We thank the reviewer for their time and constructive feedback, and appreciate their recognition of the importance and practical value of personalized research ideation. We address the concerns regarding human annotations, methodological novelty, and training dynamics below. We kindly ask the reviewer to reconsider their assessment in light of these clarifications and the updated manuscript.

W1 Human Annotator Data

It relies on human-annotated data, which is costly and prone to annotator bias.

We would like to clarify that our framework does not necessarily require human annotations, as we study both human and LLM researchers' tastes. Our main objective is to develop a framework that can learn and align with an individual researcher's preferences, rather than a universal definition of research quality.

For human researchers, collecting their preference judgments is a natural requirement for learning their individual tastes, as different researchers may have different preferences regarding research directions and ideas. In this setting, differences between researchers' judgments are not necessarily annotation bias, but may reflect the individual preferences that TIGER aims to capture.

Importantly, our framework learns from a fixed set of offline annotations and does not require additional human feedback during policy optimization. We acknowledge that collecting human preferences incurs an initial annotation cost, but our experiments with LLM researchers further demonstrate that the framework is not restricted to human supervision.

W2 Novelty and Contribution

The methodological framework lacks any innovation.

We respectfully disagree with this assessment and would like to clarify the contribution of TIGER. As discussed in the Introduction, research ideation differs from tasks such as mathematical reasoning, where correctness can be evaluated using relatively objective rewards. Research taste is inherently subjective, and different researchers may prefer different ideas even when their general scientific quality is comparable. This motivates our formulation of learning researcher-specific preferences rather than optimizing a universal quality metric.

TIGER contributes a researcher-specific training and evaluation framework for scientific ideation. Specifically, it learns a reward model from an individual researcher's offline judgments and uses this frozen model to optimize an ideation policy without requiring further annotations during training. Neutral rewriting and auxiliary scientific-quality feedback further support reward-guided optimization.

Importantly, our matched/off-target reward model and policy comparisons directly examine whether the resulting models capture individual researchers' preferences rather than merely improving general idea quality.

We have also added Table 1 in the revised manuscript to clarify the distinctions between TIGER and related approaches.

W3 Training Dynamics

The Qwen3-8B model is relatively weak; if it fails to generate suitable ideas during the rollout phase, the entire batch will lack gradients, rendering learning impossible. The research could be strengthened by including the following experiments: measuring the variance of rewards within groups, calculating the proportion of sampled groups with an "advantage" value of zero, and investigating the impact of using a stronger base policy or a larger sample size.

We appreciate the reviewer highlighting this potential optimization issue. We would like to clarify that we did not observe this issue in our training runs.

First, our reward model produces continuous-valued taste scores rather than discrete correctness rewards. Therefore, even when the generated ideas are imperfect, different rollouts can receive different reward values and provide relative learning signals for GRPO. More importantly, we examined our training logs and did not observe any sampled groups in which all samples had zero advantages. This indicates that Qwen3-8B generated sufficient reward variation for GRPO optimization in our experiments.

We agree that reward variance and the proportion of zero-advantage groups are useful diagnostics for understanding GRPO training. However, the concern about zero-advantage groups is primarily related to the choice of optimization algorithm rather than the proposed researcher-specific reward modeling framework. Our method is not inherently restricted to GRPO and could be used with alternative policy optimization methods.

We also acknowledge that stronger base policies or larger rollout groups may further affect training efficiency and performance. Our current results establish that Qwen3-8B provides sufficient reward diversity for successful optimization under the evaluated setting, rather than claiming that this configuration is optimal.

Dear Reviewer,

As the author-reviewer discussion period is approaching its end, we would appreciate it if you could take a moment to review our responses and let us know whether your concerns have been addressed. If you have any remaining questions or comments, we would be happy to respond before the discussion period ends.

Thank you very much for your time and thoughtful feedback.

Reviewer_EaovOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  • The paper studies personalized research ideation. Three LLMs and two PhDs each rate 2,000 ideas (400 requests × 5 candidates). A Qwen3.8-27B reward model is trained per rater, and a Qwen3-8B policy is trained with GRPO on the RM score minus quality, format, and length penalties.
  • The trained policies beat GPT-4.1 in 56.6–68.4% of 76 pairs under the LLM judges, and in 19/30 and 17/30 pairs under the two PhDs.
Critical strengths
  • A Interesting, practical problem of tailoring ideas to an individual researcher rather than to one universal notion of a "good idea".
  • The authors are aware of reward hacking, they use neutral rewriting and quality penalties, with component ablations.
Critical weaknesses
  1. The taste data is undocumented, and the evidence for personalization is weak.
  • The rating data is the core of the paper, yet almost nothing is said about it. It is not explained how the 400 requests were selected, what their quality is, who the two PhD annotators are, or how long they took to rate 2,000 ideas spanning ten fields. No example of a candidate idea is shown. -The only agreement analysis (Fig. 4) comes from an earlier, much smaller pilot, in which PhD 2 rated only 40 ideas. It does not report agreement between the two PhDs or test–retest consistency, so we cannot tell whether the models learn stable personal taste or annotation noise.
  • Human results are not significant, and there are no human cross-policy comparisons or ablations.
  • In Table 1, the matched reward model often beats off-target ones by less than one point. The results look more like an "LLM raters vs. human raters" split than individual taste.
  1. The evaluation is largely circular, and the baselines are weak.
  • LLM judges are the same models that trained the RMs. Ratings use a 7-dimension rubric sum, while evaluation asks "which do you prefer?".
  • The only strong baseline is GPT-4.1, GPT-5.6 Sol, prompting or profile-based personalization, and the target LLM itself are not compared.
  • Removing the quality judge makes the policy lose 90.8% of comparisons. The reward model alone can therefore be easily hacked, and much of the gain may come from the generic LLM quality penalties rather than personal taste. -Do the reward model split and the policy split overlap? This could cause possible leakage.
  1. Novelty: Methodologically, this is a direct application of personalized RLHF to ideation. Related work such as personalized reward models (VPL, P-GenRM, LDC (RL with multi-dimensional ideation rewards) ) is not discussed.
Questions for authors
  • How were the 400 requests selected from the platform? Who are the two PhD annotators, and how long did annotation take? Can you report agreement between the two PhDs, and test–retest agreement, on the full dataset?
  • How does TIGER compare with GPT-5.6 Sol, few-shot preference prompting, and directly using the target LLM as the reward?
  • The 27B reward model is trained on only about 1600 ideas. Have you studied how reward model size and policy size affect the results?
4 comments

We thank the reviewer for the constructive feedback and for recognizing the importance of our research problem. We have substantially revised the manuscript to address the concerns regarding personalization, evaluation, and methodological novelty. In particular, we have added direct human cross-policy comparisons, comparisons against the original Qwen3-8B and preference-prompting baselines, an additional 8B reward-model evaluation, and more detailed descriptions of our data collection and training procedures. We address the concerns below.

W1& Q1 Research request selection

The rating data is the core of the paper, yet almost nothing is said about it. It is not explained how the 400 requests were selected, what their quality is?

We agree rating data is one of the core components while our main contribution focuses on the pipeline and methods on how we align LLMs with research tastes using our reward model, rather than dataset or benchmark. We updated Section 3.1 to clarify the construction.

The rating contains 400 research requests and 2,000 candidate ideas. Each of the two PhD researchers and three LLM researchers rate all 2,000 ideas, yielding 10,000 annotator–idea evaluations. Human annotations use a five-point overall preference scale from −2 (strongly dislike) to +2 (strongly like). Each LLM provides seven component scores on a 0–100 scale, summed to a total of 0–700. The 400 requests are retained from 423 eligible requests after deduplication and context-quality filtering of 937 queries on an anonymized platform.

Both PhD annotators have experience reviewing at least ten papers at major ML venues and received a one-month annotation window. Candidate identities are hidden, and annotations are retained separately for each researcher. We have also included a concrete example of generated research ideas in Figure 3, with the complete original descriptions provided in Appendix B.

Who the two PhD annotators are, or how long they took to rate 2,000 ideas spanning ten fields. Who are the two PhD annotators, and how long did annotation take?

The two human annotators are PhD researchers who each have experience reviewing at least ten papers for major ML conferences, including ICLR, NeurIPS, and ICML. Each researcher was allocated a one-month annotation window to evaluate 2,000 candidate ideas. To reduce potential evaluation bias, candidate model identities and generation order are hidden during annotation.

No example of a candidate idea is shown.

We would like to clarify that Figure 3 provides a concrete example of research ideas generated before and after RL optimization, while Appendix B includes their complete original descriptions.

The only agreement analysis (Fig. 4) comes from an earlier, much smaller pilot, in which PhD 2 rated only 40 ideas. It does not report agreement between the two PhDs or test–retest consistency, so we cannot tell whether the models learn stable personal taste or annotation noise. Human results are not significant, and there are no human cross-policy comparisons or ablations.

We appreciate the reviewer highlighting the importance of distinguishing individual preferences from annotation noise. Figure 4 reports an earlier exploratory pilot, while our main study uses 2,000 ratings per researcher.

To provide stronger evidence of personalization, we have additionally introduced direct human cross-policy evaluations on newly generated ideas. Both PhD researchers evaluate the same 30 candidate pairs generated by their respective aligned policies. PhD 1 prefers their matched policy in 20/30 comparisons (66.7%), while PhD 2 prefers their matched policy in 18/30 comparisons (60.0%). Both also prefer their matched policies over the three LLM-aligned alternatives.

These results provide additional behavioural evidence that the learned preferences influence the generation of new ideas beyond the original annotation examples.

In Table 1, the matched reward model often beats off-target ones by less than one point. The results look more like an "LLM raters vs. human raters" split than individual taste.

We appreciate this concern and have added cross-policy evaluations to distinguish individual preferences from general differences between LLM and human evaluators.

As shown in the updated Table 3(b), all three LLM researchers and both human researchers prefer their corresponding aligned policies over the evaluated off-target alternatives. In particular, both human researchers prefer their own aligned policies over the other human researcher's policy, providing direct within-human evidence of individual preference differences.

These results demonstrate that differences in learned preferences can also be reflected in downstream idea generation and direct preference judgments.

W2&Q2. Evaluation protocols and baselines

LLM judges are the same models that trained the RMs. Ratings use a 7-dimension rubric sum, while evaluation asks "which do you prefer?".

We would like to clarify that our objective is to generate ideas preferred by a particular researcher. Therefore, the same target researcher intentionally provides both the offline preference annotations and the final evaluation judgments. This design directly evaluates whether the generated ideas align with the intended researcher's preferences. Importantly, this does not mean that the trained RM evaluates its own outputs. The offline annotations train a separate 27B reward model, while the final evaluation uses blinded pairwise judgments on newly generated ideas, without relying on the learned RM scores.

Furthermore, the seven-dimensional rubric scores supervise within-query preference rankings during RM training, whereas the final evaluation uses direct pairwise preference judgments. This tests whether the learned preferences transfer from rubric-based annotations to direct judgments of generated ideas. Human experiments follow the same evaluation principle.

The only strong baseline is GPT-4.1, GPT-5.6 Sol, prompting or profile-based personalization, and the target LLM itself are not compared.

We appreciate the reviewer's suggestions and have added comparisons against both direct generation and preference-based prompting using the same Qwen3-8B backbone.

Specifically, we introduce a few-shot preference-prompting baseline by providing Qwen3-8B with four preferred/less-preferred idea pairs from the researcher's training annotations. Across 30 held-out requests evaluated in both A/B orders, Terra prefers TIGER-8B in 52/60 judgments (86.7%). This demonstrates the benefit of personalized policy optimization over directly conditioning the same backbone on preference examples (Section 4.3; Appendix G.5).

We also compare TIGER-8B against direct generation and personalized-RM best-of-8 selection from the original Qwen3-8B model. TIGER is preferred in 60/76 comparisons (78.9%) against each baseline. These results suggest that the improvement cannot be explained solely by selecting higher-reward ideas from the original model.

Regarding directly using the target LLM as the reward source, our framework instead trains a frozen RM from offline annotations and reuses it throughout policy optimization. This avoids repeatedly querying the target evaluator during RL training and enables personalization for human researchers without requiring additional annotations for each rollout.

Removing the quality judge makes the policy lose 90.8% of comparisons. The reward model alone can therefore be easily hacked, and much of the gain may come from the generic LLM quality penalties rather than personal taste. -Do the reward model split and the policy split overlap? This could cause possible leakage.

We would like to clarify that we also evaluate the complementary ablation, where the personalized RM is removed while retaining the same generic quality, format, and length penalties. Full TIGER is still preferred in 51/76 comparisons (67.1%). This comparison directly demonstrates that the researcher-specific reward provides additional value beyond generic scientific-quality feedback.

Furthermore, the matched RM achieves the highest preference prediction accuracy for all five target researchers, providing independent evidence of its ability to learn individual preferences.

These results demonstrate the complementary roles of the two reward sources: the personalized RM captures individual research preferences, while the quality judge helps prevent reward exploitation and maintain scientific quality during RL training. Our objective is to combine researcher-specific preference alignment with general scientific-quality constraints, rather than relying on either reward source alone.

Regarding potential data leakage, we clarify the data splits in Sections 4.1 and 4.2. RM training uses 320 training requests and 80 validation requests, with entire requests kept together. Policy optimization uses 324 training requests and reserves 76 requests for held-out generation evaluation.

The 27B reward model is trained on only about 1600 ideas. Have you studied how reward model size and policy size affect the results?

Thank you for this question. We have investigated the effects of both reward model size and policy size. For reward model size, when evaluated on ideas generated by GPT-Terra, the 8B and 27B reward models achieved strict pairwise accuracies of 70.43% and 77.35%, respectively, a gain of 6.92 percentage points. For policy size, training of the 27B policy is still ongoing. At training step 40, it performed comparably to GPT-5.6 sol. Meanwhile, the judge showed little preference for ideas generated by the 8B policy over those generated by GPT-5.6 sol.

W3&Q3 Novelty and Our Contribution

Novelty: Methodologically, this is a direct application of personalized RLHF to ideation. Related work such as personalized reward models (VPL, P-GenRM, LDC (RL with multi-dimensional ideation rewards) ) is not discussed.

We appreciate the reviewer highlighting these relevant works and have expanded Section 2 and added Table 1 to clarify our methodological distinctions.

We agree that RL optimization is an established technique. However, our contribution is not a new general-purpose RL algorithm, but a researcher-specific training and evaluation framework for scientific ideation, where research taste is inherently subjective and cannot be evaluated using a universal correctness reward.

Specifically, VPL models latent user preferences, while P-GenRM derives personalized rewards from user history and personas. LDC focuses on controlling scientific-quality dimensions such as novelty, feasibility, and effectiveness. In contrast, TIGER learns an individual researcher's preferences directly from their ratings of candidate research ideas and trains a corresponding compact ideation policy.

Our training design separates personalized taste rewards from shared scientific-quality constraints. The researcher-specific RM scores neutrally rewritten ideas, while auxiliary penalties assess their scientific quality. Moreover, matched/off-target RM and policy comparisons, including direct human cross-policy evaluations, examine whether the resulting generator follows the intended researcher's preferences.

We have updated Table 1 to explicitly summarize these distinctions. We believe the contribution lies in formulating and empirically validating personalized scientific ideation through researcher-specific preference learning, rather than simply applying generic RLHF to improve idea quality.

Dear Reviewer,

As the author-reviewer discussion period is approaching its end, we would appreciate it if you could take a moment to review our responses and let us know whether your concerns have been addressed. If you have any remaining questions or comments, we would be happy to respond before the discussion period ends.

Thank you very much for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.