acceptodds
Under review as a conference paper at ICLR 2027

Training Reasoning Models to Generate Diverse Scientific Ideas

Abstract

One of the first steps of a research pipeline is generating ideas, and good ideation means laying out several distinct directions. Large Language Models (LLMs) are increasingly used to automate this step, but most work optimizes the quality of individual ideas, overlooking diversity. In our work, we focus on increasing the diversity of LLM generated ideas, while making sure they are high quality. We study this for document-grounded ideation, where, given prior papers, a model must propose a set of ideas that are each high-quality and clearly distinct from one another. To optimize for this, we propose training with a set-level reinforcement learning objective. In a single generation, the model writes K structured idea cards grounded in the source papers, and the set is scored jointly, where each idea's quality is weighted by its distance from the ideas written before it. This teaches the model to steer each new idea away from its predecessors. Our trained 8B model achieves higher creative utility (a measure combining quality and diversity) than Claude Opus 4.8 and GPT-6 Astra on our contamination-free benchmark. In a pilot human study with four experts, our sets are judged as promising as Claude Opus 4.8's and contain marginally more unique ideas. When a coding agent runs proof-of-concept experiments on the ideas, our model's success@k is comparable to Claude Opus 4.8's, so we see no evidence that the added diversity costs executability.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Reasoning, Reinforcement Learning, AI for Research

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XmVXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper studies document-grounded scientific ideation: given summaries of several source papers, a model must write K = 3 structured idea cards (problem, claim, method, inspiration) that are each high-quality and mutually distinct. Training uses a set-level reward adapted from Creative Utility, in which each idea’s quality is multiplied by its minimum distance to the ideas written earlier in the same generation. Quality is a pairwise impact judgment against the held-out “gold” idea of the target paper, made by SciJudge-30B; distance is an LLM-judged field-wise similarity from gpt-oss-120b. Qwen3-8B is first fine-tuned on Claude-generated idea sets with hindsight reasoning traces and then trained with GRPO. On a test set built from 2026 papers, the no-think model reaches a CU of 0.64, compared with 0.38 for Claude Opus 4.8 and 0.52 for a retrained DARLING baseline. A four-expert pilot study finds parity with Claude, and a 15-instance execution study with a coding agent finds comparable success@k.

Critical strengths
  1. A clean demonstration that in-generation, set-level diversity rewards avoid collapse. With the same base model, data and judges, impact-only RL collapses the set (distance 0.38), and DARLING collapses idea types on IdeaSeed. In contrast, the proposed objective keeps both quality and diversity high (CU 0.64, distance 0.88).
  2. External checks go beyond the training metric. The paper reports grounding, population-level type diversity, a blinded expert study and an execution study. Each is small, but together they suggest the gains are not purely reward hacking and that an 8B model reaches parity with Claude Opus 4.8 in expert judgment.
Critical weaknesses
  1. The sample-then-select baseline is missing. The most natural alternative is to draw N independent ideas from the SFT model or Claude and then select K with the same quality × distance greedy procedure used to compute CU. Without this baseline, it is unclear whether training a model to diversify within one generation is better than generating and then selecting.
  2. What is the inter-annotator agreement in the human study, and are any differences between Proposed and Claude statistically significant? How were the 60 instances selected?
  3. DARLING is trained with K = 1. How is it evaluated at K = 3? Would DARLING with the same set-format SFT be a fairer comparison?
Questions for authors

see above

3 comments

Thank you for your feedback and review! Addressing the weaknesses and questions below:

The sample-then-select baseline is missing.

In Table 12 we report set vs sampling performance. We see that set/single generation has higher diversity compared to sampling based. We also run a sample-then-select baseline. For Claude Opus 4.8, GPT 6 Astra and Qwen3-8B we sample 9 times, and use the CU greedy algorithm to pick the best 3. For scoring and selection we use the training judge and then re-score the selected set with our evaluation judges. Results:

Model3 independent samples3 ideas in one generationbest 3 of 9 (selected)
Claude Opus 4.80.330.380.38
GPT-6 Astra0.260.280.27
Qwen3-8B0.040.030.04

For Claude, sample then select (or best of n) performs at par with set level generation, still performing lower than our trained models (0.64/0.51, Table 1). We want to note this sample-then-select method is an oracle selected method. Selection uses the target ideas, which are not available at inference time. We also want to mention that this baseline is more expensive than our propsoed method e.g. this uses 9 generations per instance instead of 1.

What is the inter-annotator agreement in the human study

We present a human evaluation in Section 6.1 and Appendix D. In that each expert gives judgments on 15 independent instances, so the design has no shared items so an item-level IAA cannot be computed. We instead measure agreement at the system level. For each expert, they rank 15 instances across- three systems, and with four experiments, we get a Kendall's tau of 0.56 for "promising and unique ideas" (average pairwise kendall's tau) . All four experts agree on the base model's ranking (i.e. last). However, they split 2–2 between our model and Claude, consistent with the parity we report.

differences between Proposed and Claude statistically significant

metricProposed − Claude Opus 4.8Proposed − Qwen3-8BClaude Opus 4.8 − Qwen3-8B
Promising−0.03 [−0.25, +0.20]+0.65 [+0.40, +0.90]+0.68 [+0.42, +0.95]
Unique+0.12 [−0.05, +0.28]+0.32 [+0.12, +0.52]+0.20 [+0.00, +0.40]
Unique Promising+0.02 [−0.23, +0.27]+0.63 [+0.40, +0.87]+0.62 [+0.38, +0.85]

We report paired differences, 95% bootstrap CIs, bold means significant after Bonferroni correction over the 9 tests. We find no difference between our proposed system and Claude, which is still notable for an 8B model compared to a frontier model.

How were the 60 instances selected?

The 60 instances come from the NLP subset of our test set. Each expert chooses their area of expertise (e.g. agents, evaluation, bias and ethics etc). An LLM classifier tags each test paper with its ARR subarea. We then randomly sample 15 instances per expert that match their expertise. No step in this selection chooses ideas or instances based on their scores. Experts were also blind to the system name when giving judgments. We mention the details in Appendix D (line 1212) and will make it more clear.

DARLING is trained with K = 1. How is it evaluated at K = 3

DARLING generates one idea per call, as it was trained. For each test instance we sampled 3 single-idea rollouts and took that as the set of ideas generated from DARLING (Section 4). Since DARLING's diversity reward is computed across a group of independent samples, independent sampling is its intended way to produce several ideas.

Would DARLING with the same set-format SFT be a fairer comparison?

We trained DARLING + SFT. We retrained DARLING from the same SFT (no-think) checkpoint that our method starts from (but with k = 1). We train it with a length penalty and for the same number of steps. We see the following:

ModelInitCU>=1 WinAvg. distance
DARLING (Table 1)base0.520.590.67
DARLING + SFT (new)our SFT0.490.650.62
Proposed (think)our SFT (think)0.510.620.83
Proposed (no think)our SFT0.640.780.88

Even starting with our SFT model does not close the gap for DARLING. While it raises the best idea per set (>=1 Win goes from 0.59 to 0.65), the sets DARLING + SFT generates become less distinct (distance 0.67 goes down to 0.62). Our no-think model still beats DARLING + SFT. We will add this result to the updated draft of the paper.

Thank you again for your thoughtful review. As the discussion period is nearing its end, we would like to kindly follow up on our rebuttal and ensure that we have adequately addressed your concerns. We would greatly appreciate any further feedback you may have, and we would be happy to clarify any remaining questions!

THank you for your response. I have increased the soundness score

Reviewer_rL9zOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary
  • Task: document-grounded research ideation. Given summaries of several source papers, the model writes K=3 structured idea cards (problem gap / core claim / method / inspiration) in a single generation.
  • Method: a set-level RL reward, adapted from Creative Utility (CU). Training is SFT on Claude-Opus-4.8 ideas with hindsight-synthesized reasoning traces, followed by GRPO on Qwen3-8B, with think and no-think variants.
  • Data: training targets come from the GiantsBench train split (cs.CL, pre-2023). The test set is built with the same LLM pipeline from 2026 target papers (1,740 instances, 16 domains).
  • Validation: Main metric is CU computed with different evaluation judges (GPT-5.4-mini for quality, gemini-3.5-flash for distance). Proposed (no-think) reaches CU 0.64, vs Claude Opus 4.8 at 0.38 and DARLING at 0.52. GiantsBench comparison against GIANTS-4B and an IdeaSeed taxonomy analysis. Pilot human study: 4 annotators × 15 instances, where the model is on par with Claude. Coding-agent execution check: 15 instances, success@k comparable to Claude.
Critical strengths
  • Clear problem framing with a simple, general objective. Treating a set of ideas as the unit of generation and penalizing redundancy within one generation is well motivated. The Impact-only RL ablation shows that without the distance term the set collapses (avg. distance 0.38). This is the paper's most convincing evidence that the objective does something.
  • Reasonably thorough ablations and some candor about limitations.
  • Sampled vs. set prompting (Table 12), no-SFT, think vs. no-think, and transfer to OLMo-3.
  • The authors openly report the grounding drop and the loss of population-level type diversity.
Critical weaknesses
  1. Claims rest on LLM judges the model is trained against, and the evaluation is not convincing.
  • The training judge is weak: 65% on the authors' own validation, and it shares the impact prompt with the evaluation judge.
  • An 8B model beating real published gold ideas 50–68% of the time suggests judge exploitation, not better ideas.
  • The human study (4 annotators, 15 non-overlapping instances each, NLP only) is too small, and it shows a tie with Claude, not the CU gap.
  • The execution setting (90 min, $30, no GPU) can only test simple benchmark-style ideas, as also in the case studies show. The agent also revises its plan after seeing results and self-reports the verdict.
  • Optimizing a quality+distance metric naturally raises that metric, so the main result is not a strong contribution on its own.
  1. Limited novelty, unfair baselines, and an incomplete literature review.
  • The objective is essentially CU used as a reward in the Puri et al. (2026) multi-answer setting.
  • DARLING is trained without SFT or a length penalty. Without SFT the two objectives tie (0.51 vs 0.52), so the gain seems to come from SFT.
  • Closed models are only zero-shot, with no best-of-N or reranking.
  • GIANTS-4B is trained for similarity but evaluated on beating the gold idea, on a non-standard subset.
  • Grounding drops sharply (0.61 vs 0.98), which CU ignores.
  • Missing related work includes multi-agent ideation (VirSci / "Many Heads Are Better Than One"), ResearchTown,LDC (SFT+RL with novelty rewards), and QD-RL methods such as MoDA.
Questions for authors
  • What happens with DARLING + the same SFT, and with closed models plus best-of-N reranking?
  • How was the 2,374-instance GiantsBench subset chosen? How much do test sources overlap with training data?
  • Is "contamination-free" checked for the judges and closed baselines? Is the LLM-built benchmark validated by humans or any quality control?
3 comments
AuthorsOpenReview ↗

Part 1 of the response to Reviewer rL9z

Thanks for your review! Addressing your concerns below:

The training judge is weak [..] shares the impact prompt

The training judge (SciJudge 30B) and the evaluation judge (gpt-5.4-mini) are different models and families and were validated (Appendix C.1 Scijudge scores 0.65 and gpt-5.4-mini scores 0.74). We train against a smaller judge because a reward is queried on every rollout, increasing the cost by quite a lot, because of which we went ahead with a smaller, cheaper model trained for this task (Table 8).

The two judges share the same pairwise impact prompt. However, given that the training and evaluation judges are from different model families and have different priors, we believe they do not share similar biases. Moreover, we have other evidence in the submission that do not depend on the shared prompt and also evaluate other aspects of quality (human and execution study).

An 8B model beating real published gold ideas

We don’t believe it is exploitation since the training and evaluation judges are different model families. Moreover, we use length-penalty for training and length-controlled win-rates to avoid any obvious exploitation of the reward.

The human study (4 annotators, 15 non-overlapping instances each, NLP only) is too small

The human study does not claim that it tracks the CU gap (please see response to reviewer AYWN). The quality measure for the human study is about promising ideas, which is different from the quality measure of CU (which is more around long-term impact in terms of citations). Paired over the 60 papers, experts rate our model on par with Claude Opus 4.8 on every metric (e.g. uniquely promising ideas: +0.02 [−0.23, +0.27]) and clearly above its own base model (+0.63 [+0.40, +0.87], p < 0.0001). All 4 annotators rank the base model last.

The execution setting (90 min, $30, no GPU) can only test simple benchmark-style ideas

The execution study is meant to check a proof of concept result for the generated ideas as executed by agents. The coding agent is specifically instructed to not change the scope of the claim, the revision it does is to refine the experiment etc to draw conclusions for the claim. We agree that the results in the paper are self-reported. We also run an additional study with an audit model from a different family i.e. GPT-5.5. After the audit we see Proposed (think) still maintains all the verdicts as before, whereas our no-think model drops in success rate ( think 29% vs Claude 27%, no-think 20% at k = 3; the full table is in reviewer AYWN's response).

Optimizing a quality+distance metric naturally raises that metric,

Agreed, but along with improvements to the metric we also show the following contributions: (a) evaluating with a different judge for each component (Section 4), (b) ablations of impact-only training (where the diversity collapses) (Table 1), (c) set vs sampled generation (Table 12) (d) transfer to other model families (Olmo3, Appendix D, Table 14) and (e) human and execution studies (Table 4,5) . Also we report our optimized quantity following prior work (preference accuracy for RLHF, or distinctness for DARLING).

DARLING is trained without SFT

When comparing the DARLING and our trained pipeline without SFT, the CU is similar, but they are similar for different reasons. DARLING has a higher win-rate and lower diversity whereas our proposed model (without SFT) has a slightly lower win-rate but higher diversity, which is what the objective is meant to do.

We also retrained the DARLING baseline but this time with our trained SFT (no think) model (with k = 1). We train it with a length penalty and for the same number of steps. We see the following:

ModelInitCU≥1 WinAvg. d
DARLING (Table 1)base0.520.590.67
DARLING + SFT (new)our SFT0.490.650.62
Proposed (think)our SFT0.510.620.83
Proposed (no think)our SFT0.640.780.88

Even starting with our SFT model does not close the gap for DARLING. While it raises the best idea per set (>=1 Win goes from 0.59 to 0.65), the sets DARLING + SFT generates become less distinct (distance 0.67 goes down to 0.62). Our no-think model still beats DARLING + SFT. We will add this result to the updated draft of the paper.

AuthorsOpenReview ↗

Part 2 of the response to Reviewer rL9z

Closed models are only zero-shot, with no best-of-N or reranking.

In Table 12 we report set vs sampling performance. We see that set/single generation has higher diversity compared to sampling based. We also run a sample-then-select baseline. For Claude Opus 4.8, GPT 6 Astra and Qwen3-8B we sample 9 times, and use the CU greedy algorithm to pick the best 3. For scoring and selection we use the training judge and then re-score the selected set with our evaluation judges. Results:

Model3 independent samples3 ideas in one generationbest 3 of 9 (selected)
Claude Opus 4.80.330.380.38
GPT-6 Astra0.260.280.27
Qwen3-8B0.040.030.04

For Claude, sample then select (or best of n) performs at par with set level generation, still performing lower than our trained models (0.64/0.51, Table 1). We want to note this sample-then-select method is not possible at inference time (in real life) because we do not have access to “target ideas” to compare against (so this is oracle).

GIANTS-4B is trained for similarity but evaluated [...]

GIANTS-4B is an open source model trained on this exact data source, and it represents another standard training recipe, which is to train to generate one idea to match the gold. We compare against it to test whether that recipe produces ideas that are good and not only similar to the gold. It is not meant to be a like-for-like objective comparison. We will make that framing explicit.

Grounding drops sharply (0.61 vs 0.98), which CU ignores.

This is a limitation of our method, since we do not include it in the reward formulation. We talk about this in Appendix C.3 (line 1030). However, we do measure a faithful-CU post-hoc, where we gate each idea on our faithfulness judge (faithful = 1 if score >= 3, otherwise 0) and multiply this gate into the quality term in Equation 5, so any unfaithful idea scores 0. The full table is in reviewer AYWN's response, but a snapshot of it is:

ModelCUfaithful CU
Claude Opus 4.80.380.38
DARLING0.520.21
Proposed (no think)0.640.52
Proposed (think)0.510.44

Our model still achieves the best performance. Faithful CU also separates Proposed (think) and DARLING where they initially score nearly the same, and post-grounding gate, the latter falls from 0.52 to 0.21. We will add the table to the appendix and report grounding/faithful alongside CU in the main results.

How was the 2,374-instance GiantsBench subset chosen [...]

GiantsBench-test has 7,504 rows across 16 domains. We took the first 150 rows of each domain in the released order, with no filtering on content or scores, to keep inference and judging affordable (one of the domains only has 124 instances, which is why this is not 2400). The gold ideas between training and testing are split by year, so there is no overlap. Only 4.7% of unique sources are shared between training and testing and these are the more foundational/high cited sources like "Attention is all you need" etc.

Is "contamination-free" checked for the judges and closed baselines?

Contamination free is with respect to the policy we are training and the judge we are using for the training. No test target (which was collected between January 2026 and June 2026) was seen in the training. The knowledge cut offs for the judges and closed baselines is the following: (a) gpt-5.4-mini: August 2025 (b) SciJudge-30B: preference pairs are collected till 2024 (c) Claude-Opus-4.8: January 2026 (d) GPT-6-Astra: April 30, 2026

This means that our evaluation judge i.e. gpt-5.4-mini, which produces CU, has a cutoff before every target, so the evaluation judge cannot have seen the gold papers.

Is the LLM-built benchmark validated by humans or any quality control

The train and test IDs come from GiantsBench. Our pipeline finds the important sources, writes the summaries and the gold idea cards. Before fixing the format, we ran several rounds of internal validation for the following: the idea-card format and its level of detail; how detailed the source summaries should be; and whether the selected sources are actually important to the target paper. We can describe this process in the appendix.

Missing related work

Thank you for the pointers. For our work, we study diversity within a single model’s set of ideas instead of multi-agent systems. LDC does not focus on diversity of generated ideas (which is different from novelty). MoDA is the closest to ours, it also gates diversity on quality, but measures diversity across independently sampled responses, much like DARLING, while we optimize it within one generation (our set-vs-sampled ablation in Table 12 compares these settings directly). We will add these to our related work section.

Thank you again for your thoughtful review. As the discussion period is nearing its end, we would like to kindly follow up on our rebuttal and ensure that we have adequately addressed your concerns. We would greatly appreciate any further feedback you may have, and we would be happy to clarify any remaining questions!

Reviewer_AYWNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on the task of generating a variety of scientific research ideas given summaries of previous papers. It frames Creative Utility as a reinforcement learning reward, such that a model produces three structured 'idea cards' in a single response and the scientific impact rating of each idea is scaled by its semantic distance from the previously output ideas. Training involves supervised fine-tuning on sets of ideas generated by a teacher (optionally with synthetic reasoning traces), followed by GRPO.

The main experiments are conducted with Qwen3-8B, with additional experiments with Olmo-3-7B. The evaluation consists of a newly constructed test set of 1,740 instances across 16 domains, GiantsBench, IdeaSeed, a human evaluation with 4 experts over 60 instances, and a small execution evaluation with a coding agent. A non-thinking variant performs best, outperforming the reported trained and closed-model baselines on the automated Creative Utility measure. Human evaluation reveals similar numbers of promising ideas and uniquely promising ideas compared to Claude Opus 4.8. The authors also present large decreases on their source-dependence metric and some decreases on diversity at the population level after RL.

Critical strengths
  1. The impact-only ablation is informative for explicitly rewarding diversity.

The paper tackles an important issue: maximizing the apparent quality of individual ideas can lead to a redundant set. This is quantified in Table 1: While impact-only RL can reach a win rate of 0.43, controlled for length, the mean distance within a set is low at 0.38. The full non-thinking recipe reaches 0.50 and 0.88, respectively, with Creative Utility going from 0.38 to 0.64. This shows the value of including diversity in the training objective for the protocol used in the paper. In Table 12, the separation between independent sampling and joint prompting is also informative to separate the training effect from that of just asking for multiple ideas.

  1. The findings show promising evidence that a trained small model can provide useful suggestions, with some generalization to another model family.

The Olmo experiment replicates a significant gain in automated utility: Going from 0.17 SFT to 0.56 RL with the non-thinking model (Table 14). And it’s good to see a meaningful safeguard in different training and evaluation judges. But more importantly, the blinded expert pilot finds almost equal numbers of promising and uniquely promising ideas for the trained 8B model as for Claude Opus 4.8 (Table 4). This doesn’t say that it’s better, but being able to get at least equally good proposals from a small model is potentially useful. The authors also present some negative findings, such as reduced source dependence and population-level diversity, which inform about the limitations of the recipe.

Critical weaknesses

No evidence is shown in the evaluation that the increased utility scores result in a corresponding increase in document-grounded scientific quality as expected from the task.

The most immediately obvious issue is with the grounding analysis. Section 4 explains that this is about evaluating how important and accurate the stated document support is with respect to what those documents actually say. The judge for Prompt 8 is told explicitly that he cannot see the source documents. The only information he has is the generated insight and the model's own justification of why it was inspired. At best this can evaluate whether the information appears to depend on the stated support, but there's no test that the cited sources do in fact include the stated findings and that the statement is in fact accurate. There's nothing to stop a model giving an accurate but incorrect justification for its sources.

This is a problem because impact and diversity do not directly test this requirement. The impact judge is considering the potential future scientific influence, and the distance judge is considering differences between ideas. It is thus possible for the Creative Utility score to go up without a corresponding improvement in document-groundedness. In fact the paper itself diagnoses this issue: source dependence drops from 0.90 with non-thinking SFT to 0.61 with RL (compared to 0.98 for Claude, Table 11). It also decreases from 0.79 to 0.56 to 0.48 across the three positions in the winning model's own output (Table 15). A lack of dependence does not necessarily mean a lack of grounding and science, but it does mean that grounding must be double-checked against the actual sources.

The independent studies don't help much. Only 2.05 and 2.03 of the counts of human uniquely promising ideas are for the proposed model and Claude respectively (Table 4). Execution success@3 breaks down to 5/15 versus 4/15, where the coding agent sets up the experiment and decides the outcome (Section 6.2). The authors themselves note that some of the "supports" decisions shown are stronger than the evidence (Appendix E). These are helpful exploratory experiments, but they don't show that the larger utility scores do indeed represent a corresponding increase in accurately grounded and defensible scientific directions. I'm glad that the paper calls them pilot studies. The key point with respect to the main result is the one that's still missing.

Questions for authors
  1. Controlled analysis required for method claim. How does DARLING perform with SFT initialization using an equivalent task-specific SFT run with matched length treatment and clipping, and a clearly reported training budget? What evidence is there that isolates the value of constraining the diversity reward to a single generation, if the CU of the proposed no-SFT model (0.51) is only marginally lower than DARLING's (0.52)?

  2. Reanalysis of the independent verification. How well do changes in CU on the 60 human-scored examples predict changes in uniquely promising ideas? Can you give the paired differences (and uncertainties) for the human comparison? Was there any independent auditing for the execution study to verify that "supports" decisions were meeting the specified criteria? These would help determine the level of confidence one can place in the automated improvements.

4 comments
AuthorsOpenReview ↗

Part 1 of the response to Reviewer AYWN

Thank you for your comments! We are addressing the weaknesses and questions below:

there's no test that the cited sources do in fact include the stated findings[..]

The grounding score in the submission measures source dependence. To address this, we added a faithfulness judge (gemini-3.5-flash) that looks at the source cards and scores if the cited sources in the ideas establish what the sources say. We use a rubric that scores how well the ideas represent the source cards. This is on a scale of 1-5 (with 1 being misrepresentation, 3 being partially supported and 5 being fully supported). The two trends for the models we train are:

  • Qwen3-8B, SFT (no think), Proposed (no think) : 3.58, 4.26, 4.01
  • Qwen3-8B, SFT (think), Proposed (think) : 3.58, 4.28, 4.18

RL seems to lower the faithfulness slightly, but the proposed models still stay above the base model and near the score 4. The models with no SFT drop down to ~2 so they tend to misrepresent the sources more. We will add the full table to the appendix!

impact and diversity do not directly test this requirement. [..] Creative Utility score to go up without a corresponding improvement in document-groundedness.

We agree that in the current framing, CU does not include grounding/faithfulness against source documents. We mention in Appendix C.3 (lines 1030) as to why we do not include this in the training.

However, we can measure document-grounded CU post-hoc. With the faithfulness judge introduced above, we gate the idea to be faithful if it scores >=3 (1) else it’s not faithful (0). We add this gate as a multiplier to quality in Equation 5, so any unfaithful generation gets a score of 0. We see the following results:

ModelCUfaithful CU
Claude Opus 4.80.380.38
GPT-6 Astra0.280.28
DARLING0.520.21
Impact+diversity, no SFT0.510.19
Impact-only RL0.380.35
Proposed (no think)0.640.52
Proposed (think)0.510.44

Our models still achieve the best values under faithful CU. The difference with Claude Opus 4.8 is +0.14 (no think) and +0.06 (think) (paired over the test instances, p < 0.001). Without faithfulness, DARLING and our objective without SFT match Proposed (think) on CU, but when looking at faithful CU, our model (no think) is ahead of both by +0.31 and +0.33 resp. We can add this table in the appendix!

It also decreases from 0.79 to 0.56 to 0.48 across the three positions in the winning model's own output

We agree that the dependence of the proposed ideas on the sources decreases between SFT and RL and across the three positions. But note that the faithfulness doesn’t drop by that much (see the response above). We also check how faithfulness changes across positions for the proposed (no think):

PositionDependence/grounding (pre-normalization) (1–5)Faithfulness (1–5)
14.154.09
23.224.03
32.923.91

Later ideas seem to lean on the sources less but still represent them correctly. This is consistent with your remark that lower dependence need not mean a lack of grounding. One possible explanation (that we observed) is that the model starts having loose inspiration from the sources rather than being more dependent on them since it needs to find “new” ideas as it goes further down in the generation. With thinking, the decline is smaller (dependence 4.51 to 4.02, faithfulness 4.24 to 4.10). We will add this per-position faithfulness breakdown next to Table 15.

Only 2.05 and 2.03 of the counts of human uniquely promising ideas[..]

The human studies show no significant difference between Claude Opus 4.8 and our proposed model, which means that experts score ideas from our proposed model (8B) as high as they scored Claude's ideas. The study also measures a different axis of quality: experts judge if an idea is worth pursuing, where as the quality term is a reward model/LLM judge’s prediction of which idea would have higher scientific impact than an existing idea. We therefore do not claim that CU gains translate to expert judged gains, and we analyze the gap in our answer to Q2a. We also answer the paired differences question below.

AuthorsOpenReview ↗

Part 2 of the response to Reviewer AYWN

How does DARLING perform with SFT initialization

We train DARLING from the same SFT checkpoint as our method. The SFT checkpoint is our no-think SFT model prompted for one idea. We also use asymmetric clipping (as our method), a length penalty with the same per-idea target length, and train to 600 steps. Results on test instances, same evaluation judges, paired over instances:

ModelInitCU>=1 WinAvg. d
DARLING (Table 1)base0.520.590.67
DARLING + SFT (new)our SFT (no think)0.490.650.62
Proposed (think)our SFT (think)0.510.620.83
Proposed (no think)our SFT (no think)0.640.780.88

Even starting with our SFT model does not close the gap for DARLING. While it raises the best idea per set (>=1 Win goes from 0.59 to 0.65), the sets DARLING + SFT generates become less distinct (distance 0.67 goes down to 0.62).

What evidence is there that isolates the value of constraining the diversity reward to a single generation

The DARLING + SFT comparison we have above is the most direct evidence. Both runs share the same starting points (base and SFT models) but they only differ in where the diversity is optimized (i.e. across-rollouts vs within rollouts). Our method reaches 0.64 CU vs DARLING + SFT reaches 0.49. We also present two more observations in the submission: (a) In Table 1, while the CU for DARLING and our proposed no SFT baselines is similar, it is for two different reasons. DARLING (no SFT) has higher quality than our no SFT baseline, where our no SFT baseline has higher diversity, which is what we are claiming the single generation (set) diversity achieves. (b) In Table 12 we show the effect of the formatting, where prompting a model for 3 ideas in one generation instead of 3 independent samples raises within-set distance (Claude 0.24 to 0.80, Qwen3-8B 0.29 to 0.80). A diversity reward across samples has to work against this redundancy whereas, a reward within the set can build on the model's ability to steer each idea away from the ones it has already written.

How well do changes in CU on the 60 human-scored examples predict changes in uniquely promising ideas

The CU values for Table 4:

SystemUnique PromisingCU
Qwen3-8B1.420.02
Claude Opus 4.82.030.42
Proposed (no think)2.050.72

CU orders the systems the same way as the experts, but it's only three systems. We see that CU overstates the gap between the two strong systems: experts rate Proposed and Claude as tied, while CU separates them by 0.30. These two correlate moderately at an instance level (180 instances pooled across systems, spearman 0.29). CU does not reliably predict expert differences among the two strong systems. One possible reason is that the two measures capture aspects of quality. Experts judge whether each idea is worth pursuing whereas CU is based on an LLM judge's predicted impact of an idea relative to the published paper.

Can you give the paired differences (and uncertainties) for the human comparison?

metricProposed − Claude Opus 4.8Proposed − Qwen3-8BClaude Opus 4.8 − Qwen3-8B
Promising−0.03 [−0.25, +0.20]+0.65 [+0.40, +0.90]+0.68 [+0.42, +0.95]
Unique+0.12 [−0.05, +0.28]+0.32 [+0.12, +0.52]+0.20 [+0.00, +0.40]
Unique Promising+0.02 [−0.23, +0.27]+0.63 [+0.40, +0.87]+0.62 [+0.38, +0.85]

We report paired differences, 95% bootstrap CIs, bold means significant after Bonferroni correction over the 9 tests. We find no difference between our proposed system and Claude. We therefore do not claim an advantage over Claude, but instead only parity. Which is still notable for an 8B model compared to a frontier model.

AuthorsOpenReview ↗

Part 3 of the response to Reviewer AYWN

Was there any independent auditing for the execution study to verify that "supports" decisions were meeting the specified criteria?

We didn’t have any quantitative independent audit of the execution, the execution agent judged on its own. For the rebuttal, we run an independent, blind audit with GPT-5.5. We take all the support verdicts in the study, plus the same number of additional refutes matched by the system. For each card, the auditor sees the idea’s claim, the agent’s initial plan, its results, result files and code. It doesn’t see the agent’s verdict or which agent writes/executes the ideas. We give the auditing agent the same rules as the executor agent was given, and then ask the auditor to give a judgment.

We show Table 5 with the executor's verdicts and with the audited verdicts:

Modelk=1k=2k=3k=1 (aud)k=2 (aud)k=3 (aud)
Claude Opus 4.813%20%27%9%18%27%
Proposed (no think)11%22%33%7%13%20%
Proposed (think)10%19%29%10%19%29%

After the audit, at k=3, we see Proposed (think) and Claude still maintains all the verdicts as before, whereas our no-think model drops in success rate. We can update these results!

Thank you again for your thoughtful review. As the discussion period is nearing its end, we would like to kindly follow up on our rebuttal and ensure that we have adequately addressed your concerns. We would greatly appreciate any further feedback you may have, and we would be happy to clarify any remaining questions!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.