acceptodds
Under review as a conference paper at ICLR 2027

Reinforcing Agentic Creativity in Scientific Ideation with Night Science

Abstract

Large language models (LLMs) excel at structured, verifiable tasks, but their low-entropy bias can produce homogeneous and predictable outputs, limiting their utility for open-ended scientific ideation. Effective discovery, however, spans a broader creative spectrum: from structured _day science_ to loosely structured, serendipitous _night science_ that reaches ideas beyond those typically considered. We introduce **AI Night-Scientist**, an agentic framework that uses reinforcement learning to teach models _when_ and _how_ to depart from predictable reasoning. Grounded in cognitive science, we model creativity along three axes: _action_ (what to do and how creatively), _process_ (when to explore versus exploit), and _outcome_ (the novelty and usefulness of the resulting idea). We use these axes to train models with GRPO, exposing them to varying degrees and forms of creativity throughout training. This produces substantially more diverse scientific proposals, expanding the range of research directions by 27.8% and contribution types by 14.9% over the base model. It also improves predicted citation impact by up to 32.0 percentage points and originality by 66.2 points. These gains cannot be reproduced by simply increasing decoding temperature; instead, we find that semantic guidance specifying _what kind_ of creativity to pursue is critical. Overall, our results suggest that *creativity is a learnable, multi-level ability that can be shaped to help researchers reach ideas beyond those typically explored by LLMs*.

Primary area:
foundation or frontier models, including LLMs
Keywords:
scientific ideation, AI for research, computational creativity, creative generation, scientific discovery, post-training

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YVNfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AI Night-Scientist, an RL-trained agent for scientific ideation. The agent jointly selects actions and semantic creativity levels during multi-step reasoning, allowing it to vary how broadly it searches, which perspectives it considers, and how strongly it challenges existing assumptions. The authors use GRPO to optimize proposal quality and evaluate the framework on tasks derived from NSF awards. The experiments assess predicted citation impact, literature-grounded originality, and research-idea diversity, with comparisons against temperature control, ReAct, and several RL variants.

Critical strengths

The framework offers a useful and interpretable way to control exploration during scientific ideation. Selecting an action together with its creativity level makes the exploration strategy explicit. Learning when to use different degrees of exploration is an interesting contribution, particularly for open-ended tasks where consistently conservative or highly exploratory behavior may be ineffective.

The internal comparisons provide reasonable evidence for the proposed design. The temperature, ReAct, restricted-action, and reward ablations help distinguish the effects of the main components. Under the paper’s evaluation protocol, the results suggest that semantic creativity guidance provides benefits beyond increased sampling randomness. Considering both proposal quality and diversity also makes the study informative.

Critical weaknesses

My main concern is that the related-work discussion could more clearly establish the incremental contribution relative to existing RL-based scientific ideation methods. Relevant approaches include LDC, Learning to Generate Research Idea with Dynamic Control (2024); MoRI, Learning Motivation-Grounded Reasoning for Scientific Ideation in Large Language Models (2026); and EvoIdeator, Evolving Scientific Ideas through Checklist-Grounded Reinforcement Learning (2026). These works address dynamic generation control, motivation-grounded reasoning, and feedback-driven iterative refinement, respectively. Discussing them would help explain what is added by jointly learning action selection, action-specific creativity levels, and their scheduling.

The paper already acknowledges AI Can Learn Scientific Taste and uses its Scientific Judge. However, a more specific discussion of the Scientific Thinker generation model from that work would strengthen the comparison. The authors could explain how their process-control framework differs from directly optimizing generated ideas through comparison-based GRPO.

I consider these concerns addressable through a stronger literature discussion and clearer contribution boundaries. I do not consider substantial new baseline experiments necessary for acceptance.

Questions for authors

What capability does learning action-specific creativity levels add relative to dynamic control in LDC, reasoning optimization in MoRI, and iterative refinement in EvoIdeator? A concise comparison of their control mechanisms and training objectives would be helpful.

How do the reported paradigm and contribution-type diversity metrics relate to fluency, flexibility, and within-task ideation diversity in prior work? Please clarify the scope of the conclusions supported by the current metrics.

3 comments
AuthorsOpenReview ↗

Clarifying Our Novel Methodological Contributions Relative to Existing Work & Diversity Evaluation

We thank the reviewer for their thoughtful assessment. We appreciate their recognition of (i) the interpretability of our creativity-aligned framework, (ii) the value of learning when and how to explore during scientific ideation, and (iii) the evidence from our temperature, ReAct, and reward ablations that semantic creativity guidance provides benefits beyond sampling randomness. We are also encouraged that the reviewer considers the remaining concerns addressable through clearer positioning with related works.

What capability does learning action-specific creativity levels add relative to [LDC, MoRI, and EvoIdeator]?

We re-iterate our primary novel methodological contributions:

  1. A multi-level representation of creativity for scientific ideation: creativity within the scientific process is embedded at the action, process, and outcome-level, not just at the outcome-level (LDC, MoRI, and EvoIdeator are outcome-only and token-level)
  2. A practical implementation of this representation: we design a unique action space with varying semantic levels of creativity for agentic scientific ideation (unlike treating all actions as fixed/unidimensional as LDC, MoRI, and EvoIdeator do) →\rightarrow less to more creative search, debate, and spark (challenging assumptions in the field).
    • E.g., search ranges from familiar to distant literature, debate varies disciplinary perspectives, and spark varies the scope of assumptions challenged.
  3. We design and explore both outcome and process-level rewards (to what extent each action leads to exploration and contributing towards the final proposal) and serendipity-style training (randomly injecting actions into the trajectory)

In general, while these prior approaches learn to improve scientific ideas, they differ in where creativity is represented, what actions are available, and what the model learns to control. We summarize this below:

MethodAction Space / OperationsCreativity RepresentationLearned Control / Training Objective
LDCWriteOutcome-levelBalances novelty, feasibility, and effectiveness during sentence-level generation
MoRIWriteOutcome-levelOptimizes information gain and motivation-method alignment
EvoIdeatorWriteOutcome-levelLearns feedback-driven revision through checklist-grounded rewards and critiques
Scientific ThinkerWriteOutcome-levelGenerates ideas predicted to lead to higher citation
AI Night-ScientistSearch, Debate, Spark, WriteAction, Process, Outcome-levelJointly learns action selection, action-specific creativity levels, and their adaptive sequencing through proposal-quality and process-level rewards

Overall, we emphasize the need for both structure and a lack thereof in creative research ideation (which is a novel perspective from existing work), learning when to explore unconventional directions versus develop promising ones. With these differences clarified, we hope that it is clear that we take a significant departure from existing work on scientific creativity.

We will expand Sec. 2 to better acknowledge these complementary approaches and clarify these contribution boundaries.

How do the reported paradigm and contribution-type diversity metrics relate to fluency, flexibility, and within-task ideation diversity in prior work?

Rather than introducing new diversity metrics/categories, we adopt and adapt existing research taxonomies to evaluate two complementary aspects of scientific ideation:

  • Research paradigms: Following Chen et al. (2026), Measuring the Gap Between Human and LLM Research Ideas, we characterize the underlying research approach or conceptual move.
  • Contribution types: Drawing on Wobbrock (2012), Seven Research Contributions in HCI, we characterize the primary scholarly contribution of a proposal (e.g., empirical, methodological, theoretical, artifact-based).

For both, we measure corpus-level categorical diversity using the normalized effective number of categories, exp⁡(H)/K\exp(H)/K, where HH is Shannon entropy and K=7K=7 (Sec. 4.2; Appendix H).

These metrics are related to flexibility, as they capture breadth across conceptual categories, but differ from fluency (number of distinct ideas generated per problem) and within-task diversity (variation among multiple ideas for the same problem). Our evaluation instead measures the breadth and balance of proposal types across the test set.

As shown in Sec. 4.4.2 (Table 3b), AI Night-Scientist increases research-paradigm diversity by 27.8% and contribution-type diversity by 14.9% over Qwen3-8B. These results support broader exploration of different kinds of research ideas across problems, without implying improved within-task diversity or fluency.

We will clarify these distinctions in Sec. 4.2 and explicitly qualify the scope of our diversity claims. Thank you!

Reviewer_YVNfOpenReview ↗

Thanks for your clarification

Thanks for your detailed clarification! I will maintain my rating.

AuthorsOpenReview ↗

Thank you; Summary of Paper Modifications Based on Your Feedback

Thank you for your acknowledgement! We have added the additional requested literature review and contribution discussion into the updated manuscript (L132-140), as well as additional diversity metric clarifications (Section 4.2) and qualitative analysis (Section 4.4.2 & Table 4) based on your feedback! We hope that this has adequately addressed any further concerns you have for acceptance.

Reviewer_reRwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces AI Night-Scientist, an RL-trained agent designed to generate research proposals through a multi-step process. The agent follows a ReAct-style loop with at most five actions, choosing among Search (retrieval of arXiv papers), Debate (simulated expert discussion), Spark (a “Bit-Flip” operation that challenges existing assumptions), Write, and Stop. For Search, Debate, and Spark, the agent also selects a creativity level ranging from 1 to 5. These levels provide natural-language guidance on how each action should be performed, from focused, proposal-specific reasoning at lower levels to exploring distant research areas and alternative perspectives at higher levels. The authors distinguish three aspects of creativity: how individual actions are executed, how creativity levels are coordinated throughout the process, and the novelty and usefulness of the resulting proposal. The agent is trained using GRPO, with Qwen3-8B-Base and Qwen3-14B-Base as the underlying models. Training data consists of NSF CSE award titles.The reward is assigned to the final proposal by GPT-4.1. Specifically, each proposal is decomposed into atomic ideas, which are evaluated for precedence relative to a retrieved arXiv paper and for relevance. Feasibility is assessed at the plan level, with the minimum score used for aggregation. The study also examines two optional components: random “serendipitous” swaps of the selected action and creativity level during early training, and an additional process-level reward. For evaluation, the generated proposals are compared pairwise against reference proposals reconstructed by GPT-5.1 using award abstracts, outcomes reports, and the principal investigators’ arXiv papers. Predicted citation impact is assessed by SciJudge-30B, while originality is evaluated by GPT-5.1 with the aid of a retrieved paper. Proposal diversity is measured using an effective-number index across seven research-paradigm categories and seven contribution categories. The reported results show that Night-8B and Night-14B achieve citation-impact win rates of 29.89% and 33.18%, respectively, and originality win rates of 56.32% and 68.68%, respectively. Both models outperform RL-trained variants based on Temperature, ReAct, and Search+Write, as well as GIANTS-4B. Based on these findings, the authors argue that creativity can be learned as a multi-level capability and that explicit semantic guidance plays a greater role in the observed improvements than stochastic variation alone.

Critical strengths
  1. An interpretable way to control creativity at the action level. In my opinion, one of the main strengths of this paper is the idea of defining and controlling creativity levels through natural-language descriptions. Unlike approaches that attempt to increase model creativity by raising the sampling temperature and, consequently, introducing more randomness into the outputs, the proposed framework explicitly specifies how each action should be performed and at what level of creativity. This approach not only makes the agent's behavior more interpretable but also allows for more targeted control over creativity. The experimental results also provide support for this design choice. Overall, defining creativity through natural-language descriptions appears to be a more effective approach to guiding and controlling an agent's creative behavior than simply adjusting the sampling temperature. However, additional controlled experiments are still needed to better isolate the specific contribution of this component to the observed performance improvements.

  2. A broad experimental evaluation with useful ablations and transparent reporting. I appreciate the effort to evaluate the framework from several angles rather than relying on a single quality metric. The paper includes comparisons with zero-shot, temperature-based, ReAct, Search+Write, and GIANTS baselines, as well as experiments across two model sizes. The reward and action-swap ablations are particularly useful for understanding which design choices contribute to the observed improvements. The per-domain results, confidence intervals, and human–LLM agreement analysis also help make the empirical evidence easier to assess. Another positive aspect is the transparency of the experimental reporting. The paper provides code links, prompts, training hyperparameters, and details of the evaluation procedure. It also uses different models for training-time reward assessment and the main evaluation judgments, although this does not eliminate concerns about automated evaluation. I particularly appreciate that the authors discuss an example where the process+outcome variant produces less well-grounded claims (Appendix D) and acknowledge training instability (Appendix O).

Critical weaknesses
  1. The experiments do not fully isolate the contribution of semantic creativity levels, particularly the claimed ability to learn when to explore. My main concern is that the headline improvements combine several changes, while the incremental benefit of semantic creativity levels is considerably smaller. ReAct-RL, which uses the same actions and outcome reward but no creativity levels, already achieves 24.9% predicted citation impact and 46.6% originality, compared with 29.9% and 56.3% for Night-8B. This is a meaningful improvement, but much smaller than the gains over the zero-shot model emphasized in the abstract. The uncertainty around these differences also deserves more attention. The citation confidence intervals for ReAct-RL and Night-8B overlap substantially, while the originality intervals are only narrowly separated. A paired comparison across the 491 test problems would be more informative. More importantly, the reported intervals capture evaluation-sample uncertainty, not variation across training runs. Given the training instability acknowledged in Appendix O, results from multiple independent seeds would be important. I am particularly unconvinced by the claim that the agent learns when to deviate creatively. Definition 3.2 describes adaptive selection of creativity levels throughout a trajectory, but this behavior is not directly validated for the main outcome-only model. Figure 4 analyzes the process+outcome variant rather than the primary model, and the trajectories contain at most five actions. Fixed-level controls (always L1 or L5), random-level selection, and an analysis of the learned level-selection policy would help establish whether adaptive control is actually learned.

  2. The evaluation relies heavily on automated judgments and reconstructed references, leaving important questions about validity and possible evaluation bias. The originality evaluation is conceptually close to the training reward. Both rely on LLM judgments of novelty relative to retrieved scientific literature, although the training and evaluation judges differ. This leaves open the possibility that some of the improvements reflect optimization toward the evaluators' notion of originality rather than independently validated scientific creativity. I am also concerned about the reconstructed references. These are generated using GPT-5.1 from information that includes the principal investigators' subsequent publications. At evaluation time, originality is judged relative to the closest paper retrieved from arXiv. Since those downstream papers may also appear in the retrieval index, the references could be systematically disadvantaged by their proximity to the literature used to construct them. This possibility should be tested directly. The originality win rates should therefore not be interpreted as evidence that the generated proposals are more original than the actual funded research. A related issue is temporal leakage. The paper identifies the arXiv retrieval source but does not clearly establish that retrieval is restricted to literature available before each award's start date. Without such a cutoff, the agent may have access to research published after the corresponding NSF project began. The citation-impact evaluation is also indirect. SciJudge is used to compare proposal titles and abstracts, with an additional instruction that the reconstructed reference should be treated as earlier work. The effect of this temporal instruction is not ablated, and the paper does not establish that citation forecasting transfers reliably to this proposal-comparison setting. More importantly, feasibility and relevance—two central components of the training reward—are not separately evaluated at test time. Predicted citation impact alone does not establish whether a proposal is scientifically useful or executable. The qualitative examples raise further concerns about possible evaluator preferences. Appendix D includes a process+outcome proposal with unsupported quantitative claims and invented constructs, while the main-model example also contains questionable design choices. A systematic analysis of unsupported claims, invented terminology, citation accuracy, proposal length, and feasibility would be valuable, particularly for the main model. Finally, the human evaluation is too limited to validate the main empirical comparisons. It involves 33 proposal pairs, with only 15 shared items used for inter-annotator agreement. More importantly, the sample covers Qwen3-Base, GPT-4.1, and Night-8B, but not the strongest competing RL baselines. A targeted, blinded human evaluation of these comparisons would significantly strengthen the paper.

Questions for authors

The following questions directly affect my assessment of the paper. I would particularly appreciate responses to Questions 1–3, which concern the central empirical claims.

  1. Is adaptive creativity-level selection actually responsible for the improvement? Could you provide fixed-level (always L1 and always L5) and randomly selected level controls, ideally with multiple independent training seeds? A paired comparison between Night-8B and ReAct-RL on the 491 evaluation problems would also help establish the reliability of the incremental gains. For the primary outcome-only model, could you report the distribution of selected creativity levels by action and reasoning turn, and examine whether these selections depend on the evolving trajectory state? This would provide direct evidence for the claim that the agent learns when, rather than only how, to explore creatively.

  2. Could the evaluation references or retrieval setup introduce systematic bias? What is the exact arXiv snapshot date, and are Search, reward computation, and originality evaluation restricted to literature available before each NSF award began? Since the reconstructed references are partly based on the PIs' subsequent publications, how often does the originality evaluator retrieve those same publications or closely related sources? Could you report a sensitivity analysis excluding reconstruction-source papers from the reference-side retrieval?

  3. Do independent evaluations confirm the improvements over the strongest baselines? Could you report separate test-time feasibility and relevance evaluations for Night-8B and ReAct-RL, alongside originality assessments from an independent model family or evaluation protocol? A targeted, blinded human comparison of Night-8B against ReAct-RL and Temp-RL would be particularly valuable, as the current human evaluation does not cover these central comparisons. Could you also examine unsupported quantitative claims, invented terminology, hallucinated citations, and proposal length to rule out superficial sources of higher originality scores?

  4. How are action swaps implemented, and which model does the no-swap ablation represent? What is the precise decay formula for the initial swap probability of 0.5 with γ = 0.001? Under some conventional decay schedules, the probability would remain substantial after 170 training steps. Are externally swapped actions included in the GRPO optimization as ordinary policy actions, or are their contributions masked or otherwise corrected? Also, does the "+ No swap" row in Table 3(b) correspond to the outcome-only or process+outcome model? The differences reported in Section 4.4.2 appear to use the outcome-only result. Could you also provide citation-impact and originality results for the no-swap setting?

6 comments
AuthorsOpenReview ↗

Clarifying Potential Misunderstanding: ReAct + RL Does Use Fixed Creativity Levels & Direct Automatic + Human Comparison b/w ReAct-RL and Night-8B (Gains Stable)

We thank the reviewer for their detailed feedback. We appreciate their recognition of our interpretable, action-specific creativity control, the benefits of semantic guidance over sampling randomness, and our broad evaluation, useful ablations, transparent reporting. We provide additional evidence and clarifications below.

The experiments do not fully isolate the contribution of semantic creativity levels.

We clarify a potential misunderstanding: our ReAct-RL baseline uses the same action space, semantic instructions, reward, and trajectory budget. We simply modified the prompts to include the following creativity levels by default (effectively a fixed per-action, creativity-level ablation); we clarify this in the updated draft:

ActionReAct-RL instructionNight-Scientist level
SearchBackground information and adjacent methodsc=2
DebateCompeting perspectives and overlooked assumptionsc=3
SparkDramatically challenge conventional thinkingc=5
WriteIdentical promptc=1 (fixed)

Thus, ReAct-RL already serves as a fixed-level control against adaptive creativity-level selection, explaining why it is our strongest ablation. We clarify this in the revised draft.

Existing component isolations (Sec. 4.3; Table 3). Results below are impact / originality win rates (%) for Qwen3-8B unless noted:

ComponentComparisonImpact / Originality
Agentic processZero-shot0.46 / 2.53
Selectable levels (no RL)ReAct vs. Creative2.33 / 6.54 vs. 1.89 / 4.25
RL-trained agentReAct → ReAct-RL2.33 / 6.54 → 24.94 / 46.60
Temp → Temp-RL2.07 / 0.69 → 11.52 / 19.82
Creative → Night-8B1.89 / 4.25 → 29.89 / 56.32
Fixed vs. selectable levelsReAct-RL vs. Night-8B24.94 / 46.60 vs. 29.89 / 56.32
Semantic levels vs. stochasticityTemp-RL (levels control temperature)11.52 / 19.82
Action spaceSearch + Write + RL24.47 / 33.33
Reward designOutcome vs. process + outcome29.89 / 56.32 vs. 26.20 / 61.61
Model scaleNight-8B → Night-14B29.89 / 56.32 → 33.18 / 68.68

These 10 component-isolation comparisons demonstrate the breadth of our evaluation.

We also directly compare Night-8B against ReAct-RL on 350 matched test problems (700 proposals). Below are Night-8B’s win rates against ReAct-RL (w/ 95% CI) with and without any unsupported claim/data artifacts:

ProposalsOriginalityCitation Impact
Original68.30% [63.23, 72.97]68.59% [63.52, 73.25]
Rewritten67.44% [62.34, 71.95]69.16% [64.12, 73.79]

These results provide strong additional evidence for Night-Scientist's gains, with consistent 67-69% win rates across both metrics. It also shows that our gains are robust even when excluding unsupported artifacts. This advantage also cannot be attributed to longer proposals: ReAct-RL proposals average 2,843 words, whereas Night-8B proposals average 2,345 words.

Additional blinded human evaluation. We evaluated 25 blinded pairs (50 proposals) from Night-8B and ReAct-RL, asking domain experts to assess originality and scientific significance, considering correctness and feasibility for the latter:

MethodOriginalitySignificance
ReAct-RL8%36%
AI Night-Scientist-8B92%64%

We observe a strong preference for Night-Scientist on originality, with a more moderate preference for significance, where feasibility and execution effort also influenced judgments.

Evaluator feedback and post-hoc analysis further revealed that ReAct-RL repeatedly applies similar multimodal approaches across unrelated research problems, whereas Night-Scientist explores more distinct directions:

Research problemReAct-RL key ideaNight-Scientist key idea
Gaze controlMultimodal gaze predictionReconfigurable task-affordance network
Math error diagnosisMultimodal text/math analysisErrors as evolving cognitive phenomena
CyberbullyingMultimodal text/image/network modelLinguistic cues of abuse escalation
Teen cyber safetyMultimodal social-data knowledge graphEmergent trust recalibration
AI companionsMultimodal behavioral/physiological modeling"Cognitive teasing" through evolving limitations

These examples illustrate a qualitative difference in research mechanisms and problem formulations, further supporting our finding that adaptive creativity guidance enables exploration beyond conventional agentic reasoning.

Finally, we correct a typo: "33 proposal pairs" should be 60 pairs (Table 12). Including our additional study, we now report 85 human-evaluated pairs (170 proposals), a substantial undertaking given the complexity and interdisciplinary scope of the proposals.

AuthorsOpenReview ↗

Clarifying Evaluation Setup and Goals (Reconstructed Proposals Serve as References for Relative Method Analysis); Retrieval-Overlap Hit-Rate and Sensitivity Analysis; Citation Impact Discussion

The evaluation relies heavily on automated judgments and reconstructed references.

We would first like to clarify the purpose of our reconstructed references, beyond our domain-expert human judgements on 170 proposals. As stated in Sec. 4.1 and Appendix K: "These reconstructions are not intended to reproduce the original NSF proposals. Rather, they provide standardized, evidence-grounded references for research directions that were actually funded and subsequently pursued."

Thus, we do not claim that our originality win rates establish that generated proposals are more original than the human-generated funded research. Our goal is to compare different scientific ideation methods consistently on a challenging, long-horizon task, using reconstructed proposals as matched reference points (as direct comparisons across all methods would be expensive).

Importantly, the references are provided substantially richer information than any other method:

  • Generated proposals: Only the NSF award title (besides GIANTS, which receives two papers)
  • Reconstructed references: Award abstract, detailed project outcomes report, PI information, and associated PI-authored publications.

With this informational advantage, the references remain challenging comparison targets: the zero-shot and other agentic baselines achieve substantially lower win rates against them (Table 3a).

Additional retrieval-overlap analysis. We investigated whether the originality evaluator systematically retrieves the same publications used to construct reference proposals:

Reference publicationsExact title hitsHit rate
Grounding papers16 / 4913.25%

Using fuzzy title matching (threshold = 0.9) produces identical results. Thus, direct overlap between retrieved evaluation papers and reconstruction-source publications is relatively uncommon and has very minimal impact on the overall:

MethodOriginality changeCitation Impact change
Zero-shot-0.14 pp+0.02 pp
ReAct w/ RL-0.93 pp-0.27 pp
Temp w/ RL-1.16 pp+0.20 pp
Night-8B+0.24 pp+0.43 pp

Our NSF awards span 2018 onward, so some knowledge of the awards and associated publications is likely present in the models' parametric knowledge. We therefore intentionally do not enforce strict award-year retrieval cutoffs for our agents, reward computation, reconstruction, or evaluation. Our task is not to recreate the proposals; rather it is to generate new proposals conditioned on the long-term research problems which these proposals target. Thus, all methods are evaluated under the same retrieval conditions. As shown above, this does not meaningfully bias our evaluation. Furthermore, our direct Night-8B vs. ReAct-RL comparisons above eliminate dependence on reconstructed references and continue to show strong improvements.

Predicted citation impact and scientific significance. Assessing scientific value and soundness is an open-problem for long-term proposals whose ideas cannot yet be experimentally validated. We therefore assess scientific value at final evaluation time using complementary automatic and human judgments on 170 proposals:

  • Predicted citation impact: We use SciJudge (Tong et al., 2026) as a proxy for downstream value, following GIANTS. Our human study found 76.7% human–LLM agreement on impact preferences and 80.0% inter-annotator agreement (κ=0.625\kappa=0.625) (Sec. 4.2). This supports a proxy for perceived impact, not a direct estimate of eventual citations. We retain SciJudge's temporal instruction for consistency with its original protocol (Appendix G), but have not ablated it.
  • Scientific significance: In a blinded human study against ReAct-RL (25 pairs), evaluators considered correctness, feasibility, and potential value. Night-Scientist was preferred in 64% of significance comparisons, and 92% for originality (above).

Our primary goal is to address the lack of creativity in scientific research agents, while acknowledging the difficulty of jointly optimizing novelty and soundness. Our reward (Sec. 3.3.1) therefore assesses atomic ideas through precedence, feasibility, and relevance to encourage novel, grounded directions, not merely distance from literature. The feasibility and relevance checks discourage impractical and off-topic proposals, without establishing technical correctness.

We further test evaluator biases through meaning-preserving rewrites, proposal-length comparisons, and a technical-flaw error analyses (below). Night-Scientist shows slightly higher rates of some technical flaws, but its originality advantage persists after unsupported claims are removed, and experts still prefer it for significance. These are complementary proxies, not experimental validation; stronger soundness rewards for long-term ideation remain future work.

AuthorsOpenReview ↗

Independent Evaluations Confirm Night-8B Improvements + Evaluation is Robust; Technical Flaw Analysis; Minor Clarifications

Do independent evaluations confirm the improvements over the strongest baselines? Could you examine unsupported claims, technical flaws, proposal length, and test-time feasibility?

Our additional direct comparison automatic and human evaluations above and in Sections 4.4.1, 4.2, and K further confirm improvements over the strongest baselines. We include additional analyses to further demonstrate the robustness of our gains against the strongest baselines:

Additional technical-error evaluation. We also introduced an independent, error-focused pairwise assessment using GPT-6.1 Sol. We first validated this evaluator by deliberately injecting fundamental but non-obvious conceptual errors into previously preferred proposals. For example, claiming that an object's absolute mass can be uniquely recovered from its free-fall trajectory, despite free-fall acceleration being independent of mass. After introducing these errors, 99% of previously preferred proposals lost their preference under the flaw-focused evaluation, supporting the evaluator's sensitivity to substantive technical mistakes.

We then assessed 700 original proposals for technical flaws:

Flaw categoryReAct + RLAI Night-Scientist
Unsupported data0.58%4.61%
Unsupported theoretical guarantees1.44%4.61%
Mathematical/conceptual inconsistencies0.86%5.76%
Supervision/training-objective mismatch0.29%1.15%
Component/interface integration0.00%0.29%
Physical/hardware incompatibility0.29%3.17%
Computational/resource feasibility0.29%0.29%

We already acknowledge in L506-526, Appendix D, and Table 8 that AI Night-Scientist exhibits slightly higher rates of certain technical flaws, particularly unsupported data and theoretical guarantees (a trade-off). However, these results should be considered alongside the rewrite robustness and expert evaluation: the originality advantage persists when unsupported quantitative claims are removed, while experts still prefer Night-8B on originality and scientific significance in 64% of paired comparisons.

These findings highlight a minor trade-off between exploring unconventional research directions and maintaining methodological rigor, which also motivates our separate feasibility and relevance rewards (Sec. 3.3.1) and in general, our focus on long-term proposal-level ideation.

Proposal-evolution analysis. We added 2 qualitative trajectory analyses (Appx E.1-E.2, L1037-1125). Unlike Temp-RL and ReAct-RL, which largely repeat their initial proposals, both Night-8B variants meaningfully revise their proposed mechanisms based on intermediate exploration. These samples showcase the potential of our multi-level representation to support more substantive exploration and revision.

How are action swaps implemented, what is the precise decay schedule, and how are externally swapped actions handled in GRPO?

With an initial swap probability of p0=0.5p_0=0.5, γ=0.001\gamma=0.001, and batch size B=32B=32, our decay is linear per training batch (so it reaches 0 after 16 training steps): pk=max⁡(0,  0.5−0.032k)p_k = \max\left(0,\; 0.5 - 0.032k\right).

Injected action tokens remain in the trajectory context, but are excluded from policy-gradient, entropy, and KL losses, as these tokens are not generated by the model (following similar decisions as Search-R1 with retrieved literature). However, the remaining trajectory tokens and rewards are included.

We isolate no swap on the process-outcome reward model to explore its impact on the reward ablation’s diversity and originality gains: Paradigm diversity: 0.853 → 0.844 (-0.009); Contribution diversity: 0.362 → 0.358 (-0.004); Citation Impact: 0.262 → 0.202 (-0.06); Originality: 0.616 → 0.595 (-0.021).

Removing swaps reduces performance across all four measures, supporting the value of early exposure to unfamiliar creative actions. We will clarify the model configuration and comparison in Table 3(b). We also clarify that the GRPO training instability we mention in Appendix O is not specific to our framework; it is a common phenomenon and active research area across GRPO-based methods [1], and we follow the same training standards as other works in the area (e.g., DeepSeek-R1; Search-R1).

Overall, we hope that our additional direct comparisons, blinded human evaluation, retrieval-overlap audit, and robustness + qualitative analysis address the reviewer's central concerns. Together with our existing component ablations, these results support our main conclusion: existing approaches to scientific ideation are insufficient for creative and having a rich action space that provides additional degrees of creative control at the action, process, and outcome-level is more effective, interpretable strategy. Thus, both day and night science should be represented within AI-assisted research methods.

[1] On GRPO Collapse in Search-R1: The Lazy Likelihood-Displacement Death Spiral; Deng, et al. 2025.

Reviewer_gQsjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AI Night-Scientist trains a proposal-writing agent to choose search, debate, assumption inversion, and writing actions, with natural-language creativity levels controlling how exploratory selected actions should be. GRPO uses proposal-level novelty, feasibility, and relevance rewards, with optional early action swaps and process rewards. The paper evaluates proposals generated from NSF award titles using model-based pairwise judgments, category-diversity metrics, and a small human agreement study.

Critical strengths
  1. The paper addresses an interesting question: how can scientific agents explore unconventional directions while remaining useful and grounded? Representing creativity through interpretable, action-specific instructions is a concrete and promising approach.
  2. The comparisons with RL-trained ReAct and temperature-controlled agents are valuable. The results suggest that the design of the action interface affects what an agent can learn under the same outcome reward. This interaction between the harness and RL is an interesting empirical finding.
Critical weaknesses
  1. The reward is not sufficiently validated as a proxy for useful scientific novelty. Difference from retrieved literature can reflect an unsound idea rather than a valuable advance.
  2. The human study does not directly validate gains over the strongest RL baseline. It measures agreement on impact and originality against reconstructed references, rather than comparing Night-Scientist with RL-trained ReAct. A blinded comparison jointly assessing novelty, correctness, and feasibility would make the claimed improvement more convincing.
  3. Separate training and evaluation judges do not fully address shared evaluator biases. This separation is useful, but judges may still reward persuasive presentation over substance. Testing whether scores withstand meaning-preserving rewrites and decrease when technical flaws are introduced would strengthen evaluator validation.
  4. The benefit of adaptive creativity control remains unclear. Existing comparisons do not separate adaptive switching from consistently using strong creative instructions. Fixed-level and fixed-schedule baselines with the same actions, reward, and budget would isolate this contribution.
Questions for authors
  1. Have you audited high-reward proposals for technical flaws, infeasible plans, or overlooked prior work? Does higher reward correspond to expert-rated novelty while preserving correctness and feasibility?
  2. Could you provide a blinded comparison of Night-Scientist and RL-trained ReAct (or other baseline) on a modest subset of existing outputs, separately rating novelty, technical correctness, and feasibility?
  3. Do evaluator preferences remain stable under meaning-preserving rewrites, and do scores decrease when substantive technical flaws are introduced? How sensitive are originality judgments to using multiple retrieved papers?
  4. How does the learned controller compare with fixed creativity levels or simple fixed schedules using the same action space, semantic instructions, reward, and budget?
6 comments
AuthorsOpenReview ↗

Assessing Scientific Impact/Significance in Final Evaluation + Blind Human Preference Study b/w ReAct + RL and Night-8B

We thank the reviewer for their thoughtful comments, and are glad that they found our approach to address an “interesting question” in an "interpretable [...] concrete and promising" manner, with "valuable" and "interesting" empirical findings. We hope that our response below adequately addresses your specified weaknesses and questions:

The reward is not sufficiently validated as a proxy for useful scientific novelty. Difference from retrieved literature can reflect an unsound idea [...].

We prioritize assessing scientific value in our final evaluation using both automatic and human evaluation:

  • Predicted citation impact: Predicted citation impact: We use SciJudge, trained to predict citation impact, as a proxy for scientific value, with 76.7% human-LLM agreement on human perceived impact (Sec. 4.2).
  • Scientific significance: We add a human-rated significance evaluation considering correctness, feasibility, and potential value. In blinded comparisons against ReAct-RL (50 proposals), AI Night-Scientist was preferred for significance in 64% and originality in 92% (below).

While AI Night Scientist does evaluate significance/impact, we point out that our work’s primary purpose is to address the lack of scientific creativity in research agents (especially since assessing scientific significance using an LLM-Judge while maintaining novelty is an open problem [Si 2025]). Thus, our reward was intentionally designed to prioritize agentic creativity using precedence, feasibility, and relevance. The following are excerpts from each metric’s reward prompt:

  • Precedence: “ [...] Assess whether the idea challenges existing paradigms or practices and whether it offers a unique contribution to the field.”
  • Feasibility: “how practical the idea is to implement, considering the resources available, the complexity of the idea, the ethical implications [...], and any potential challenges [...]. If an idea is too broad or vague, it may be infeasible to implement effectively.”
  • Relevance: “how well the idea addresses the problem [...]. Consider whether the idea is directly aligned with the problem statement and whether it offers a clear approach to addressing it.”

Thus, our reward does not simply assess the difference from retrieved literature; rather, it uses it to additionally ground the model’s judgment (especially for proposals on topics less represented within the model’s parametric knowledge). Together, all three dimensions and our final evaluation helps assess scientific significance. This is supported by the below empirical analysis. We leave designing an automatic reward for assessing long-term scientific value at an early ideation stage for future work.

blinded comparison of Night-Scientist and RL-trained ReAct (or other baseline) on a modest subset of existing outputs?

We conduct an additional human evaluation on blind pairs between these two methods (50 proposals; 25 pairs), where we instructed the evaluator to assess significance with correctness & feasibility in mind (since unoriginal ideas are much more feasible):

MethodOriginalitySignificance
ReAct + RL8%36%
AI Night-Scientist (8B)92%64%

We observe a strong preference for AI Night-Scientist on originality, with evaluators noting that Significance was influenced by the feasibility of the idea as it required more effort to execute; they also noted that in general, significant jargon is used by both methods, making the proposals time-consuming to evaluate. Based on feedback from participants and post-hoc analysis, ReAct-RL also produced similar ideas for very different proposals:

Research problemReAct-RL Key IdeaAI Night Scientist Key Idea
Gaze control in natural viewingA "multi-modal gaze prediction model"A task-affordance network that "dynamically reconfigures representations of objects, agents, and task goals"
Diagnosing errors in short-answer mathA "multi-modal diagnostic framework [...] leveraging both textual and mathematical structure analysis"Errors as "dynamic cognitive phenomena unfolding across student interaction sequences,"
Detecting cyberbullyingA "multi-modal machine learning framework"Detecting "subtle linguistic cues signaling the escalation from neutral to abusive exchanges"
AI co-pilot for teens' cyber safetyAn LLM + knowledge graph "trained on a multi-modal dataset of social media interactions"Online risk as "emergent trust recalibrations" between teens, peers and platforms
Mind perception in AI companionsA "multi-modal approach, combining behavioral tasks, physiological measures, and computational modeling""Cognitive teasing": AI with "purposeful, iteratively evolving cognitive limitations"

This demonstrates AI Night-Scientist’s clear originality strength which is even further captured by domain-expert evaluation.

AuthorsOpenReview ↗

Stable Automatic Preference for Night-8B over ReAct + RL w/ Meaning-Preserving Rewrites & Technical Correctness Error Analysis

Do evaluator preferences remain stable under meaning-preserving rewrites, and do scores decrease when substantive technical flaws are introduced?

We conducted meaning-preserving rewrites on 700 proposals (350 for AI Night-Scientist-8B and Qwen3-8B + ReAct w/ RL each), where any projected quantitative results or unsupported claims were removed from all proposals (in order to avoid any potential bias the automatic evaluation may have towards “good projected results”; only the fundamental ideas presented in the proposal are preserved). Here are the AI Night-Scientist win-rates (with confidence intervals) when compared directly with ReAct-RL on both the original and rewritten proposals:

Table A:

OriginalityCitation Impact
Original68.30% [63.23, 72.97]68.59% [63.52, 73.25]
Rewritten67.44% [62.34, 71.95]69.16% [64.12, 73.79]

This demonstrates that our results still hold under meaning-preserving rewrites that intend to further even the playing field; AI Night-Scientist consistently outperforms ReAct-RL across both originality and citation impact, with win rates remaining largely unchanged (67-69%). We note that this cannot be attributed to longer proposals: on average, ReAct-RL proposals are 2,843 words; AI Night-Scientist are 2,345 words long. These new direct comparison results support our existing direct comparisons results in Section 4.4.1 (L400-404).

Furthermore, we add an additional pairwise assessment of technical challenges or flaws using GPT-6.1 Sol. We validated it on comparisons where we purposely injected fundamental but non-obvious conceptual flaws into the proposals (e.g., “The physical-property encoder will uniquely recover each freely falling object's absolute mass from its visual trajectory [...]” →\rightarrow Free-fall acceleration does not depend on mass. Different masses can produce identical trajectories). We injected them into proposals that originally won against the other method; after injection, 99% of these proposals lost. This demonstrates the reliability of the metric.

Using the same metric, we assessed 700 of the original proposals pairwise and categorized the nature of the technical flaws. Here are the percent of proposals identified to have technical flaws for each of the same two methods:

Table B:

Flaw categoryDescriptionReAct + RLAI Night-Scientist
Unsupported dataInferring data not determined by available observations0.58%4.61%
Unsupported theoretical guaranteesPromising potential formal guarantees without sufficient justification1.44%4.61%
Mathematical/conceptual inconsistenciesIncompatible concepts or setups0.86%5.76%
Supervision/training-objective mismatchLearning objectives require signals inconsistent with training setup0.29%1.15%
Component/interface integrationMissing or incompatible dependencies between key method components0.00%0.29%
Physical/hardware incompatibilityRequired capabilities incompatible with stated hardware/equipment0.29%3.17%
Computational/resource feasibilityComputational demands conflicting with stated resources/budgets0.29%0.29%

As observed here, AI Night-Scientist does exhibit a minor increase in technical flaws (specifically unsupported data/theory), however with Table A, we see that removing these does not cause the scores to meaningfully drop. Thus, factors like unsupported data or statements of potential theoretical guarantees are not significant considerations for originality or impact.

We did not observe any significant differences in the originality judgements being provided multiple papers (and providing additional tokens result in significantly higher cost).

We would also like to note that not only do our rewards and evaluation setups use different judges, but they prioritize different aspects of ideation and use an independent, fine-grained vs. comparative + holistic evaluation setup, respectively.

AuthorsOpenReview ↗

Clarifying Potential Misunderstanding: ReAct + RL Does Use Fixed Creativity Levels

How does the learned controller compare with fixed creativity levels or simple fixed schedules using the same action space, semantic instructions, reward, and budget?

We would like to note that our ReAct-RL baseline features the same action space, semantic instructions, reward, and budget. We simply modified the prompts to include the following creativity levels by default (effectively a fixed per-action, creativity-level ablation). By “no corresponding Ca\mathcal{C}_a” (L347), we meant that a set of creativity levels were not provided to the agent; instead we assigned a fixed level to each action based on the nature of the action; we clarify this in the updated draft):

ActionReAct-RL (fixed instruction)Corresponding AI Night-Scientist level
Search"background information, adjacent methods, and related work"c=2
Debate"stress-test a proposal [...] competing perspectives, and overlooked assumptions"c=3
Spark"dramatically breaks conventional thinking and your own assumptions"c=5:
WriteIdentical promptc=1 (fixed)

We would also like to note all of our baselines/ablations which isolate the core components of our framework (Section 4.3):

Component isolatedComparison (same reward and budget)Result (Qwen3-8B; citation / originality)
Agentic processZero-shot (no actions, retrieval, or levels)0.46 / 2.53
Selectable levels (no training)ReAct (one fixed level per action) vs. Creative (selectable semantic levels)2.33 / 6.54 vs. 1.89 / 4.25
RL-trained controllerReAct → ReAct + RL2.33 / 6.54 → 24.94 / 46.60
Temp → Temp + RL (no semantic creativity)2.07 / 0.69 → 11.52 / 19.82
Creative → AI Night-Scientist1.89 / 4.25 → 29.89 / 56.32
Learned level selection vs. fixed per-action levelsReAct + RL vs. AI Night-Scientist24.94 / 46.60 vs. 29.89 / 56.32
Semantic levels vs. stochasticityTemp + RL (levels → decoding temperature only)11.52 / 19.82
Action spaceSearch + Write + RL (no debate or spark)24.47 / 33.33
Reward designRout vs. Rpo (Table 3c)e.g., Night-8B: 29.89 / 56.32 vs. 26.20 / 61.61
Model scaleNight-8B → Night-14B29.89 / 56.32 → 33.18 / 68.68

Thus, we hope these 10 component isolations sufficiently demonstrate the comprehensiveness of our evaluation and effectiveness of our framework.

Thank you so much again for your feedback! We hope that the following clarifications and additional experiments have adequately addressed your concerns:

  • Validating scientific significance in final evaluation (both human and automatic evaluation)
  • Blinded human evaluation between ReAct + RL and Night-8B, where Night-8B beats on originality and scientific significance; clear gains on diversity.
  • Scores withstand meaning-preserving rewrites; technical flaw judge drops with even subtly injected technical flaws; demonstrates minimal novelty-technical correctness trade-off
  • Clarified that ReAct + RL does use fixed creativity levels, assigned per action. Isolates the fixed vs. adaptive creativity level selection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Request for Reviewer Follow-up to Rebuttal; Summary of Rebuttal and Paper Updates

We thank all reviewers for their thoughtful feedback! We have updated the paper with the requested clarifications and supporting analyses that support our original framing, methodological claims, and findings. Specifically, we incorporated:

  1. Expanded literature review (YVNf) → Clarified our contributions relative to LDC, MoRI, EvoIdeator, and Scientific Thinker (Sec. 2, L132-140).

  2. Fixed-creativity baseline clarification (gQsj, reRw) → Clarified that ReAct-RL already uses fixed per-action creativity levels, isolating the benefit of selectable creativity levels (Sec. 4.3, L366-368).

  3. Direct Night-8B vs. ReAct-RL evaluations (gQsj, reRw) → Added automatic and blinded human comparisons, alongside qualitative case studies (Secs. 4.4.1–4.4.4, L421-501).

  4. Evaluation robustness analyses (gQsj, reRw) → Added proposal rewrite-robustness analyses, error and length analyses, and a retrieval-overlap audit (Appendix A, L765-813; Appendix M, L1643-1649).

We would greatly appreciate it if Reviewer reRw could please respond to our rebuttal and, if their concerns have been adequately addressed, kindly consider updating their scores accordingly.


For the AC and reviewers' convenience, we briefly reiterate our contributions and summarize how we addressed the concerns.

Our work targets an overlooked challenge in AI-assisted scientific discovery: learning when and how to reason creatively during scientific ideation, instead of treating creativity as only a property of the output idea.

  1. Multi-level creativity representation: We represent creativity at the action, process, and outcome levels.
  2. Action-specific creative control: Our agentic action space (Search, Debate, Spark, Write) supports varying semantic creativity levels, for learning which actions to perform, how creatively, and when to explore versus develop promising directions.
  3. Reinforcing creative exploration: We investigate outcome/process-level rewards. Relative to its base model, Night-8B improves originality by +53.79 pp and predicted citation impact by +29.43 pp; relative to temperature-based RL, gains remain +36.50 pp and +18.37 pp, respectively.

Reviewers gQsj and reRw: Independent Evaluation and Robustness

Both reviewers requested stronger validation against our ReAct-RL ablation and further evaluation robustness analyses. We addressed these through:

  • Fixed vs. selectable creativity: Clarified that ReAct-RL already uses fixed per-action semantic instructions, providing an existing control for learned creativity-level selection.
  • Direct paired evaluation: Across 350 matched problems (700 proposals), Night-8B wins 68.30% on originality and 68.59% on predicted citation impact, with 95% confidence intervals.
  • Blinded human evaluation: Across 25 pairs (50 proposals), experts prefer Night-8B 92% for originality and 64% for scientific significance, considering correctness and feasibility.
  • Evaluator robustness: Our 700-proposal error analysis identifies slightly higher technical error rates for Night-8B (0.29-5.76%), but after meaning-preserving rewrites of them (removing unsupported claims from both methods), win rates remain 67-69%. The flaw evaluator was validated to reverse 99% of preferences after injecting substantive conceptual errors, indicating that the infrequent errors within Night-8B proposals are not significant for reversing preference.

Reviewer reRw: Retrieval & Evaluation Setup Clarifications

  • Reference/retrieval bias: Clarified that reconstructed references are for assessing relative gains between methods. Our retrieval audit found only 16 overlaps (3.26% of samples) with reconstruction-source publications; we show that our evaluation gains are robust to these samples.
  • Proposal evolution: Added 2 qualitative trajectory analyses showing that Night-8B meaningfully revises its research mechanisms through intermediate exploration, whereas RL baselines largely repeat earlier proposals (Appx. E.1-E.2, L1037-1125).
  • Implementation details: Clarified action-swap decay, gradient masking, and the no-swap ablation configuration (Sec. 3.3.2).

Reviewer YVNf: Clear Contribution Distinctions

  • Related work: Expanded Sec. 2 to distinguish our learned action-specific creativity control from approaches focused on generation quality, reasoning optimization, or iterative refinement.
  • Diversity evaluation: Clarified that our metrics capture corpus-level conceptual flexibility, not within-task diversity or fluency. Our original results demonstrate +27.8% research-paradigm and +14.9% contribution-type diversity, complemented by new qualitative comparisons (Sec. 4.4.2).

We hope these additional analyses and clarifications adequately address the reviewers' concerns and further support the value of learning creative control within scientific reasoning.

Thank you again for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.