acceptodds
Under review as a conference paper at ICLR 2027

Toward Domain-specific Innovation Discovery in Computer Science Research

Abstract

Recent advances in Large Language Models (LLMs) have substantially improved scientific hypothesis formulation and research idea generation. However, existing approaches often produce superficial recombinations of prior solutions, while external literature resources rarely provide the fine-grained, structured evidence required to develop technically coherent ideas for complex and interdisciplinary research problems. To address these limitations, we present CSMuse, an evidence-grounded idea-generation system that leverages cross-domain knowledge. CSMuse incorporates three key designs: (1) domain-specific scientific knowledge construction, (2) evidence-grounded idea generation and validation, and (3) cross-domain multi-agent ideation and refinement. Evaluations across multiple LLM backbones and research domains demonstrate that \system consistently generates high-quality, novel ideas and outperforms state-of-the-art automated scientific discovery systems.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
AI for Scientific Discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bctBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CSMuse, an idea-generation system for complex and interdisciplinary computer science research. The system has three main components: (1) a structured paper-summary dataset built from premier conferences across multiple ACM CCS domains, where each paper is normalized into five fields (Problem, Background, Design, Evaluation, Conclusion); (2) domain-specific knowledge graphs constructed from these summaries, used to retrieve traceable evidence for generated ideas; and (3) a cross-domain multi-agent workflow in which a seed idea is generated by a primary-domain agent, critiqued by a critic agent, revised independently by domain-specific agents using evidence from their respective knowledge graphs, and finally integrated and novelty-refined. The authors evaluate CSMuse on three domains (Computing Methodologies, Networks, Computer Systems Organization) using GPT-5.5 and DeepSeek-V4-Pro, comparing against AI Scientist and VIRSCI, with additional ablation studies on evidence grounding and cross-domain refinement.

Critical strengths

1、The paper accurately identifies and targets a real limitation in existing cross-domain idea-generation systems. The authors explicitly note that simply aggregating independently generated ideas does not ensure meaningful cross-domain transfer and may produce incoherent or technically incompatible combinations. CSMuse addresses this through domain-specific knowledge graphs, evidence retrieval, and cross-domain multi-agent refinement. This is consequential because it grounds the system's design in a clearly articulated failure mode of prior work, rather than proposing a generic multi-agent pipeline. 2、The structured paper-summary dataset and domain-specific knowledge graphs provide a traceable, reusable foundation for evidence-grounded idea generation. By normalizing papers into five structured fields and constructing graphs with explicit entity and relation types, the authors preserve mechanism-level design information that raw PDFs or abstracts often omit. This is consequential because it enables the evidence-grounding and retrieval components that distinguish CSMuse from purely parameter-based LLM idea generation.

Critical weaknesses

1、The benefits of cross-domain knowledge transfer are inconsistent across domains, and the paper does not adequately explain this. In the pairwise comparisons, the variant without cross-domain knowledge slightly outperforms the full system in Computing Methodologies (51% vs. 49%), and the "Refinement alone" variant also outperforms "Retrieval and Refinement" in CM and CSO. While the authors provide a finer-grained analysis using CD/HD and LJ scores, the overall evidence for cross-domain transfer in CM remains weak. This is consequential because cross-domain knowledge transfer is one of the paper's three core contributions, and a mixed result undermines the claim that CSMuse "consistently outperforms" baselines. 2、The paper does not report basic statistics about the constructed knowledge graphs, nor does it provide concrete examples of retrieved evidence being used in the final ideas. The paper claims to be "evidence-grounded," but without information on graph scale (number of nodes and edges), coverage, or extraction accuracy, and without examples showing how retrieved evidence actually shapes the revised ideas, the reader cannot verify that the retrieval component is doing meaningful work rather than simply providing additional context to the LLM. This is consequential because evidence grounding is the paper's second core contribution.

Questions for authors

1、The cross-domain ablation results in Computing Methodologies show that the variant without cross-domain knowledge slightly outperforms the full system. Can you explain why cross-domain transfer appears less effective in CM than in Net and CSO? Is this due to the nature of the domain, the quality of the retrieved evidence, or some other factor? 2、The Overall Idea Score combines the LLM-as-a-Judge score and the CD/HD ratio. Can you clarify the dimension and range of the CD/HD ratio, and explain why this particular combination was chosen over alternatives?

1 comment
AuthorsOpenReview ↗

Response to Reviewer bctB

We thank the reviewer for the thoughtful assessment and constructive suggestions, and for recognizing the value of our structured knowledge representation and evidence-grounded cross-domain design. We have revised the manuscript, with changes highlighted in blue. Below, we address each concern and question.

Weakness 1: Inconsistent benefits across domains

To better characterize these differences, we have added paired analyses in Section 6.5 and a dimension-level comparison in Figure 6. These analyses clarify where cross-domain integration contributes most and how its novelty gains relate to technical soundness and feasibility. We discuss the results in our response to Question 1 below.

Weakness 2: Knowledge-graph statistics and concrete evidence-use examples

Thank you for highlighting the need to make the evidence-grounding process more transparent. We have expanded the description of the constructed knowledge graphs in the revised manuscript and added a worked example illustrating the complete workflow.

Q1: Why does cross-domain transfer appear less effective in Computing Methodologies than in Networks and Computer Systems Organization?

The CM result reflects a smaller benefit from cross-domain integration under the pairwise evaluation: the single-domain Retrieval and Refinement variant is preferred in 51% of comparisons, versus 49% for CSMuse. This near-even result contrasts with the stronger pairwise performance in Networks and CSO.

To understand this difference, we added a comparison across the six evaluation dimensions in Figure 6. CSMuse achieves higher average scores for novelty, mechanism specificity, and evaluation quality across all three domains, alongside lower scores for technical soundness and feasibility. The novelty improvements are +0.28, +0.32, and +0.35 points in CM, Networks, and CSO, respectively. Thus, cross-domain transfer improves the assessed novelty of CM ideas, although this improvement does not translate into a higher pairwise preference rate.

One possible explanation is that the usefulness of external knowledge depends on how naturally it connects to the target problem. Networks and CSO problems may offer more direct opportunities to combine architectural and system-level mechanisms from different domains. For some CM problems, incorporating external mechanisms may require additional assumptions or implementation complexity, offsetting their novelty benefits.

These results highlight the value of cross-domain transfer in generating more novel and detailed research ideas, while identifying technical soundness and feasibility as priorities for further development. In future work, we will implement and empirically evaluate these ideas to assess their technical soundness and practical potential.

Q2: What are the dimension and range of CD/HD, and why is it combined with the LLM-as-a-Judge score in this way?

Thank you for requesting this clarification. Both HD and CD are computed from cosine similarities:

  • Historical Dissimilarity (HD) is the average cosine similarity between a generated idea and its three most similar paper summaries in the historical corpus (2021–2025). Lower HD indicates less semantic overlap with prior work and therefore greater potential novelty.
  • Contemporary Dissimilarity (CD) is the average cosine similarity between a generated idea and its five most similar paper summaries in the contemporary corpus (2026). Higher CD indicates greater alignment with independently published contemporary research.

Each is dimensionless and has a theoretical range of [-1,1].

We define the Overall Idea Score as OIS=LJ5×CDHD\mathrm{OIS}=\frac{\mathrm{LJ}}{5}\times\frac{\mathrm{CD}}{\mathrm{HD}}, where LJ/5\mathrm{LJ}/5 normalizes the five-point judge score. The two factors provide complementary perspectives: LJ assesses the proposed idea across six quality dimensions, while CD/HD considers its relationship to historical and contemporary literature. When CD and HD are positive, the ratio favors ideas that align with contemporary research while exhibiting less overlap with historical work.

We chose a multiplicative formulation so that the literature-based factor scales the normalized quality score, rather than contributing a separate additive term requiring a mixing weight. This is a heuristic composite measure, rather than a calibrated probability or an independently validated measure of scientific value. We also report LJ and the embedding-based analyses separately so that the results can be assessed beyond the aggregate OIS.

Reviewer_4WnoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents CSMuse, an evidence-grounded research idea generation system for complex and interdisciplinary computer science research. CSMuse integrates three key components: constructing structured paper summaries and domain-specific knowledge graphs, retrieving literature evidence to support idea generation and refinement, and employing cross-domain multi-agent collaboration to improve technical coherence and feasibility. Specifically, a seed idea is generated and critiqued, domain-specific agents propose evidence-supported revisions, and an integration agent synthesizes compatible suggestions. The system further uses embedding-based similarity to identify overlaps with prior work and refine idea novelty. The authors evaluate CSMuse across three computer science domains using GPT-5.5 and DeepSeek-V4-Pro, comparing it with AI Scientist and VIRSci. The results show that CSMuse achieves higher overall idea quality scores than both baselines across the evaluated domains and models. Ablation studies further investigate the contributions of evidence retrieval, iterative refinement, and cross-domain knowledge transfer, although the benefits of individual components vary across domains and evaluation metrics.

Critical strengths
  1. The paper addresses a practically relevant problem and explores the integration of domain knowledge with multi-agent collaboration. The paper focuses on generating scientific research ideas that are more innovative, technically sound, and relevant to specific research domains, which is a practically valuable research direction. CSMuse integrates a domain-specific knowledge graph, retrieval-augmented generation, and iterative multi-agent refinement into a unified framework, aiming to leverage knowledge and evidence from existing literature to guide idea generation and improvement. This design provides a meaningful direction for exploring how to improve the quality of LLM-based scientific idea generation.

  2. The experiments cover multiple computer science domains and different underlying models, providing empirical evidence for the proposed approach. The paper evaluates CSMuse across three computer science domains and two different underlying models. The results show that CSMuse generally outperforms the selected baselines in terms of the proposed Overall Innovation Score (OIS). The paper also conducts ablation studies to investigate the effects of retrieval, iterative refinement, and cross-domain knowledge transfer. Although the current evaluation metrics have limitations, these experiments provide some empirical support for the effectiveness of the framework across different domains and model settings, offering a useful experimental basis for further research on domain-knowledge-enhanced scientific idea generation systems.

Critical weaknesses
  1. Limited methodological novelty. The proposed CSMuse framework consists primarily of initial idea generation, critic-based evaluation, multi-agent refinement, knowledge retrieval, and novelty screening. However, existing research on scientific idea generation has already explored similar design principles, including critic-based iterative refinement, multi-agent collaboration, and retrieval-augmented idea generation. Compared with these existing approaches, the paper does not sufficiently clarify the substantive methodological innovations of CSMuse or explain what distinguishes the interactions among its components from prior work. Although the experimental results show that CSMuse outperforms the selected baselines, they do not sufficiently establish the novelty of the proposed methodology itself.

  2. Limitations of the evaluation framework. The paper primarily relies on LLM-as-a-Judge to assess idea quality and embedding-based cosine similarity to measure the distinctiveness of generated ideas from prior work and contemporary research directions. However, these metrics have important limitations. LLM-based judgments may be influenced by model preferences and the subjectivity of evaluation criteria, while textual similarity does not necessarily capture differences in research problems, technical mechanisms, or substantive innovation. Furthermore, the paper combines these metrics into the OIS as its primary evaluation measure without sufficiently validating whether this composite metric reliably reflects the actual value of scientific innovation. For a system designed to generate high-quality research ideas, the current evaluation evidence is insufficient to substantiate the paper’s central claims regarding idea quality and novelty.

Questions for authors
  1. Clarification on methodological novelty. The proposed CSMuse framework combines critic-based evaluation, multi-agent collaboration, retrieval-augmented refinement, and novelty screening. Given that similar design principles have been explored in prior work on scientific idea generation, could the authors clarify the key methodological differences between CSMuse and existing approaches? In particular, what specific capabilities or benefits arise from the proposed framework that cannot be readily achieved by combining existing techniques?

  2. Validation of the evaluation metrics. The evaluation primarily relies on LLM-as-a-Judge and embedding-based cosine similarity, which may not reliably capture the substantive novelty and scientific value of generated research ideas. Could the authors consider a more structured approach to novelty assessment, such as retrieving relevant literature and using agents to conduct rubric-based or structured analyses of the similarities and differences in research problems, technical mechanisms, and contributions, rather than relying primarily on cosine similarity? Such an approach could provide a more reliable assessment of novelty and help establish whether the reported improvements in OIS reflect meaningful improvements in research idea quality.

  3. Examples of generated research ideas. I could not find concrete examples of research ideas generated by CSMuse in the paper. Could the authors provide several representative examples, ideally accompanied by case studies illustrating how the framework generates an initial idea, identifies its limitations, and refines it through knowledge retrieval and multi-agent collaboration? These examples would help readers better understand the framework’s practical behavior and assess whether its refinement process leads to substantively more innovative and technically sound research ideas.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 4Wno

We thank the reviewer for the thoughtful assessment and constructive suggestions, and for recognizing the practical relevance of evidence-grounded scientific idea generation and the breadth of our experiments. We have revised the manuscript, with changes highlighted in blue. Below, we address each concern and question.

Weakness 1: Limited methodological novelty

We emphasize the contribution of innovative knowledge representation and provide evidence for both human comprehension and the reliability of auto-research. We have clarified how this representation supports the interactions among the agents and distinguishes the framework from a generic combination of retrieval and multi-agent refinement. We elaborate on these differences in our response to Question 1.

Weakness 2: Limitations of the evaluation framework

To complement the LLM-based and embedding-based metrics, we have recruited four PhD students to conduct a blind evaluation of the generated ideas. Responses are currently being collected, and we will report the evaluation protocol and results once data collection is complete. This evaluation will provide additional evidence about whether the improvements measured by our automated metrics correspond to ideas that experts consider novel, technically credible, and actionable.

Q1: Clarification on methodological novelty

The key distinction of CSMuse lies in how scientific knowledge is represented and used throughout the refinement process. On the other hand, it also provides evidence for both human comprehension and the reliability of auto-research.

First, CSMuse converts papers into a consistent structured representation and organizes their research problems, motivations, key ideas, and system components through explicit relations. This representation captures how a mechanism addresses a research problem and how its components implement the underlying idea, making these connections available as traceable evidence for subsequent ideation.

Second, CSMuse organizes this structured representation into domain-specific graphs that support the corresponding domain agents. Following criticism of a seed idea, each agent retrieves evidence from its own domain graph and develops revisions informed by that evidence. The integration agent then synthesizes these revisions into a coherent proposal. Domain specialization is therefore supported by distinct, explicitly organized knowledge sources, rather than solely by assigning different roles through prompts.

Together, these design choices connect mechanism-level literature evidence with critique-guided, cross-domain refinement. They enable researchers to inspect the knowledge informing a proposed revision and examine how mechanisms from different domains contribute to the final idea. We emphasize the contribution of innovative knowledge representation and provide evidence for both human comprehension and the reliability of auto-research.

Q2: Validation of the evaluation metrics

Thank you for suggesting another approach to novelty assessment. Retrieving relevant literature and comparing research problems, technical mechanisms, and contributions through an explicit rubric is a promising direction for making novelty judgments more interpretable and substantively grounded. We also recognize that relying solely on LLM-based judgments may leave some aspects of the evaluation difficult to interpret.

In our evaluation, cosine similarity serves as a complementary measure by providing a consistent, quantitative estimate of semantic proximity between generated ideas and paper summaries, enabling comparisons across methods and domains. Within CSMuse, similarity screening also helps identify potentially overlapping prior work for the novelty agent to examine during refinement. In this way, it supports literature-informed revision as part of the generation process, rather than serving as a standalone measure of scientific novelty.

To further assess the quality of the generated ideas, we have initiated the blind expert evaluation and the evaluation results have been incorporated into the revised manuscript. We believe that combining quantitative measures with expert assessments can provide a more comprehensive perspective on idea quality. The reviewer’s suggestion of rubric-based literature comparison is also valuable for further strengthening the interpretability and depth of our evaluation.

Q3: Examples of generated research ideas

We have added a worked example in Figure 2 and Section 3 of the revised manuscript. The example illustrates the complete workflow, from initial idea generation and criticism to evidence retrieval, domain-specific revision, integration, and novelty refinement. It also provides details of the retrieved evidence and its role in shaping the revisions, making the framework’s practical behavior easier to inspect.

Reviewer_zqaHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents CSMuse, an automated idea-generation system designed for computer science research. To address issues in existing LLM-based discovery systems—such as lack of fine-grained evidence grounding and shallow cross-domain integration—CSMuse proposes a three-part pipeline:

Converting CS conference papers (2021–2025/2026) into structured JSON summaries.

Building domain-specific Knowledge Graphs (KGs) via GraphRAG to store fine-grained schema relations (Problem, Key Idea, System Component, etc.).

Deploying a multi-agent workflow (Seed Agent, Critic, Domain-Specific Agents, Fusion/Integration Agent, and Novelty Agent) to generate, critique, retrieve evidence for, and iteratively refine research ideas.

Evaluations are conducted across three ACM domains (Computing Methodologies, Networks, Computer Systems Organization) using embedding-based metrics (CD/HD) and LLM-as-a-Judge protocols.

Critical strengths

The idea of extracting fine-grained, operational mechanisms from PDF papers into JSON schema and building GraphRAG-based domain knowledge graphs is well-motivated for complex computer science domains.

The appendix provides detailed and well-structured prompt templates for paper extraction, criticism, fusion, and novelty checking.

Critical weaknesses

1)Incremental Combination of Existing Frameworks: The core components of CSMuse—GraphRAG for retrieval, multi-agent critic/refinement workflows, and embedding-based similarity filtering—are existing techniques. Combining multi-agent interaction with RAG is now a standard paradigm in automated scientific discovery (e.g., ResearchAgent, VIRSCI, SciAgents).
Lack of Deep Technical Innovation: The paper lacks algorithmic innovation in how agents reason over cross-domain knowledge; it relies almost entirely on prompt engineering to enforce structured behavior rather than introducing novel agentic architecture or learning mechanisms.

2)Contradictory/Mixed Ablation Findings: In Section 6.3 and Figure 3, the pairwise win-rate comparisons show that Refinement alone or Retrieval + Refinement (Single Domain) frequently ties with or even beats the full CSMuse system (e.g., Retrieval & Refinement slightly outperforms CSMuse in CM at 51% vs. 49%, and Refinement alone outperforms Retrieval & Refinement in CSO). This undermines the paper's core claim that cross-domain multi-agent fusion and KG evidence grounding provide consistent, decisive gains.
Over-reliance on Heuristic Thresholds: The similarity-based novelty refinement (§5.3) relies on arbitrary thresholds (tproblem=0.5t_{\text{problem}}=0.5, tmechanism=0.8t_{\text{mechanism}}=0.8) using SPECTER embeddings. There is no sensitivity analysis or justification provided for these hyperparameter choices, nor proof that cosine similarity on embeddings captures technical novelty vs. mere lexical overlap.

3)Circular Evaluation / LLM-as-a-Judge Bias: The paper uses DeepSeek-V4-Pro both to generate evaluation problems/summaries/graphs (§6.1) and as the evaluator/judge (§6.1, §6.3). Evaluating LLM-generated ideas using the same LLM backbones introduces severe self-preference and tautological bias.
Unrealistic Baseline Performance / Metric Inflation: The reported LLM-as-a-Judge scores for CSMuse routinely reach near-perfect scores (~4.8 to 4.94 out of 5 in Tables 8–10), which strongly suggests metric saturation or prompt-induced scoring inflation rather than genuine, actionable scientific breakthroughs.

4)Missing Baseline & Paradigm References: The literature review misses several critical recent benchmarks and frameworks in AI-driven scientific discovery and agentic graph reasoning (e.g., AgentLaboratory, IdeaAgent, or recent benchmarks assessing hallucination in automated ideation).Anachronistic / Predictive References: The reference list includes multiple citations dated 2026 (e.g., ACM 2026, IEEE 2026, OpenAI 2026, DeepSeek 2026, ACM Digital Library 2026), which appears artificially future-dated or misformatted for a conference submission.

Questions for authors

Have you conducted any blind evaluation with domain experts (e.g., PhD students or professors in Systems/Networking) to verify whether the top-scoring generated ideas are technically sound, non-trivial, and actionable?

How do you explain the fact in Figure 3 where LLM with Refinement beats LLM with Retrieval and Refinement in CSO (54% vs 46%), and Retrieval and Refinement beats CSMuse in CM (51% vs 49%)? Does this indicate that cross-domain knowledge transfer sometimes introduces noise rather than value?

How sensitive are the novelty results to the selection of tproblem=0.5t_{\text{problem}}=0.5 and tmechanism=0.8t_{\text{mechanism}}=0.8?

2 comments
AuthorsOpenReview ↗

Response to Reviewer zqaH (Part 1/2)

We thank the reviewer for the detailed assessment and constructive suggestions, as well as for recognizing the motivation behind our fine-grained knowledge representation and the clarity of our prompt templates. We have revised the manuscript, with changes highlighted in blue. Below, we address each concern and question.

Weakness 1: Contribution and distinction from existing frameworks

The core contribution of our work is a systematic, structured representation of research papers that consistently captures each paper’s research problem, motivation, key idea, and underlying mechanisms. This representation provides explicit evidence for idea generation and makes the logical relationships within and across papers more transparent. To the best of our knowledge, this is the first work to introduce such a formalized paper abstraction specifically for research idea generation.

Specifically, CSMuse converts papers into a consistent structured representation and organizes their research problems, motivations, key ideas, and system components into explicit relations. This representation preserves how a proposed mechanism addresses a problem and how its components implement the underlying idea, providing more detailed and traceable evidence for subsequent ideation.

CSMuse further organizes this knowledge into domain-specific graphs. Following criticism of a seed idea, each domain agent retrieves evidence from its corresponding graph and proposes revisions, which an integration agent then synthesizes. The distinction is therefore not simply the assignment of different expert roles: domain specialization is supported by explicitly organized knowledge sources and retrieved evidence.

Weakness 2: Mixed ablation findings

We agree that the pairwise results do not support a claim that every component improves every idea or every evaluation dimension. We have retained these results and added more detailed analyses in Sections 6.4–6.5 and Figures 5–6. We discuss these findings in our response to Question 2 below.

Weakness 3: Evaluation bias and high judge scores

We appreciate this concern. In the revised evaluation, generation and judging use different models: ideas generated by DeepSeek-V4-Pro are evaluated by GPT-5.5, while ideas generated by GPT-5.5 are evaluated by Claude-Fable-5. As reported in Table 1(b), CSMuse achieves higher average LLM-as-a-Judge and Overall Idea Scores than both baselines across the three domains under the additional generation–judge configuration.

Regarding the high aggregate scores, we further examined the results across individual evaluation dimensions. We observed clear differences among the criteria: novelty, mechanism specificity, and evaluation quality generally received higher scores, whereas technical soundness and feasibility received relatively lower scores. This variation suggests that the evaluation captures different aspects of idea quality rather than assigning uniformly high scores across all criteria. Nevertheless, we acknowledge that LLM-based evaluation has inherent limitations, and the scores should be interpreted as comparative measures under the adopted evaluation protocol.

Weakness 4: Related work and reference formatting

We have expanded the related-work discussion to include additional relevant systems, including Agent Laboratory and IdeaAgent. We have also corrected references in which website access years had been incorrectly recorded as publication years. We appreciate the reviewer for identifying these issues.

AuthorsOpenReview ↗

Response to Reviewer zqaH (Part 2/2)

Q1. Have you conducted any blind evaluation with domain experts to verify whether the generated ideas are technically sound, non-trivial, and actionable?

Thank you for this valuable suggestion. We recruited four PhD students specializing in networking to conduct a blind evaluation of the generated ideas. The evaluation results have been incorporated into the revised manuscript, and the corresponding questionnaire has been included in the supplementary materials.

Q2. How do you explain the cases where Refinement outperforms Retrieval and Refinement, or Retrieval and Refinement outperforms CSMuse? Can cross-domain transfer introduce noise?

We have added a more qualified interpretation of the benefits of retrieval and cross-domain integration.

Regarding retrieval, we acknowledge that its benefits are not necessarily reflected in higher pairwise preference in every case. Nevertheless, retrieval provides explicit links to source knowledge, allowing researchers to inspect the supporting evidence and assess how the generated ideas relate to existing work. Thus, its value extends beyond its potential contribution to idea generation, particularly in enhancing evidence traceability and supporting subsequent human verification.

To better understand the cross-domain results, we added a dimension-level comparison between CSMuse and the single-domain Retrieval and Refinement variant in Figure 6. Across all three domains, CSMuse receives higher average scores for novelty, mechanism specificity, and evaluation quality, but lower scores for technical soundness and feasibility. The novelty gains are +0.28, +0.32, and +0.35 points in CM, Networks, and CSO, respectively.

Taken together, these results suggest that cross-domain transfer can contribute to generating more novel and detailed research ideas, while also introducing challenges in technical soundness and feasibility. This trade-off highlights both the potential and the limitations of cross-domain integration. In future work, we plan to implement and empirically evaluate the generated ideas to further assess their technical soundness and practical potential.

Q3. How sensitive are the novelty results to tproblem=0.5t_{\mathrm{problem}}=0.5 and tmechanism=0.8t_{\mathrm{mechanism}}=0.8 ?

We have added sensitivity analyses in Appendix C.3 and clarify that the two thresholds serve different purposes.

The problem-similarity threshold filters candidate papers before mechanism comparison. As shown in Table 11, thresholds of 0.4, 0.5, and 0.6 retain, on average, 99.9%, 98.1%, and 74.4% of problem–paper pairs, respectively. Across these tested settings, the reported rankings and top-three mechanism-similarity results remain unchanged across domains and ablation variants. Thus, these comparison results are stable within the tested range. We use 0.5 as a conservative setting that removes a small fraction of low-similarity candidates while preserving broad coverage.

The mechanism-similarity threshold identifies potentially overlapping prior work to provide feedback for novelty refinement. Tables 12–14 report how the number of retrieved candidates changes with this threshold. Stricter thresholds substantially reduce the available feedback: for example, in CM, the Vanilla LLM variant retrieves an average of 68.80 candidates at 0.8, 7.64 at 0.85, and 0.34 at 0.9. A lower threshold admits many more candidates and is less selective. We therefore adopt 0.8 as a practical trade-off, rather than claiming it to be an optimal threshold.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.