acceptodds
Under review as a conference paper at ICLR 2027

A Survey of Human-AI Collaboration for Scientific Discovery

Abstract

Artificial intelligence (AI) is increasingly integrated into scientific discovery processes, such as protein design, gene analysis, and materials research, significantly improving research efficiency in these fields. While much of the recent literature emphasizes fully automated pipelines, it is crucial to acknowledge that scientific discovery is inherently a creative and high-stakes endeavor. Therefore, it relies heavily on human expertise for judgment and guidance, especially in the face of uncertainty. Despite the rapid growth in human-in-the-loop and collaborative systems, the field still lacks a unifying survey that explains how humans and AI collaborate throughout the scientific discovery life cycle. In this paper, we present a systematic review of human–AI (HAI) collaboration for scientific discovery. Specifically, we identify four representative roles that humans and AI can assume, including Informer, Explorer, Evaluator, and Controller. Using this lens, we then distill common HAI collaboration patterns across three distinct stages in the scientific discovery process (i.e., observation, hypothesis, and experiment). Finally, we identify key gaps in existing approaches and outline future research directions for developing trustworthy, role-aware HAI collaboration systems in scientific discovery.

Primary area:
interpretability and explainable AI
Keywords:
Human-AI Collaboration, Scientific Discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JRFpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper surveys 65 studies on human-AI collaboration in scientific discovery and proposes four roles—Informer, Explorer, Evaluator, and Controller—to describe how humans and AI work together.

Critical strengths
  1. Structured literature review: The paper systematically reviews 65 studies and organizes them into a unified framework.

  2. Broad coverage: The paper covers human-AI collaboration across different scientific domains and discusses potential future directions.

Critical weaknesses

1. Limited novel contributions for ICLR. The paper is mainly a descriptive survey, with its primary contribution being a taxonomy of four roles. While this is useful for organizing existing work, I find it difficult to see what new insights it offers beyond summarizing the literature. The contribution seems insufficient for an ICLR research paper.

Questions for authors

None

Reviewer_Jx29OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a taxonomy of the scientific discovery process and the roles humans and AI can occupy in the process. The AI scientist literature is surveyed to construct this taxonomy, and then common patterns of human-AI interaction are identified. In particular, it is noted that across the process, humans and AI tend to occupy specific relative roles, with humans typically occupying judgement-sensitive roles. Finally, several new or emerging directions are proposed on the basis of these observations, including the potential benefit of adaptive HAI interfaces and the need to consider these implications in the context of an AI-driven lab.

Critical strengths

I think the paper clearly explains the methodology and the rationale behind decisions made. The field of AI systems/agent design is often very discrete, and published literature uses varying terminology. The work the authors have done to collate and categorize the field is a valuable contribution.

Critical weaknesses

My first concern is with the coding procedure used for taxonomy construction. It would help to provide some rationale behind why this is a useful taxonomy. I think the methodology presented justifies it as a natural one (i.e. it is constructed by the natural labels human readers apply when reading papers), but it would help to understand why considering Informers vs Explorers is useful. Or drawing some conclusions about why things naturally group this way.

Secondly, I find much of the Discussion section weakly supported by the presented taxonomy and data. The first claim is reasonable, but the generative UI argument reads to me is unrelated to the rest of the paper, besides the broader framing.

Questions for authors
  1. Could you please provide Figure 3 (or the data) after normalization? It is unclear if the share of AI controllers/evaluators is actually increasing over time relative to the increase in total volume (ie. if there are 10% controllers and <10 papers you wouldn't expect to see any controllers).
  2. Can you offer stronger motivation for the proposed taxonomy? You note in the Discussion that most studied papers emphasize data analysis vs lab experimentation, but that is not captured in the taxonomy. Similarly, an Explorer stage that designs experiments vs proposes hypotheses seems meaningfully different. I am not suggesting reshaping the taxonomy to account for these things, but some justification for why this is the correct level of abstraction would help.
Reviewer_wVCxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper surveys human–AI collaboration for scientific discovery. It introduces four functional roles: Informer, Explorer, Evaluator, and Controller, which can be assigned to either humans or AI, and examines their pairings across observation, hypothesis, and experiment stages. The authors describe a literature selection and coding procedure, report inter-coder agreement, and identify recurring collaboration patterns. The paper concludes with research directions concerning reliable evaluation and control, generative interfaces, adaptive role assignment, embodied experimentation, and the long-term implications of AI-assisted science.

Critical strengths
  1. A useful framework for making human participation explicit. Applying the same functional roles to humans and AI provides a practical vocabulary for distinguishing different forms of collaboration. In particular, separating candidate generation from scientific evaluation, and artifact evaluation from workflow control, helps characterize responsibilities that an autonomy-based classification may obscure. This makes the survey useful for readers comparing collaborative systems across scientific domains.

  2. A reasonably well-documented coding methodology. The paper provides inclusion criteria, a screening flow, operational definitions, and rules for resolving ambiguous role boundaries. Each paper is independently coded by three authors, with reported Krippendorff’s α values of 0.78 for roles and 0.85 for stages. The explicit requirement that paired actions participate in an actual collaboration episode is a methodological strength: it avoids treating every combination of human and AI roles within a system as an observed interaction.

Critical weaknesses
  1. The distinctive contribution relative to closely related surveys is insufficiently established. Even for the surveys already cited, the comparison largely identifies differences in emphasis: prior work focuses on AI capabilities, autonomy levels, or general collaboration principles, whereas this paper focuses on role allocation in scientific discovery. This motivates the perspective but does not establish its added analytical value. The paper does not show, through concrete comparisons, which important collaboration structures existing frameworks fail to capture or which findings become apparent specifically through the proposed role-pair taxonomy. Although the discussion of coarser and finer role definitions supports the taxonomy’s internal coherence, it does not substitute for comparison with established frameworks. Moreover, directly relevant works such as Zhao et al. [1] and Hasib et al. [2] deserve explicit discussion. Overall, the paper needs to distinguish its new insights from familiar observations, such as AI generating candidates while humans evaluate them, and demonstrate why its framework materially advances our understanding of scientific human–AI collaboration.

  2. Several interpretations exceed what the descriptive evidence establishes. Section 3.2 interprets differences between stages as dynamic role reconfiguration as uncertainty decreases and task structure increases. However, comparing role frequencies across heterogeneous systems does not establish within-workflow transitions or their causes. Similarly, Section 4 attributes the scarcity of AI-Evaluators to technical limitations, although Appendix B excludes internal AI evaluation unless it enters a visible human–AI interaction. That is a defensible scope choice, but it makes the observed scarcity partly dependent on the coding definition. The preprint citation threshold may introduce additional selection effects. Finally, Table 5 lists paper-level roles and stages, rather than the episode-level pairings needed to reconstruct Figure 2. These issues weaken the connection between the reported counts and the broader conclusions about how collaboration develops and why particular roles remain uncommon.

[1] Zhao et al. 2025. Human–AI Collaboration for Scientific Discovery. Cambridge University Press.
[2] Hasib et al. 2025. A Process-Centric Survey of AI for Scientific Discovery Through the EXHYTE Framework.

Questions for authors
  1. How does the proposed role-pair framework differ from directly related scientific collaboration and process-centric surveys? Can the authors identify a concrete finding or design implication that becomes apparent through this framework but is obscured by those alternatives?

  2. What exactly is counted in each cell of Figure 2: collaboration episodes, distinct role pairs per paper, or another unit? Can the authors provide the corresponding records and explain how repeated episodes within one paper are aggregated? Table 5 alone does not make these counts reproducible.

  3. Is the claim of dynamic role reconfiguration supported by documented transitions within individual systems, or is it an interpretation of aggregate differences between stages? Likewise, what evidence distinguishes technical limitations from corpus selection and coding scope as explanations for the scarcity of AI-Evaluators?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.