acceptodds
Under review as a conference paper at ICLR 2027

Deep Cognition: A Transparent, Fine-Grained Multi-Agent Deep Research System for Human-Agent Collaboration

Abstract

Deep research systems can synthesize information across hundreds of sources, yet they operate under an "input–wait–output" paradigm that offers no visibility into intermediate decisions and no mechanism for mid-process correction. We introduce Deep Cognition, a multi-agent system that renders the research process transparent, controllable, and interruptible through fine-grained bidirectional dialogue. Users can steer individual subagents, inject domain knowledge, and redirect reasoning at any checkpoint. A controlled within-subjects study (n=50, spanning six professional domains) shows that Deep Cognition significantly outperforms commercial baselines on interaction quality while achieving competitive report quality. Automated evaluation across six benchmarks demonstrates state-of-the-art research synthesis—achieving RACE 52.01 on DeepResearch Bench (vs. 48.88 Gemini, 46.98 OpenAI), 0.715 rubric coverage on ResearcherBench (vs. 0.703 OpenAI), and 50.0% accuracy on GAIA Level 3 (vs. 26.9% OpenAI)—with ablations confirming that both transparent architecture and user expertise are necessary. Large-scale deployment analysis (2,461 sessions, 194 users) provides convergent evidence at scale: users spontaneously develop rubric-based quality signals that trigger persistent quality improvements, and exhibit phase-dependent strategy shifts that mirror controlled study findings—demonstrating that structured human reward shaping fundamentally improves deep research outcomes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Decision making, reinforcement learning, and control (e.g., hierarchical RL, multi-agent systems, agentic AI), Human interaction in AI systems (e.g., human-in-the-loop, affective computing)

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_f7zxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Deep Cognition is a multi-agent system for deep research tasks that exposes plans, subagents, queries, evidence, and drafts so users can intervene mid-process. It has a 2B evidence router and a pipeline that uses an LLM to simulate an expert. The authors report a 50 people user study, benchmark results on 5 datasets and controlled feedback-timing experiments and analysis of 2461 sessions.

Critical strengths

1- Mid-process, artifact level steering of deep research agents is a timely and useful problem. 2- The feedback timing controls in Table 5 and Appendix B fix the feedback content and budget, which is the right experimental setup. 3- Real deployment data and a user study with professionals are rare in this area.

Critical weaknesses

1- Numbers contradict each other across tables: (1) Table 5 reports 0.7681 (structured) and 0.7583 (post-output). Table 10, which claims the same setup, reports 0.6912 and 0.6861. (2) ResearcherBench in Table 4 (0.7281) equals the post-output refinement score in Appendix B.1. (3) The Claude expert row in Table 6 gives 0.5942 on ScholarQA-CS, while DC in Table 4 gives 0.7652. (4) It is unclear which configuration produced which headline number. 2- Table 3 claims n=50, but every mean (e.g., 2.231, 4.385, 4.538, 4.35, 4.73) is a multiple of 1/26. That points to 26 ratings per system, not 50. Please clarify. 3- In Table 6, every expert condition scores below "None" on ScholarQA-CS (0.594–0.658 vs. 0.667), so expert interaction hurts there. The abstract cites only the ResearcherBench and DeepResearch gains, and the text never discusses this. 4- Benchmark comparisons are unfair due to unequal expert feedback, inconsistent judges and GAIA splits, while autonomous DC trails or ties commercial baselines. 5- The timing claim is overstated. The gap is only about 0.005. 6- The user study is biased toward DV due to the interaction metrics favor DC by consutrction since the baselines can't be interrupted. 7- Deployment analysis is weak. A "4.7× quality shift" on a 1–5 scale isn't explained. The judge is an LLM (Claude Sonnet 4.6), the comparison uses 25 within-session pairs, and nothing is controlled.

Questions for authors

1- Which backbone, expert, and checkpoint produced each number in Tables 4, 5, 6, and 10? Why do Tables 5 and 10 disagree for the same setting? 2- How many participants and evaluators participated in Table 3? What was the agreement? 3- Why do all expert conditions hurt on ScholarQA-CS?

Reviewer_aBZnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission addresses the challenge of making human guidance effective in long-horizon, open-ended deep research, moving beyond the opaque input–wait–output workflow of existing autonomous systems. The core idea is to decompose the research process into an interruptible and transparent multi-agent workflow, involving Planning, Clarification, Browsing, and Writing, allowing users to intervene and contribute domain knowledge during execution. The framework also introduces a learned 2B feedback-conditioned evidence router to improve the efficiency and quality of evidence delivery to the writer. The evaluation covers five benchmarks (ResearcherBench, ScholarQA-CS, DeepResearch Bench, GAIA, and BrowseComp-ZH), alongside a controlled within-subjects human study and an analysis of large-scale deployment logs.

Critical strengths

S1. Strong evaluation involving real users.

The submission includes a well-designed, controlled within-subjects human study, complemented by an analysis of large-scale deployment logs. This combination is particularly valuable for evaluating human-in-the-loop deep-research systems, where practical usability and the effectiveness of human interventions may not be adequately captured by automated benchmarks alone.

S2. Effective and efficient evidence routing.

The introduction of a learned 2B feedback-conditioned evidence router is a useful technical contribution. The reported reduction of approximately 28% in writer input tokens, alongside improved output quality, suggests that the router can improve evidence utilisation while reducing context-processing costs.

Critical weaknesses

W1. The generalisability of the evidence router across different model families remains unclear.

The 2B evidence router appears to be developed and evaluated primarily within the Qwen model family. It is unclear whether its effectiveness transfers to other backbone models, particularly proprietary models such as Claude or Gemini, which may differ substantially in their ability to interpret, prioritise, and synthesise retrieved evidence. This raises concerns about whether the reported improvements reflect a generally applicable evidence-routing mechanism or are partly specific to the chosen model configuration.

W2. The observational results risk being interpreted as causal evidence.

The quality improvements following rubric introduction in Section 5.5 are appropriately described as observational. However, some of the presentation, particularly the use of "Causal Effect" in Figure 4, suggests a stronger causal conclusion than the experimental design can support. Without a controlled intervention or an appropriate causal identification strategy, the observed improvements could also reflect confounding factors, such as changes in user behaviour, task composition, or system usage over time.

Questions for authors

Q1 Cross-model generalisation: How well does the 2B evidence router generalise to different writer backbones, particularly non-Qwen models such as Claude or Gemini? Could the authors provide cross-model experiments to establish whether the routing improvements are backbone-independent?

Q2 Causal interpretation of the rubric intervention: Could the authors clarify the interpretation of the results in Section 5.5 and the "Causal Effect" label in Figure 4? What evidence supports a causal relationship between rubric introduction and the observed quality improvements, rather than an observational association?

Reviewer_ErCWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces Deep Cognition, a multi-agent deep research system that is designed to add human intervention to autonomous research agents. The authors that if the agent misunderstands the research objective, users may have limited opportunities to correct its direction during execution. Deep Cognition addresses this limitation by enabling fine-grained human intervention throughout the research process, including planning, web browsing, evidence selection, and report generation.The central hypothesis is that structured mid-process feedback can improve research quality while reducing computational overhead compared with mechanisms that has less structured or post-structured feedback.

The authors evaluate Deep Cognition on five benchmarks: three research-synthesis benchmarks (ScholarQA-CS, ResearcherBench, and DeepResearch Bench), as well as GAIA and BrowseComp-ZH. They also conduct a human study involving 50 domain professionals and analyze 2,461 deployed sessions. The experimental results suggest that structured mid-process feedback can improve research coverage and computational efficiency relative to alternative feedback strategies, while a learned evidence router can improve evidence selection and context efficiency.

Critical strengths
  1. Well-motivated problem with practical relevance. The paper identifies a real limitation of autonomous deep research systems: users have limited opportunities to correct research direction during execution. For example, users can review the plan, redirect or pause individual browsing subagents. This extends plan-first designs.
  2. Provides a useful comparison of feedback strategies. One of the paper’s strongest contributions is its comparison of structured mid-process feedback, unstructured feedback, post-output feedback, and autonomous execution. The results suggest that structured feedback can improve research coverage while requiring fewer computational resources compared to less structured feedback mechanisms. This provides useful evidence about how the timing and structure of human feedback affect research performance.
Critical weaknesses
  1. Inconsistency in human-study reporting raise concerns about the reliability of the evaluation. The paper reports a human study involving 50 domain professionals. But several reported statistics appear inconsistent with this n=50 size. For example, numbers in Table 3 reports such as 4.385 and 4.538, which are not consistent with averages of 50 individual integer-valued ratings.

  2. Robustness to incorrect or biased feedback is not evaluated. The system gives users strong influence over plans, evidence and synthesis, and knowledge injection. But my understanding is that all controlled experiments use a competent, good-faith simulated expert. Feedback may also be incomplete, factually incorrect, or biased toward a preferred conclusion. The proposed evidence router conditions evidence selection on feedback, while the context-retention mechanism prioritizes previously cited evidence. These mechanisms could potentially reinforce confirmation bias by favoring evidence consistent with earlier feedback or intermediate reports while deprioritizing contradictory information.

Questions for authors

How does Deep Cognition handle incorrect, conflicting, or confirmation-biased human feedback?

How is end-to-end latency measured in the human-interaction experiments? Does the reported execution time include the time spent waiting for users to review intermediate results and provide feedback?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.