acceptodds
Withdrawn from ICLR 2027

PANINI-DR: Controlled Deep Research Through Generative Semantic Workspaces

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Deep research requires connecting multiple steps of search, reasoning, and computation, with intermediate findings guiding what to investigate next. As investigations grow, agents must maintain research state across model calls and check findings before errors propagate into subsequent work. We introduce Panini-DR, a multi-agent harness that separates coordination of the overall investigation from the research and verification of individual subquestions. A shared workspace, built on the Generative Semantic Workspace representation, records subquestions, their dependencies, findings, and supporting evidence. Researchers investigate individual subquestions, while a separate verification agent checks proposed answers against those subquestions and their evidence. Verification feedback guides further investigation, and recorded findings inform subsequent assignments. We evaluate Panini-DR on BrowseComp-Plus, FRAMES, and MoNaCo against seven baseline implementations using a shared language-model backbone and frozen corpora. Panini-DR achieves the strongest average performance across the three benchmarks, with strong gains on harder questions. On BrowseComp-Plus, Panini-DR achieves **73.3%** accuracy, outperforming the next-best baseline by **16.0** percentage points, at higher inference cost. Its advantage is strongest on harder investigations, correctly answering **26 of 50** hard questions—nearly twice as many as the next-best baseline. Our ablation studies further show that structured planning and persistent state improve performance, as does verification combined with evidence checks. To assess whether correct answers are supported by the research performed, we also analyze the evidence retained by each system. Panini-DR achieves the highest answer recoverability from saved evidence on all three benchmarks, while complementary claim-support audits show stronger evidence-backed answers. Accounting for this evidence changes how systems compare and further strengthens Panini-DR's advantage over existing approaches.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Deep research, LLM agents, harness-level chain of thought, agent planning, structured workspaces, evidence verification, grounded question answering.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wqGTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PANINI-DR is a multi-agent research harness that organizes subquestions, dependencies, findings, and evidence in a shared workspace. Research agents investigate individual subquestions, while a verifier checks candidate findings and provides feedback for further investigation; admitted findings then inform later research and computation. The system is evaluated on BrowseComp-Plus, FRAMES, and MoNaCo using a shared gpt-oss-120b backbone, and it achieves the strongest average answer quality among the compared systems, with particularly large gains on harder questions.

Critical strengths

The improvement on BrowseComp-Plus is substantial and consistent across repeated runs. Specifically, PANINI-DR averages 72.7% accuracy, compared with 58.0% for both SLEUTH and Open Deep Search. The paper also includes useful controls, such as structured state versus prose memory, a common answer-recovery policy, and repeated SLEUTH sampling at comparable estimated computation. The last control is particularly convincing: even with repeated sampling, SLEUTH only reaches 64.7%, remaining below PANINI-DR’s 73.3% reference result.

The evidence-quality analysis is also valuable. The common-reader experiment and the claim-support audit complement the final-answer accuracy. I also appreciate the authors’ discussion of related approaches, inference costs, and practical failure cases.

Critical weaknesses
  • The incremental contribution of the proposed execution protocol remains insufficiently established, and the experimental evidence does not adequately validate the stated contributions. While the paper discusses related concepts (e.g., dependency-aware planning in LLMCompiler, persistent state in IterResearch), my concern is whether this specific integration—and its claimed components such as the shared workspace and typed subquestion review—provides a sufficiently distinct advantage over simpler baselines. The current ablations evaluate the entire package rather than its individual mechanisms. For instance, the planning/state ablation only compares the full package against prose memory, without isolating its advantage over a simpler dependency-aware implementation. More critically, as shown in Table 11, a reduced system with answer recovery already matches the full system's 73.3% BCP accuracy. While the full system yields better evidence support (Obs.) and uses fewer prompt tokens, it remains unclear which specific architectural contributions (e.g., the typed resolver, dedicated Reader, or the complex verification loop) are actually driving these gains.

  • Reproducibility heavily depends on execution details that are difficult to inspect without the implementation. The manuscript promises a public repository only after the review process, but I could not locate an accessible anonymized implementation link in the PDF or supplementary materials. The appendix usefully documents important exceptions: research outside the ready frontier is permitted, unsuccessful verification can lead to a WARNED admission, and final-answer recovery can produce an answer even when the target remains unresolved. Notably, this recovery mechanism supplies 40 of the 110 correct BCP answers. While these design choices are not inherently problematic and the paper includes recovery controls, the inability to inspect the exact execution policy during review makes it difficult to fully verify and interpret the headline results.

  • The verification ablation does not fully separate the quality of feedback from the additional research effort it triggers. Enabling verification improves BCP accuracy by 16 points, but the median prompt usage also rises from 453k to 1,042k tokens, and the introduction of evidence-attachment checks adds another confounding variable. This establishes the effectiveness of the complete policy, but not its superiority over a baseline that simply spends the same extra computation on more search or refinement. The matched-computation SLEUTH comparison evaluates the overall system rather than this specific mechanism. Furthermore, generality is uncertain: the primary experiments use a single backbone and frozen corpora, and the system shows smaller gains on FRAMES and a lower raw MoNaCo F1 than HyMem.

Questions for authors
  • While the paper lists related work, the core components (persistent state, verification, planning) are shared with several baselines (e.g., SLEUTH, Omni-Decision). Could the authors provide a concrete example (e.g., a specific query) demonstrating how PANINI-DR's specific design (such as the GSW-based typed subquestions and admission policy) leads to a different and better execution trajectory than these baselines? Which experiment best isolates this specific architectural advantage?

  • Given the complexity of the execution details, I strongly encourage the authors to provide the anonymized implementation and a subset of reasoning trajectories in the supplementary materials. Specifically, the execution configurations and rules governing WARNED admissions, reopening dependent findings, and answer recovery, along with concrete examples of these paths, should be made available for inspection during the review period to verify the reported results.

  • Beyond the existing system-level computation control, could the authors provide a focused comparison between verifier-guided research and additional search or refinement at approximately matched expenditure?

Reviewer_tz84OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

PANINI-DR is a multi-agent deep research harness that decomposes the task in two parts - orchestration and research/verification of subproblems. It does so by utilizing Generative Sematic Workspaces (Rajesh et al 2026) and repeated verification. The main motivation for the work is to separate maintaining the research state from the research task. The approach is evaluated on BrowseComp-Plu, FRAMES, and MoNaCo - which are widely used challenging DR benchmarks.

Critical strengths
  1. The approach of using multiple agents is logical and the chosen GSW representation encodes necessary dependency graphs that make deep research manageable. The subagents also use a reasonable strategy of separating reading and reasoning which allows them to arrive at an accurate answer, followed by verification and answering the query.

  2. They authors do a fairly comprehensive and transparent evaluation using multiple metrics and challenging benchmarks. Notably, the improvement on BCP (hard) is double the next best baseline.

  3. The paper studies an important problem of understanding how to build effective harnesses for the DR task.

Critical weaknesses
  1. My core concern is lack of new insights beyond prior work. Although the method is sound (as stated in the strenghts), none of the papers contributions are novel. The orchestration agent, the subagents, verification, etc have all been well studied in prior literature. The paper itself attributes its contributions to prior work like GSW (representation), IterResearch and SLEUTH (persistent state), DeepVerifier and Marco DR (verification), etc. The ICLR community would not benefit from this paper because the readers would not know which design principles or insights they can build upon.

  2. I also have several concerns about the empirical results:

2a. The method convincingly only improves over baselines on BCP. Table 21 (CI tests), show that the improvements on FRAMES could just be chance (CI interval includes 0) and two baselines convincingly beat Panini DR on MoNaCo. The authors state this fact in the paper.

2b. Some settings were chosen using the same questions they report scores on (App. D.2.3, Table 13; App. D.3.3, Table 18). This raises concerns about the fairness of evaluation.

2c. I'm not fully convinced by baseline adaptations. One of the baseline "produce 76 empty outputs and 7 abstentions" - it seems like the baseline was not setup correctly.

2c. PANINI DR uses 5.68 PFLOPs compared to SLEUTH which uses 0.78. The compute accuracy tradeoff is not justified well.

2d. Although the authors acknowledge some of this in the limitations, they use only one model, a very small subset (150 examples of the main benchmark) and a custom retriever. This makes their approach hard to compare with published numbers and leaderboards. This is a major limitation and the claims of the entire paper are justified using this weak setting.

  1. The paper could benefit from substantial restructuring. All of the ablations are moved to the appendix which makes it really hard to follow critical details of the paper. The authors propose a simple idea that can be described a lot more concisely. The main paper only has two tables, a reader of the main paper would never be able to understand the results properly. For example, in Appendix S.1.2 we find that most of the gain on BCP comes from the Finalizer fallback. Overall the main text "presents" a convincing story that the appendix does not support.

  2. The related work does not talk about many recent trained DR systems like DR Tulu and Tongyi DR which demonstrate strong performance. A comparison against these methods would help position this approach better or reporting their published results would better position the paper.

Questions for authors
  1. With recovery outputs withheld, the full system scores 46.7%. That is below SLEUTH (57.3%) and below the reduced ON harness (58.0%). Can the authors clarify this behavior?

  2. When verification is removed, the number of tokens drop drastically. Does this not confound the comparision? I'm unsure if this is a fair comparison and request the authors for clarification.

Reviewer_JGp7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents PANINI-DR, a multi-agent deep research system for deep research tasks built on prior Generative Semantic Workspaces systems. It decomposes questions into dependent subquestions, uses Researcher agents to resolve them and uses a Verifier to check intermediate findings before recording them in a persistent workspace for subsequent research. The paper evaluates the proposed system against seven agentic research baselines on BrowseComp-Plus (BCP), FRAMES, and MoNaCo, using a shared gpt-oss-120b backbone and retrieval infrastructure.

Critical strengths
  1. The paper is well written and provides experiments and ablations.
  2. The ablations show that structured state and verification improve performance for the evaluated LLM.
Critical weaknesses
  1. The authors did not highlight the novelty in this work, as the described techniques seem to be commonly used today.

  2. Weak comparison in the evaluation. . The paper shows a substantial gain only for the BrowseComp-plus (BCP) benchmark, however the paper does not include the state-of-the-art results on this benchmark. Panini-DR achieves an overall score of 73.3%, and all the experiments are based on gpt-oss-120b, released in 2025. (Note gpt-oss-120b achieves 42.9%). GPT-5 using a relatively simple retrieval agent gets 70.1%, as reported in the original BrowseComp-plus paper. In addition, Sliders (Joshi et al., 2026) using GPT4.1/GPT4.1-mini and RLM (Zhang et al., 2026) with GPT-5/GPT-5-mini achieve about 90% on this same benchmark.

On FRAMES, the improvement over SLEUTH is only 1.9 percentage points. On MoNaCo, PANINI-DR achieves 0.549 F1, substantially below HyMem’s 0.612.

The MoNaCo result is particularly concerning given the paper’s motivation. The authors note that PANINI-DR produces more precise collection answers, but this does not translate into higher F1. This raises questions about whether its decomposition and verification mechanisms support exhaustive evidence gathering, rather than merely verifying individual findings. It would be useful to understand whether PANINI-DR’s lower recall arises from retrieval failures, incomplete question decomposition, or its verification procedures

  1. PANINI-DR answers are not all supported by its evidence. Table 2 shows that there is a gap between PANINI’s answers with evidence and without evidence. PANINI-DR’s observed-support scores are noticeably below its original answer scores on FRAMES (66.1% vs 79.1%) and MoNaCo (0.458 vs 0.549). A deep researcher should always be able to back up their answers with evidence. The system needs to be improved before it is trustworthy.

The paper does not provide enough information to understand how the results on other systems are computed.

Questions for authors
  1. Could the authors provide a failure-mode analysis of the paper and baselines to distinguish the benefits of decomposition, persistent state and verification over the baselines?
  2. The paper reports token consumption, but for the proposed deep research system, cost and latency metrics are important and should be reported.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.