- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper presents wastehub, a search info reuse framework that allows deep research agent to save and reuse useful search results. instead of discarding web searches and webpage visits, it stores three types of information: unvisited search results, omitted webpage content, and source-appraisal information. The stored information is selectively reused during later research steps to improve report quality and token efficiency. Experiments on deep research benchmarks (DRB and DRB II) show improvements in report quality and reduced resource consumption compared with baselines.
Critical strengths
-
Clear motivation for practical benefits. This is a real issue for deep research agents, where most search information is discarded during search and summarization.
-
Reasonable design with general applicability. The framework is a modular and general design and evaluated across multiple agent frameworks and models.
Critical weaknesses
-
The baseline selection is questionable. Evaluation mainly compares with PRM-F and PAE, which introduce additional feedback or exploration overhead. It is not clear why these two are most related direct comparison, instead of existing persistent-memory or retrieval-based mechanisms. Moreover, WasteHub does not show token savings over basic ReAct agent. Comparisons with simple memory-based methods would strengthen the work.
-
The performance improvement seems marginal relative to the additional complexity. The report-quality improvements over PAE are only 0.0089 on DRB and 0.0161 on DRB-II. Although statistically significant, their practical importance remains unclear. It is questionable whether these small improvements justify the extra system complexity. Also, does the framework gain applies to frontier LLMs like GPT5.6 or GPT6?
-
Unclear attribution of latency improvements. The evaluation shows end-to-end latency reductions, but the system relies on external search and webpage retrieval services. The authors use the same backend configurations, but authors do not clearly explain how variability of external service latency is controlled. This makes it difficult to attribute latency improvements specifically to WasteHub.
Questions for authors
-
How does WasteHub differ from memory or retrieval-based mechanisms? Would a simpler memory cache achieve comparable results? Why are PRM-F and PAE the primary baselines? Given that WasteHub does not save tokens compared with ReAct, what is its practical advantage over the base agent?
-
Can the authors provide evidence that the relatively small score improvements lead to clear differences in final report quality?
-
How is external search and webpage retrieval latency variability controlled or measured? Can the authors report separate measurements for model inference, external tool calls, and WasteHub/framework-specific latency?
3 comments
Round1-Part2
Response to W2: Gains, overhead, and frontier models
Gains relative to added complexity. Score increments should be interpreted in terms of both the metrics and improvements in report content. DRB uses scores normalized relative to a reference report. With a fixed reference score, the same normalized gain requires a larger raw-score improvement at higher scores. On DRB-II, a gain of 0.0161 corresponds to an increase of 1.61 percentage points in the rubric pass rate averaged across tasks. At the content level, WasteHub improves Citation Accuracy from 48.78% to 63.95% and average Effective Citations from 6.67 to 8.19 over PAE (Table 4), reflecting improved source support in the reports.
In terms of resources, Table 1 and Table 2 show that, relative to Base, WasteHub achieves larger quality gains with a smaller increase in cumulative input tokens than PRM-F and PAE. At similar report quality, PAE-III uses about 2.01 / 1.85 times as many cumulative input tokens as WasteHub (Appendix H). These measurements include WasteHub's additional generation calls, showing that information reuse achieves report quality close to that obtained through additional exploration, with lower resource usage.
In terms of system design, WasteHub keeps the original planner and tool interfaces unchanged. The type ablations in Table 3 and stage ablations in Appendix G support the contributions of the information types and reuse stages. The added unified storage and retrieval control (Table D2) further shows that, with comparable total token usage, full WasteHub achieves higher mean scores on both metrics, supporting the additional quality benefits of its overall design over simple storage and retrieval.
Table D2. Unified storage and retrieval control
| Setting | DRB Overall | DRB-II TotalScore |
|---|---|---|
| Unified storage and retrieval | 0.4740 | 0.3152 |
| Full WasteHub | 0.4861 | 0.3380 |
Applicability to frontier backbones. Appendix D reports WasteHub's gains with Qwen3.5-397B-A17B, Kimi-K2.5, and DeepSeek-V4-Flash. To address your question, we have added experiments using GPT-6 Sol as the main agent, keeping the ReAct framework, tools, and auxiliary models unchanged.
Table D3. Results with GPT-6 Sol as the backbone
| Method | DRB Overall | DRB-II TotalScore |
|---|---|---|
| Base | 0.5302 | 0.3554 |
| PAE | 0.5351 | 0.3592 |
| WasteHub | 0.5460 | 0.3731 |
With GPT-6 Sol, WasteHub achieves higher mean scores on both metrics than Base and PAE (Table D3).
Response to Q2: Improvements in report quality
Table 4 shows that, compared with PAE, WasteHub improves Citation Accuracy by 15.17 percentage points and provides an average of 1.52 additional source-supported citations per report, indicating improvements in both citation reliability and the number of effective citations.
The automatic score gains in the main experiments have statistical support and show a consistent direction across evaluator models. The main experiments use 5 independent runs, with task-level paired tests yielding p = 0.028 and p = 0.004 on the two benchmarks, respectively (Appendix E). When three evaluator models re-evaluate the same reports, WasteHub's gains over PAE remain within 0.0073 to 0.0104 and 0.0136 to 0.0194, respectively (Appendix L, Table 17).
We also conducted blind pairwise human evaluation by 3 independent raters, covering all 100 DRB tasks and 132 DRB-II tasks. We use each method's complete reports from its first run, hide method identities, and randomize presentation order. Raters compare factual coverage of the core question, source support, and analytical adequacy. For each preference judgment, raters must identify concrete content differences and verify the relevant original sources. They may choose a tie when they cannot clearly distinguish the reports. Results are determined by the majority judgment for each task, with a tie recorded when there is no majority.
Table D4. Blind human evaluation (task counts)
| Benchmark | Prefer WasteHub | Prefer PAE | Tie |
|---|---|---|---|
| DRB | 56 | 24 | 20 |
| DRB-II | 78 | 29 | 25 |
| Total | 134 | 53 | 45 |
Human raters prefer WasteHub on more tasks on both benchmarks (Table D4). These judgments, grounded in concrete content differences and source verification, together with the improvements in citation quality, support discernible quality improvements in the final reports.
Round1-Part3
Response to W3 & Q3: External-service variability and latency attribution
Web retrieval cannot fully eliminate variability from network conditions and real-time service load. To reduce its influence on method comparisons, we use the same search and page-fetching backends, concurrency settings, caching, and retry policies, keeping concurrency within the capacity of the experimental deployment. Methods are executed in randomized order within the same batch for each task, with 5 runs per method, to reduce the effects of service-load changes and execution order.
We also record the start and end times of each call. Table D5 breaks down latency into model calls from the original framework, external search and page fetching, and framework and WasteHub auxiliary processing. On both benchmarks, model calls account for approximately 80% of WasteHub's end-to-end latency.
Table D5. End-to-end latency breakdown (seconds per task)
| Latency category | DRB Base | DRB WasteHub | DRB-II Base | DRB-II WasteHub |
|---|---|---|---|---|
| Original framework model calls | 1850.0 | 1710.4 | 2305.9 | 2298.7 |
| External search and page fetching | 370.0 | 365.0 | 575.0 | 580.0 |
| Framework and WasteHub auxiliary processing | 25.7 | 124.2 | 30.4 | 158.7 |
| End-to-end total | 2245.7 | 2199.6 | 2911.3 | 3037.4 |
On DRB, total model-call latency remains lower than Base even after including auxiliary calls, and its reduction is close to the reduction in end-to-end latency. Thus, the average latency improvement in these runs is primarily in model calls, while the combined time spent on external tools and non-model framework processing differs little.
Manuscript revisions
Considering all reviewers' feedback and the completeness and coherence of the manuscript, we have made the following revisions, all marked in blue:
- Section 3.3: Clarified the rationale for baseline selection.
- Section 5: Added discussion of memory-related baselines, such as A-MEM, and clarified their connections to and differences from our work.
- Appendix G: Added the fixed-trajectory replay experiment with Synthesis residuals.
- Appendix N: Added direct comparisons with task-conditioned summarization, ordinary page retrieval, and A-MEM.
Round1-Part1
We sincerely appreciate the time and effort you devoted to reviewing our paper, and your detailed, constructive suggestions.
Response to W1 & Q1: Baselines, distinctions, and practical benefits
Choice of PRM-F and PAE. WasteHub aims to improve report quality across deep research frameworks through a plug-and-play mechanism. We therefore compare enhancement strategies that can be naturally integrated into existing research workflows across frameworks. PRM-F improves research decisions through feedback, while PAE supplements evidence through additional exploration, representing two general enhancement strategies.
Distinction from memory and retrieval mechanisms. Memory mechanisms focus on storing, organizing, and updating information. For example, A-MEM organizes memory through note linking and updating. Retrieval mechanisms focus on recalling existing material relevant to current needs. WasteHub focuses on information omitted during search and visit: task-relevant information that is not retained in the native research state. It contributes a mechanism to preserve this information and reuse it at appropriate stages. Based on current needs, existing content, and information status, it selects residuals to guide verification, supplement evidence, or qualify claims, and plugs into different research frameworks.
Practical advantage over the base ReAct agent. WasteHub obtains larger quality gains with a smaller increase in input tokens. In the original main experiments, relative to Base, WasteHub increases cumulative input tokens by 17.0% / 17.1% on DRB / DRB-II and improves aggregate scores by 0.0148 / 0.0284. PAE increases input tokens by 67.9% / 43.2% and improves aggregate scores by 0.0059 / 0.0123. Thus, compared with additional exploration, reusing existing information improves the base agent's report quality with less additional input overhead.
Comparison with simpler memory. We have added A-MEM [1] and the Base Memory configuration from its original paper as direct controls. Base Memory retains note construction and retrieval but removes note linking and updates to existing memories, representing a simpler storage-and-retrieval approach. Both configurations use the public memory module adapted for use within a single ReAct task. Information is written after search and visit calls and retrieved before subsequent research and writing.
Table D1. Memory baseline comparison
WasteHub achieves higher mean scores on both metrics than the simpler Base Memory configuration (Table D1). With comparable total token usage, it also achieves higher mean scores on both metrics than A-MEM.
[1] Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-Mem: Agentic Memory for LLM Agents. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, 2025.