- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
- This paper introduces EviWeave-Agent, a framework for risk-controlled evidence acquisition in tool-using agents, motivated by the concern that multiple tools may provide seemingly corroborating evidence originating from shared upstream sources.
- The framework integrates provenance-aware evidence fusion, active tool querying, and hierarchical answer release, with policy-level calibration formulated as a joint risk–coverage–cost optimization problem.
- The authors evaluate the approach on CorruptEvidenceBench, a synthetically constructed benchmark spanning three task domains and six corruption conditions.
- Results report improvements in correct-task coverage, factual risk, and tool-query cost relative to several controlled baselines. Ablations further examine the contribution of dependency modeling, reliability estimation and calibration.
Critical strengths
- Relevant consideration of correlated evidence.
- The paper raises a legitimate concern that multiple tools may provide seemingly independent corroboration while relying on common upstream information. But there is a lack of examples here which makes it difficult to follow why is this even an issue.
- The distinction between independent evidence and duplicated support is conceptually relevant to the assurance of agentic systems.
- Joint treatment of risk, coverage, and acquisition cost.
- The attempt to coordinate evidence fusion, acquisition, and answer specificity within a common risk-control formulation is potentially useful. - The distinction between factual errors, unsupported claims, and over-specificity is also sensible, although the broader value of this integration remains insufficiently demonstrated.
Critical weaknesses
- Weak problem motivation and questionable research significance.
- My primary concern is that the paper appears solution-driven without adequately establishing the significance of the underlying problem. Yes, correlated evidence is a legitimate consideration, but the authors do not convincingly demonstrate its prevalence or consequential impact in real-world tool-using agents.
- There is an elaborate framework to introduce substantial complexity through provenance modeling, active acquisition, hierarchical release, and risk calibration. However, this becomes a little hard to appreciate without clarity on the fundamental challenge that we face today with agents.
- What substantive improvement does EviWeave-Agent provide over contemporary, simpler approaches such as RA-RAG (Hwang et al., EMNLP 2025)
- Can the authors demonstrate these benefits using independently implemented baselines and realistic retrieval tasks, rather than predominantly synthetic evaluations and internal framework variants?
- The overall contribution therefore feels disproportionate to the problem being addressed, and the paper does not establish a sufficiently compelling research impetus.
- Unconvincing experimental validation and baselines.
- The evaluation relies almost entirely on a synthetic benchmark with predefined provenance, fixed answer hierarchies, and constrained corruption conditions.
- More concerningly, the comparisons largely involve simplified or ablated variants of the proposed framework rather than independently implemented contemporary agentic systems (Provenance, Fixed fusion, Selective etc.).
- The ReAct + self-consistency baseline is also described as a symbolic proxy rather than a verified LLM-agent implementation.
These limitations significantly weaken the credibility and generalizability of the empirical claims.
Questions for authors
-
Problem significance: Can the authors provide concrete empirical evidence that the problem formulation is indeed a significant and consequential failure mode in contemporary tool-use agents? Why is the proposed joint framework necessary over simpler provenance approaches such as tracking, de-duplication, or some form of weighted fusion?
-
Baseline validity: Why are the main comparisons largely limited to symbolic or internal framework variants? In particular, how can the ReAct + self-consistency results substantiate comparative claims without an independently implemented LLM-based agent?
-
Real-world validation: How representative is CorruptEvidenceBench of actual agentic workflows? Can the authors substantiate the reported improvements on realistic retrieval tasks with independently obtained experimental results?