acceptodds
Withdrawn from ICLR 2027

Checked, Not Just Cited: Source-Grounded Verification for Multi-Agent Handoffs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-agent systems let agents use one another's findings without rereading the underlying material. In handoffs where the receiving agent lacks source access, however, a convincing report may contain genuine quotations yet draw unsupported conclusions. We study a source-grounded verification signal that combines quotation matching with model-based reasoning checks at the handoff boundary, where the source remains. We test it on BrowseComp-Plus by selecting reports that increasingly persuade the receiver. Across nine sender–receiver pairings, verification loses precision more slowly than receiver adoption, retaining 78–84% under the strongest selection compared with 54–67% for adoption, although it accepts only 14–36% of reports. The reasoning checks catch unsupported conclusions even when quotations match the source. When fed back to receivers, the signal improves their decisions, especially under stronger selection. We incorporate this signal into AgentCite, a handoff protocol that informs receiver decisions and triggers source-side re-investigation when quotations are missing. On three tool-based delegation tasks across three model families, AgentCite without re-investigation matches the strongest baseline within 0.3 points with 23–55% shorter receiver messages; re-investigation lifts mean accuracy 1.8–4.4 points above it at extra tool cost. These results support source-grounded verification for agent handoffs without transferring the full source context to the receiver.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, LLM agents, agent communication

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Nd3fOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a protocol for checking the reliability of evidence provided in a multi-agent setting from an agent that reviews source material and sends a message quoting the source material to another agent who does not have access to the source material. The protocol purports to use a verifier that both confirms the presence of the quoted material in the source and engages in reasoning to confirm that the quoted material supports a claim conveyed in a message. Experiments on constructed pairings of correct and plausible incorrect (distractor) answers and three decision tasks may show modest improvements in precision and accuracy.

Critical strengths
  • This is a nice setup and framework, adding reasoning verification to quotation verification to support claims transmitted in a multi-agent system
Critical weaknesses
  • despite a reasonable framing, the paper is extremely badly explained and very hard to follow. Despite multiple reads, I was unable to evaluate the core results. It appears to be heavily AI written and not explanatory. I can't evaluate the core findings or compare those to the literature.
Questions for authors
  • How have you implemented reasoning in the verifier and what did you do to evaluate the reliability of reasoning?
Reviewer_Dom6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AgentCite, a source-grounded verification framework for multi-agent handoffs in which a sender has access to source materials while the receiver does not. The framework combines quotation matching with LLM-based reasoning verification to assess whether cited evidence supports the conclusions in a sender's report. Verification feedback is then communicated to the receiver without transferring the full source context.

The authors first evaluate the verification signal under persuasive report selection on BrowseComp-Plus, showing that verification maintains higher precision than receiver adoption, albeit at substantially lower coverage. They further integrate the signal into AgentCite, which supports conditional source-side re-investigation when quotations cannot be verified. Experiments across three model families and three tool-based delegation benchmarks show competitive accuracy with shorter receiver messages, while optional re-investigation provides additional accuracy improvements at increased computational cost.

Critical strengths

The paper addresses a relevant reliability challenge in multi-agent systems, where receiving agents must rely on information summarized by other agents without direct access to the original sources.

The paper demonstrates that genuine quotations do not necessarily support the conclusions drawn from them. Separately checking quotation validity and reasoning consistency is a meaningful design choice.

The experiments cover nine sender–receiver combinations, persuasive selection, component analyses, and three tool-based delegation benchmarks. The inclusion of multiple baselines and paired uncertainty estimates strengthens the empirical study.

Critical weaknesses

AgentCite primarily combines established components, including quotation matching, LLM-based evidence verification, structured reasoning dependencies, and conditional re-investigation. Similar ideas have been explored in citation verification, fact-checking, and tool-assisted self-correction. The paper could more clearly establish what fundamentally distinguishes AgentCite from existing verification pipelines.

While quotation matching is relatively deterministic, reasoning verification depends on model judgments that may introduce errors or inherit biases from the sender. The paper also reports substantial verifier-dependent variation in precision. Stronger analysis of verifier reliability and robustness across different verifier capabilities would strengthen the claims.

AgentCite initiates additional investigation when quotation matching fails, but unsupported reasoning with valid quotations does not trigger the same process. Moreover, improvements from re-investigation may primarily result from additional computational resources rather than the proposed verification mechanism. Stronger compute-matched controls would help isolate the contribution of the triggering strategy.

The evaluation primarily considers single-sender, single-receiver handoffs with relatively structured evidence certificates. It remains unclear how well the framework extends to multi-hop delegation, conflicting reports from multiple agents, or long-horizon tasks involving dynamically changing sources.

Questions for authors
  1. What are the main methodological differences between AgentCite and existing citation-verification or tool-assisted self-correction methods? Which components account for the observed improvements?

  2. How sensitive is AgentCite to the capability and reliability of the verifier? Would a weaker verifier significantly degrade the final receiver's decisions?

  3. Can the authors provide end-to-end cost and latency comparisons that include verification calls, source-side exploration, and conditional re-investigation?

Reviewer_rZTNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-agent handoffs where one agent investigates source material and another must act only on its report. AgentCite asks the sender for a structured certificate, checks whether its quotations occur in the source, and uses an LLM to judge whether they support the conclusion. The feedback informs the receiver and can trigger further investigation. Reasoning feedback improves over quotation-only feedback, but end-to-end AgentCite matches existing baselines without re-investigation; its accuracy gains require additional tool use and occur mainly on factual retrieval.

Critical strengths
  • The paper identifies an important failure mode: a report can contain genuine quotations while drawing a conclusion that they do not support.
  • The fixed-report experiments isolate the value of reasoning feedback. Under the strongest selection pressure, full feedback improves fresh Qwen receiver accuracy by 3.9 points over quotation-only feedback.
  • The evaluation includes nine sender-receiver pairings, three model families, relevant baselines, and paired uncertainty estimates.
Critical weaknesses
  1. My main concern is limited technical novelty. Khan et al. (2024) already studies source-asymmetric persuasive oversight with quotation verification; ALCE/AIS checks whether citations support claims; Natural Program uses explicit premise graphs; and CRITIC uses verification to guide correction. Appendix L.2 also finds that the dependency-graph verifier is tied with a simpler semantic check plus quotation matching. The novelty is therefore mainly the combination of existing ideas at a multi-agent handoff.

  2. The end-to-end gains are not isolated from additional search or from quotation matching. AgentCite without re-investigation is tied with the strongest baselines, while the improved version adds up to 16 tool steps and 91–100k exploration tokens on eligible BrowseComp tasks. Re-investigation is triggered by missing quotations, not failed reasoning checks, and there is no quotation-only AgentCite ablation. A confidence-triggered baseline also matches verification-triggered continuation for Flash in an earlier setup.

  3. The reasoning check is an LLM judgment without human support annotations. Its precision also depends strongly on the verifier: replacing Flash with Qwen reduces pooled precision from roughly 79–81% to 62–65%. Its reliability across models and domains is therefore unclear.

Questions for authors
  • How does AgentCite compare with confidence-triggered or random re-investigation using the same additional tool and token budget?
  • How much of AgentCite's cooperative-task performance remains with quotation matching but no reasoning check?
  • How well do the reasoning-check labels agree with human judgments of evidential support?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.