acceptodds
Withdrawn from ICLR 2027

Evidence-Inference Reconstruction: When The Evidence Is Recalled But The Reasoning Goes Wrong

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern multi-hop LLM agents are equipped with built-in mechanisms to detect errors in intermediate reasoning steps. Such errors trigger corrective actions from these agents, which mostly follow the paradigm of retrying the steps or the reasoning trajectories. Not only are these retries expensive, we present in this paper that they are also potentially unnecessary. To this end, we introduce Evidence-Inference Reconstruction (EIR), which uses structured state to guide one retrieval trajectory, accumulating source evidence in the process. We show that as long as the relevant evidence has been collected, EIR is capable of generating the correct answer in a single final model call even if erroneous evidence has been mixed in due to incorrect intermediate reasoning steps. In one evaluation, using Haiku 4.5 and GPT-4.1 Mini, we evaluate EIR on matched 1,000-question subsets of HotpotQA, 2WikiMultiHopQA, and MuSiQue, showing that EIR improves Answer F1, the overlap between the model’s and the correct answer, over the baseline by 8.3–32.8 points, Agentic SSR by 10.6–29.1 points, and Reflexion by 1.1–15.9 points. Additionally, we show that EIR averages 4.85 total model calls per question, compared with 35.29 for Agentic SSR and 12.41 for Reflexion. Together, these results corroborate EIR’s central premise: separating evidence retrieval from the final answer model call can improve answer accuracy while utilizing substantially less computation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model agents, agentic retrieval, multi-hop question answering, test-time computation, evidence-grounded reasoning, retrieval-augmented generation, self-correction

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hSsZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-hop QA agents that read the right paragraphs but still answer wrong, and argues that retry-based correction (an agentic port of SSR, and Reflexion) is costly and often unnecessary.

The proposed EIR runs one retrieval trajectory where each step updates a requirement list and an evidence-citing conclusion list and picks the next paragraph, while the verbatim sentences go into a separate store.

A fresh final call then answers from only the question and the stored sentences, without the lists or the provisional answer. The authors call this the evidence-inference boundary.

Experiments use 1,000-question subsets of HotpotQA, 2Wiki and MuSiQue in the distractor setting with Claude Haiku 4.5 and GPT-4.1 Mini, plus a corpus-search extension.

EIR beats Direct, Agentic SSR and oracle-stopped Reflexion on answer F1 at about 4.85 calls per question, and an error analysis separates incomplete-evidence errors from inference errors.

The evaluation is careful, but the core design, answering with a separate reader over everything collected, was already proposed and ablated by IRCoT, and the paper's own controls suggest that's where the gain comes from.

Critical strengths
  1. Careful, paired and transparent evaluation. All methods run on the same questions with paired bootstrap CIs, exact McNemar tests and Holm correction, costs are reported next to accuracy, and negative results (the non-significant Haiku MuSiQue case, the corpus-setting loss) are reported openly. This matters because the measurements themselves can be trusted, even where I disagree with the interpretation.

  2. A useful diagnostic that separates retrieval failures from inference failures. Supplying the missing gold paragraphs recovers about 27% of incomplete-evidence errors, while re-answering on the same evidence recovers about 1%. It shows that retrieval completeness, not retrying, is the main lever, and others can reuse the decomposition.

Critical weaknesses
  1. The core idea isn't new, and the closest prior work isn't cited. IRCoT (Trivedi et al., ACL 2023) already answers with a separate reader over all collected paragraphs on the same three benchmarks, and reports that taking the answer from the retrieval-time reasoning works worse. Checklist-style gap tracking is in FAIR-RAG (cited in one line), and PRISM, ReSP and Search-o1 also separate evidence collection from answering. Related work is one paragraph, and the only competitors are the authors' own port of a math self-refine method and 2023 Reflexion. This matters because the 'new agentic paradigm' framing doesn't hold up, and beating weak retry baselines doesn't show EIR beats existing retrieve-then-read agents.

  2. The claimed mechanism is never isolated, and the baselines look handicapped. Direct's action prompt seems to hold only the latest paragraph, it reads only about 2 to 3 paragraphs on average, and it gets a smaller output budget and stricter parsing. The paper's own controls point away from the boundary: a single Full Pool call over all provided paragraphs nearly ties EIR on average at about a quarter of the cost, and the list-feedback control keeps most of EIR's gain over Direct. No variant gives the final call the lists or the provisional answer, which is the ablation the thesis needs. If the gain is just 'let the answerer see all the text', the result is expected. The abstract also overclaims, e.g. it counts the non-significant +1.1 over Reflexion as an improvement.

Questions for authors
  1. In the main distractor evaluation, what exactly is in the action prompts of Direct, Agentic SSR and Reflexion: only the latest paragraph, or all previously read paragraphs? Do they use the same output limits and the same lenient action extraction as EIR?
Reviewer_S9FTOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a failure mode in multi-hop retrieval-based question answering: even when a model’s reasoning trajectory contains the correct evidence needed to answer the question, erroneous intermediate evidence can still lead it to generate an incorrect answer. The paper proposes a training-free method, Evidence-Inference Reconstruction (EIR), which maintains a requirement list and a conclusion list during retrieval and generates the final answer in a separate call using the accumulated source text after retrieval ends, thereby improving answer accuracy. The method also achieves lower costs than some approaches that require explicit correction of reasoning trajectories.

Critical strengths
  1. The paper examines the causes of failure in multi-hop retrieval-based question answering in considerable detail. It categorizes and analyzes failure causes in terms of the correctness of retrieved evidence and reasoning conclusions, and investigates whether additional evidence can help recover from some failures. This analysis has research value.
  2. The method has an intuitive design. It explicitly maintains text-based requirement and conclusion lists while preserving the original source evidence for final answering, and achieves clear improvements in answer accuracy. The experiments report metrics beyond accuracy, including calls, token usage, and costs, providing useful practical analysis.
Critical weaknesses
  1. The method has limited novelty. The idea of independently generating the final answer from accumulated evidence after retrieval was already proposed in IRCoT[1]. The present paper has not explained its substantive differences from this design.
  2. The paper aims to reduce the influence of erroneous intermediate evidence arising during reasoning on the final answer, but it does not sufficiently distinguish among these intermediate errors. EIR cannot check for conflicts between conclusions in the conclusion list. For example, when mutually contradictory conclusions appear in the list, the method cannot detect and resolve these conflicts at the semantic level. In addition, according to Table 14, among the 2,758 cases that fail Exact Match (EM), 1,510 already include the required supporting paragraphs in the collected evidence. The mechanism underlying these failures remains unexplained.

[1] Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.

Questions for authors
  1. IRCoT already employs independent QA-based answers derived from retrieval, and it has conducted an ablation study to assess their necessity. Compared to this design, what is the substantive new contribution of EIR?
Reviewer_izcyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EIR separates evidence collection from final answering in multi-hop QA. The agent uses requirement and conclusion lists to guide retrieval, while the environment preserves the original source sentences. A final call receives the question and collected sentences without the earlier reasoning. The method requires no additional training.

Critical strengths

Keeping source text separate from its interpretation is a useful, straightforward design. Under the reported supplied-paragraph protocols, EIR has significant Answer F1 gains over Direct in all six settings and over oracle-stopped Reflexion in five. The paired comparisons, corpus-search extension, and diagnostic experiments add substance. In particular, adding missing annotated evidence repairs 335 of 1,248 selected failures, versus 17 from repeating the answer call on unchanged evidence. These are meaningful system results, even if they do not fully identify the mechanism.

Critical weaknesses

The central explanation is not yet isolated. EIR changes retrieval state, available evidence, stopping, and the final answering call together. The list-feedback ablation tests retrieval guidance, while Appendix H.4 compares retrieval-stage submissions with a fresh final call on accepted-submission subsets. Neither holds evidence and the final answering opportunity fixed while changing only access to earlier reasoning. The paper acknowledges that these are comparisons of complete methods; the stronger claim about discarding faulty reasoning still needs a direct control.

Interface and budget differences also affect comparability. Direct gets six total actions, whereas EIR can take six reads, submit, and then answer separately. In corpus search, EIR accepts actions embedded in prose while Direct and Reflexion require pure JSON. Haiku Reflexion rejects 46.58% of actions, and EIR still answers after forced empty submissions. The valid-output analysis remains positive, so formatting cannot explain all gains. Still, the present comparison mixes reasoning, interface compatibility, and answering opportunities.

Questions for authors

On frozen trajectories, could you compare equally budgeted final calls using the same evidence, with versus without earlier reasoning? Giving Direct and Reflexion the same final-reader opportunity would help separate retrieval quality from reconstruction, including trajectories that ended forcibly.

Could you repeat a focused comparison with a shared parser, citation format, read/action budget, and final-answer opportunity? Reporting the remaining accuracy gain and actual cost would directly address the protocol concern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.