acceptodds
Under review as a conference paper at ICLR 2027

Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents

Abstract

Language agents that interleave reasoning and tool use degrade sharply as reasoning chains lengthen, even when each individual step is easy. We trace this to context dilution: an agent's investigative state (what it has confirmed, what it suspects, and what it still needs) lives only implicitly in a growing context window, where early discoveries are buried under later retrievals. We introduce SLEUTH, which makes this state explicit and actionable through a structured epistemic working memory: the agent maintains Confirmed Facts grounded to sources, Active Hypotheses ranked by evidence, and Open Questions that directly drive its next action. Across five multi-hop benchmarks and five established baselines, SLEUTH's advantage grows with difficulty, from +5 points on HotpotQA to +11 on 4-hop chains, surpassing Reflexion without multiple episodes. Analyzing where the remaining gap lies, we identify the evidence sufficiency problem: agents often find the answer but fail to commit, exhausting their budget on needless verification. A lightweight commitment trigger fixes this, but only when the agent already maintains structured state: the identical trigger applied to an unstructured agent yields no improvement, isolating organized epistemic state as the necessary condition for effective commitment. Finally, enforcing protocol adherence on a weaker model recovers up to +19 points on the hardest problems, showing that the way an agent organizes its reasoning, more than raw model capability, is the active ingredient for scaling multi-hop reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, multi-hop reasoning, epistemic working memory, tool use, context management

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_s3nbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SLEUTH, a prompting-based framework for maintaining within-episode epistemic working memory in language agents. After tool use, the agent records sourced confirmed facts, ranks candidate hypotheses, and maintains open questions that guide subsequent actions. A budget-dependent commitment nudge encourages synthesis after approximately 70% of the turn budget, subject to a minimum fact count. Experiments cover three QA datasets, with MuSiQue separated into three hop-depth settings, using Claude Sonnet 4.6 and GLM-5. The reported results show consistent improvements over the evaluated agent baselines, and component and stopping-policy ablations investigate the contributions of state organization and commitment.

Critical strengths
  1. Useful empirical gains from a lightweight intervention. On four-hop MuSiQue, SLEUTH improves Sonnet EM from 42.0% to 53.1% over ReAct. The per-question comparison reports 59 wins and 14 losses, consistent with the approximately 11.1-point gain. The paired bootstrap analysis and corresponding F1 improvements provide evidence that the advantage is meaningful and not solely an exact-match formatting effect. A method that achieves these improvements without training has practical value.
  2. Substantial ablation coverage of the proposed design. The paper evaluates unstructured notes, component subsets, commitment timing, alternative stopping strategies, and retrieval and turn-budget sensitivity. In particular, Facts-Only reaches 49.9% on four-hop MuSiQue, full SLEUTH reaches 53.1%, and removing the nudge reduces SLEUTH to 49.1%. ReAct receives essentially no benefit from the tested nudge. These controls provide an informative starting point for understanding the interaction between evidence organization and commitment.
Critical weaknesses
  1. The evidence supports the performance improvement more strongly than the central mechanistic claims. The 3.2-point four-hop gain over Facts-Only is important to establishing the value of the full tripartite schema, but the reported significance analysis concerns SLEUTH versus ReAct rather than this increment or the structure-by-nudge interaction. A negligible nudge effect for one unstructured baseline does not establish organized epistemic state as a general necessary condition for effective commitment. Conditional accuracy also depends on which questions a policy elects to answer, and gold-answer occurrence in a history is not sufficient evidence of a completed reasoning chain. Finally, Sonnet's within-MuSiQue gains over ReAct are approximately 11.9, 11.7, and 11.1 points at two, three, and four hops: they remain substantial but do not increase with depth. Comparing HotpotQA to MuSiQue additionally changes the dataset and context distribution. These distinctions matter because the claimed scientific contribution goes beyond a successful prompting recipe.
  2. Reporting inconsistencies and confounded cross-model interventions weaken the supporting analysis. Under the stated definitions, EM should equal the answered fraction times conditional accuracy. In Tables 2 and 8, EM of 53.1% and timeout of 11.1% imply conditional accuracy of about 59.7%, rather than 58.9%; the no-nudge row similarly implies about 65.2%, rather than 64.6%. The four-hop SLEUTH categories in Table 15 sum to 100.7%, which is too large to be explained by rounding mutually exclusive categories. These discrepancies may be reporting errors, but they affect the central stopping analysis. Separately, the GLM-5 adherence intervention combines prompt adaptation and runtime retries, and Appendix M states that rejected turns do not consume the 20-turn budget. The 19-point improvement, therefore, does not isolate adherence from prompt changes and additional model calls. This concern does not apply to the primary Sonnet results, which use neither adaptation nor runtime enforcement.
Questions for authors
  1. Please reconcile Tables 2, 8, and 15 using counts from the same evaluated trajectories: correct answers, all answered questions, timeouts, and mutually exclusive error categories. Were any rows computed from different runs or with additional exclusions?
  2. What are the paired confidence intervals for full SLEUTH versus Facts-Only and for the structure-by-nudge interaction? Please provide the exact nudge text and explain how its fact-count condition is handled for ReAct and whether further searches remain possible. Would an unstructured-note control with matched commitment and source-extraction instructions retain a nudge benefit? This would help distinguish the proposed schema from more general evidence organization.
  3. What is GLM-5's accuracy with the adapted prompt alone, before runtime enforcement? Does the improvement persist when rejected generations count toward the total model-call budget? Please provide the adapted prompt, validation rules, retry counts, and token accounting that includes rejected responses.
  4. Were the nudge threshold and the best timings reported in Table 11 selected independently of the final evaluation questions? If not, can you provide a held-out or fixed-threshold result and appropriately qualify the difficulty-scaling claim?
5 comments
AuthorsOpenReview ↗

Gentle reminder: author–reviewer discussion period ending in a few days

Dear Reviewer s3nb,

We wanted to send a brief reminder that the author–reviewer discussion period concludes in a few days.

In our response, we reconciled Tables 2, 8 and 15 from deduplicated trajectories with one timeout definition (reported EM changes by at most 0.2 points); gave the exact commitment texts; reported paired confidence intervals for SLEUTH vs. Facts-Only and the structure × commitment interaction; added a matched-instruction Notes control; separated prompt adaptation from enforcement on GLM-5; and ran a held-out check of the commitment threshold on MuSiQue training-split questions, where turn 14 keeps its advantage over ReAct (+8.5 [+4.8, +12.2]).

If these updates address your concerns, we would be glad to hear it, and we would be grateful if you would consider reassessing your rating accordingly. If there are any remaining questions or concerns, we would be happy to address them during the remaining discussion period.

Thank you again for the time and effort you put into reviewing our work.

Best regards,

Authors

I thank the authors for a detailed response and for the substantial new work in Appendix M. Several of my points are now settled, and I want to be explicit about which, before explaining why my assessment has not moved.

Settled. Q1: the reconciliation is complete and convincing — the duplicate rows from the resumed run (407–408 for 405 questions) and the inconsistent timeout definition explain the discrepancies I reported, and with a single definition EM = (1 − timeout) × conditional accuracy holds while the Table 15 categories sum to 100%. Q3: the accounting is now transparent, and I accept that the GLM-5 gain is an effect of the retry policy rather than of adherence. Q2's factual requests — the exact commitment texts, the trigger conditions, the paired intervals — are all answered.

Not settled. My concern was never that these numbers were unreported; it was what they would show once reported. The new controls show it.

First, the matched-instruction Notes control recovers most of the effect the paper attributes to organization: 64.3 / 64.1 / 62.8 / 52.1 / 75.9 against SLEUTH's 66.2 / 66.5 / 63.7 / 53.1 / 76.0. The residual is +1.4 [+0.5, +2.3] pooled, individually significant on two of five splits. Second, and more damaging to the thesis, that residual does not grow with depth: +2.4 at 2-hop, +1.0 at 4-hop, +0.1 on 2Wiki. The central claim is that typed epistemic organization is what makes long chains tractable. If the advantage of the full schema over a flat record carrying the same instructions is largest at two hops and shrinks at four, the data point the other way. Third, the interaction the paper needs — structure × commitment — is +3.7 [−1.0, +8.4], and the authors have correctly withdrawn the necessity claim, which was the paper's distinctive scientific claim.

Two further points on the new experiments. The GLM-5 result, once rejections are charged, is −0.2 on 3-hop; please report the paired interval for that difference, and state in the caption of Table 17 that this configuration is a retry-policy intervention rather than a result about model capability. And the held-out threshold check runs on MuSiQue's training split, where turn 14 reaches 60.8 EM against 53.1 on the evaluation split of the same hop count — a different difficulty regime; on that holdout the selection procedure preferred turn 6 (60.2) to turn 14 (60.8), so the threshold surface is flat. A flat landscape is consistent with the instruction helping and the specific operating point not being identifiable.

On reporting: I would ask the authors to state in the paper that the submitted Tables 2, 8 and 15 were computed on trajectories containing duplicate rows, and whether any qualitative conclusion beyond the ≤0.2 EM shift depended on the affected rows.

Assessment. What remains is a well-executed empirical study of two instructions — source extraction and commitment — plus a small, non-scaling increment from typed structure. That is worth reporting, and the revision is honest about it. But the paper has not only become smaller in what it establishes; the claim that survives no longer matches the frame it is written in, and a reader who takes the title and abstract as the contribution will be misled by them. I am therefore keeping my rating at 2, and I would not have moved it upward even if every remaining question had been closed.

I would genuinely encourage a resubmission reframed around the matched-Notes comparison. The +9.6 to +10.4 points that source extraction and commitment add to a plain notebook is the most useful number in this paper, and it deserves to be the headline rather than the control that deflates the headline.

AuthorsOpenReview ↗

Response to Reviewer s3nb (follow-up)

We sincerely thank the reviewer for the careful reading of the new experiments and for stating precisely which points are now settled. We deeply appreciate the reviewer's active engagement during the discussion phase, and we are committed to improving the paper regardless of the outcome. We have made the three additions the reviewer requested to the revised PDF (Appendix M).

GLM-5 paired interval and the Table 17 caption. With every rejected generation charged to the budget, the paired 95% bootstrap interval for SLEUTH minus ReAct on GLM-5 is [−3.8, +3.3] on 3-hop, so the two are statistically indistinguishable there; the corresponding intervals are [+1.1, +6.1] on 2-hop and [+0.2, +10.6] on 4-hop. We added these intervals to the caption of Table 17, which now also states that the runtime check is a retry-policy intervention on how generations are executed, not a result about model capability.

Deduplicated trajectories. We agree with the reviewer that the paper should state this explicitly. Appendix M now states that the submitted Tables 2, 8 and 15 were computed from trace files in which three SLEUTH configurations (commitment at turn 8, commitment at turn 14, no commitment) contained duplicate records from a resumed run (407–408 records for 405 questions), and that episodes answering on the final turn were counted as timeouts. No qualitative conclusion depended on the affected rows: reported EM changes by at most 0.2 points, and every comparison keeps its direction. With the deduplicated trajectories and one timeout definition, SLEUTH has a lower timeout rate and higher conditional accuracy than ReAct on every MuSiQue split (4-hop: 8.1% vs. 17.5% and 57.8% vs. 50.9%), and commitment at turn 14 still gives the highest EM in the sweep.

The matched-Notes comparison. We agree with the reviewer that the gain from source extraction and commitment over a plain notebook (+9.6 [+7.5, +11.7], +9.7 [+7.2, +12.2] and +10.4 [+6.9, +13.8] on 2-, 3- and 4-hop) is among the most useful results of the study, and we will give it a more prominent place in the final version, alongside the comparison of the complete configuration with the matched control (+1.4 [+0.5, +2.3] pooled). We also agree with the reviewer's reading of the held-out threshold check: it shows that the commitment instruction keeps its benefit on unseen questions, both at an independently selected threshold and at turn 14 (+8.0 [+4.2, +11.8] and +8.5 [+4.8, +12.2] over ReAct), and the revised appendix states that it does not establish turn 14 as optimal.

We thank the reviewer again for the constructive and detailed feedback throughout the discussion, which has made the paper's claims considerably more precise.

Reviewer_H7AbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SLEUTH, a prompting-based framework for epistemic working memory in language agents. The agent explicitly maintains confirmed facts, ranked hypotheses, and open questions, updating them after each tool call to guide subsequent actions. A commitment trigger encourages the agent to answer before exhausting its turn budget. Experiments on HotpotQA, MuSiQue, and 2WikiMultiHopQA, using Claude Sonnet 4.6 and GLM-5, show improvements over reasoning-agent baselines and simpler memory variants, particularly on longer reasoning chains.

Critical strengths
  • Well-motivated working-memory formulation. Explicitly separating what an agent knows, suspects, and still needs to resolve is a sensible response to information becoming buried in a growing interaction history. Evaluating this organization across different reasoning depths makes the motivation concrete.

  • Straightforward and understandable method. The facts–hypotheses–questions structure is intuitive, and its connection to subsequent tool selection is easy to follow. The prompting-based implementation makes the central idea accessible without requiring model training or architectural changes.

Critical weaknesses
  • Limited comparison with dedicated memory methods. The paper includes five reasoning baselines, including Reflexion, as well as Notes and structured-memory ablations, so the comparison is not restricted to agents without memory. However, these do not fully establish the advantage over existing within-episode memory and context-management approaches. A comparison with an appropriate method such as HiAgent or CAT, both discussed in the related work, under matched retrieval and resource budgets would better isolate the benefit of SLEUTH’s particular organization.

  • Several interacting choices and a narrow evaluation scope. Although the central structure is simple, the complete system combines sourced facts, hypothesis ranking, question-driven actions, monotonic fact retention, a commitment trigger, and model-specific prompt adaptation and runtime enforcement. The component and nudge ablations are useful, but the GLM-5 comparison bundles prompt adaptation with enforcement, leaving their separate contributions unclear. More broadly, the five reported evaluation settings come from three multi-hop QA datasets, with MuSiQue divided into three hop-count splits. All use closed-context retrieval with supplied paragraphs. This leaves generalization to other agent tasks, noisy or contradictory sources, and settings requiring revision of previously confirmed facts unresolved.

Questions for authors
  1. Can the advantage over dedicated within-episode memory methods be established under matched tools, token budgets, and stopping rules? The current ablations isolate parts of SLEUTH, but do not substitute for this external comparison.

  2. How should the monotonic fact-retention rule behave when newly retrieved evidence contradicts an earlier confirmed fact? Evidence on a non-QA agent task would strengthen generality.

1 comment
AuthorsOpenReview ↗

Gentle reminder: author–reviewer discussion period ending in a few days

Dear Reviewer H7Ab,

We wanted to send a brief reminder that the author–reviewer discussion period concludes in a few days.

In our response, we compared SLEUTH with dedicated within-episode memory methods (HiAgent, CAT and the BATS Budget Tracker) under matched tools, budgets and stopping rules; separated prompt adaptation from runtime enforcement on GLM-5 with full generation and token accounting; tested the append-only fact policy with injected contradictory evidence; added a transfer test on HoVer claim verification (+7.7 points over ReAct); and reported controls that separate the interacting design choices.

If these updates address your concerns, we would be glad to hear it, and we would be grateful if you would consider reassessing your rating accordingly. If there are any remaining questions or concerns, we would be happy to address them during the remaining discussion period.

Thank you again for the time and effort you put into reviewing our work.

Best regards,

Authors

Reviewer_2WUFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a prompting method for LLM agents that improves multi-hop reasoning at test time. The framework guides the agent to record sourced facts, rank the responses, and list questions to consider. The authors report that this simple technique demonstrates higher acc. on various benchmarks with frontier models.

Critical strengths
  • The proposed method improves accuracy over the tested prompt baselines on selected settings without additional training. On a practical side, this could be of great use in improving agents' performance at test time. The ablations and reported costs help assess its practical value.
Critical weaknesses

Apart from the gains that the authors present, I'm yet to be convinced that the paper presents a meaningful contribution to the literature.

  1. Limited Novelty: To the best of my knowledge, the cited paper Hu et al., (2025) already studies a similar approach. I'm also aware of a similar work [1] that extends this line of work. The novelty of SLEUTH is that it adds a 'notebook' that manages the context in a more structured way, but it does not compare with the more recent baselines (all baselines in Tab 1 are from before 2024). If the aim of the paper was to show that the notebook structure enables better test-time improvement than the current practice, it should have compared with relevant works.

  2. Experimental Scope: The paper studies the framework on two frontier models (to show that their results transfer across model families), but does not elaborate on the criteria for selecting the models. Considering this is a paper without training, and largely empirical, I believe a sweep across a large number of models, benchmarks, and test-time choices (e.g., token budgets) should have been conducted. More importantly, it compares with old baselines that do not reflect the current line of work, making it difficult to assess the proposed method. To add, while the paper reports the costs, it does not compare the methods under an equal token budget, and thus does not show whether there could be a simpler method on par with SLEUTH. Lastly, the authors do not report the error bars across multiple seeds, which weakens the evidence for the proposed method's robustness.

[1] https://arxiv.org/html/2511.17006v1

Questions for authors
  1. I request that the authors compare SLEUTH against more recent context-managing frameworks such as HiAgent. (Since it's cited in the paper) What technical constraints, if any, prevent adapting these methods to your experiments? Ideally, I would like SLEUTH to be compared against these recent baselines with matched budgets.

  2. Could you report mean scores and their standard deviation / standard error across repeated runs for full SLEUTH and Facts-Only? I have more questions (e.g., how many runs were used). (e.g., BATS averages its Budget Tracker results across three runs.)

  3. Could the authors please elaborate on the criteria for selecting these models? Would SLEUTH’s benefit persist across models with different baseline scores and costs? Given the limited timeframe, I do not expect the authors to be able to finalize the experiments by the end of the discussion period.

3 comments
AuthorsOpenReview ↗

Gentle reminder: author–reviewer discussion period ending in a few days

Dear Reviewer 2WUF,

We wanted to send a brief reminder that the author–reviewer discussion period concludes in a few days.

In our response, we added adaptations of HiAgent, CAT and the BATS Budget Tracker under the same tools and turn cap (SLEUTH has the highest EM on every split); compared methods at similar average token consumption (SLEUTH with 15 turns reaches 52.1 EM at 47.3K tokens, vs. 42.0 for ReAct at 50.8K); ran three full runs (SLEUTH 54.0 ± 0.8 vs. ReAct 41.6 ± 0.8 on 4-hop MuSiQue); added two open-weight models from further families; and clarified how SLEUTH's prospective, typed state differs from compression-based memory.

If these updates address your concerns, we would be glad to hear it, and we would be grateful if you would consider reassessing your rating accordingly. If there are any remaining questions or concerns, we would be happy to address them during the remaining discussion period.

Thank you again for the time and effort you put into reviewing our work.

Best regards,

Authors

Thank you for presenting us with the additional results. I believe that the paper really needed multi-seed results, as it is a very common standard for good papers. That being said, I still believe that the paper has great potential, and I sincerely think the authors have put great effort into this submission. However, I'm not sure if we can disetnagle the benefits of the proposed method's core components from others, despite the fact that the authors have provided a lot of evidence to show it. So while my evaluations regarding the ablation are mixed, I will take into consideration of what the other reviewers suggest. However, my main concern lies elsewhere, as I still remain unsure of the paper's contribution at large. I will keep my initial scores but will discuss with the rest of the reviewers during the following period.

We sincerely thank the reviewer for the careful engagement with the new results, for recognizing the potential of the paper and the effort behind it, and for the willingness to discuss it further with the other reviewers. We deeply appreciate the reviewer's active engagement during the discussion phase, and we are committed to improving the paper regardless of the outcome. We address the two remaining points below.

Disentangling SLEUTH's components. Every variant in our ablation differs from SLEUTH only in its system prompt, with the same tools, retrieval, answer-format instruction and 20-turn cap, so each comparison isolates one design element, and the key comparisons hold over three full runs with paired intervals. (i) The typed state beyond a fact record. Facts-Only keeps SLEUTH's source-grounded fact record and its commitment and extraction rules but drops ranked hypotheses and open questions; adding them back improves EM by +4.4 [+2.8, +6.0] on 3-hop and +4.5 [+2.2, +6.9] on 4-hop MuSiQue. No reduced variant recovers this gain: Facts-Only, Facts+Hypotheses and Facts+Questions all stay at least 3.2 points below SLEUTH on HotpotQA and every MuSiQue split, so hypotheses and questions act together. (ii) Commitment from the evidence state. On 4-hop, the commitment instruction raises SLEUTH's EM by +4.0 [+0.5, +7.4], while on ReAct, which keeps no evidence record, its effect is statistically indistinguishable from zero (+0.2 [−3.0, +3.5]). (iii) Source extraction and commitment versus an unstructured record. Given to an unstructured notes agent, these two instructions add +9.6 to +10.4 points, and SLEUTH's typed state adds a further significant +1.4 [+0.5, +2.3] pooled over five benchmarks. (iv) Not an artifact of enforcement or budget. The primary Sonnet results use no prompt adaptation or runtime enforcement, and at similar average token consumption SLEUTH with 15 turns reaches 52.1 EM against 42.0 for ReAct with 20 turns. Together, these controls attribute SLEUTH's +12.3-point gain over ReAct on 4-hop MuSiQue (three runs, [+9.3, +15.5]) to identifiable design elements, each measured with its own interval.

The contribution at large. We see the contribution in three parts. First, a diagnosis of why agents fail on long chains. We separate two failure modes of implicit state management, which the outcome decomposition measures directly. One is information loss, where evidence found early is not used. The other is the evidence sufficiency problem: without a commitment instruction, 24.0% of SLEUTH's 4-hop episodes time out, and the gold answer string already appears in the trajectory of 92% of them. Second, a prompting-only method that addresses both. SLEUTH keeps a prospective, typed epistemic state (source-grounded facts, ranked hypotheses and open questions that determine the next tool call) and commits from that state. This differs from the recent methods the reviewer pointed us to. HiAgent and CAT compress past interactions retrospectively, and the BATS Budget Tracker reports the remaining budget. Under the same tools, budget and stopping rule, SLEUTH has the highest EM on every benchmark (4-hop MuSiQue: 53.1 vs. 49.4 for the BATS Budget Tracker, 42.2 for CAT and 36.3 for HiAgent). The Budget Tracker targets the same timeout failure as our commitment instruction, and its gap to SLEUTH shows that the commitment needs an explicit evidence record to draw on. Third, evidence that the gains are robust. SLEUTH improves over the strongest established baseline on all five benchmarks (+4.7 on HotpotQA, +10.4 to +11.9 on 2- to 4-hop chains). It does so across three full runs, at similar average token consumption, on four model families with ReAct baselines from 22.7 to 42.0 EM, and on held-out questions and a claim-verification transfer task (HoVer, +7.7 over ReAct). Because SLEUTH needs no training, extra episodes or search, it can be added to an existing agent directly, and we will release the code, prompts and per-question results for all experiments.

We thank the reviewer again for the constructive feedback, which has substantially strengthened the paper, and we would be glad to answer any further questions during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.