acceptodds
Under review as a conference paper at ICLR 2027

The Context Gathering Decision Process: A POMDP Framework for Agentic Search

Abstract

A coding assistant asked about an unfamiliar repository cannot fit it into its prompt, so it searches: it issues a query, reads the result, and issues another. Harnesses run this phase, context gathering, by appending each passage to the prompt. One LLM call then chooses the next query, remembers what earlier rounds found, and decides when to stop. None of the three can be changed separately. We formalize the phase as a Context Gathering Decision Process, a finite-horizon POMDP over a fixed corpus, which separates two decisions the fused call makes together. The first is whether the agent's notes are rewritten into a bounded form after each observation, or accumulate. The second is who rewrites them: the policy, prompted to summarize itself, or the orchestrator, the code that runs between LLM calls. We call the latter an Orchestrator-Managed State (OMS). Rewriting raises accuracy over four published harnesses and three question-answering domains. Ownership does not change accuracy. It changes whether the agent declines a question its corpus cannot answer, where correct abstention rises points under OMS. A stopping rule reading the state in code ends stagnant searches early, with no LLM call. The gain from rewriting does not hold on two frontier policies. Accuracy and abstention therefore come from different parts of a harness, a reason to expose the update and the stopping rule as orchestrator interfaces. More broadly, pieces of an agent loop usually written as prompt engineering can be separated, measured and replaced without retraining.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Partially Observable Markov Decision Processes, Sequential Decision Making, Agentic Search, Retrieval-Augmented Generation, Context Management, Long-Horizon Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_e2VVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A good framing, and good question to ask. Relevance is present, but there are inconsistencies int he results.

Critical strengths

The question is useful: which part of an agent loop (query choice, memory, stopping) drives which outcome? The ACT/OBSERVE/UPDATE/STOP split is a clean way to ask it.

The state-fidelity analysis (B.5) is strong.

Critical weaknesses

There is no guarantee that the abstentin result is from calibration and not from abstention from OMS? There are multiple cases where OMS refuses more on quesitons not having answers. Additionally, a policy given no corpus at all scores 94.3 on LoCoMo's unanswerable stratum, because abstaining is the right answer there (B.11, p.24). The stratum rewards caution on its own. Abstention rate on unanswerable questions alone therefore cannot show calibration. The paper needs a threshold-free measure, such as selective accuracy or a utility that scores abstentions and errors on all questions, and it should compare arms at matched abstention rates.

Since the "ownership" effect is really an effect of the extractor prompt. The authors write: "We never ran the reverse condition, an OMS-style extractor prompt placed inside that call, so we do not claim that our particular instruction is what matters" (p.8, lines 417 to 418). So the paper never tests the claim in its title. A single control (the OMS extractor prompt placed inside the policy's own call) would separate "who writes the state" from "what the instruction says".

The Table 2 headline doesn't reproduce, the same rerun window is thin, and token cost swings widely.

OMS fails on both coding and multi-hop, the two settings closest to the paper's own motivation.

"The state term is a weak threshold knob and a decisive structural one" (B.8) is circular. Without OMS the growth condition is true by construction, so the gate becomes a repetition detector. That shows the gate needs some measure of state change, not that OMS is the right one.

Questions for authors
  1. Can you run the missing control, the OMS extractor prompt inside the policy's own call? Without it, how do you separate ownership from prompt content?
  2. Can you report a selective-prediction measure (accuracy vs coverage, or a utility over all questions) so that higher abstention is not counted as better calibration by default?
  3. Why does OMS not improve Iter-RetGen, the strongest native harness, in any domain? What is the gain over the best native system per domain?
  4. The abstract says rewriting no longer beats intact frontier harnesses, yet Table 19 shows +10.2 and +7.8 for gpt-5.6-terra on LoCoMo and MuSiQue. Which is right?
Reviewer_yFfdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper formulates context gathering over a static corpus as a finite-horizon POMDP and separates the agent loop into action selection, observation, state update, and stopping. It introduces an orchestrator-managed bounded state (OMS) and a programmatic stagnation gate, evaluating their effects on answer quality, abstention, and token costs.

Critical strengths
  1. The evaluation spans four harnesses, three domains, and multiple models, with question-clustered bootstrap intervals, multiple-comparison correction, frozen-threshold testing, and reruns within the same time window to address model drift.
  2. The paper addresses a practical problem and separates state updates and stopping rules into independent interfaces, making memory management and budget control easier to analyze and modify without retraining.
  3. The paper clearly reports negative results and limitations, including weaker performance with structured states, limited recovery on deeper synthetic chains, a 6.7-point deficit to model-managed state with repository tools, and inconsistent gains over native harnesses with frontier models.
Critical weaknesses
  1. Appendix B reads like a collection of rebuttal experiment logs, retaining commands such as bash rebuttal/...(???).
  2. The main text is overly compressed, while key experimental details and qualifications are scattered across a lengthy appendix, making it difficult to assess how the evidence supports and limits the central claims.
  3. The claimed importance of the state-change signal is weakly supported, removing it leaves 194 of 225 settings unchanged.
Questions for authors
  1. Does the ownership advantage persist when instructions, visible inputs, output budgets, and write mechanics are matched?
  2. In the transfer experiment, how does the frozen gate compare with a version without the state-change term under the same OMS configuration?
  3. Which types of information does OMS preserve or lose, and how do these differences explain when it improves or reduces performance?
Reviewer_LLQPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper investigates harness design for LLMs (with an emphasis around Q&A). Specifically, a core part of the harness is context gathering i.e., constructing a natural language context observable by the policy to condition it's next action - the decision making.
  • While, both phases (context gathering and decision making) is fused into a single policy/harness, the paper advocates for separate interfaces.
  • Technically, the paper proposes a 'Orchestrator-Managed State' (OMS), which is a bounded-scratchpad (the 'state') that is updated as the policy model progresses through the task. Implementation-wise, this is performed by an LLM to update the scratchpad.
  • The paper evaluates their approach with four choices of standard harnesses (e.g., ReAct, MemGPT) and on three datasets (LoCoMo, MuSiQue, SWE-QA-Pro) and find promising results: their OMS approach provides reasonable gains across most configurations.
Critical strengths
  1. Insight to use a working memory: Using a bounded scratchpad is a well-justified way to maintain some memory while completing the tasks. The policy's context window does not rot over task progress.
  2. Results that transfer across harness: The authors evaluate their algorithm along with multiple harnesses and show reasonable gains across most settings.
Critical weaknesses

Major Concerns

1. Connection between initial theoretical justification of CGDP and algorithm/implementation

  • The paper's contributions are two-fold:
    • (a) Defining a "CGDP" process (i.e., context gathering)
    • (b) Algorithmic contribution (Algorithm 1; how/when update happens)
  • The justification for CGDP is elaborate (which is somewhat reasonable) and leads to claim of proposing a "textual bounded state" (a bounded scratch pad / working memory). I believe this topic has received considerable attention in literature (more details in Concern#2) and appears to diminish the proposed contribution.
  • More importantly, this framework leads in practise appears to amount to something rather simple: an LLM at each step to summarize and compress observation-dependent context (Line 9, Algorithm 1). Based on the prompt (Section E.2), this appears to be a more elaborate version of a naive extractor baseline (L384) "summarize what you have learned so far".
  • All in all, my concern here that the justification of CGDP (which I believe has overap with prior works) amounts to an LLM call (the state summarizer) at each turn. Are there specific insights from the CGDP analysis (a) that is carried over to the prompt or algorithm design (b)?
  • Minor related comment: L453 mentions the paper's contributions on two independent choices - how often the update step is run, and who writes. The latter is clear. However, can the authors clarify the frequency to the update rule, since Algorithm 1 appears to indicate it being run every step. In fact, L092 also mentions "that it runs after every observation* - the claims are contradictory.

Minor Concerns

2. Bounded Textual State: Significance given prior works

  • The authors advocate for a POMDP definition of the context-gathering process. Importantly, this amounts to maintaing a bounded textial state by the policy while performing the task.
  • I believe there are many works that operate on the exact idea. For instance:
    • Training-based methods
      • MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent (2025)
      • MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents (2025)
      • (ReSum-GRPO) ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization (2025)
    • Training-free methods (harness design)
      • MemGPT: Towards LLMs as Operating Systems (2023) - already cited/baselined by the paper
      • Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models (2023)
      • (ReSum - summary tool) ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization (2025)
  • Overall, the idea of a bounded working memory (the textual state) proposed by the authors has received considerable prior attention in literature. The paper would benefit from differentiating the working memory proposed by the paper and prior works.

3. Setting: Tasks that admit bounded state

  • While results appear promising, the tasks on which the authors evaluate on appear particularly elegant for some assumptions the approach makes.
  • Specifically, the bounded state involves limiting to 10 items or 500 output tokens. This appears perfectly reasonable for keeping track of certain facts during multi-hop retrieval for Q&A tasks.
  • However, this bounded-ness remains unclear for more general tasks (e.g., Needle-in-haystack variants, long-horizon multi-step reasoning).
Questions for authors

Clarification requests:

  1. (Concern 1) Update rule - this appears to amount to a LLM-based prompt to write/summarize the state. Are there specific insights you carry over from the CGDP analysis in crafting the prompt?
  2. (Concern 2) How is the proposed notion of bounded textual memory different from prior works on scratchpad or working memory?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.