acceptodds
Under review as a conference paper at ICLR 2027

From Reflection to Discovery: Active Knowledge Acquisition for Training Context

Abstract

Most existing large language models (LLMs) are expensive and time-consuming to adapt after deployment, especially when a task requires newly produced information or niche domain knowledge. Recent work has shown that LLMs can iteratively update their own context to adapt to downstream tasks without updating their weights. However, relying solely on their intrinsic knowledge limits their ability to acquire missing task-relevant information: feedback may identify errors without providing the knowledge needed to correct them. In this paper, we equip these LLMs with Wikipedia search and browser tools to actively acquire missing information during context updates. Simply adding these tools to standard sequential context training can degrade performance. We therefore adopt a beam-search-style procedure that explores multiple candidate contexts and retains states selected by validation feedback, including the previous best context. We evaluate this combination on low-resource translation (FLORES+), healthcare (HealthBench), and reasoning tasks (LiveCodeBench and Humanity's Last Exam), observing gains over the evaluated non-information-seeking baselines. Further analyses examine data efficiency and hyperparameter sensitivity on low-resource translation, and show transfer of the optimized context from Gemini-2.5-Flash to Gemini-3-Flash.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Recursive Improvement, Agent, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vGeDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for adapting frozen LLMs by actively acquiring external knowledge to optimize their context. An optimizer uses task feedback and search tools to update a persistent knowledge store, while beam search explores alternative updates and retains candidates based on validation performance. Experiments cover low-resource translation, healthcare, coding, and knowledge-intensive reasoning, alongside an evaluation of cross-model context transfer.

Critical strengths
  • The motivation is sound: error feedback may identify failures without providing the knowledge needed to correct them, making external knowledge acquisition a reasonable strategy for context optimization.
  • Contexts optimized with Gemini-2.5-Flash also improve Gemini-3-Flash, providing evidence of cross-model reusability.
  • The evaluation spans several distinct task families, including low-resource translation, healthcare, coding, and knowledge-intensive reasoning.
Critical weaknesses
  • The baselines primarily comprise simplified variants of the proposed framework; comparisons with established retrieval- or search-augmented approaches are missing.
  • All evaluated models belong to the Gemini family, limiting the generalizability of the findings.
  • The core methodological novelty is not clearly established, given prior work on feedback-driven context updates, structured knowledge stores, and beam search for prompt optimization.
  • The presentation would be strengthened by a clearer and more coherent organization.
Questions for authors
  • Could the authors provide direct empirical comparisons with representative established and recent methods for context optimization and search-based knowledge acquisition?
  • Why does information seeking hurt Seq-IS but help BeamSearch-IS? Is branching essential, or would validation-based rejection of harmful updates suffice? Additional experiments could help clarify this distinction.
  • Given the capabilities of current strong LLMs, what practical settings does this approach target, particularly when their remaining failures may not be readily resolved through additional knowledge retrieved from the web?
Reviewer_FMsYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Existing methods that sequentially refine the input context without updating model parameters rely on the model’s internal knowledge to optimize the context, which limits their effective deployment across various domains, including high-stakes settings.

To address this limitation, the authors propose a beam-search-style procedure that incorporates agentic techniques.

They evaluate the proposed method across diverse domains, including low-resource translation and healthcare, to demonstrate its effectiveness.

Critical strengths
  1. The proposed framework is clearly motivated by the limitations of existing methods identified through a sound preliminary study, and its effectiveness is demonstrated across diverse domains.
  2. The proposed method achieves strong performance compared with existing baselines across a range of domains.
Critical weaknesses
  1. The experimental comparisons include only a subset of the baselines discussed in the related work section. The authors should either explain why state-of-the-art baselines that incorporate external knowledge, including RAG-based methods, as well as a broader range of agent-based methods, were excluded or include them in the experiment. Such comparisons would provide stronger evidence of the proposed method’s effectiveness and help justify the high inference cost associated with agentic reasoning.
  2. The authors claim that the proposed method is transferable based on the results in Table 3. However, these experiments are limited to models within the same Gemini family that exhibit similar distributions. While the results provide some evidence of transferability, they are insufficient to support a broader claim. Additional experiments across different model families would be helpful to establish whether the proposed method generalizes in this respect.
Questions for authors
  1. What evidence supports the characterization of existing methods as “passive” in the conclusion?
1 comment

Since the authors have not responded to my concerns, I am maintaining my original assessment.

Reviewer_jEBoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a beam-search-style context search procedure that explores multiple context candidates and selects the best one to retain by validation feedback. The proposed method is well-motivated by the observation of context pollution and local optima, and the effectiveness is corroborated by experiments on low-resource translation, healthcare, and reasoning tasks.

Critical strengths
  1. The paper motivates the design of the proposed BeamSearch-IS with two concrete observations on context pollution and local optima;
  2. The experiments are well-documented and seem to be reproducible;
  3. The effectiveness of the proposed procedure is supported with experiments on three task domains (translation, healthcare, and reasoning);
Critical weaknesses
  1. The effectiveness of the proposed method and the original motivation are only weakly connected: while the two concrete observations on context pollution and local optima are appreciated, neither effects are quantified nor correlated with downstream performance/efficiency; the effectiveness of the proposed method, although motivated by the two observations, is only supported via downstream performance improvement, not via any quantifiable improvement on context pollution or local optima, so the connection between the original motivation and the observed effectiveness are relatively weak;
  2. The method, experiments, and the following analyses compound error analysis and context editing: as motivated in the introduction, one critical challenge for context edit is that "the training feedback identifies that the executor is wrong but does not itself contain the missing knowledge needed to repair the context" (line 51-52). Therefore, given the execution trace and feedback, the process of performing a context edit involves first identifying what is missing from feedback then actively retrieving relevant external sources to edit the context. But the method branches on the whole process, failing to differentiate these two capabilities, especially given error analysis is previously observed to be a more major bottleneck than retrieval and context editing and leaving the hypothesis that the improvement may only come from multiple sampling of error analysis untested (which has little to do with active knowledge acquisition);
Questions for authors
  1. Does the proposed method quantitatively improve the "context pollution" and "local optima" phenomenon and how strong is the improvement correlated with downstream performance improvement?
  2. Can the experiment/analysis ablate the separate effect of error analysis and context editing, for example, by branching only on the edits (conditioned on the same error analysis generated in one-shot manner)? If the ablation shows similar improvement, this can further strengthen the claims and effectiveness about context learning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.