- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework for adapting frozen LLMs by actively acquiring external knowledge to optimize their context. An optimizer uses task feedback and search tools to update a persistent knowledge store, while beam search explores alternative updates and retains candidates based on validation performance. Experiments cover low-resource translation, healthcare, coding, and knowledge-intensive reasoning, alongside an evaluation of cross-model context transfer.
Critical strengths
- The motivation is sound: error feedback may identify failures without providing the knowledge needed to correct them, making external knowledge acquisition a reasonable strategy for context optimization.
- Contexts optimized with Gemini-2.5-Flash also improve Gemini-3-Flash, providing evidence of cross-model reusability.
- The evaluation spans several distinct task families, including low-resource translation, healthcare, coding, and knowledge-intensive reasoning.
Critical weaknesses
- The baselines primarily comprise simplified variants of the proposed framework; comparisons with established retrieval- or search-augmented approaches are missing.
- All evaluated models belong to the Gemini family, limiting the generalizability of the findings.
- The core methodological novelty is not clearly established, given prior work on feedback-driven context updates, structured knowledge stores, and beam search for prompt optimization.
- The presentation would be strengthened by a clearer and more coherent organization.
Questions for authors
- Could the authors provide direct empirical comparisons with representative established and recent methods for context optimization and search-based knowledge acquisition?
- Why does information seeking hurt Seq-IS but help BeamSearch-IS? Is branching essential, or would validation-based rejection of harmful updates suffice? Additional experiments could help clarify this distinction.
- Given the capabilities of current strong LLMs, what practical settings does this approach target, particularly when their remaining failures may not be readily resolved through additional knowledge retrieved from the web?
Since the authors have not responded to my concerns, I am maintaining my original assessment.