- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work presents a human-in-loop agentic framework, Policy Iteration with Human Feedback (PIHF), for improving in-context learning of LLMs. The framework incorporates five agents, including a Causal Investigator (CI), a Curator, a Critic, a Verifier, and an Executor. In experiments, the proposed method was based on GPT LLMs and mainly compared with GEPA.
Critical strengths
- The proposed method shows a clear efficient superiority over its baseline method GEPA.
- The proposed method shows its improvement over cold-start prompting on GPT-5.6.
Critical weaknesses
- The selection of LLMs in the experiments are limited and based on the private GPT family. More LLMs, in particular the open weight ones, with different sizes could be useful for readers in model selection.
- Only three tasks, BIG-Bench Extra Hard (BBEH), HoVer, and LiveBench-Math, are involved in the main experiments.
Questions for authors
- What are the consideration of the backbone model selection?
We thank the reviewer for the generous time spent reviewing our paper and for the constructive feedback on open-weight models, model sizes, and backbone selection.
We added two new sets of replicated experiments on HoVer (300 protected claims, 3 independent runs each; General Response Items 3 and 5) alongside an existing open-weight experiment from our submission:
DeepSeek-V4.1-Flash-1017) scores 226, 189, and 220 / 300 (mean 211.7, SD 19.9) vs. 182 / 300 for P0. Policies learned by PIHF-MCP transfer zero-shot to an open-weight executor without re-optimization.138 / 300, 46.0%) in every run (187, 199, and 183 / 300; mean 189.7, SD 8.3), while scaling with investigator capability (200.0 to 206.3 with Sol-medium).The investigator and executor serve different roles with different cost profiles:
Our evaluation spans four distinct settings: multi-hop retrieval (HoVer), multi-step heterogeneous reasoning across 23 task families (BBEH Mini), competition mathematics (LiveBench-Math). Because our central claim concerns how we work with capable models during optimization (handing the investigative loop to the model via tools and state control), we focused the rebuttal window on deep, replicated controls that isolate this mechanism rigorously: 12 full HoVer optimization runs across 4 configurations, 15 protected 300-claim tests (including 3 open-weight transfer runs), and 12 BBEH format-control evaluations.