- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the problem in recursive self-improvement that a winner strategy in historical replay may not show improvement in the futures task. To solve this, the authors propose a framework that registering all strategies first, freezing the winner after replay, and then test the winner and original strategy on new tasks. Based on this framework, the authors define replay gain, live gain and their difference. The authors further define regret and the ranking of the replay winner on the new tasks. The authors validate their claim in the experiment with multiple LLMs.
This paper studies a very important problem in recursive self-improvement, how to use the information in replay to update the strategy.
What I concern most is the novelty and significance of this paper. Since the observation that replay winner may not perform good in the future tasks and auditing on additional tasks are relatively standard, it is hard for me to believe the contribution of this paper is enough for ICLR.
- Could the authors clarify what contribution the proposed framework provides compared to regular additional auditing or holdout evaluation?
Critical strengths
This paper studies a very important problem in recursive self-improvement, how to use the information in replay to update the strategy.
Critical weaknesses
What I concern most is the novelty and significance of this paper. Since the observation that replay winner may not perform good in the future tasks and auditing on additional tasks are relatively standard, it is hard for me to believe the contribution of this paper is enough for ICLR.
Questions for authors
- Could the authors clarify what contribution the proposed framework provides compared to regular additional auditing or holdout evaluation?