- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper examines how effectively LLM agents can post-train other LLMs, distinguishing between two capabilities: execution, or implementing a specified strategy, running experiments, and producing artifacts; and strategy exploration, or exploring alternative approaches to post-training. An analysis of 1,338 post-training trajectories from PostTrainBench suggests that current agents are capable executors but struggle to explore alternatives once they have committed to a strategy. Additional analyses suggest that providing more context, increasing the reasoning budget, and introducing human intervention do not reliably produce meaningful strategic diversity.
Critical strengths
-
The paper addresses an important question about LLM agents’ ability to explore. Success on complex, long-horizon tasks depends on exploring the space of possible approaches, beyond generating or executing a single plausible solution. Distinguishing execution from strategy exploration provides a useful lens for assessing agent capabilities.
-
The analysis goes beyond final benchmark scores. Examining agents’ execution processes and strategic choices helps explain why they succeed or fail at post-training. This process-level perspective provides information that aggregate performance metrics alone cannot capture.
Critical weaknesses
The main concern is the novelty of the findings and their specificity to post-training. Although the question is important, the paper does not clearly establish what studying it in this setting reveals beyond existing observations that LLMs produce limited diversity and become anchored to earlier assumptions or responses in multi-turn and long-horizon tasks. Relevant prior work includes:
-
LLMs Get Lost in Multi-Turn Conversation (https://arxiv.org/abs/2505.06120), which analyzes how early assumptions and mistakes persist across conversational turns, causing models to become increasingly unreliable.
-
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs (https://arxiv.org/abs/2603.03308), which studies how conversational history biases subsequent generations.
-
Escaping the Echo Trap: On Credit Assignment Failure in Multi-turn LLM Self-Reflection (https://aclanthology.org/2026.acl-long.1636/), how multi-turn self-reflection can reinforce repeated strategies and suppress exploration, and proposes a modified optimization method to address the issue.
Without a clearer distinction from these findings, the contribution appears primarily to be a case study of known limitations in the post-training setting. This limits my assessment of its conceptual novelty, despite the value of the empirical analysis.
Questions for authors
Follow-up on the weakness, how do simple exploration baselines, such as independent resampling or restarting without prior trajectory context, affect strategic diversity and final performance? Do agent harnesses that explicitly encourage alternative strategies alleviate the observed exploration failure? These comparisons would help establish whether the limitation is intrinsic to the models or substantially shaped by the interaction setup.
Dear Reviewer WAGi, thanks again for your detailed comments. We have addressed your concerns as fully as possible by clarifying the annotation and pair-construction protocol, the intervention controls, and the interpretation of token usage, and by revising the corresponding parts of the manuscript. As the rebuttal deadline is approaching, we would be grateful if you could review our response and let us know whether any concern remains. We would very much appreciate the opportunity to discuss your feedback further.