- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a method for terminal-agent post-training, which replays recorded trajectories to generate executable workspaces via per-turn container snapshots for counterfactual rollouts. The method generates a environments from different public trajectories, uses Qwen3.7-Max to produce solution rollouts, which are then used to fine-tune Qwen3.5-27B. The paper further proposed two modifications of GRPO for policy optimization: pivotal-anchored policy optimization (PAPO) and contrastive branch advantage optimization (CBAO). The model is evaluated on Terminal-Bench 2.1 and EvoCode-Bench v2, where it improves upon the base model.
I have flagged the paper because I think it violates the guidelines since it does not contain the required AI use statement while showing clear signs of AI usage. As indicated by the low confidence, I was assigned this submission outside my expertise, so I may not be able to properly assess the paper, but I'd recommend to reject the paper in its current form.
Critical strengths
- Improves upon the base model
Critical weaknesses
- My biggest concern is that the submission does not contain the required AI use statement, which violates the guidelines. Especially, since the gray text and captions in Figures 5 and 6 suggest that the plots were generated with generative AI and the shown numbers are not the real numbers.
- The method requires a better model to generate a dataset, which is then used to fine-tune a weaker model, which may not be available. In general, the pipeline seems complex (5 stages) and there is no information about the additional computational effort. This raises the question why not just use a better model directly, e.g. Qwen3.7-Max? Does the relatively small improvement justify the additional effort? Biggest performance improvement comes from SFT while improvements from CBAO and PAPO are much smaller.
- Some parts are hard to follow due to the (dense) writing style, for example the whole Appendix A is hard to follow containing sentences with language that is hard to comprehend like "The framework therefore benefits from stronger source agents while retaining an explicit information ceiling imposed by the traces".
Minor:
- All papers are Arxiv preprints although some were accepted at conferences
- There is already a paper called "RePO: Replay-Enhanced Policy Optimization" (https://arxiv.org/abs/2506.09340), so it may be better to use a different name/acronym to avoid confusion
- ToC for Appendix: A after J, which probably only needs another compile
- Also no reproducibility and ethics statement
Questions for authors
- Can you please elaborate why the proposed method should be used instead of simply using a better model? The overall approach seems quite complex and I am unsure whether the relatively small improvements justify the additional effort.
- How large is the additional computational effort of the method compared to the baselines?
- Could you please clarify the actual contributions and position the paper more clearly towards related work?