- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper is trying to provide a new method for auto-generating terminal tasks, and then collect how the agents finish the tasks, and based on the trajectories to train the model. And the issue to fix is clear: the current environment synthesis may already give the key hints to the agents, so that the agents may skip exploring and verifying, which may lead to useless training data. That's why the authors propose TerminalHorizon, and there are 2 steps: 1. rebuild the environment, where agents can try to finish the task first and check if the key work has already been completed or not. 2. Extend the tasks, based on one goal, to add new requirements so that the later work can rely on previous results and give multi-stage solutions. Using this methodology, the authors generated 1500+ task envs, filtered 2800+ trajectories, and used them to fine-tune the Qwen3.5 model. For the experiment part, they tested the trained model based on the teminal operation, agent tasks and general reasoning tasks and demonstrate the validity of their methods.
Critical strengths
- This paper tries to solve a real issue and also provide a proper solution. For complex tasks, the agent may not need to complete them; the environment may already provide the results.
- The authors not only evaluate the model on terminal tasks, but also extend it to coding-agent tasks, agentic tasks in real-world production settings, and general reasoning, and show good performance improvement, which means their method works for more than one evaluation system.
Critical weaknesses
Lack of demonstration of how the pipeline components actually help improve performance. Although the entire pipeline works, I don't know how each part of the pipeline plays its part. For example, if we ablate the env reconstruction, will that impact the final results? Or if we ablate the extension, will the performance get better or worse? Their Individual contributions to downstream training are not clear.
Questions for authors
I like the general idea of this paper, and I don't have any other questions.