acceptodds
Under review as a conference paper at ICLR 2027

Recursive Synthesis for Long-Horizon Terminal Tasks

10 views

Abstract

High-quality long-horizon training data for terminal agents are costly to produce because the instruction, environment, reference solution, and verifier must remain aligned. Existing synthesis pipelines typically generate each task independently. We study how accepted executable tasks can be reused across rounds while checking for execution and specification errors. Recursive Synthetic Terminal Tasks (RST) applies this process to complete executable task bundles. In each round, RST extends the executable workflow of an accepted seed task, realigns the environment, verifier, and public instruction, validates the resulting candidate in a fresh sandbox, and reuses accepted candidates as seeds for the next round. Across 15 rounds, RST produces 37,484 accepted tasks while maintaining stable validation rates and broad domain and rewrite-operator coverage. Meanwhile, the median reference solution grows from 67 to 374 lines, the static command count grows from 40 to 245, and DeepSeek-V4-Pro pass@4 falls from 90% in R1 (round 1) to 2.5% in R15 (round 15). Supervised fine-tuning (SFT) on successful rollouts improves Qwen3.5-27B and Qwen3.5-122B-A10B across Terminal-Bench 2, Terminal-Bench Hard, and Long-Horizon Terminal Bench, with gains of up to 10 percentage points. Each updated checkpoint generates rollouts for the next SFT stage. Verifier-based proximal policy optimization (PPO) further raises Qwen3.5-27B to 49.81%, 32.00%, and 22.07% on the three benchmarks, corresponding to relative gains of 20.9%, 41.2%, and 21.9% over the base model. These results show that accepted executable tasks can serve both as training data and as seeds for generating progressively harder tasks. Code and the complete Terminal-Bench Hard evaluation subset are available at https://anonymous.4open.science/r/recursive-synthesis-terminal-tasks-005F. Synthesized tasks, trajectories, and checkpoints will be released upon publication.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Terminal Agents, Recursive Task Synthesis, Long-Horizon Tasks, Synthetic Training Data, Agent Post-Training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RwGJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a pipeline for extending tasks using an LLM by having them modify existing tasks. Static checks gate whether a modified task is accepted. SFT on successful rollouts on these new tasks improves model performance.

Critical strengths
  • The pipeline generates more complex tasks from known-good tasks using LLMs to extend the tasks. The generated tasks require longer reference solutions, and SFT on successful rollouts on these tasks improves model performance.
Critical weaknesses
  • It's not clearly established whether this specific method provides any gain over baselines, since there are no comparisons to baselines with equal amounts of compute. What is missing is a clean comparison to other tasks using the same token budget, successful rollouts from the same model, and running in the same harness. The comparison in Table 6 is not that, and shows the issue if this is not matched (e.g., the performance of Qwen3.5 drops if you train on Nemotron trajectories).
  • None of the generated tasks are released, making it difficult to assess whether they are useful or are underspecified or obscure. There are three indications for why the generated tasks might be of poor quality. 1. The modifications (Tab. 11) are adding chores on top of tasks without adding intellectual challenge. 2. The task generation model is instructed to not put information in the instructions that can be discovered from the workspace. Whether something is discoverable is therefore wholly up to the judgement of a single model, which is the same model that designed the task. 3. The task quality control from App. E.1 is not implemented as described. Instead, only rigid, hard-coded checks (regex, length checks, count of the number of numbered steps) are used for issues instead of more thorough methods. Especially the last point is worrying, since the quality checks are very crude. Running the “quality checks” on TB-hard would reject 100% of tasks. Just 80% would be blocked by the regex detecting the numbered lists (^\s*(?:\d+[\.\)]|\-\s+\*\*) from synthesis_pipeline.py).
Questions for authors
  • What if you train on other successful rollouts, e.g., on successful Qwen rollouts on unmodified TMax-15K tasks? And how does this perform compared to other baselines?
  • How did you ensure the tasks are of good quality beyond the metrics from the paper?
  • Can you release the generated tasks?
8 comments

W1: We thank Reviewer RwGJ for the careful reading and constructive feedback. We agree that controlled comparisons are important for evaluating RST. We clarify our existing experiments and provide two additional comparisons using the same rollout model and harness.

1. Existing Controlled Ablation. Section 4.1 (Table 1) presents a 2×2 ablation using the same generation model, operator library, validation pipeline, and generation budget. With 3,000 training examples per condition and identical SFT settings, RST achieves 45.32% on TB2, compared with 43.07% for the strongest alternative. This supports the benefits of solution-first rewriting and recursive task reuse.

2. Purpose of Table 6. Table 6 evaluates existing public SFT datasets under identical training conditions, rather than comparing their original data-generation pipelines. We use the same base model, SFT settings, evaluation setup, and exactly 36,363,275 assistant-loss tokens for each dataset. RST achieves 46.82% on TB2, compared with 42.70% for the strongest public baseline. However, these datasets were originally collected using different rollout models and harnesses. We therefore conducted the following additional comparisons.

3. Additional Controlled Experiments. Following the reviewer's suggestion, we compared RST with TMax-15K [1] using the same Qwen3.5-27B rollout model, harness, and inference configuration.

(1) Controlled RL Comparison. We trained Qwen3.5-27B on 14,601 tasks from TMax-15K and an equally sized subset of RST tasks sampled from the 15 rounds (R1–R15). Both conditions used identical model initialization, rollout settings, and PPO configurations (Section 4.7 and Appendix H).

Table. Controlled RL Comparison on Terminal-Bench 2.

RL Training DataTasksPPO StepsSeed 42 (%)Seed 43 (%)Seed 44 (%)Mean ± Std (%)
TMax-15K14,6017048.31 (43/89)49.44 (44/89)44.94 (40/89)47.57 ± 2.34
RST (Ours)14,6017048.31 (43/89)47.19 (42/89)49.44 (44/89)48.31 ± 1.12

(2) Controlled SFT Comparison. We collected successful trajectories from TMax-15K and RST R1–R3 using the same Qwen3.5-27B rollout configuration: SGLang, Harbor, Daytona, temperature 1.0, top-p 0.95, up to 100 turns, and 8,192 generation tokens per call (Appendix H).

We selected 5,000 successful trajectories from each dataset and fine-tuned Qwen3.5-27B using identical SFT settings.

Table. Controlled SFT Comparison on Terminal-Bench 2.

SFT Training DataSuccessful TrajectoriesSeed 42 (%)Seed 43 (%)Seed 44 (%)Mean ± Std (%)
TMax-15K5,00042.70 (38/89)40.45 (36/89)39.33 (35/89)40.82 ± 1.72
RST (R1–R3, Ours)5,00043.82 (39/89)42.70 (38/89)43.82 (39/89)43.45 ± 0.65

Why We Do Not Match Tokens in These Additional Comparisons. Unlike Table 6, we intentionally match the number of tasks for RL and the number of successful trajectories for SFT, rather than forcing the same total token count. This is because task difficulty can affect how many steps an agent takes, how long its trajectories become, and what the model learns from them. Forcing equal token counts could require truncating or resampling trajectories, changing these natural differences. Our goal here is to compare the training value of the tasks and trajectories produced by each dataset under the same model, harness, and training configuration.

Results. RST achieves 48.31% vs. 47.57% in RL (+0.74 percentage points) and 43.45% vs. 40.82% in SFT (+2.63 percentage points). Notably, RL achieves substantially higher absolute performance than SFT from the same Qwen3.5-27B base model (41.20 ± 1.72). This may relate to the previously observed ~20% gap between pass@8 and pass@1, suggesting that RL can unlock existing capabilities by shifting probability toward successful solutions. In contrast, SFT is limited by the coverage of fixed trajectories, although our scaling results show that increasing SFT data can further improve performance.These findings suggest that combining SFT and RL may lead to better performance, as SFT helps the model learn more solution strategies, while RL improves its ability to use them successfully. Exploring this combination could be an interesting direction for our future work.

We appreciate the reviewer's suggestion and will include these experiments in the revised appendix.

References

[1] Ivison, H., et al. (2026). Tmax: A Simple Recipe for Terminal Agents. https://arxiv.org/abs/2606.23321

W2: We thank the reviewer for carefully examining our code and raising these concerns. We would like to clarify the following points and provide additional experimental evidence.

1. Task Complexity. Our 40 operators do more than simply add extra steps or outputs. For example, debugging, test failure repair, state migration, and Git forensics require agents to identify problems, understand their causes, and develop solutions. As shown in Appendix B.5, these operators introduce new challenges rather than simply increasing task length.

2. Requirement Discoverability. We agree that the generation model may incorrectly assume that certain information is available in the workspace. To address this, our pipeline checks the alignment between task instructions, workspace files, reference solutions, and verifiers. When inconsistencies are found, the pipeline can repair them rather than simply rejecting the task. Importantly, our independent human evaluation (Appendix G, Table 13) shows that 90–94% of RST tasks are publicly solvable, compared with 86–88% for single-pass baselines.

3. Quality Control. The regex and length checks mentioned by the reviewer are preliminary filters designed to prevent overly detailed instructions and trivial rewrites, rather than general measures of task quality. These filters are designed for RST task generation, not for evaluating existing benchmark tasks like TB-hard. Therefore, rejecting TB-hard tasks does not mean those tasks are of poor quality. Beyond these filters, we check operator compatibility, instruction–solution–verifier consistency, environment configuration, and whether reference solutions pass verification in fresh Daytona sandboxes. For recoverable failures, our pipeline uses validation feedback to repair the task, with at most two repair rounds, followed by revalidation. Furthermore, independent LLM and human evaluations (Appendix G, Tables 12–13) show that 86–92% of RST tasks are release-ready, compared with 82–84% for the baselines.

4. Additional Experiment: To directly evaluate whether our quality checks can detect hidden requirements, we sampled 20 tasks from each of R1, R5, R10, and R15, resulting in 80 tasks. For each task, we created two versions with the same target requirement:

  • Discoverable: The requirement can be found in the public instruction or workspace.
  • Hidden: The requirement is still checked by the verifier but cannot be found in either the public instruction or workspace.

We then applied our quality checks to both versions. A successful check should accept the discoverable version and flag the hidden version.

Table. Hidden Requirement Detection Across Recursive Rounds.

RoundTask PairsDiscoverable AcceptedHidden FlaggedFalse PositivesFalse NegativesAccuracy
R12020/20 (100%)20/20 (100%)00100%
R52020/20 (100%)20/20 (100%)00100%
R102020/20 (100%)20/20 (100%)00100%
R152020/20 (100%)20/20 (100%)00100%
Overall8080/80 (100%)80/80 (100%)00100%

Results. Our quality checks correctly accepted all 80 discoverable versions and flagged all 80 hidden versions, achieving 100% accuracy across 160 test cases. This performance remains consistent from R1 to R15, suggesting that hidden-requirement detection remains effective across recursive rounds. More importantly, the results show that our checks can identify requirements missing from publicly available information, beyond simply filtering instruction formats. In our full pipeline, detected inconsistencies can also enter a bounded repair process, followed by revalidation.

Task Release. We agree that releasing generated tasks is important for independent assessment. As stated in our paper, the synthesized tasks and trajectories will be released upon publication.

We appreciate the reviewer's constructive suggestions and will include these additional experiments and clarifications in the revised appendix.

Q1: Comparison with Successful Rollouts from Other Tasks.

We thank the reviewer for this helpful question. To directly address this concern, we conducted a controlled SFT experiment using 5,000 successful Qwen3.5-27B trajectories collected from unmodified TMax-15K tasks and RST (R1–R3), respectively. Under the same rollout and training settings, RST achieves 43.45 ± 0.65% on TB2, compared with 40.82 ± 1.72% for TMax-15K (+2.63 percentage points). This suggests that the benefit of RST comes not simply from training on successful trajectories, but also from the additional learning challenges introduced by recursive task generation. Furthermore, Table 6 compares RST against five public SFT datasets under the same training-token budget, where RST achieves 46.82%, outperforming the strongest baseline at 42.70% (+4.12 percentage points). Together, these results show that RST provides useful training signals beyond those obtained from existing tasks and successful rollouts.

Reviewer_kWnpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes RST, a pipeline that generates increasingly difficult terminal tasks by repeatedly rewriting accepted tasks. Starting from 639 TerminalWorld tasks, each round selects an accepted task and one of 40 hand-designed rewrite operators, such as configuration repair or log diagnosis. It first extends the reference solution, and then updates the verifier, instruction, and environment to match. A rewritten task is kept only if its reference solution passes the verifier in a fresh sandbox, and accepted tasks serve as seeds for the next round. Over 15 rounds, the pipeline produces 37,484 tasks; the median reference solution grows from 67 to 374 lines, and DeepSeek-V4-Pro pass@4 drops from 90% to 2.5%. Fine-tuning Qwen3.5-27B and Qwen3.5-122B-A10B on their own successful trajectories from the first three rounds improves results on Terminal-Bench 2, Terminal-Bench Hard and Long-Horizon Terminal Bench. PPO on the full task pool further improves Qwen3.5-27B.

Critical strengths
  1. Reusing accepted tasks as seeds and extending them round by round is an interesting way to obtain harder tasks. Since each round adds work on top of an already validated task, the pipeline produces tasks with gradually increasing difficulty.

  2. The increase in difficulty is supported by multiple measures. In addition to longer reference solutions and more verifier assertions, two fixed solvers show consistent performance drops across rounds; for example, DeepSeek-V4-Pro pass@4 falls from 90% to 2.5%. Task quality is assessed with human audits.

  3. The downstream gains are consistent across two model sizes and three benchmarks, and the results are averaged over three training and evaluation runs with standard deviations reported.

Critical weaknesses
  1. The rewrites appear limited in what they can change. The 40 rewrite operators are hand-designed and may not align with the policy's weaknesses. And since each rewrite builds on the parent task’s environment, later tasks tend to remain variants of the original task, mostly with more required outputs. For example, in the case study of Appendix B.5, the core objective stays the same from the seed to R15, and later rounds mainly add deliverables such as diagnosis and validation reports.

  2. Validation only checks that the reference solution passes the verifier. It does not check that incorrect or incomplete solutions are rejected.

Questions for authors

Please refer to the weaknesses part.

4 comments

W1: We thank the reviewer kWnp for this constructive feedback. We acknowledge the limitations of the current operator library and provide additional clarification and experimental evidence.

1. Operator Design. Our 40 operators are motivated by real-world terminal workflow challenges. TerminalWorld [1] studies 80,870 terminal recordings, while Terminal-Bench 2.0 [2] analyzes 3,800 command failures. Accordingly, our operators cover five families: environment/runtime, build/execution, data/artifacts, configuration/state, and diagnostics (Appendix D, Table 11). Rather than merely adding deliverables, they introduce challenges such as debugging failures, migrating states, and verifying intermediate results. Although the core objective may remain unchanged, the required workflows and recovery strategies become more complex.

2. Task Difficulty. We focus on improving action reliability in long-horizon workflows. The approximately 20-percentage-point gap between Pass@1 and Pass@8 for Qwen3.5-27B on TB2 motivates our focus on improving execution reliability across repeated attempts. Improvements in LHTB partial credit further indicate better progress on challenging tasks.

3. Scalability and Extensibility. RST generated 37,484 validated tasks and 327,189 trajectories, which we plan to release after review. These can support SFT, RL, and further synthesis. We view the current 40 operators as an initial library and envision a dynamic operator library that evolves based on observed model failures, automatically introducing new operators to address model-specific weaknesses.

4. Additional Experiment: To examine this direction, we conducted a controlled experiment using existing RST trajectories. Following [2, 3], we analyzed Qwen3.5-27B's failed trajectories on TMax-15K [4] and designed a multi-label rubric covering command execution, state management, error recovery, and result verification. Each successful RST trajectory was scored from 0 to 2 per dimension. We then prioritized trajectories addressing the model's observed weaknesses, particularly recovery and verification.

We used the same candidate pool as the original random-selection experiment: all successful R1–R3 trajectories. From this pool, we selected 11,368 trajectories based on their failure-aware scores, matching the original 36,363,275 assistant-token budget through loss masking (Section 4.6). We used identical SFT settings and evaluated on TB2.

Table. Random vs. Failure-Aware Trajectory Selection.

Training DataTrajectoriesTB2 Success Rate (%)
Original RST (Random Selection)11,53146.82 ± 1.30
Failure-Aware Selection11,36849.44 ± 1.12

Results. Failure-aware selection improves TB2 success from 46.82% to 49.44% (+2.62 percentage points) under the same token budget. This suggests that targeting model-specific weaknesses can improve training effectiveness without generating additional tasks. While this experiment focuses on trajectory selection, it also motivates our future direction of developing a dynamic operator library that adapts to observed model failures.

We appreciate the suggestion and will include this experiment in the revised appendix.

References

[1] Chu, Z., et al. (2026). TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks. https://arxiv.org/abs/2605.22535

[2] Merrill, M. A., et al. (2026). Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. ICLR 2026. https://arxiv.org/abs/2601.11868

[3] Luo, Y., et al. (2026). Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification. https://arxiv.org/abs/2609.38812

[4] Ivison, H., et al. (2026). Tmax: A Simple Recipe for Terminal Agents. https://arxiv.org/abs/2606.23321

Q2: Validation only checks that the reference solution passes the verifier. It does not check that incorrect or incomplete solutions are rejected.

We thank the reviewer for raising this concern. We would like to clarify that each of our 40 rewrite operators has its own anti-shortcut strategy, which are included in the task and verifier design (Appendices C, D, and F). These checks aim to prevent agents from taking shortcuts, such as creating fake programs, skipping compilation, hard-coding answers, reporting fake benchmark results, reusing old files, or modifying tests to make them easier. However, we agree that testing only the correct solution is not enough to show that these checks work.

Additional Experiment. We sampled 60 validated tasks, with 15 tasks each from R1, R5, R10, and R15. For each task, we created three types of invalid solutions:

  1. Incomplete: Remove a required output file.
  2. Incorrect: Change the result to an incorrect value while keeping the expected format.
  3. Shortcut: Skip a required step but still produce a result that looks correct. For example, use a fake program, copy an old result, or directly write the expected answer without running the required computation.

We manually confirmed that all 180 solutions were invalid and tested whether the original verifiers could reject them.

Table. Verifier Robustness Against Invalid Solutions.

Solution TypeSamplesRejection Rate (%)
Incomplete Solution60100
Incorrect Solution60100
Shortcut Solution6096.67
Overall18098.89

Results. The verifiers rejected 178 out of 180 invalid solutions (98.89%). All incomplete and incorrect solutions were rejected. Only two shortcut solutions passed the checks. We examined both cases and found why they passed:

  • Case 1: Fake System Tool. One task required the agent to isolate a faulty cluster node. Instead of doing this, the agent created a fake crm_node command that returned normal-looking output. The verifier checked the command output but did not check whether the node was actually isolated.

  • Case 2: Prohibited Git Command. Another task required the agent to create a Git tag using specific commands and explicitly prohibited git tag -a. However, the agent used git tag -a and produced the correct final tag. Since the verifier only checked the final Git state, it could not tell which command was used.

These shortcut patterns were also very rare in our additional audit of 10,000 trajectories randomly sampled from R1–R3. The fake-tool shortcut appeared in only 3 trajectories (0.03%), and the Git command shortcut appeared in only 2 trajectories (0.02%). Before releasing the dataset, we will test more tasks for shortcuts, fix the identified verifier problems, and remove any tasks that cannot be reliably checked. In particular, we will add checks for actual system states and ensure that required execution steps can be verified.

We appreciate the reviewer's suggestion and will include these results and improvements in the revised appendix.

Reviewer_kWnpOpenReview ↗

Response to rebuttal

Thank you for the detailed response and additional experiments. The verifier robustness study addresses my second concern well. I will maintain my positive score.

We sincerely thank the reviewer for the thoughtful and encouraging response. We are truly grateful that you took the time to carefully consider our additional experiments and engage with our rebuttal. Your concerns have helped us strengthen our work, and we are especially glad that the verifier robustness study addressed your questions. We deeply appreciate your constructive feedback, continued support, and the time and effort you have devoted to reviewing our paper.

Reviewer_A13LOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose Recursive synthetic Terminal Tasks (RST) to generate high-quality long-horizon training data for terminal agent training. RST reuse accepted executable tasks to extend more tasks. It extends the reference solution and model supervised fine-tuned with the data showed significant gain over the base models.

Critical strengths

The paper presents a recursive reuse of complete executable terminal tasks to extend the number of accepted tasks and their complexity measured by length and command counts. The authors presents ablations to isolate generation vs. reseeding, evaluates task systhesis quality (acceptance and difficulty), and controls task diveristy. The proposed method shows broader improvements on two models.

Critical weaknesses

The propsed methold is limited to terminal testing. Whether it can be generalized to broader use cases are not explored.

Questions for authors

Did authors test the model improvement by removing low quality data after the model-based quality audit?
Also given the limited data, it might be worth to filter with human review as well.

3 comments

We sincerely thank Reviewer A13L for the careful reading and helpful feedback. We address the comments below.

W1: The proposed method is limited to terminal tasks, and its generalizability to broader applications has not been explored.

We thank the reviewer for this helpful suggestion. We agree that our original experiments focus on terminal tasks. To explore whether RST can work in other domains, we conducted an additional experiment on MiniWoB++ web tasks.

Tasks. We selected four tasks covering different web interactions:

  • email-inbox: Find and act on an email.
  • multi-layouts: Complete forms with different layouts.
  • navigate-tree: Navigate a folder tree to find a target.
  • book-flight-nodelay: Select a flight using autocomplete and a date picker.

We sampled 3 instances per task, giving 12 seed tasks.

Experimental Setup. We adapted RST's five operator families into 18 web-specific operators, covering environment changes, multi-step actions, data processing, state management, and error diagnosis. We used the same solution-first pipeline and DeepSeek-V4-Pro generator. A generated task is accepted only if its reference solution passes twice, while both a no-op execution and the parent's reference solution fail.

For RST, we perform 3 recursive rounds, generating 2 candidates per seed in each round and using one valid task for the next round. For Non-Recursive R3, we apply the exact same three operators used in the corresponding RST lineage, but combine them into a single rewrite, generating 2 candidates per original seed. This allows us to compare recursive and non-recursive rewriting under the same operator choices. We evaluate task difficulty using GPT-5.6-sol with a 40-step limit and 3 independent rollouts per task.

Table. Generalization Results on MiniWoB++ Tasks.

MethodValid TasksOracle StepsAgent StepsAgent Success (%) ↓
Original Seeds12/124.678.02100.00
RST R122/246.367.65100.00
RST R220/247.6511.4681.67
RST R321/248.2412.7966.67
Non-Recursive R318/247.1410.9675.93

Results. RST maintains 83–92% task validity across rounds. As tasks become more complex, oracle steps increase from 4.67 to 8.24, while agent success drops from 100% to 66.67%. Compared with Non-Recursive R3 using the same operators, RST R3 achieves higher task validity (87.5% vs. 75.0%) and generates tasks with lower agent success rates (66.67% vs. 75.93% agent success). These results suggest that RST can also work on web tasks, with recursive rewriting helping to increase difficulty while keeping tasks valid.

We appreciate the reviewer's suggestion and will include this experiment in the appendix.

Q1: Did authors test the model improvement by removing low quality data after the model-based quality audit?

We thank the reviewer for this helpful suggestion. We agree that training data quality is important. To study its impact, we conducted an additional experiment on quality-based trajectory filtering.

Quality Evaluation. Unlike the task-level audit in Appendix G, this experiment focuses on trajectory quality. We use GPT-6-Astra to score each trajectory on four dimensions: action redundancy, error recovery, execution efficiency, and trajectory uniqueness. Each dimension receives a score from 1 to 5, and the final score is their average.

Experimental Setup. We rank the 11,531 RST trajectories from R1–R3 by quality score and remove the lowest-scoring 10%, 20%, or 30%. We then fine-tune Qwen3.5-27B using the same training settings and 36.36M assistant-loss tokens, with resampling and loss masking to match the token budget. We also include a random filtering baseline.

Table. Impact of Quality-Based Trajectory Filtering on Terminal-Bench 2.

Training DataRemovedRetained TrajectoriesTB2 Success Rate (%)
Original RST SFT0%11,53146.82 ± 1.30
Quality Filtered10%10,37847.57 ± 0.65
Quality Filtered20%9,22548.69 ± 1.72
Quality Filtered30%8,07245.32 ± 1.30
Random Filtered20%9,22544.57 ± 2.34

Results. Removing the lowest-quality 20% of trajectories achieves the best performance (48.69%), improving over the original RST data (46.82%) and random filtering (44.57%). However, removing 30% reduces performance to 45.32%. These results suggest that selecting higher-quality trajectories can further improve RST training, while removing too much data may hurt performance.

We thank the reviewer for this suggestion and will include the additional experiment in the revised appendix.

Q2: Also given the limited data, it might be worth to filter with human review as well.

We thank the reviewer for this helpful suggestion. As reported in Appendix G (Table 13), we already conducted a blinded human audit of generated tasks using two independent reviewers and a third adjudicator. The results show that 86% of sampled R15 tasks were rated release-ready.

To further validate our trajectory quality assessment, we conducted an additional human evaluation.

Experimental Setup. Using the trajectories collected for the quality-filtering experiment in Q1, we divided them into three groups based on their GPT-6-Astra quality scores: High Quality (top 20%), Medium Quality (40th–60th percentile), and Low Quality (bottom 20%). We randomly sampled 20 trajectories from each group, resulting in 60 trajectories. Two independent human reviewers evaluated each trajectory using the same four quality criteria as GPT-6-Astra, scoring from 1 to 5 in 0.25-point increments, without seeing the model scores. We then averaged the two reviewers' scores and calculated Spearman's rank correlation with GPT-6-Astra scores.

Table. Human Validation of GPT-6-Astra Trajectory Quality Assessment.

Quality GroupScore PercentileSamplesGPT-6-Astra Score (Mean)Human Score (Mean)Absolute DifferenceSpearman ρ
High QualityP80–P100204.644.550.09—
Medium QualityP40–P60203.453.540.09—
Low QualityP0–P20202.492.640.15—
OverallAll sampled groups60———0.921

Results. Human ratings closely follow GPT-6-Astra scores, with small mean differences (0.09–0.15) and a strong Spearman correlation (ρ = 0.921) across all 60 trajectories. These results support the use of model-based quality scores for selecting training trajectories. We appreciate the reviewer's suggestion and will include this additional evaluation in the revised appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.