acceptodds
Under review as a conference paper at ICLR 2027

Repairing Failed Coding-Agent Trajectories for Cost-Efficient Training Data Expansion

Abstract

Coding-agent trajectories provide supervision for model training and evidence for failure analysis, yet collecting them through repeated rollouts is costly and often produces unsuccessful runs. Existing curation pipelines largely retain solved trajectories or regenerate complete runs, leaving action-level reuse of failed trajectories insufficiently explored. We introduce , a framework for augmenting coding-agent trajectory datasets by recomposing failed runs. localizes anomalous actions, replaces them with compatible spans from successful trajectories, and verifies the resulting repository states. For the first research question (RQ1), the strongest checker reaches 93.72 anomaly F1, while a compact 3B variant retains 87.77 F1 and powers the remaining pipeline. In RQ2, 18.23% of failed trajectories pass build and executable functional tests, and 11.78% yield verified alternative implementations, outperforming the evaluated local-replacement baselines. In RQ3, three-seed training on a 16,000-record source-verified, state-aligned corpus reduces held-out action negative log-likelihood by 22.05% relative to the donor-context baseline. A separate resource-normalized evaluation finds that the marginal cost of acquiring an accepted output is 3.9–210 lower than complete rollout with the evaluated agents. These results show that trajectory recomposition recovers useful training supervision from failed coding-agent runs without generating complete replacement rollouts.

Primary area:
datasets and benchmarks
Keywords:
coding agents, trajectory recomposition, data augmentation, software engineering, dataset

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TbvxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReComp, a framework for repairing failed coding-agent trajectories to generate additional training data at a lower cost. ReComp first identifies anomalous actions in failed trajectories, retrieves compatible action spans from successful trajectories, and recomposes them through a learned planner. The repaired trajectories are then executed and verified to ensure correctness. Experimental results show that ReComp can successfully repair a portion of failed trajectories, reduce training action NLL, and lower data acquisition costs compared to generating complete new trajectories.

Critical strengths

1. Novel and cost-efficient trajectory augmentation approach.

The paper proposes an interesting approach to reusing failed coding-agent trajectories by replacing anomalous actions with compatible actions from successful trajectories, reducing the cost of collecting additional training data.

2. Comprehensive experimental evaluation.

The paper evaluates anomaly detection, trajectory repair, downstream training utility, and data acquisition costs, providing empirical evidence for the effectiveness of the proposed framework.

Critical weaknesses

1. Limited improvement over Retrieval Top-1.

ReComp only improves the strict acceptance rate from 17.18% to 18.23% compared to Retrieval Top-1. This relatively small improvement raises concerns about whether the additional complexity of the planner is justified.

2. Lack of strong downstream training baselines.

The downstream training experiments lack comparisons with strong data selection and augmentation baselines, making it difficult to demonstrate the effectiveness of ReComp-generated training data over existing approaches.

Questions for authors

Mentioned in Critical Weaknesses.

Reviewer_hj57OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces ReComp, a framework to synthesize datasets for coding agents by recomposing failed trajectories. The authors evaluate the framework through three separated research questions(anomaly localization, verified recomposition, and downstream training utility and cost), and experiments on NanoSWE and SWE-smith trajectories show improvements over the evaluated baselines in each.

Critical strengths
  1. The paper tackles a practically relevant problem, reusing costly bad cases that are usually discarded, and does so without generating new rollouts, which is a valuable research direction.
  2. The RQ3 comparison is well controlled. The Donor and ReComp arms share identical target actions and target token counts and differ only in the conditioning context. The shuffled-context ablation further keeps the failed-recipient contexts but breaks their pairing with target actions, and it performs worse than both arms, which shows that the gain comes from state–action alignment rather than from mere exposure to failure contexts.
Critical weaknesses
  1. ReComp is presented as a complete pipeline for repairing failed agent trajectories, but the experiments evaluate its three stages independently under idealized conditions, and there is no end-to-end experiment. The only checker-gated run appears in the cost appendix, is not reported as an effectiveness result, and its outputs are never used for training. Since each stage relies on the output of the previous one, such an experiment is necessary to assess how errors accumulate. For example, with construction-defined windows RQ2 repairs 18.23% of failed trajectories, whereas the only checker-gated run, reported in the cost appendix, accepts 95 of 2,649 (3.6%)
  2. The RQ3 evaluation has two gaps. First, the test set only uses failed contexts, which is the same format as the ReComp training data. So we do not know whether the ReComp model also does well in normal contexts. The authors should report the NLL of both models on a test set with successful contexts. Second, NLL only measures how well the model predicts one reference action. It does not show whether the model can actually solve tasks. The only execution result improves from 1.39% to 2.12%, which is about five more successes per seed out of 769. The paper should report end-to-end agent performance, such as the resolve rate.
  3. (Minor) Many terms are used before they are defined. For example, the abstract reports results for RQ1 to RQ3 without saying what they are, and terms such as “donor”, “recipient” and “source-verified” are only explained in Section 3 or the appendix.
Questions for authors
  1. Why is the RQ3 corpus not built from the outputs of the full pipeline?
  2. what is the NLL of the Donor and ReComp models on a test set with success contexts?
  3. In RQ3, the context ends before the anomalous action, and the code state is the same as in the successful trajectory. The model therefore never sees a state where an error has already been made. In what sense does this teach the model to recover from failure states, and have you evaluated it on states after an error?
Reviewer_dJkuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReComp, a framework for repairing failed coding-agent trajectories through action-level recomposition instead of full rerollout. Given a failed recipient trajectory and a solved donor trajectory, ReComp first localizes suspicious action spans using a learned anomaly checker, then retrieves compatible donor spans and stitches them into the failed trajectory before executing and verifying the resulting repository state. The paper studies three research questions: anomaly localization, executable trajectory recomposition, and downstream training utility. Experiments on NanoSWE and SWE-smith-derived datasets show that ReComp can recover executable repaired trajectories at relatively low marginal cost and that recomposed trajectories improve downstream action modeling compared to training only on donor trajectories.

Critical strengths
  1. The paper studies a practically important problem. Failed coding-agent trajectories are expensive to collect, and leveraging them instead of discarding them is a meaningful direction for agent training and data efficiency.

  2. The downstream utility experiments are interesting. The matched comparison between donor-context training and recomposed-context training suggests that aligning corrective actions with failure states may provide additional supervision benefits.

  3. The writing is clear and easy to follow.

Critical weaknesses
  1. The overall framework is fairly complex, involving checker training, retrieval, compatibility planning, execution replay, and endpoint verification. It is therefore difficult to isolate which components are truly necessary for the reported gains, and several design choices appear heavily engineered.

  2. The evaluation focuses heavily on trajectories derived from SWE-style coding-agent environments. It remains unclear whether the recomposition strategy would generalize to broader agent settings where actions are less structured or repository states are harder to align.

  3. The downstream improvements, while consistent, are still relatively modest in absolute terms. For example, endpoint replay success remains low overall, and it is unclear how much these gains would translate into stronger real-world coding-agent performance.

Questions for authors

Please refer to weaknesses part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.