acceptodds
Under review as a conference paper at ICLR 2027

XRepoSkill: Learning Transferable Skills for Software Engineering Agents

Abstract

Software engineering agents increasingly use reusable skills distilled from prior experience to resolve repository-level issues, yet such skills often fail to transfer across repositories. A central challenge is that a behavior appearing in a successful trajectory is not necessarily responsible for the successful outcome: it may be genuinely useful, merely incidental, or simply a recurring habit of the model. We introduce XRepoSkill, a trajectory-based approach for learning transferable skills. We represent a skill as a collection of rules, each specifying what action to take and when to take it during issue resolution. XRepoSkill first contrasts successful and failed trajectories of the same agent on the same issue and derives candidate rules from where their execution paths diverge. Each rule is paired with an executable predicate that enables its prescribed behavior to be evaluated systematically on other trajectories. A rule is verified based on its association with successful issue resolution and retained only when its prescribed behavior recurs across multiple repositories; repository-specific variants of the same behavior are then consolidated into transferable rules. For a new issue, XRepoSkill selects relevant rules to guide the agent. We learn skills from publicly released trajectories on the official SWE-bench Verified leaderboard and evaluate them on SWE-bench Pro and DeepSWE using three backbone LLMs from different vendors; none of the evaluation repositories appears in the skill-learning trajectory pool. Against three recent skill learning methods, XRepoSkill achieves the highest issue resolution rate in all six benchmark–LLM combinations. In particular, on the challenging long-horizon DeepSWE benchmark, XRepoSkill improves issue resolution by 10.3 percentage points over the same agent without learned skills and by 5.0 points over the strongest skill-learning baseline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Software Agent, Skills, LLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7C1ZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper proposes XRepoSkill, an LLM-based pipeline that extracts rules from successful–failed coding-agent trajectories, scores their association with historical success, merges recurring rules across repositories, and injects selected rules into the agent prompt. Experiments report improvements on SWE-bench Pro and DeepSWE.

Critical strengths

The evaluation uses repository-disjoint benchmarks and identical underlying agents across methods. The repeated DeepSWE experiment suggests that the final rule prompt can improve agent performance.

Critical weaknesses
  1. The claimed rule verification is observational and heavily confounded by backbone capability. Successful and failed trajectories are generated by different models, with successful trajectories intentionally selected from stronger models. Historical association therefore does not establish that a rule causes improved resolution.

  2. Rule generation, predicate construction, effect estimation, clustering, and evidence scoring reuse the same trajectory pool. There is no genuinely held-out rule-validation set or correction for adaptive multiple testing. Furthermore, gain and z-score are not used to reject ineffective rules, despite the paper describing retained rules as verified.

  3. The evaluation lacks the most important simple baseline: a manually written or automatically selected generic software-engineering checklist with matched prompt length and inference calls. The most frequently selected rules are standard practices such as reproducing the failure and inspecting tests. Without this baseline, the necessity of XRepoSkill’s elaborate learning pipeline is unclear.

Questions for authors
  1. Does XRepoSkill outperform a compact generic SWE checklist under matched prompt length and selection-call budgets?
  2. Do rule effects remain after controlling for backbone identity and evaluating predicates on entirely held-out issues?
  3. Are the improvements over the strongest baseline statistically significant, particularly the 2.1-point gain on SWE-bench Pro?
3 comments

Thank you for your constructive feedback. We have addressed all your questions as follows.

[Q1] Does XRepoSkill outperform a compact generic SWE checklist under matched prompt length and selection-call budgets?

Yes. Following your suggestion, we have added a generic SWE checklist baseline on DeepSWE with DeepSeek, matching both prompt length and selection-call budget.

We prompted the skill-learning LLM to generate 36 generic SWE practices from its prior knowledge, without access to any trajectories. For each issue, the DeepSeek backbone selects up to three items using the same Stage 3 selection procedure as XRepoSkill, with the injected guidance matched in length to XRepoSkill's average.

The generic checklist achieves 49.6% resolution, compared with 46.0% for mini-SWE-agent and 58.4% for XRepoSkill. Thus, XRepoSkill outperforms the generic checklist by 8.8 percentage points under matched prompt length and selection-call budgets, demonstrating the added value of trajectory-learned, issue-adaptive guidance beyond generic SWE best practices.

[Q2] Do rule effects remain after controlling for backbone identity and evaluating predicates on entirely held-out issues?

Thank you for the question. We conducted two complementary analyses to examine whether rule associations remain robust after controlling for backbone identity and generalize to held-out issues.

(1) Controlling for backbone identity. We re-estimated each rule's association with issue resolution using both issue and backbone fixed effects, with standard errors clustered by issue:

yt=βrpr(t)+αissue(t)+γbackbone(t)+εt.y_t = \beta_r p_r(t) + \alpha_{\mathrm{issue}(t)} + \gamma_{\mathrm{backbone}(t)} + \varepsilon_t.

Here, βr\beta_r captures the association between predicate satisfaction and issue resolution after accounting for issue difficulty and backbone identity. We used the adjusted estimates and their z-scores in place of the original rule evidence scores.

Table R3: Rule evidence before and after controlling for backbone identity.

MetricIssue only (Eq. 1)Issue + backbone
Rules with positive estimates24/3628/36
%Resolved, DeepSWE w/ DeepSeek58.457.5

After adjustment, the median estimated association is +3.9 percentage points, and 15 of the 19 most frequently selected rules have positive estimates. Importantly, using backbone-adjusted evidence preserves downstream resolution performance (57.5% vs. 58.4%).

(2) Held-out issue validation. We randomly held out 100 of the 500 issues and reconstructed the rule library using only the remaining 400 issues, including rule discovery, predicate construction, evidence estimation, and cross-repository clustering. The learned rules and predicates were frozen before evaluation on the held-out issues.

The reconstructed library contains 29 transferable rules. Of the 20 rules with positive training-set gains, 18 (90.0%) remain positive on held-out issues, including all seven rules with training gains above +10 percentage points. Overall, 24/29 rules (82.8%) preserve the sign of their estimated associations.

Together, these results support the robustness of the learned rule associations to backbone adjustment and issue holdout. While these associations are observational rather than causal, our downstream evaluation independently demonstrates the effectiveness of the resulting skill guidance on unseen repositories.

[Q3] Are the improvements over the strongest baseline statistically significant, particularly the 2.1-point gain on SWE-bench Pro?

Six of the eight comparisons of XRepoSkill against mini-SWE-agent and the three skill-learning baselines across both benchmarks remain statistically significant at the 5% level after applying Holm–Bonferroni correction to the p-values from the eight McNemar tests.

XRepoSkill achieves the highest resolution rates on both benchmarks, with statistically significant improvements over both leading skill-learning baselines on at least one benchmark each. Specifically, XRepoSkill significantly outperforms STAIR on DeepSWE and Trace2Skill on SWE-bench Pro. Its gains over STAIR on SWE-bench Pro (+2.1 pp) and Trace2Skill on DeepSWE (+5.0 pp) do not reach statistical significance.

Furthermore, gain and z-score are not used to reject ineffective rules, despite the paper describing retained rules as verified.

Sorry for being unclear. We clarify that gain and z-scores quantify observational associations rather than establish statistical significance or causality. XRepoSkill deliberately uses these scores for issue-adaptive rule selection, not as hard retention thresholds, because a rule may be useful in specific contexts despite a negative average estimated gain.

To validate this design choice, we added an ablation that removes rules with negative gains, reducing DeepSWE resolution from 58.4% to 54.9% with DeepSeek as the backbone. Separately, removing cross-repository generalization reduces resolution to 51.3% (Table 4). These results support retaining rules based on cross-repository recurrence and using evidence scores for issue-adaptive prioritization. We will clarify that “verification” refers to empirical behavioral evidence and cross-repository support, rather than formal statistical or causal verification.

Thank you for the detailed response. All my concerns have been addressed, and I will raise my score accordingly.

Reviewer_7oETOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper solved the reusable issues when applying successful skills distilled from agent trajectories to new repositories, as it has been observed that a successful trajectory may be useful, incidental.

The paper separates rule discovery from rule verification. It proposes a systematic approach to discovering, evaluating, and selecting reusable coding-agent guidance. The method uses trajectories from 12 backbone models on 500 SWE-bench Verified issues. Combining trajectory contrasts, executable predicates, and cross-repository consolidation is sufficiently substantive.

Evaluation covers 200 SWE-bench Pro tasks and all 113 DeepSWE tasks with three deployment models. Evaluation repositories are absent from the learning pool. Average resolution reaches 53.8% on SWE-bench Pro, versus 47.8% without skills and 51.7% for the strongest baseline; on DeepSWE, it reaches 44.2%, versus 33.9% and 39.2%, respectively. XRepoSkill has the highest score in every model–benchmark combination, including one tie.

Critical strengths
  1. The method is proved to have improvements extend to unseen repositories, different programming language scope and different model vendors. This makes the results solid and general and makes the results substantially consequential then just improvements on certain restricted repositories.

  2. Consistent gains in a strict transfer setting. No evaluation repository is in the pool, gains hold across vendors and unseen languages, and the repeated-run on the DeepSWE result reducing concerns that the headline is merely a lucky run.

Critical weaknesses
  1. The most concern I had is table 1 results, which is single-run with no intervals. On SWE-bench Pro the margin over STAIR is 2.1 points on average and zero for DeepSeek, which is within the ±2.5-point noise the authors themselves measure in RQ3. The "all six combinations" framing is weaker than it looks. So this raise the alarm again whether it is a lucky run or not.

  2. Most results use one run per task and configuration; repeated evaluation covers only one model–benchmark setting. The relatively small advantage over the strongest SWE-bench Pro baseline needs paired uncertainty estimates before it supports a strong superiority claim.

  3. Skills do not overcome the backbone’s underlying limitations. On DeepSWE, MiMo with skills reaches only 17.7%, substantially below DeepSeek without skills at 46.0%. The appropriate comparison is the improvement within each backbone, rather than comparing one model with skills against a different model without them.

  4. Model capability can confound historical rule verification. Appendix A.1 preferentially pairs successful trajectories from stronger models. Verification controls for issue difficulty, but does not explicitly control for backbone identity. A behavior could correlate with success partly because stronger models exhibit it more often, but the paper does not isolate this correlation properly.

Questions for authors
  1. How do rule gains and rankings change after accounting for backbone-model identity or using within-backbone comparisons? Do the most frequently selected rules retain positive associations?

  2. What are the paired confidence intervals for improvements over the strongest baselines in Table 1, particularly on SWE-bench Pro?

  3. Three models from different vendors are useful evidence, but all are tested with high reasoning effort. This does not establish effectiveness for smaller models, models without extended reasoning, or substantially different coding capabilities. The 12 models used to generate historical trajectories are not 12 deployment-model evaluations.

3 comments

Thank you for your constructive feedback. We have addressed all your questions as follows.

[Q1] How do rule gains and rankings change after accounting for backbone-model identity or using within-backbone comparisons? Do the most frequently selected rules retain positive associations?

Thank you for the question. Backbone differences are used for candidate rule discovery (Stage 1), whereas rule verification (Stage 2) evaluates associations on other trajectories. Nevertheless, backbone identity may still confound these associations. We therefore re-estimated each rule's association with issue resolution using both issue and backbone fixed effects, with standard errors clustered by issue:

yt=βrpr(t)+αissue(t)+γbackbone(t)+εt.y_t = \beta_r p_r(t) + \alpha_{\mathrm{issue}(t)} + \gamma_{\mathrm{backbone}(t)} + \varepsilon_t.

Here, βr\beta_r captures the association between predicate satisfaction and issue resolution after adjusting for issue difficulty and backbone identity. We used the adjusted estimates and their z-scores in place of the original rule evidence scores.

Table R1: Rule evidence before and after controlling for backbone identity.

MetricIssue only (Eq. 1)Issue + backbone
Rules with positive estimates24/3628/36
Rank correlation with Eq. 1 (gain / z)–0.56 / 0.49
%Resolved, DeepSWE w/ DeepSeek58.457.5

After controlling for backbone identity, the number of rules with positive estimated gains increases from 24/36 to 28/36, although rule rankings change (rank correlations of 0.56 for gain and 0.49 for z-score). The adjusted estimates have a median of +3.9 percentage points.

Importantly, 15 of the 19 most frequently selected rules have positive backbone-adjusted associations, and downstream resolution remains nearly unchanged (57.5% vs. 58.4%). These results indicate that the majority of frequently selected rules remain positively associated with successful resolution despite backbone adjustment.

[Q2] What are the paired confidence intervals for improvements over the strongest baselines in Table 1, particularly on SWE-bench Pro?

Thank you for the question. We computed paired 95% confidence intervals (CIs) for improvements over the strongest baselines in Table 1, alongside McNemar tests with Holm–Bonferroni correction for multiple comparisons.

XRepoSkill achieves the highest observed resolution rates on both benchmarks, with statistically significant improvements over each of the two leading skill-learning baselines on at least one benchmark. Specifically, it significantly outperforms STAIR on DeepSWE (Holm-adjusted p = 0.004) and Trace2Skill on SWE-bench Pro (Holm-adjusted p = 0.033).

For the strongest baseline on each benchmark, the paired improvements are +2.1 pp (95% CI: [−0.4, 4.8]) over STAIR on SWE-bench Pro and +5.0 pp (95% CI: [−0.2, 10.2]) over Trace2Skill on DeepSWE. Both point estimates favor XRepoSkill, although the intervals include zero.

Our repeated-run evaluation (Table 3) provides complementary evidence of stability. Across four runs on all 113 DeepSWE issues with DeepSeek, XRepoSkill achieves 58.6 ± 2.49% resolution versus 49.3 ± 2.69% for Trace2Skill (95% CIs). The non-overlapping intervals provide additional evidence that the advantage on this setting is not driven solely by a favorable single run.

[Q3] Three models from different vendors are useful evidence, but all are tested with high reasoning effort. This does not establish effectiveness for smaller models, models without extended reasoning, or substantially different coding capabilities. The 12 models used to generate historical trajectories are not 12 deployment-model evaluations.

Thank you for the comment. We used high reasoning effort in our main evaluation because long-horizon software engineering tasks such as DeepSWE require substantial reasoning for repository understanding, failure diagnosis, and code repair. To examine whether XRepoSkill also benefits agents with reduced reasoning effort, we conducted additional experiments on DeepSWE using GPT-5.6-Luna with reasoning disabled and low reasoning effort.

Table R2: Within-backbone improvements across reasoning levels on DeepSWE.

Reasoning effortmini-SWE-agentXRepoSkillImprovement
No reasoning1.8%2.7%+0.9 pp
Low reasoning10.6%15.0%+4.4 pp
High reasoning (Table 1)44.2%56.6%+12.4 pp

XRepoSkill improves resolution at all three reasoning levels, including a +4.4 pp gain under low reasoning. Notably, without reasoning, the unmodified agent resolves only 1.8% of DeepSWE issues, highlighting the substantial reasoning demands of these long-horizon tasks and motivating our use of high reasoning effort in the main evaluation. The smaller absolute gain in this setting is consistent with XRepoSkill's role as procedural guidance, which complements rather than replaces the backbone's reasoning capabilities.

Regarding backbone capability, our evaluation compares methods within each backbone. For example, on DeepSWE, XRepoSkill improves MiMo from 11.5% to 17.7% (+6.2 pp), despite its lower absolute performance than DeepSeek. These results show that XRepoSkill can benefit agents beyond high-reasoning configurations while remaining dependent on their underlying capabilities.

Thank you for the additional analyses.

The backbone-adjusted rule estimates and the similar downstream performance substantially reduce my concern that the method depends on backbone identity confounding the original evidence scores. The reduced-reasoning experiments also broaden the demonstrated scope, even though the no-reasoning improvement is comparatively small ( 1) and does not establish transfer to different model families or sizes.

Overall, the rebuttal strengthens my assessment of the method’s robustness.

Reviewer_gsn5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how software engineering agents can learn cross-repository transferable skills from historical trajectories. The core issue is that observing a certain behavior in a successful trajectory does not suffice to show that this behavior is related to the successful outcome. The authors therefore introduce failed trajectories to learn transferable skills that are related to successful outcomes. The paper represents a skill as a collection of rules, each rule specifying when to do something and what to do, and accompanied by an executable predicate used to automatically judge whether other trajectories exhibit the behavior prescribed by the rule. First, the paper compares successful and failed trajectories generated by the same agent on the same issue using different backbone large language models, and extracts candidate rules from their divergence. It then statistically keeps, as repository-independent transferable rules, the rules that are independently verified across multiple repositories and represent the same behavior. The main contribution of the paper is discovering candidate rules from the behavioral divergence between successful and failed attempts, and verifying the rules using independent trajectory evidence. The paper uses the public trajectories of SWE-bench Verified for learning, and on SWE-bench Pro and DeepSWE compares against three backbone large language models, one agent, and three skill-learning baselines; it also examines the method's generalization ability, stability, and the role of each component through cross-language tests, repeated runs, and ablation experiments.

Critical strengths

The paper points out that not all behaviors in successful trajectories are related to the successful outcome, and proposes a corresponding method to extract valuable behaviors to form transferable rules. An interesting point is the use of executable predicates for cross-trajectory evaluation; the cross-repository experimental results are somewhat persuasive.

Critical weaknesses

The paper uses executable predicates to judge whether other trajectories exhibit the behavior prescribed by a rule, and the accuracy of the predicates is key to extracting transferable rules. Although the final cross-repository experimental results do show an improved issue resolution rate, the paper does not provide any experimental statistics related to the executable predicates themselves. For example, Section 2.4 represents "before editing any file, first run the failing test" as the ordering relation between a test run and the first edit (before(run_test, first_edit)), but it does not actually explain how it ensures that the identified test indeed corresponds to the current failure.

Questions for authors
  1. How does an executable predicate express the complete semantic information of a rule, and how accurate is it? If one merely uses simple ordering relations to judge whether other trajectories exhibit the behavior of satisfying this rule, is such a scheme too simplistic?

  2. Why is SkillOpt's average task resolution rate on DeepSWE even lower than that of the agent without any skills? If you reproduced it yourself, how do you guarantee that the reason your method is better is not merely that the baselines were reproduced poorly? Also, your method is trained on Python data, so why, in the cross-language generalization experiments, is the effect only worse on the Python data than the compared methods, while better on the other languages?

4 comments

Thank you for your constructive review. We have addressed all your questions as follows.

[Q1] How does an executable predicate express the complete semantic information of a rule, and how accurate is it? If one merely uses simple ordering relations to judge whether other trajectories exhibit the behavior of satisfying this rule, is such a scheme too simplistic?

Thank you for raising these concerns.

Predicate expressiveness. Executable predicates are not intended to encode the complete semantics of natural-language rules. Instead, they serve as behavioral signatures for trajectory verification and cross-repository rule matching in Stage 2, while the agent receives the full natural-language rules during issue resolution. Predicates are not limited to simple ordering: they also support action existence, counting, and argument matching. For example, the rule “survey definitions before editing” is represented as before(search("def "), first_edit) ∧ count(search("def ")) ≥ 3.

Predicate accuracy. We have conducted two complementary human evaluations:

(1) Rule–predicate fidelity. Two annotators independently assessed all 36 mappings against their corresponding natural-language rules, agreeing on 34/36 cases (94.4%). After resolving disagreements, all 19 most frequently selected rules had faithful predicates (100%), and 34/36 mappings (94.4%) were faithful overall.

(2) Trajectory-level accuracy. We randomly sampled 100 trajectories, each paired with a randomly selected rule from the 19 most frequently selected rules. Annotators examined the full natural-language rule, issue context, and original trajectory to independently determine whether the prescribed behavior was followed. Against these human judgments, predicates achieved 97.0% accuracy, 95.2% precision, and 97.6% recall. These results directly demonstrate the reliability of predicates as behavioral signatures on the sampled trajectories.

Regarding the reviewer's example, before(run_test, first_edit) captures whether testing precedes editing but does not itself verify that the test reproduces the specific failure. This issue-dependent semantic judgment is guided by the full natural-language rule provided to the agent. Such abstraction may introduce occasional mismatches, which our trajectory-level evaluation explicitly measures. Moreover, predicate satisfaction alone does not establish a rule's usefulness: Stage 2 further evaluates its association with successful resolution on trajectories beyond the originating pairs and requires corroborating evidence across repositories.

[Q2a] Why is SkillOpt's average task resolution rate on DeepSWE even lower than that of the agent without any skills? If you reproduced it yourself, how do you guarantee that the reason your method is better is not merely that the baselines were reproduced poorly?

We inspected SkillOpt's learned skills and found explicit patch-submission instructions specific to the SWE-bench Verified protocol used in the trajectory pool. These instructions can be incompatible with DeepSWE's submission procedure and contribute to negative transfer, illustrating the transferability challenge XRepoSkill aims to address. Importantly, SkillOpt's performance decrease (31.3% vs. 33.9%) is not statistically significant (McNemar p = 0.417).

We used the official baseline implementations and prompts. For SkillOpt, we disabled task re-execution to adapt it to the common offline-trajectory setting, since online re-execution would introduce additional rollouts unavailable to other methods. We will explicitly label this adaptation “SkillOpt (offline)” to distinguish it from the original method.

As a sanity check, STAIR and Trace2Skill both significantly outperform mini-SWE-agent on SWE-bench Pro (McNemar p = 0.007 and 0.040). Importantly, XRepoSkill also achieves higher average resolution rates than both STAIR and Trace2Skill on each benchmark, so its overall advantage is not solely attributable to SkillOpt's offline adaptation. For transparency and reproducibility, we have publicly released our baseline implementations and adaptations for independent inspection.

[Q2b] Also, your method is trained on Python data, so why, in the cross-language generalization experiments, is the effect only worse on the Python data than the compared methods, while better on the other languages?

Thank you for the question. XRepoSkill does not consistently underperform on Python, as the premise suggests. As shown in Table 2, on SWE-bench Pro, it trails STAIR by only 1.0 percentage point while outperforming Trace2Skill by 1.0 point. On DeepSWE, it outperforms STAIR by 5.9 points and matches Trace2Skill.

The larger relative gains on other languages are consistent with XRepoSkill's design. While Python-derived experience can benefit all methods on Python tasks, XRepoSkill explicitly verifies candidate behaviors on independent trajectories and retains only those recurring across repositories, reducing reliance on repository-specific conventions and implementation details. This favors general issue-resolution strategies that can remain useful across programming languages, explaining its stronger relative gains on languages absent from the learning corpus.

Thank you for the detailed rebuttal. The additional predicate evaluations and explanations address some of my concerns. However, my main concern remains: the rule verification relies on correlations between predicate satisfaction and task success, without adequately controlling for differences in backbone LLM capabilities. Therefore, it remains unclear whether the identified behaviors actually contribute to successful issue resolution. Given this unresolved soundness concern, I will maintain my original rating.

Thank you for the follow-up. To directly address this concern, we summarize below the additional analyses we conducted during the rebuttal on whether the rule evidence is robust to backbone capability and whether the learned rules contribute to issue resolution.

(1) Controlling for backbone capability. Backbone differences are used for candidate rule discovery (Stage 1), whereas rule verification (Stage 2) evaluates associations on other trajectories. Nevertheless, backbone capability may still confound these associations. We therefore re-estimated each rule's association with issue resolution using both issue and backbone fixed effects, with standard errors clustered by issue:

yt=βrpr(t)+αissue(t)+γbackbone(t)+εt.y_t = \beta_r p_r(t) + \alpha_{\mathrm{issue}(t)} + \gamma_{\mathrm{backbone}(t)} + \varepsilon_t.

Here, βr\beta_r captures the association between predicate satisfaction and issue resolution after adjusting for issue difficulty and backbone identity, so differences in backbone capability are absorbed by γbackbone(t)\gamma_{\mathrm{backbone}(t)}. We used the adjusted estimates and their z-scores in place of the original rule evidence scores.

MetricIssue only (Eq. 1)Issue + backbone
Rules with positive estimates24/3628/36
%Resolved, DeepSWE w/ DeepSeek58.457.5

After controlling for backbone capability, the number of rules with positive estimated gains increases from 24/36 to 28/36, with a median adjusted estimate of +3.9 percentage points. Importantly, 15 of the 19 most frequently selected rules have positive backbone-adjusted associations, and using backbone-adjusted evidence preserves downstream resolution performance (57.5% vs. 58.4%).

(2) Direct evidence that the learned rules contribute to resolution. Beyond these associations, our downstream evaluation compares methods within each backbone: the same backbone resolves the same issues with and without the learned rules. On DeepSWE, XRepoSkill improves DeepSeek from 46.0% to 58.4% (+12.4 pp), GPT-5.6-Luna from 44.2% to 56.6% (+12.4 pp), and MiMo from 11.5% to 17.7% (+6.2 pp). Because the backbone is held fixed in each comparison, these gains cannot be attributed to differences in backbone capability. Moreover, a generic SWE checklist under matched prompt length and selection-call budgets achieves only 49.6% with DeepSeek, compared with 58.4% for XRepoSkill, demonstrating the added value of trajectory-learned, issue-adaptive guidance beyond generic SWE best practices.

Together, these results show that the rule associations are robust to controlling for backbone capability, and that the learned rules improve issue resolution when the backbone is held fixed. We will add the backbone-adjusted analysis to the revised paper. We hope this response addresses your remaining concern, and we would be grateful if you could reconsider your assessment in light of these results.

Comments on the paper

1 comment

Dear Reviewers and Area Chair,

Thank you for your constructive feedback. We have carefully addressed all the reviewers' concerns in our responses, including providing additional experimental evidence and analyses.

As the discussion period approaches its end, we would greatly appreciate it if the reviewers could review our responses and share any further feedback. We also hope the additional evidence can be taken into account when reassessing the paper.

Thank you again for your time and consideration.

Best, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.