acceptodds
Under review as a conference paper at ICLR 2027

RepairChemist: Cross-Language Repair Skill Distillation for Low-Resource Program Repair

Abstract

Large language models (LLMs) have made substantial progress in program repair for high-resource programming languages (PLs) such as Java, yet a clear performance gap remains for languages with comparatively scarce repair data, such as Rust. Existing approaches use fine-tuning or program translation to alleviate this gap, but may introduce additional adaptation costs or semantic inconsistencies. In this work, we propose RepairChemist, a cross-language repair knowledge transfer framework that abstracts repair knowledge from high-resource PLs into reusable repair skills and guides program repair in low-resource PLs. Specifically, RepairChemist distills defect patterns and repair actions from bug-fix pairs in high-resource PLs, and organizes the resulting skills into skill groups based on statement types, forming a structured skill library. During low-resource program repair, RepairChemist retrieves applicable skills for the current defect based on its statement types and defect patterns to guide patch generation. Experiments on xCodeEval and SWE-bench Multilingual show that RepairChemist consistently improves repair performance across low-resource PLs and LLMs in both procedural and agentic repair settings, without parameter updates or program translation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Program Repair, Low-Resource Programming Languages, Cross-Language Knowledge Transfer, Repair Skill Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kwE5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RepairChemist, a framework that transfers program-repair knowledge from a high-resource language (Java) to lower-resource languages without fine-tuning or translation. It distills bug-fix pairs into language-agnostic repair skills (defect patterns and repair actions) organized by statement types for structured retrieval, effectively guiding both procedural and repository-level repair agents. Across multiple languages (Kotlin, Ruby, Rust, Go) on xCodeEval and SWE-bench Multilingual, RepairChemist with Qwen3-30B achieves a 76.41% resolved rate on xCodeEval, notably outperforming direct repair (61.78%) and the strongest baseline LANTERN (71.85%).

Critical strengths

The paper's overall method is complete and its application path is clear, and it does not stop at single-function repair. The authors cover procedural workflows and repository-level agents, examine the overlap between source-language training problems and target evaluation problems, and provide ablations of the skill library, grouping, and sampling mechanisms.

Critical weaknesses

The significance of the research task is limited: currently, even the experimental results of the qwen3-30B model on the low-resource languages mentioned in the article all exceed 60%; does this mean that the generalization ability of current LLMs is already able to solve this problem fairly well?

Questions for authors

see weeknesses.

3 comments

Thank you for the insightful review. We hope that our response below addresses your concern.

W1: Significance of low-resource language repair.

We agree that current LLMs have demonstrated certain capabilities in Low-Resource Program Repair. However, even with an average resolved rate exceeding 60%, Qwen3-30B still fails to repair nearly 40% of the tasks. RepairChemist improves its average resolved rate from 61.78% to 76.41% (+14.63pp), substantially enhancing its practical utility for low-resource program repair. We believe this research problem remains important for three reasons.

First, although existing LLMs have made progress on function-level repair tasks, they still face substantial challenges in more complex repository-level repair tasks[1, 2]. For example, Qwen3-30B achieves an average resolved rate of only 17.06% across the three target languages on SWE-bench Multilingual. Second, smaller models generally have more limited repair capabilities in low-resource languages. These models remain valuable in practical software development scenarios with limited computational resources, offline deployment requirements, or privacy constraints. Finally, as programming language ecosystems continue to evolve, emerging languages (e.g., Mojo, Cangjie, and MoonBit) and new language features are constantly introduced, making it difficult for models to fully acquire knowledge of these languages and features solely from their existing training data.

[1] Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving (NeurIPS 2025)

[2] On the Evaluation of Large Language Models in Multilingual Vulnerability Repair (TOSEM 2026)

Thank you for your explanation; I stand by my score.

Thank you for your support. We sincerely appreciate the time and effort you have devoted to helping improve the quality of our manuscript.

Reviewer_1JTTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RepairChemist, a cross-language repair knowledge transfer framework for low-resource programming languages. The method distills language-agnostic defect patterns and repair actions from Java bug-fix pairs into reusable skills. These skills are organized by statement groups and retrieved to guide patch generation in target languages. Experiments on xCodeEval and SWE-bench Multilingual show improvements across several low resource PLs and LLMs in both procedural and agentic repair settings.

Critical strengths

• The paper explores a useful direction for low-resource program repair. Reusing repair knowledge from high-resource languages avoids the need for model fine-tuning or code translation.

• RepairChemist compares with three types of baselines across different low-resource languages and LLMs. The ablation studies provide evidence for the effectiveness of the skill library and statement-type grouping.

Critical weaknesses

• The methodological novelty is relatively limited. The framework mainly combines existing techniques, including LLM-based knowledge abstraction and hierarchical skill retrieval. The paper does not clearly establish how its skill distillation mechanism advances beyond existing knowledge reuse methods, such as ExpeRepair.

• The evaluation only considers Java as the source language for skill distillation. It is unclear whether skills distilled from other programming languages, even low-resource PLs, can improve repair performance on different target languages.

Questions for authors

• The paper highlights the limitations of fine-tuning-based methods but does not include them as baselines. Could authors compare the performances and costs of fine-tuning and skill distillation?

• Can authors provide further analysis of the skill distillation and retrieval mechanisms? Such as, how much does the Keep/Propose/Refine strategy contribute? How many skills are retrieved in different target languages?

5 comments

Thank you for the positive feedback and constructive suggestions. We hope that the following responses clarify the questions you raised

W1: Methodological novelty and distinction from ExpeRepair.

We apologize that our original presentation may not have sufficiently highlighted the key novelty of RepairChemist. Our main contribution lies in using language-agnostic repair skills as the medium for knowledge transfer, establishing a cross-language transfer framework that leverages repair experience from high-resource languages to low-resource languages. This provides an effective pathway to improving program repair in low-resource languages. Experimental results demonstrate consistent performance improvements across multiple low-resource programming languages.

We thank the reviewer for suggesting a clearer distinction between RepairChemist and ExpeRepair. ExpeRepair distills general repair experiences and heuristic strategies from historical repair trajectories, but does not explicitly specify their applicability conditions. Moreover, these experiences may retain language-specific implementation details, limiting cross-language reuse. In contrast, RepairChemist differs in three key aspects. First, applicability-aware repair knowledge. RepairChemist explicitly characterizes defect patterns (When) and corresponding repair actions (How), specifying the conditions under which the repair knowledge applies. Second, language-agnostic knowledge abstraction. RepairChemist removes language-specific syntax and implementation details to enable cross-language knowledge reuse. Third, language-agnostic skill retrieval. RepairChemist uses fundamental statement types shared across programming languages and target defect patterns to enable cross-language skill retrieval. The relevant discussion has been added to the revised manuscript (Line 132).

W2: Extension to other source languages.

We further distill repair skills from 200 Rust bug-fix pairs in the xCodeEval training set and apply them to Rust and Ruby. As shown in the table, RepairChemist outperforms Direct Repair by 5.36pp on Rust and 4.96pp on Ruby, achieving resolved rates of 70.73% and 63.27%, respectively. These results demonstrate that repair skills distilled from Rust can be transferred across languages. The relevant discussion has been added to the revised manuscript (Lines 495 and 851).

MethodRust (%)Ruby (%)
Direct Repair65.3758.31
RepairChemist (Rust-distilled)70.7363.27
Improvement (pp)+5.36+4.96

Q1: Comparison with fine-tuning methods.

Thank you for your constructive suggestion. We include HELO-APR [1], a fine-tuning-based cross-language program repair method, as an additional baseline. As shown in the table below, RepairChemist outperforms HELO-APR by 5.86pp on Rust. HELO-APR was fine-tuned on three NVIDIA A100 80GB GPUs, while RepairChemist performed skill distillation using the GPT-5.5 API. Under our experimental setup, HELO-APR required 3.52× as much fine-tuning time as RepairChemist required for skill distillation. During inference, RepairChemist incorporates retrieved skills into the context, resulting in 3.12× the token consumption of HELO-APR.

Compared to fine-tuning, RepairChemist offers two key advantages: (1) Cross-Model and Cross-Language Adaptability. Distilled skills can be reused across different repair models and target languages without separately fine-tuning model parameters for each model-language combination. (2) Incremental Extensibility. RepairChemist can incrementally incorporate new repair skills to cover additional defect patterns without further model fine-tuning. Furthermore, RepairChemist can be combined with fine-tuning: RepairChemist transfers general repair knowledge across languages, while fine-tuning enhances the model's ability to repair language-specific errors. The relevant discussion has been added to the revised manuscript (Lines 318 and 769).

[1] HELO-APR: Enhancing Low-Resource Program Repair through Cross-Lingual Knowledge Transfer (ISSTA 2026)

Methodresolved rate (%) ↑Training / Distillation Time ↓Inference Tokens ↓
Direct Repair65.37–2380.91
HELO-APR70.2420 h 11 min2257.36
RepairChemist76.105 h 44 min7050.52

Q2: Keep/Propose/Refine contribution and number of retrieved skills.

Thank you for your constructive suggestion. We provide additional analyses of the skill distillation strategies and skill retrieval across different target languages.

(1) Keep/Propose/Refine Strategy. We conduct an ablation study to evaluate the effectiveness of the Keep/Propose/Refine strategy. Retaining only Propose (removing both Keep and Refine) increases the number of skills to 4,353, leading to a sharp increase in skill counts for certain statement types and substantially higher retrieval context overhead, making this setting impractical for standard repair evaluation. Retaining only Keep and Propose (removing Refine) increases the number of skills from 191 to 954 and reduces the average repair success rate from 76.41% to 72.91% (−3.50pp). These results demonstrate that Keep prevents redundant skill generation by reusing existing skills, while Refine further reduces redundancy by consolidating existing skills, thereby mitigating its negative impact on repair performance. The relevant discussion has been added to the revised manuscript (Lines 421 and 443).

Method# SkillsRust (%)Ruby (%)Kotlin (%)Avg. (%)
w/o Refine95471.7172.8974.1272.91
RepairChemist (Full)19176.1075.8077.3276.41

(2) Skill Retrieval Across Target Languages. As shown in the table below, repair tasks in Kotlin, Ruby, and Rust retrieve 136, 107, and 102 distinct skills, respectively. These results indicate that RepairChemist's skill library can provide candidate repair skills for tasks across different target languages. The relevant discussion has been added to the revised manuscript (Line 890).

MetricKotlinRubyRust
Retrieved Skills136107102

Dear Reviewer 1JTT,

Thank you for your time and thoughtful feedback on our paper. As the discussion period draws to a close, we would appreciate knowing whether our rebuttal has adequately addressed your concerns. If you have any remaining questions, we would be happy to provide further clarification. We sincerely appreciate your time and consideration.

Best regards,
Authors

Reviewer_WasyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RepairChemist, which (1) distills language-agnostic repair skills from well-known programming languages (Java) and (2) uses those skills to improve performance in low-resource program repair. RepairChemist is evaluated on xCodeEval, using the Java subset as a source for skill distillation and Rust, Ruby, and Go for testing (43, 44 and 42 tasks, respectively). Different baselines are evaluated, as well as agentic repair (+ agentic coding on SWE-Multilingual).

Critical strengths
  • The paper considers many existing methods (6 published methods) and baselines.
  • The distilled skills seem reasonable token-friendly and help smaller models.
Critical weaknesses

The evaluation set is really tiny, so every instance that flips has a big impact on the final performance. Any language-agnostic skill will fix generic programming errors, rather than errors specific to those low-resource languages (which might incidentally be the most interesting part to fix). Since the skills are written by a strong teacher model, it might just be teacher-to-student transfer, rather than Java-to-X.

Whereas different repair baselines were evaluated, no generic (domain-agnostic) meta-learning methods were compared to evaluate specifically the repair skill distillation approach. The used baselines (like ExpeRepair) might simply be hurt by the choice of model (see also the first point on teacher-to-student distillation). The pairs (baseline, weak model) and (direct repair, stronger model) were compared, but not (baseline, stronger model). This is a significant gap in the evaluation. Figures 2 and 3 already show that better models benefit less from the domain-agnostic skills.

Questions for authors
  • Did you investigate the effect of teacher-to-student distillation (using a weaker teacher to distill skills)?
  • Did you investigate the delta with baseline methods on stronger models?
  • How sensitive is RepairChemist to specific errors being available in the "training" set?
4 comments

Thank you for your detailed review and constructive suggestions. We hope that our responses below adequately address your concerns.

W1: Evaluation set size, language-specific errors, and knowledge transfer sources.

We appreciate your insights regarding evaluation scale, language-specific errors, and knowledge transfer. We address these points through additional experiments and analyses below.

(1) Evaluation Scale. Our evaluation employs two widely used public code repair benchmarks, comprising 861 tasks from xCodeEval and 129 tasks from SWE-bench Multilingual. In code repair and Code Agent evaluation, benchmarks with relatively limited numbers of tasks are also common. For example, QuixBugs, Terminal-Bench 1.0, and AgentIssue-Bench contain 40, 80, and 50 tasks, respectively. Moreover, with Qwen3-Coder-30B, RepairChemist improves performance by 14.63pp and 12.39pp on the two benchmarks, respectively, netting 131 and 16 additional successful repairs, further supporting the reliability of our findings.

To further address your concern, we extend our evaluation to the full Rust subset of Multi-SWE-bench, comprising 239 repository-level repair tasks. The results show that RepairChemist improves the resolved rate of mini-SWE-agent from 5.9% to 10.9%, providing additional evidence of its effectiveness on repository-level repair tasks. The relevant discussion has been added to the revised manuscript (Line 407).

(2) General and Language-Specific Errors. We agree that language-agnostic skills primarily facilitate the transfer of general repair knowledge. However, we argue that general programming errors remain an important challenge in low-resource language program repair. This is because such errors encompass diverse defect patterns and repair strategies, which manifest in different code structures and implementations across languages, making them difficult to identify and fix. Consequently, models need to acquire relevant repair knowledge from extensive bug-fix pairs, yet the scarcity of repair data in low-resource languages limits their ability to learn and apply such knowledge [3,5]. Meanwhile, some language-specific errors involve explicit syntactic or type constraints, which can be mitigated through fine-tuning or compiler feedback [1,2,4,6].

(3) Source of Knowledge Transfer. RepairChemist aims to distill repair knowledge from Java bug-fix pairs without requiring a stronger distillation model. To validate this, we evaluate Qwen3-Coder-30B, DeepSeek-V4-Flash, and GPT-5.5 under a setting where each model serves as both the skill distillation model and the repair model. Qwen3-Coder-30B and DeepSeek-V4-Flash achieve performance comparable to GPT-5.5-distilled skills on xCodeEval (75.56% vs. 76.41% and 88.80% vs. 88.95%, respectively; see Q1). Meanwhile, GPT-5.5 achieves an average improvement of 2.32pp across all three target languages (see the table below). These results suggest that RepairChemist does not require a stronger model for skill distillation to achieve competitive repair performance. The relevant discussion has been added to the revised manuscript (Line 487).

MethodRustRubyKotlinAvg.
Direct Repair95.6097.9691.6995.08
RepairChemist98.5499.4294.2597.40

[1] Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language (COLM 2026)

[2] RustAssistant: Using LLMs to Fix Compilation Errors in Rust Code (ICSE 2025)

[3] HELO-APR: Enhancing Low-Resource Program Repair through Cross-Lingual Knowledge Transfer (ISSTA 2026)

[4] Rust-lancet: Automated Ownership-Rule-Violation Fixing with Behavior Preservation (ICSE 2024)

[5] Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models(EMNLP 2023)

[6] TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer (ICML 2021)

W2: More baselines and stronger-model comparisons.

Thank you for suggesting broader baseline comparisons, including general experience learning methods and evaluations on stronger models. We have conducted additional experiments to address these concerns, as detailed below.

(1) Comparison with General Experience Learning Methods. We introduce ACE [1], a domain-agnostic experience learning method, as an additional baseline. As shown in the table below, RepairChemist consistently outperforms ACE across Rust, Ruby, and Kotlin, achieving a higher average resolved rate (76.41% vs. 69.54%). These results demonstrate that, compared to general experience learning methods, RepairChemist more effectively guides program repair in low-resource languages by structuring historical repair experiences into defect patterns and repair actions. The relevant discussion has been added to the revised manuscript (Lines 263 and 312).

MethodRustRubyKotlinAvg.
Direct Repair65.3758.3161.6661.78
ACE71.7168.2268.6969.54
RepairChemist76.1075.8077.3276.41

[1] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models (ICLR 2026)

(2) Baseline Comparison with Stronger Models. We further evaluate representative baselines, including ExpeRepair, using DeepSeek-V4-Flash. The results show that RepairChemist continues to outperform existing baselines when using a stronger model. Detailed results and analysis are provided in Q2.

Q1: Skill distillation with weaker models.

To assess RepairChemist's dependence on the distillation model, we use Qwen3-Coder-30B and DeepSeek-V4-Flash for both skill distillation and repair inference. The results on xCodeEval show that self-distilled skills achieve average resolved rates comparable to those obtained with GPT-5.5-distilled skills for both models (75.56% vs. 76.41% and 88.80% vs. 88.95%, respectively). These findings demonstrate that RepairChemist does not rely on a stronger distillation model. Even when the same model is used to extract repair skills from Java bug-fix pairs and apply them during repair, RepairChemist achieves comparable repair performance. The relevant discussion has been added to the revised manuscript (Line 487).

Distillation ModelRepair ModelRustRubyKotlinAvg.
GPT-5.5Qwen3-Coder-30B76.1075.8077.3276.41
Qwen3-Coder-30BQwen3-Coder-30B75.1275.5176.0475.56
GPT-5.5DeepSeek-V4-Flash93.1784.5589.1488.95
DeepSeek-V4-FlashDeepSeek-V4-Flash93.6684.2588.5088.80

Q2: Comparisons on stronger models.

Thank you for your suggestion. We have conducted additional baseline comparisons using DeepSeek-V4-Flash, with the results presented in the table below. The stronger model already achieves a high resolved rate, and some methods underperform Direct Repair on certain languages, such as ExpeRepair and ThinkRepair on Rust, and LANTERN on Kotlin. This may be due to interference introduced by additional repair experience guidance or code translation. In contrast, RepairChemist consistently outperforms existing baselines across all three languages, achieving an 8.6% relative improvement on Ruby, where the model's repair performance remains comparatively weaker. The relevant discussion has been added to the revised manuscript (Lines 365 and 748).

MethodRustRubyKotlinAvg.
Direct Repair92.2077.8484.9885.01
ExpeRepair90.7383.0985.9486.59
ThinkRepair91.2282.2285.6286.35
LANTERN92.6879.0084.6685.45
RepairChemist93.1784.5589.1488.95

Q3: Sensitivity to specific errors in "training" data.

Thank you for raising this important question. We re-distill the skills using a different set of Java bug-fix pairs of the same size, while keeping all other experimental settings unchanged. The results show that skills distilled from the two sets of Java bug-fix pairs achieve comparable repair performance on xCodeEval (76.41% vs. 77.65%). This may be attributed to two factors. First, different sets of Java bug-fix pairs may share many common defect patterns, allowing both to yield effective repair skills. Second, for defect patterns not covered by the skill library, RepairChemist can still leverage the model's own repair capabilities through the fallback skill to generate patches, thereby mitigating the impact of incomplete defect pattern coverage on repair performance. The relevant discussion has been added to the revised manuscript (Lines 491 and 837).

Distillation DataRustRubyKotlinAvg.
Original Java Bug-Fix Data76.1075.8077.3276.41
Alternative Java Bug-Fix Data77.5678.7276.6877.65

Dear Reviewer Wasy,

Thank you for your time and thoughtful feedback on our paper. As the discussion period draws to a close, we would appreciate knowing whether our rebuttal has adequately addressed your concerns. If you have any remaining questions, we would be happy to provide further clarification. We sincerely appreciate your time and consideration.

Best regards,
Authors

Dear Reviewer Wasy,

With only one day remaining in the discussion period, we sincerely look forward to your follow-up. If you have any remaining questions, we would be happy to provide further clarification. We sincerely appreciate your time and consideration.

Best regards,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their valuable feedback and constructive suggestions. Based on this feedback, we made the following revisions to our paper:

1. Expanded Baseline Comparisons and Benchmark Coverage

  • Added ACE as a general experience learning baseline. (Lines 263 and 312, Reviewer Wasy)
  • Added a comparison with HELO-APR in terms of performance and costs. (Lines 318 and 769, Reviewer 1JTT)
  • Added baseline comparisons using the stronger model DeepSeek-V4-Flash. (Lines 365 and 748, Reviewer Wasy)
  • Extended the repository-level evaluation to include 239 Rust tasks from Multi-SWE-bench. (Lines 407 and 794, Reviewer Wasy)

2. Expanded Ablation Study and Skill Retrieval Analysis

  • Added ablation experiments and analysis of the Keep/Propose/Refine strategy. (Lines 421 and 443, Reviewer 1JTT)
  • Added an analysis of the number of distinct skills retrieved for each target language. (Line 890, Reviewer 1JTT)

3. Robustness of Skill Distillation

  • Added experiments with different skill distillation models, including settings where the same model performs both skill distillation and program repair. (Lines 487 and 809, Reviewer Wasy)
  • Added an experiment comparing repair performance using two different distillation datasets. (Lines 492 and 837, Reviewer Wasy)
  • Added an experiment using Rust instead of Java as the source language for skill distillation. (Lines 494 and 851, Reviewer 1JTT)

In the revised manuscript, all modifications are highlighted in blue. We once again thank all reviewers for their time and effort in helping us improve our manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.