acceptodds
Withdrawn from ICLR 2027

REPAIR: Learning What to Optimize for Efficient Ascend Kernels

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

High-performance kernels are essential for fully exploiting modern accelerators. Large language models (LLMs) offer a promising route to automated kernel generation and optimization, but struggle with the specialized requirements of Ascend kernels. Existing end-to-end approaches, including reinforcement-learning-based and agent-driven optimization, yield limited performance gains in this setting. We argue that a key challenge lies in coupling optimization decision-making with code implementation, requiring models to jointly determine what to optimize and how to implement the corresponding changes. To address this challenge, we propose REcursive kernel oPtimizAtion by Identification and Revision (REPAIR), a recursive optimization framework that explicitly decouples these two responsibilities. At each iteration, Identification selects an actionable optimization direction without modifying the kernel, while Revision takes this decision as input and implements the corresponding code changes. Building on this separation, we leverage publicly available agents to generate data for both stages and train a compact LLM with reinforcement learning specifically for Identification. This design focuses RL on learning which optimization to pursue, rather than jointly learning optimization decisions and complex code modifications, allowing the compact model to specialize in decision-making while delegating implementation to Revision. Experiments on 253 distinct Ascend kernels from MultiKernelBench show that REPAIR achieves up to 34.8× speedup over the original kernel code, outperforming the 9.5x for the baseline Codex agent.

Primary area:
reinforcement learning
Keywords:
Kernel optimization, Reinforcement learning, Large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QjmTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents REPAIR, a recursive optimization framework for Ascend kernels. REPAIR decouples the optimization process into two stages: identification, which diagnoses the current kernel and proposes an actionable direction for the next round of optimization; and revision, which carries out the actual kernel code changes according to the plan generated in the identification stage. REPAIR instantiates the identification module with a (relatively) small LLM and delegates the revision to off-the-shelf coding agents powered by larger LLMs. It focuses on learning the identification LLM to propose effective and actionable plans via data generation and OPSD. Experimental results on MultiKernelBench show that the identification LLM can learn high-quality optimization suggestions that simultaneously preserve correctness.

Critical strengths
  1. This work identifies the core challenges of Ascend kernel optimization in entangled optimization planning and execution, with the proposed decoupling framework providing a novel aspect to approach the challenge.
  2. The experimental results demonstrate the empirical strength of learning a (relatively) lightweight optimization proposal module.
Critical weaknesses
  1. Some details of the identification module. In the abstract, it states that "Identification selects an actionable optimization direction"; However, in Sec 4.1, it seems that "Consequently, interdependent changes are combined into one internally consistent plan rather than selected independently" suggests that the generated plan might be composed of multiple possible directions. Is there any constraint on the scope and coverage of the plan in terms of direction, or any empirical results suggesting that one direction per turn is better than one plan that points to multiple edits?

  2. While the paper claims that REPAIR is conducted for multiple iterations, there are no experimental details or results on the test-time performance across multi-turn runs.

  3. Though the authors argue that "a key challenge lies in coupling optimization decision-making with code implementation, requiring models to jointly determine what to optimize and how to implement the corresponding changes.", it seems to me that this challenge is not unique to Ascend kernels (e.g., complex CUDA kernels from KernelBench level 2 & 3); thus, limiting the experiment to Ascend kernels only hinders the general applicability of the proposed framework.

Questions for authors

Please see the weaknesses above.

1 comment
AuthorsOpenReview ↗

Short response to major concern

Thank you for these insightful questions. Regard your question about details of the identification module: one optimization direction does not necessarily correspond to a single code modification. In kernel optimization, meaningful improvements often require several interdependent changes, such as adjusting task ownership, data residency, and memory transfers together. Identification therefore produces one coherent optimization plan, which Revision implements as a whole while preserving the necessary constraints. This also motivates the recursive design: once a performance bottleneck is addressed, the kernel's execution characteristics change, and the next optimization decision should be made based on the resulting implementation rather than the original one. We agree that multi-turn optimization trajectories are important for demonstrating this mechanism empirically. While Ascend C provides a particularly compelling setting due to its specialized programming requirements and limited training data, the underlying challenge of separating optimization decisions from implementation also arises in complex CUDA kernels. Exploring this generality is a valuable direction for future work.

Reviewer_Q9nHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes REPAIR, a two-stage framework for optimizing Ascend C kernels on Huawei Ascend 910B4 NPUs. Existing methods usually ask a single model to decide what to optimize and rewrite the kernel code at the same time, which often fails on Ascend C because of complex memory management and very little public training data.

To fix this, the authors split kernel optimization into two steps: Identification (proposing an optimization plan) and Revision (editing the code). They use external agents (Codex and Claude Code) to generate candidate plans, pick the fastest working one for each operator as the golden label, and train small open-source models for the Identification step using SFT and On-Policy Self-Distillation (OPSD).

Critical strengths
  1. Separating high-level optimization planning from low-level Ascend C coding makes sense, since small open-source models easily break syntax when editing long host and kernel files directly.
  2. The 7 Ascend optimization categories in Section 4.1 and the hardware work breakdown in Equation (6) and Appendix B.2 are clear and practical.
Critical weaknesses

W1. The numbers in Table 1, Figure 2, Section 5, and Appendix A contradict each other.

  1. Table 1 (Page 8) and Figure 2 (Page 9) show different validation results for the exact same setup (Qwen3-8B + Codex). Their training numbers are identical (Comp.: 0%, 100%, 100%; Speedup: 0, 53.3x, 37.6x), but on the validation set (n = 10):
    • SFT + Codex compilation rate is 10.0% (1/10) in Table 1, but 90% (9/10) in Figure 2(c) (and Section 5.3 Line 426 also says SFT compiles on most evaluation operators).
    • OPSD + Codex compilation rate is 100.0% in Table 1, but 90% in Figure 2(c).
    • OPSD + Codex speedup is 34.8x in Table 1 (and in the Abstract/Intro), but 38.9x in Figure 2(d).
  2. The dataset size keeps changing across the paper:
    • Section 5.1 (Page 7, Line 346) and the columns in Table 1 say there are 253 operators in total (243 train + 10 val, 972 candidates). But just 12 lines later on the same page (Line 362), in Table 1's caption, and in Appendix A.1 (Page 13, Line 658), it says 243 operators in total and 233 training examples (932 candidates).

W2. Table 1 is missing the teacher agent baseline under the two-stage setup. The main claim in Section 5 (RQ1) is that splitting Identification and Revision works better than end-to-end generation. However, Table 1 only compares an untrained end-to-end Codex agent (9.5x) against the trained Qwen3-8B + Codex pipeline (34.8x), while the untrained Qwen3-8B + Codex baseline gets 0%. Since the training labels actually came from running external agents (Codex / Claude Code) in the two-stage Identification + Revision setup—and Section 6 (Line 481) admits that Qwen3-8B does not beat that teacher agent—the authors should have included the two-stage teacher agent (Codex/Claude Identification + Codex Revision) in Table 1. Without it, we cannot tell how the trained 8B model compares to its teacher, or whether the two-stage split itself helps without training.

W3. The validation set is too small (n = 10). Out of 243 (or 253) operators, 233 (or 243) are used for training and only 10 are left for validation (and Figure 3 shows these same 10 operators are also used to check validation loss during training, with no separate test set). Evaluating on just 10 operators is too small to draw reliable conclusions.

Questions for authors
  1. Why do Table 1 (Page 8) and Figure 2(c, d) (Page 9) have different validation numbers for Qwen3-8B (SFT Comp.: 10.0% vs. 90%; OPSD Comp.: 100.0% vs. 90%; OPSD Speedup: 34.8x vs. 38.9x)? Also, is the dataset 253 total (243 train) or 243 total (233 train)?

  2. How does the teacher agent (Codex / Claude Code) perform in the two-stage Identification + Revision setup in Table 1? Also, how is Speedup averaged across operators, and why does SFT get 0* speedup in Figure 2(d) when 90% of its validation kernels compile?

1 comment
AuthorsOpenReview ↗

Short response to major concern

Thank you for your careful examination and constructive feedback. We appreciate your observations regarding the reported experimental results and will conduct a more detailed review of the underlying records and evaluation procedures to ensure clarity and consistency. Regarding the teacher-agent baseline, our central motivation is to investigate whether the optimization knowledge demonstrated by capable coding agents can be transferred into a compact model that learns to identify valuable hardware-level transformations, while delegating implementation to a separate executor. We agree that comparing the learned Identification model with its teacher under the same two-stage framework is important for understanding the effectiveness of this learning process. Your suggestions regarding baseline comparisons and broader validation also provide valuable guidance for further strengthening the experimental evaluation.

Reviewer_upfgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes REPAIR, a recursive framework that leverages LLMs to optimize Ascend kernels. The authors argue that existing end-to-end approaches, including reinforcement learning-based and agent-driven methods, achieve limited improvements because they require the model to simultaneously identify optimization opportunities and implement the corresponding code modifications. To address this limitation, REPAIR decouples these responsibilities into two distinct stages: an Identification stage, which determines actionable optimization directions without modifying the kernel, and a Revision stage, which implements the proposed changes. These stages are applied iteratively to progressively improve kernel performance.

Critical strengths
  1. S1. Separation of optimization planning and implementation. The idea of decoupling the agent responsible for identifying optimization opportunities from the agent responsible for implementing code modifications is intuitive and well motivated. I find this design choice promising, as it could potentially simplify the optimization process and allow each component to focus on a more specific task. However, as discussed in the weaknesses below, I believe additional experimental evidence is needed to establish the effectiveness of this separation.

  2. S2. Interesting problem and practical application. The paper addresses an interesting and practically relevant problem by exploring how LLM-based agents can be used for low-level kernel optimization. Improving the efficiency of hardware-specific kernels is an important application, and investigating automated approaches to this problem is a valuable research direction.

Critical weaknesses
  1. W1. Clarity of the problem formulation. Although the problem formulation appears technically well structured, I found it somewhat difficult to follow, particularly for readers who may not be familiar with Ascend kernel optimization. The authors could simplify the presentation and provide a more intuitive explanation of the optimization objectives, constraints, and overall workflow before introducing the formal details. A concrete example illustrating how a kernel is progressively optimized through the proposed framework would also improve accessibility.
  2. W2. Limited discussion of the optimization strategy. The proposed framework reminds me of recent AutoResearch approaches, where agents iteratively improve an existing solution based on feedback. However, REPAIR appears to follow a relatively local and incremental optimization strategy, conceptually closer to gradient descent than to broader exploration-based search. The paper would benefit from a clearer discussion of the underlying search strategy and how it compares with alternative approaches that explicitly explore multiple candidate solutions or optimization trajectories.
  3. W3. Unclear exploration–exploitation trade-off. One of my main concerns is the lack of evidence that the proposed framework explores a sufficiently diverse set of kernel implementations. Since the Identification stage iteratively proposes optimization directions based on the current solution, it is unclear whether the framework can move beyond local improvements and discover fundamentally different implementations. How does REPAIR balance exploiting known optimization opportunities with exploring alternative optimization strategies? The authors should provide additional analysis or experiments to characterize the diversity of the generated kernels and the framework's ability to avoid suboptimal solutions.
  4. W4. Limited validation set. The use of only 10 samples for validation appears rather limited, although this may depend on the diversity and complexity of the considered kernels. It is unclear whether these samples constitute an exhaustive or representative evaluation of the target optimization scenarios. The authors should better justify the selection and size of the validation set, clarify its coverage, and discuss whether the reported improvements generalize to a broader range of kernels and workloads.
  5. W5. Insufficient experimental evaluation and benchmarking. The experimental evaluation does not appear sufficiently comprehensive to fully support the effectiveness and generality of the proposed framework. In particular, I would expect a more extensive benchmarking process covering a broader range of optimization scenarios, stronger baselines, and more detailed ablation studies. Additional experiments would help establish whether the reported improvements are consistent across different kernel types and whether they can be attributed specifically to the proposed architectural choices.
  6. W6. Limited comparison with related optimization frameworks. The paper does not sufficiently discuss or compare REPAIR with existing approaches to iterative program optimization, evolutionary search, or agent harness optimization. I see a conceptual connection between these research directions, as they all involve repeatedly proposing, evaluating, and refining candidate solutions. A more comprehensive discussion of related approaches, together with experimental comparisons where applicable, would help clarify the novelty and advantages of REPAIR.
  7. W7. Unclear methodological contributions of the Identification component. While I understand the motivation for separating Identification and Revision into two components, it is less clear what specific methodological innovations make these components effective. In particular, the paper does not sufficiently explain how the Identification component is designed to consistently select useful optimization directions, beyond the general decision to separate planning from implementation. I encourage the authors to provide a clearer breakdown of the algorithmic contributions and explain which design choices are responsible for the observed improvements. Targeted ablation studies would also help distinguish the benefits of the two-stage decomposition from those of the individual components.
Questions for authors
  1. Q1. Relationship to agent harness optimization. The proposed approach reminds me of recent work on agent harness optimization, such as Meta-Harness (https://arxiv.org/pdf/2603.28052v1), where iterative optimization is used to improve agentic systems. Could the authors clarify how REPAIR relates to this line of research and why such approaches are not discussed or considered as baselines? Although the optimization targets differ, the underlying principles appear conceptually related, and a comparison would help better position the contribution.
  2. Q2. Avoiding local optima. Given the iterative nature of REPAIR, how do the authors ensure that the optimization process does not become trapped in local optima? Does the framework incorporate mechanisms for exploring alternative optimization trajectories, maintaining diverse candidate solutions, or revisiting earlier decisions? It would be valuable to understand whether the Identification stage can propose substantially different implementations rather than only incremental improvements to the current kernel.
1 comment
AuthorsOpenReview ↗

Short response to major concern

Thank you for the thoughtful feedback.

Our central motivation is to make kernel optimization decisions explicitly learnable, rather than treating optimization as repeated code generation. In Ascend C, a promising optimization strategy may fail due to implementation difficulties, making it challenging to distinguish poor decisions from poor execution. REPAIR addresses this by separating the reasoning about what should change from how to realize that change. Importantly, an optimization plan may involve coordinated structural transformations, such as reorganizing task ownership to enable on-chip data reuse, rather than merely incremental code edits. Recursion then allows each subsequent decision to reflect the changed performance bottleneck. We agree that understanding how this decision-oriented approach interacts with broader exploration, particularly through multiple optimization trajectories, is an important research question. Your suggestions on stronger comparisons and more comprehensive evaluation will help us develop this direction further.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.