acceptodds
Under review as a conference paper at ICLR 2027

Experience Is Not Enough: Knowledge-Grounded Evolutionary Search for AscendC Kernel Generation

Abstract

LLM agents have become capable kernel engineers on mainstream GPU stacks, but they struggle on emerging accelerators such as Ascend NPUs, whose programming model is barely represented in pretraining data. Recent NPU agents address this gap largely through experience: they accumulate memory from their own generation and optimization trials and reuse it on later tasks. We argue that experience alone is not enough for kernel generation. An agent can only remember what it has tried, and it rarely tries a mechanism it does not know. As a result, such memory offers little help when a new operator calls for a different mechanism, and even familiar operators raise unseen situations as optimization proceeds. We present AscendEvolve, which grounds every step of an evolutionary kernel search in a large external knowledge base built from platform documentation, toolkit headers, and operator repositories. Because this corpus far exceeds any context window, AscendEvolve indexes it and routes each agent to a small, task-specific slice of evidence instead of the corpus itself. The retrieved knowledge seeds diverse implementation strategies, guides repair when a candidate fails, and constrains how strong candidates are recombined. On 53 Ascend C operators from CANN-Bench spanning four difficulty levels, AscendEvolve achieves an overall score of 71.39, improving over Reflexion and OpenEvolve by 11.6% and 17.2%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Kernel generation, Agentic reasoning, self-improving

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_F13YOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents AscendEvolve, an LLM-based system for Ascend C kernel generation. It argues that experience-based agents struggle to discover implementation mechanisms outside their existing knowledge. The method combines hierarchical, version-aware retrieval from platform sources with tree-based exploration and mechanism-level crossover, using execution feedback to guide subsequent retrieval.

Critical strengths

A well-motivated approach to an important practical problem. Combining version-aware platform evidence with execution feedback directly addresses unfamiliar APIs and implementation constraints. Grounding both exploration and repair makes the approach useful beyond initial code generation.

Critical weaknesses
  1. Comparison fairness and reliability are insufficiently established. The paper does not clearly report matched search budgets, total inference costs, baseline knowledge access, or variability across independent runs. Comparisons with closely related knowledge-augmented Ascend systems are also missing, limiting evidence of advancement over existing approaches.
  2. The distinctive methodological contribution is not sufficiently isolated. Removing the entire knowledge base establishes its usefulness, but does not identify the benefits of version-aware retrieval or bidirectional feedback over simpler retrieval augmentation.
Questions for authors
  1. Please clarigy what search budgets, model-call/token costs, and knowledge access each method receive.
  2. Under matched budgets and knowledge access, does AscendEvolve outperform a simpler retrieval-augmented baseline? Which gains specifically depend on version filtering and experience-guided retrieval?
  3. Prior work documents reward hacking in kernel generation, including bypassed computation and misleading timing measurements [1]. Were final kernels independently checked on hidden or freshly sampled inputs, including the case-specific numerical correction in Figure 8? Please clarify how timing integrity and attribution to the submitted NPU kernels were verified.

[1] Wei Liu, Jiawei Xu, Yingru Li, Longtao Zheng, Tianjian Li, Qian Liu, and Junxian He. Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generations. ICML 2026.

Reviewer_phrVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AscendEvolve combines version-aware platform knowledge retrieval, multi-line exploration, and mechanism-level crossover for Ascend C kernel generation. It evaluates 53 CANN-Bench operators and reports improved aggregate scores over Reflexion and OpenEvolve.

Critical strengths

1. Structured platform evidence. The method separates the executable reference's semantic requirements from the installed environment's API constraints, while retaining source identity and version information during retrieval.

2. Evaluation at multiple levels. The experiments cover 53 operators across four difficulty levels and report build, runtime, correctness, and performance coverage separately. The knowledge-base ablation and stage-wise case studies document both improvements and failure cases.

Critical weaknesses

1. The additional methodological contribution is not sufficiently established. AccelOpt combines search with optimization memory; AKG Kernel Agent uses document-driven knowledge, hierarchical retrieval, and evolutionary search; AscendOptimizer combines hardware feedback with optimization-pattern retrieval. Table 3 tests only the whole knowledge base, and Table 4 lacks an equal-budget continued-search control. The evidence therefore does not isolate the value of version-aware/experience-guided retrieval and crossover beyond additional knowledge and search.

2. The modified evaluation does not sufficiently establish gains in fully correct, efficient kernels. Eq. (4) replaces operator-level compilation credit with case-level successful-run credit and replaces CANN-Bench's HAP score with clipped square-root speedup. Although the hardware-specific motivation is understandable, the impact on method rankings is not assessed. Reporting the fraction of final kernels passing all cases and paired performance would clarify the aggregate gain.

Questions for authors
  1. Under matched knowledge access and total search cost, what benefits remain from the proposed retrieval design and crossover compared with simpler knowledge-augmented search and continued Stage 1 exploration?

  2. What fraction of final kernels pass all cases, and what are the paired speedups on a fixed common set of correct cases? Does the ranking persist under reasonable alternative scoring choices?

  3. What is the unified Stage 2 eligibility rule? Figure 16 requires a fully correct parent, whereas Figures 8, 11-13, and 15 allow partially correct parents. How is Table 4's adoption rate defined?

  4. Which artifact and evaluator configuration produce the 20/20 final result in Figure 7, given that synth_009 is reported at 16/20? Please provide the corresponding per-case replay results.

Reviewer_FBSxOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper aims to address the challenge of generating and optimizing kernels for novel accelerators like the ASCEND accelerator, specifically when an LLM agent’s platform knowledge and past optimization experience are insufficient. It makes a case for how past experience is insufficient for this. Its key idea is to augment evolutionary search with retrieval from documentation, headers, and vendor implementations, using this information to propose implementation strategies, repair failures, and combine promising candidates.

Critical strengths
  • Tackles an important and unsolved challenge in using agents for kernel generation, especially new accelerators
  • The paper is quite easy to read and clearly written.
Critical weaknesses

My major concern with this work is in its novelty. The major contribution of this work seems to be the observation that prior optimization experience is not enough. However, this is quite straightforward and understandable. A range of prior works also use additional available information like rough programming guides, API definitions, installed headers, reference implementations, and vendor-maintained operator libraries For example, AccelOpt, EvoKernel, AgenticCann etc., also use variations of this as a part of their mechanism. Although the paper acknowledges this related work, it does not make the technical distinction sufficiently clear

If the major contribution is terms of how to use this additional information that is more effective, the paper needs to be positioned in this manner, such that the ideas and novelty of how to do this is more apparent. During the comparisons, it is also unclear how the computational budgets compare.

Second, there are limited quantitative comparisons with many prior works such as AccelOpt, EvoKernel, AgenticCann, HAWK, etc. As the number of papers and prior literature in this area grows, it is increasingly important to make novel contributions clear and demonstrable. The baselines compared against also seem quite weak. Comparing the proposed approach against an experience-based agent with straightforward retrieval over the full corpus, would help isolate the benefit of the proposed retrieval and search design. Removing the knowledge base entirely is a substantially weaker baseline for this purpose.

Third, many of the results are somewhat underwhelming. For example, compared to Reflexion, the proposed approach seems only marginally better. If the demonstrated improvements are significant, the numbers need to be placed in sufficient context. The key aspects of the design that actually generate the benefits must be clarified.

Questions for authors

Please comment on my points under weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.