acceptodds
Withdrawn from ICLR 2027

AscendAgent: Evolving Context to Fit the Harness in CANN Kernel Optimization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

High-performance accelerator kernels are fundamental to efficient large language model (LLM) inference, yet optimizing them still requires substantial systems expertise. Existing approaches often optimize workloads independently, making it difficult to reuse experience across input patterns and hardware. We present \ascendagent, a kernel optimization framework that evolves reusable knowledge across workloads while keeping the optimization harness fixed. It organizes API documentation, community implementations, and technical discussions into conditional, composable transformations with expected effects and supporting evidence. \ascendagent operates through two mechanisms. (1) Within each workload, a frozen knowledge snapshot guides evolutionary search over a program graph. The harness proposes changes, checks correctness, and measures latency. Compatible programs share evaluation evidence and explored continuations, while LLM-guided mutations explore changes beyond existing knowledge. (2) After each source search, workflow replay uses recorded proposals and execution outcomes to revise transformation preferences, application conditions, and reusable combinations. Independent confirmation determines which updates guide later workloads. Model weights, the workflow, and the evaluator remain fixed, and each new workload starts with fresh local search state. We evaluate \ascendagent with five language models on four kernel benchmarks spanning Ascend NPUs and NVIDIA GPUs under a budget of candidate compilations per task. With \MainModel, \ascendagent achieves mean best speedups of 1.40 on CANN-Bench and 1.75 on TritonBench-G, compared with 1.21 and 1.36 for the same harness with frozen initial knowledge.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Kernel Optimization, Self-Improvement, AI agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fYUNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AscendAgent, an LLM-guided kernel-optimization system that keeps its search harness fixed while updating a persistent knowledge base across workloads. External documentation, community implementations, and technical discussions are organized into conditional transformations with expected effects and provenance. Within a workload, the system performs evolutionary search over a program graph, compiling and measuring every candidate under a 40-compilation budget. After source workloads, replay compares expected and observed effects and proposes changes to transformation priorities, applicability conditions, and reusable combinations. Updates must pass independent source-side confirmation before they are transferred to held-out targets. The paper evaluates five language models and five search methods on CANN-Bench, KernelBench, MultiKernelBench, and TritonBench-G across Ascend NPUs and NVIDIA GPUs.

Critical strengths
  1. The paper is unusually careful about separating workload-local search from cross-workload knowledge transfer. Frozen-KB Harness and AscendAgent share the same target-time controller, program graph, evaluation reuse, model, tasks, and compilation budget; only the inherited knowledge differs. Program archives and local statistics are reset between workloads, and held-out target outcomes cannot update or select the knowledge snapshot.

  2. The experimental scope is broad. It covers five models, four benchmarks, two hardware ecosystems, and three repeated runs. Failed compilations and final confirmation consume the same budget, and the paper reports terminal speedup, first-hit cost, success rate, run variation, threshold sensitivity, and inclusive learning cost. The results are also presented with appropriate qualifications: AscendAgent has strong terminal speedups, while AB-MCTS-M often reaches the threshold earlier or on more targets.

Critical weaknesses
  1. The reusable knowledge may contain benchmark-specific solution information, but the paper does not provide a sufficient overlap audit. The initial context is collected from community implementations, public repositories, documentation, and technical discussions, while the evaluated benchmarks also contain public kernel implementations and common operator families. The pilot study explicitly uses community implementations for the same types of workloads. No repository-level deduplication, source-to-benchmark mapping, near-duplicate analysis, or experiment removing community code is reported. A retrieved transformation or implementation close to a benchmark solution could directly improve search and would be difficult to distinguish from generalizable optimization knowledge.

  2. The knowledge-evolution mechanism is only evaluated through aggregate downstream outcomes. The paper does not report how many updates are proposed, accepted, rejected, narrowed, merged, or transferred; which transformation families change; or whether the stated mechanism predictions are confirmed. Table 3 shows that updated knowledge outperforms frozen knowledge in terminal speedup, but it does not reveal whether the gain comes from refined conditions, changed priorities, new advice, or repeated source-side tuning. Since source learning and confirmation cost up to 932 minutes, stronger comparisons with alternative ways to use the same source-workload budget would be needed to attribute the gain specifically to replay-based knowledge revision.

  3. The main benchmark matrix supports a more limited conclusion than the framing suggests. AscendAgent does not dominate all objectives or models. With Kimi K3, AB-MCTS-M has better first-hit cost and success rate on CANN-Bench. With GPT-5.6 Sol, AscendAgent is not the best CANN method, and other methods lead several CUDA cells. The main table reports only three runs and no paired statistical tests. Differences are often small after rounding, particularly in the macro speedup. The evidence supports a competitive method with strong terminal quality, but not a generally superior optimizer.

  4. Reproducibility is currently weak for a result matrix of this scale. The paper explicitly states that the current package is a manuscript draft and not a validated benchmark archive. It does not provide the implementation, prompts, initial and evolved knowledge bases, source ordering, accepted update records, benchmark manifests, or measured traces required to verify the results. The method depends on many implementation choices, including proposal allocation, repair, profiling, evidence compatibility, replay, and confirmation. Formal pseudocode is helpful, but it is not enough to reproduce the reported millions of candidate-level decisions and hardware measurements.

Questions for authors

See Weaknesses

Reviewer_v6syOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper optimizes accelerator kernels via evolving, reusable optimization knowledge. It pairs program search with a knowledge base. Knowledge is frozen within each workload. Execution-trace replay proposes updates that require independent confirmation on separate source workloads before transfer. Model weights, workflow, and evaluator stay fixed, and program archives reset per workload, so experience transfers only through knowledge.

Critical strengths

Updating reusable knowledge while fixing the model and harness cleanly separates cross-workload learning from local program search.

Critical weaknesses
  1. The paper includes KernelFoundry alongside the measured baselines in Figure 3, but the legend labels it “KernelFoundry (est.).” I could not find an explanation of how this estimate was obtained or whether the KernelFoundry results in Table 2 were measured or estimated. It is difficult to judge whether the comparison is made on a consistent basis.
  2. The transfer experiments in Figure 3 use 24 source workloads and 48 targets. If these sets are disjoint, this would require at least 72 distinct Ascend workloads, while the only Ascend benchmark described has 53 tasks. The results also seem difficult to reconcile with a 48 task subset of CANN-Bench: Table 8 reports Frozen-KB solving 36 to 43 of 53 tasks, which would imply at least 64.6% success on any 48 task subset under the same evaluation conditions, compared with the reported 43.75%.
  3. The paper’s statements about hardware measurements seem difficult to reconcile with the main results. Beyond the reproducibility statement on p. 10, the AI-use statement says “No new hardware experiments were performed during this revision” and “Unmeasured 40-attempt cells remain NM.” However, Table 2’s 5 × 5 × 4 × 3 matrix contains no NM cells.
  4. The paper reports a budget of 40 compilations for the pilot study, but six of Table 1’s eight correctness rates are not exact multiples of 2.5%. Some of these values could be explained by rounding, but it is unclear to me whether the rates were rounded, averaged over multiple runs, or computed using a different denominator.
  5. Frozen-KB performs worse than plain MCTS on both platforms. This suggests that part of the reported improvement may come from overcoming a weak initial knowledge base. It is unclear how much benefit the evolved knowledge provides over search without a knowledge base. The direct comparison between AscendAgent and MCTS seems central to establishing this benefit, but it is not sufficiently emphasized.
Questions for authors
  1. How were the KernelFoundry estimates obtained, and are its Table 2 results measured or estimated?
  2. Could you provide the source and target workload partitions and reconcile Figure 3’s Frozen-KB success rate with Table 8?
  3. Which Table 2 entries were measured on hardware, and how do they align with the reproducibility and AI-use statements?
  4. How were Table 1’s correctness rates computed?
  5. How much does AscendAgent improve over plain MCTS under matched settings, and what evidence attributes this gain to evolved knowledge?
Reviewer_XDUuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AscendAgent, a framework for kernel optimization with evolving reusable knowledge. Each knowledge item contains a transformation, the condition under which it applies, its expected effect, and supporting sources. Within a single workload, the agent proposes and implements changes with a frozen knowledge base. After the agent finishes a workload, the proposed changes are evaluated by comparing their expected and actual outcomes, and the knowledge base is updated with the revisions that pass an independent confirmation step. The paper reports that the evolved knowledge base achieves higher speedups than the same harness with a frozen knowledge base.

Critical strengths

The paper targets a scenario where LLM kernel generation genuinely fails. Most work in this area is CUDA-only; here the evaluation runs on Ascend NPUs with CANN-Bench alongside three CUDA benchmarks. The study in Table 1 quantifies and highlights the gap between platforms: correctness drops from 80–90% on NVIDIA to 38–68% on Ascend, introducing a real research question for kernel design.

The knowledge representation and the update rule are carefully designed. Each entry pairs a transformation with the condition under which it applies, rather than a finished kernel only. Each proposal also records the expected effect, so replay can check whether a change worked for the reason predicted, providing more explainability to the system.

Critical weaknesses

The method wins on one metric and loses on the other two. On CANN-Bench, AscendAgent does not show a clear advantage over baselines. For example, with Kimi K3, AscendAgent has the best speedup, but AB-MCTS-M reaches the 1.05× threshold sooner and on more tasks, and Table 8 shows AB-MCTS-M solving more tasks in each of the three runs. Moreover, a similar observation happens in TritonBench-G. Therefore, the performance of AscendAgent is questionable.

No ablation study isolates the mechanism claimed as the contribution. The paper’s pitch is that the knowledge base changes while all other components are fixed, and Figure 2 breaks this evolution into four update types. However, the ablation in Table 3 is for four end-to-end systems. Although comparing the last two shows that evolved knowledge helps in aggregate, no run separates the four update types, so the mechanism the paper actually proposes is never tested on its own.

The paper has no related work section, and it never discusses the sub-area it belongs to. The introduction positions AscendAgent only against general agent self-improvement (e.g., AFlow and Darwin Gödel Machine). Still, its closest neighbours are LLM kernel agents that accumulate knowledge across tasks, a line that includes KernelEvolve (arXiv:2512.23236), KernelSkill (arXiv:2603.10085), EvoKernel (arXiv:2603.10846), and AdaExplore (arXiv:2604.16625). These should be discussed and compared. Otherwise, the paper’s positioning is unclear.

Questions for authors

Which workloads produced the knowledge used in Tables 2 and 3? Appendix C says the transfer experiment uses 48 targets and 24 source workloads per platform, but I do not find a source for them. Are they from the benchmark itself? Could the authors state the source/target splits for every table and confirm that no evaluated task was also a source workload?

How can the individual components be shown to work? See weakness 2, an ablation study on each component of the system is helpful for readers to understand the mechanism and contribution of the design.

Why does AscendAgent trail AB-MCTS-M on C40 and R40, and what would close the gap? See weakness 1: AscendAgent leads only on terminal speedup. Is this a property of the method, for example, knowledge retrieval costing early exploration, or a tuning issue?

Does the agent keep improving with a larger budget? All results use B = 40, and the curves in Figure 3 have not flattened at 40. Reporting results with larger budgets would show how the advantage changes: does it grow or saturate?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.