acceptodds
Under review as a conference paper at ICLR 2027

Klineage: Recovering the Missing When of Kernel Optimization by Deoptimizing Experts

Abstract

LLM-based agents are increasingly used to generate GPU kernels, but they often struggle to determine when an optimization is sound because its required code state and dependencies are implicit in expert implementations. We introduce Klineage, which learns this missing “when” knowledge from expert kernels: instead of relying on forward rollouts, Klineage walks expert implementations backward through validation-gated simplifications and reverses each accepted step into a reusable optimization skill. Each skill records not only the optimization intent, but also when to apply the optimization technique, including where it applies in code, what conditions made it valid, what effect it has, and what failures its assumptions avoid. A downstream LLM materializes these skills on new code surfaces under the same compile/correctness/profile gate. This guidance on when to apply each optimization can help downstream models to generate higher-performance kernels. On five expert workloads across two NVIDIA architectures, these lineage-derived skills serve as an effective optimization curriculum, exceeding recent memory-based LLM-kernel baselines in both final kernel quality and optimization efficiency under the same fixed budget. We also demonstrate that the Klineage framework extends beyond NVIDIA GPUs to Ascend NPUs.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
GPU kernel optimization, LLM agents, procedural memory, automated skill construction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qX11OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces KLineage, a framework for constructing reusable, conditional optimization memory by progressively deoptimizing expert kernels. Accepted simplifications are reconstructed as validated forward transitions and abstracted into SkillCards containing optimization intents, code anchors, implementation carriers, preconditions, risks, and verification evidence. During online optimization, an agent retrieves skills according to the target workload, language, platform, and current optimization state, then applies them under compilation, correctness, and profiling checks.

Critical strengths
  1. Deoptimizing expert implementations provides a concrete and potentially valuable approach to acquiring optimization knowledge. The paper identifies a practical limitation of kernel-generation agents: knowing an optimization technique does not imply knowing whether the current program state satisfies its prerequisites. Constructing intermediate states through deoptimization, followed by validated forward reconstruction, offers a practical way to expose dependencies that are implicit in the final expert implementation.
  2. The evaluation includes meaningful transfer evidence beyond reconstruction of the source workloads. The paper explicitly excludes the transfer operators and broader benchmark problems from skill induction. It reports a 1.73× geometric-mean speedup over the no-skill baseline on five H100 transfer targets. On the Ascend benchmark, the number of correctly solved problems increases from 68 to 100, while the number outperforming the reference increases from 22 to 93.
Critical weaknesses
  1. The comparisons change expert information, knowledge representation, validation, and session organization simultaneously. They therefore demonstrate the effectiveness of the complete system, but do not isolate the benefits of backward deoptimization or explicit preconditions.
  2. Failed or slower intermediate candidates are expected during search and do not themselves establish a limitation of forward search. Although AdaExplore provides a search-based baseline, superiority over stronger population-based evolutionary search with comparable expert information remains unestablished. Efficiency claims also require considering offline induction costs and differences in caching and search budgets.
Questions for authors
  1. Isolate the benefits of backward construction and explicit conditions. Using the same expert sources and comparable context budgets, how does KLineage compare with directly summarized SkillCards without deoptimization, and with otherwise identical cards that omit preconditions and conditional scope?
  2. Does the advantage persist under stronger search and comparable total costs? Compare against population-based evolutionary search given the same expert information, including a broader-budget setting.
  3. How similar are the held-out benchmark tasks to the expert kernels used for skill induction? While the paper states that evaluation tasks do not contribute to induction, does it analyze overlap in implementation sources, computational structure, or reusable suboperators? A breakdown of transfer gains by source–target similarity would help clarify the scope of generalization.
3 comments
AuthorsOpenReview ↗

Response (1/2)

Thank you for your valuable comments.

Q1. Isolating backward construction and explicit conditions.

Section 4.3 and Appendix C, Table 9 compare conditioned skills with direct expert-source prompting under a 10 USD budget, where source-prompted agents spent most of their context reasoning through C++ templates and MMA layouts without recovering the full optimization structure. To separate backward deoptimization, explicit preconditions, and conditional retrieval under identical expert information and comparable context budgets, we evaluate five variants on all 100 MultiKernelBench Fusion tasks on Ascend with DeepSeek-V4.1-Flash and Codex (Section 4.6, Table 5). Importantly, Without KLineage is not a zero-guidance baseline: in both Section 4.5 and Section 4.6, it receives the exact same five expert kernel source files as few-shot examples, without SkillCards. Direct-summary SkillCards summarize those same expert sources using the identical schema and roundtrip validation without Guided Deoptimization, while the two condition ablations keep full implementation guidance and session setup:

VariantChange from full KLineageCorrectPaired speedup vs full
Full KLineageComplete construction and reuse pipeline1001.00×
Without conditional retrievalRetrieve full cards without workload or state filtering920.81×
Without explicit conditionsRemove precondition text from the retrieved cards890.73×
Direct-summary SkillCardsSummarize the same expert sources into validated cards without deoptimization850.58×
Without KLineageProvide the same 5 expert sources directly as few-shot examples680.30×
  1. Structured SkillCards improve substantially over raw expert few-shot prompting. All five variants use the same five expert kernels, yet raw few-shot prompting solves 68 of 100 tasks at 0.30× paired speedup vs. full KLineage, whereas all SkillCard variants solve 85 to 100 tasks at 0.58× to 1.00×.
  2. Explicit preconditions and conditional retrieval mainly drive performance recovery. Removing either keeps the same implementation guidance and session structure, reducing correctness from 100 to 92 or 89 while lowering paired speedup more sharply from 1.00× to 0.81× or 0.73×.
  3. Backward deoptimization outperforms direct summarization. Using the same expert sources, schema, and validation gate without deoptimization drops correctness from 100 to 85 and paired speedup from 1.00× to 0.58×. Guided Deoptimization exposes intermediate code states and ordering dependencies that stay entangled in the final expert code.

Q2. Stronger search and comparable total costs.

We compare KLineage with the official OpenEvolve evolutionary search framework on all 100 Ascend MultiKernelBench Fusion tasks using DeepSeek-V4.1-Flash, with population size 5, migration interval 10, and the default five-island setup (Section 4.7, Table 6). For fairness, OpenEvolve receives the exact same five expert kernel sources and hardware descriptions in context under 12 USD and 60 USD total online API budgets across the 100 tasks:

Online budget for 100 tasksMethodCorrect / 100Geomean speedup vs PyTorch
12 USDOpenEvolve + expert sources170.29×
12 USDKLineage871.72×
60 USDOpenEvolve + expert sources250.48×
60 USDKLineage1002.47×
  1. Evolutionary search is orthogonal to KLineage. Our core focus is how to automatically acquire reusable optimization skills offline, while online Scope-Conditioned Reuse can also incorporate evolution-based methods in future work. In this experiment, raising OpenEvolve's online budget from 12 to 60 USD increases its correct kernels from 17 to 25 and speedup from 0.29× to 0.48×, compared with 87 and 100 correct kernels and 1.72× and 2.47× speedups for KLineage. This gap mainly reflects that OpenEvolve is built around LLM endpoint calls without a full loop-based agent harness such as Codex, whereas Table 5 in Q1 isolates the gain from KLineage's skills under the same Codex harness. We see integrating techniques like evolution-based search into our system for more efficient online skill reuse as future work.
  2. Total cost remains small once offline induction is amortized. Skill induction is a one-time offline cost per platform: 197 USD on Ascend 910B, 131 USD on H100, and 92 USD on RTX PRO 6000. Across the 110 Ascend tasks in this paper, covering the five main kernels, five transfer operators, and 100 MultiKernelBench tasks, the 197 USD offline cost comes to 1.79 USD per task.
AuthorsOpenReview ↗

Response (2/2)

Continued: Q3.

Q3. Source–target similarity and the scope of generalization.

Section 4.5 evaluates five held-out transfer operators: KDA, Sparse Attention, Top-P, Fused Add-RMSNorm, and GQA. None of them are used during skill induction, which draws only from GEMM, Conv2d, FMHA, GDN, and Top-K:

Transfer targetShared computational basisKey difference from induction kernels
KDADelta-rule recurrence with GDNDifferent state update and gating recurrence
Sparse AttentionAttention and softmax with FMHAIndexed block or token gather instead of dense tiles
Top-PSelection and filtering with Top-KCumulative probability threshold and prefix scan
Fused Add-RMSNormVectorized reduction patternsNo normalization kernel in the five expert sources
GQAAttention computation with FMHAGrouped KV-head broadcasting layout

Although these operators share low-level primitives with the expert kernels, their indexing, dataflow, and synchronization differ substantially. Both without KLineage and with KLineage receive the same five expert kernels in context, isolating the transfer benefit of conditioned skills: 1.73× speedup on H100, and 1.41× speedup on Ascend while also producing a working Sparse Attention kernel where the baseline fails.

To measure similarity on the larger benchmarks, we also inspected the reference programs for all 100 KernelBench Level 2 tasks on NVIDIA and all 100 MultiKernelBench Fusion tasks on Ascend, which use the same 100 operator compositions:

Computational familyKernelBench Level 2MultiKernelBench FusionRelation to the 5 expert kernels
GEMM / Linear compositions3737Same base family as GEMM plus fused epilogues
Conv2d compositions2121Same base family as Conv2d plus fused ops
Conv3d compositions12123D convolution; no 3D expert in library
ConvTranspose2d compositions1111Transposed stencil; no ConvTranspose expert
ConvTranspose3d compositions19193D transposed convolution; no 3D or transpose expert

Out of the 100 tasks, 58 combine GEMM or Conv2d with downstream activations, normalizations, reductions, pooling, or residual adds; for instance, task #14 fuses matrix multiplication with scaling and row-wise sum, and task #92 fuses Conv2d with GroupNorm, nonlinearities, residual addition, and LogSumExp. The other 42 tasks are built around Conv3d, ConvTranspose2d, or ConvTranspose3d, which have no matching operator among the five expert kernels. Even so, KLineage produces 100 out of 100 correct kernels on both platforms. Compared with without KLineage, which receives the same five expert sources as few-shot examples, KLineage raises Ascend correctness from 68 to 100 and speedup over PyTorch from 0.63× to 2.09×, and raises NVIDIA speedup over torch.compile from 1.05× to 1.32×. Getting all 42 Conv3d and ConvTranspose tasks right shows that lower-level conditioned skills, such as GM-to-UB ping-pong staging, vectorized aligned transfers, tiling and launch partitioning, and explicit DMA/Vector/Cube synchronization, transfer beyond the source operator families.

Dear Reviewer qX11,

Thank you again for your time and constructive feedback on our submission. As the author–reviewer discussion deadline (Nov 18 AoE) approaches, we wanted to gently check in to see whether our rebuttal and revised manuscript have addressed your concerns and might impact your rating.

Please let us know if any question remains or if there is anything else we can clarify before the discussion period closes. We would greatly appreciate your feedback!

Reviewer_a5TbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Klineage, a framework that extracts reusable knowledge about when to apply kernel optimizations from expert implementations. Starting from an optimized expert kernel, it performs stepwise deoptimization under validation constraints, reconstructs accepted simplifications as forward optimization transitions, and abstracts them into structured SkillCards. Each SkillCard records an optimization intent, code-level implementation guidance, applicability conditions, expected effects, and verification evidence. Skills are admitted to the library after roundtrip validation and retrieved according to the target workload, programming language, hardware platform, and previously applied optimizations. The experiments evaluate the collected SkillCards against two baseline methods on five operators, compare expert-source prompting with extracted SkillCards, and examine transfer to other GPUs and broader benchmarks. Separate skill libraries are constructed for NVIDIA GPUs and Ascend NPUs. The authors report improvements in generated-kernel performance and, in some settings, correctness.

Critical strengths
  1. Deoptimizing expert kernels and then reconstructing and validating the optimizations checks whether the current backbone can regenerate the optimized code. Using this process to organize optimization trajectories and transfer them to other operators is an interesting idea.
  2. Workload, programming language, hardware platform, and previously applied optimizations are explicit retrieval dimensions, with retrieval based on verified rather than merely declared scope. These dimensions provide a useful reference for organizing reusable kernel-optimization knowledge.
Critical weaknesses
  1. The paper emphasizes when to apply optimizations, but the experiments compare final performance, which does not sufficiently support this claim. The comparison with existing methods’ representations of applicability is also insufficient, and it remains unclear whether differences in applicability decisions cause the performance gaps.
  2. The paper lacks ablation experiments demonstrating the contributions of guided deoptimization and scope-conditioned reuse. Ablations of SkillCards are also missing.
Questions for authors
  1. What evidence isolates the benefit of applicability conditions from that of additional expert guidance? Are ablations available for guided deoptimization, scope-conditioned retrieval, and key SkillCard fields?
  2. How is the comparison fair when Klineage has access to expert solutions but the other methods do not? Are comparisons with AdaExplore and AccelOpt available on the broader benchmarks in Section 4.5.1?
  3. Figure 9 shows deoptimization chains, but how does a specific code edit lead to the inferred conditions and a complete SkillCard? Please illustrate this connection with one concrete example.
  4. Are the reference implementations timed in Table 6 exactly those used for skill induction and expert-source prompting in Section 4.3? Please clarify any differences.
6 comments
AuthorsOpenReview ↗

Response (1/2)

Thank you for your careful review.

Q1. Separating conditioned skills from expert guidance.

Section 4.3 and Appendix C, Table 9 (Table 7 in the original submission) compare KLineage against direct expert-source prompting under the same $10 online budget. In the trajectories we inspected, agents given raw expert source spent most of their context reasoning through C++ templates and probing MMA layouts without recovering the full set of optimizations.

To separate preconditions, conditional retrieval, and Guided Deoptimization while keeping expert information fixed, we compare five variants on all 100 MultiKernelBench Fusion tasks on Ascend using DeepSeek-V4.1-Flash and Codex under matched context and online budgets (Section 4.6, Table 5). Importantly, Without KLineage is not a zero-guidance baseline: in both Section 4.5 and Section 4.6, it receives the exact same five expert kernel source files as few-shot examples, without SkillCards. All five variants therefore start from the same five expert kernels:

VariantChange from full KLineageCorrectPaired speedup vs full
Full KLineageComplete construction and reuse pipeline1001.00×
Without conditional retrievalRetrieve full cards without workload or state filtering920.81×
Without explicit conditionsRemove precondition text from the retrieved cards890.73×
Direct-summary SkillCardsSummarize the same expert sources into validated cards without deoptimization850.58×
Without KLineageProvide the same 5 expert sources directly as few-shot examples680.30×

These results show three points:

  1. Validated SkillCards help much more than raw expert few-shot code. Given the same five expert kernels, Without KLineage solves 68 of 100 tasks at 0.30× paired speedup relative to full KLineage, whereas every SkillCard variant solves 85 to 100 tasks at 0.58× to 1.00×.
  2. Conditional retrieval and explicit preconditions directly affect when optimizations succeed. Both condition ablations keep the full implementation guidance in anchor and carrier, yet correctness drops to 92 and 89 and paired speedup drops to 0.81× and 0.73×. Without preconditions or state-based retrieval, the agent often applies optimizations before their layout, alignment, or staging requirements hold, leading to compile or test failures and slower fallback code.
  3. Guided Deoptimization works better than direct summarization. Direct-summary SkillCards use the same expert sources, schema, and roundtrip validation, raising correctness from 68 to 85 and paired speedup from 0.30× to 0.58×, but still trail full KLineage at 100 correct and 1.00×. Summarizing only the final expert kernel leaves intermediate code states and ordering dependencies implicit.

Q2. Expert access and comparisons with AdaExplore and AccelOpt.

First, in Section 4.3, Section 4.5, and Section 4.6, Without KLineage already receives the same five expert kernel implementations as few-shot examples. KLineage's gains in these experiments come from turning those implementations into conditioned, roundtrip-tested skills rather than from privileged access to expert code.

Second, AdaExplore and AccelOpt build memory from the model's own forward rollouts: AdaExplore records failure rules from broken or slow MCTS attempts, and AccelOpt keeps high-level trajectory summaries. Neither captures the positive code-state requirements, such as tile alignment, UB buffer lifetime, or prior phase splitting, needed to apply multi-step expert optimizations.

We also ran AdaExplore and AccelOpt on all 100 Ascend MultiKernelBench Fusion tasks from Section 4.5.1 using DeepSeek-V4.1-Flash under the same $24 total online budget, giving both baselines the exact same five expert kernel sources and hardware descriptions in context:

MethodCorrect / 100Faster than PyTorch / 100Geomean speedup vs PyTorch
KLineage100932.09×
AdaExplore + expert sources1780.35×
AccelOpt + expert sources23110.48×

Even with the same expert code in context, both baselines remain far behind KLineage. Monolithic reference code combined with failure rules or rollout summaries still leaves the agent guessing when an expert optimization fits the current kernel, which frequently causes AscendC synchronization and buffer-shape errors.

AuthorsOpenReview ↗

Response (2/2)

Continued: Q3 and Q4.

Q3. How preconditions and a complete SkillCard are derived from a code edit.

For each accepted deoptimization step, we reverse the edit into a forward transition τi:Ki−1→Ki\tau_i: K_{i-1} \rightarrow K_i and give the induction LLM one example SkillCard for the schema, the code diff between Ki−1K_{i-1} and KiK_i, the surrounding code, and the compile, test, and profiling logs.

Take smem_transpose in GDN's cumsum chain in Appendix G, Figure 9, where removing the shared-memory transpose slows the intermediate kernel from 0.012 ms to 0.034 ms, and forward reconstruction achieves a 2.9× speedup over the naive kernel in Appendix H, Table 11. In Ki−1K_{i-1}, threads scan along the sequence axis across a strided [chunk_seq, num_heads] layout. In KiK_i, each chunk is loaded cooperatively and transposed into padded shared memory [num_heads, chunk_seq + pad] before the prefix scan:

// Before, K_{i-1}: strided scan across [chunk_seq, num_heads]
for (int t = 0; t < CHUNK; ++t) {
    acc += buf_in[t * NUM_HEADS + h]; // non-unit stride along scan axis t
    buf_out[t * NUM_HEADS + h] = acc;
}
// After, K_i: cooperative transpose into [num_heads, chunk_seq + pad]
smem_tile[h][t + pad(t)] = gmem_in[chunk_base + t * NUM_HEADS + h];
__syncthreads(); // contiguous per-head rows enable coalesced or warp-local scan

From this diff and the 0.034 ms to 0.012 ms speedup, Skill Induction produces the smem_transpose card:

  • intent: Stage chunked strided inputs through transposed shared memory so the scan axis is contiguous per head.
  • anchor: Chunked loop doing sequential or prefix accumulation along a non-unit-stride tensor axis.
  • carrier: Cooperative coalesced load into smem[head][chunk_seq + pad], CTA barrier, contiguous scan along chunk_seq, and transposed store.
  • preconditions: First, the scan axis is strided in memory while an orthogonal parallel axis such as head or channel is contiguous. Second, num_heads * (chunk_seq + pad) * sizeof(T) fits in CTA shared memory without hurting occupancy. Third, all threads in the cooperative load reach uniform CTA barriers before scanning and before reusing the buffer.
  • effect and evidence: Removes strided global accesses and sets up contiguous rows for warp_shuffle_scan, verified at 0.034 ms to 0.012 ms and 2.9× over naive.
  • risk: Shared-memory bank conflicts if chunk_seq is a multiple of 32 without padding, and out-of-bounds reads on tail chunks when seq is not divisible by chunk_seq.
  • scope and ver: Verified on gdn/cumsum in CUDA on sm90, and admitted after an independent agent given only this card reproduces the speedup and passes numerical checks starting from Ki−1K_{i-1}.

Online, the agent checks whether the current kernel scans or reduces along a strided axis over a tile that fits on-chip memory before applying this card.

Q4. Consistency of the reference implementations.

Yes. For every workload and platform, the reference kernel timed in Appendix C, Table 8 (Table 6 in the original submission) is the exact same expert implementation used for offline skill induction and for expert-source prompting in Section 4.3, Table 9 (Table 7 in the original submission). Only how the agent receives that optimization knowledge changes: KLineage receives conditioned skills induced from that kernel, whereas the baseline receives its source code directly.

Dear Reviewer a5Tb,

Thank you again for your detailed review and insightful questions on our work. With the author–reviewer discussion period closing soon (Nov 18 AoE), we wanted to kindly follow up to see whether our responses and additional experiments have resolved your concerns and might be reflected in your evaluation.

If any point remains unclear or if there is anything further we can clarify before the deadline, please let us know. We would really appreciate your thoughts!

Reviewer_aaZROpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that LLM kernel agents know what optimizations exist but don't know when to use them. And this knowledge of condition is implicit in expert kernels. The paper therefore introduces KLineage, which recovers the knowledge of using optimizations under conditions by walking an expert kernel backward through "deoptimization steps", and stores the "inverse" of each accepted step as a validated forward optimization transition. The resulting lineage forms an executable curriculum from naive kernels to expert kernles. Then, these transitions are summarized into skills for further usage in inference time to boost performance. The method is evaluated on H100, RTX PRO 6000, and Ascend 910B.

Critical strengths
  1. I think the core idea is well motivated and novel in this setting. Using "deoptimization" to generate a lineage from naive kernels to expert kernels is very interesting.

  2. The baseline audit in Appendix D is strong and rigorous. For example, the authors found that several AdaExplore submissions were wrappers that caught the kernel-call failure and returned the PyTorch reference, confirmed this by replacing the fallback with a raise, and then wired an adapter to recover the real underlying kernels so as not to under-credit the baseline.

  3. The experiment is comprehensive, and I think the Ascend evaluation is the most convincing part. The “model lacks platform priors” argument is substantive for AscendC, and the supporting observations (lower admission rate, more deoptimization steps to reach a correct naive kernel) is valuable and meangingful . Moreover, the ablation studies are convincing.

Critical weaknesses
  1. I think authors should provide more information about how the whole framework looks like. At least for me, how you deoptimize kernels, create skill cards and reuse them in inference are not very clear
Questions for authors

Could you please provide more case studies for weakness 1? Thanks.

2 comments

Thank you for your thoughtful comments on KLineage.

KLineage builds on existing coding-agent harnesses such as Claude Code and Codex through a thin tool layer for artifact management, benchmarking, and skill retrieval across accelerator backends. As shown in Figure 2, the workflow has two offline stages, Guided Deoptimization and Skill Induction, and one online stage, Scope-Conditioned Reuse.

In Guided Deoptimization (Section 3.2), we start from an expert kernel in a reference repository such as FlashInfer. An LLM removes or simplifies optimizations one step at a time to produce intermediate kernel versions. Every candidate is compiled, tested for correctness, and profiled until the kernel reaches a naive implementation, such as a CUDA-core GEMM written with standard i, j, and k loops. This produces a verified chain of step-by-step code transitions.

In Skill Induction (Section 3.3), KLineage reverses each accepted simplification into a forward optimization transition, groups related steps, and summarizes them into conditioned SkillCards. Each card records the optimization intent, code anchors, implementation carriers, preconditions, expected effect, risks, scope, and verification record. Every candidate card then goes through roundtrip validation: starting from a deoptimized kernel, a separate coding agent receives only the card and tries to reconstruct the removed optimization, pass numerical tests, and recover the performance gain. Only cards that pass this check enter the library, filtering out steps that are too coarse for a comparable agent to apply reliably.

In Scope-Conditioned Reuse (Section 3.4), the online agent starts from a PyTorch operator and generates a kernel for the target accelerator. At each step, it retrieves admitted SkillCards matching the verified workload, language, hardware platform, and current optimization state. It uses the preconditions to decide whether an optimization applies to the current code and uses the anchors and carriers to implement it, keeping the fastest valid kernel found within the budget.

Case Study 1: Offline Induction on GDN. Appendix G, Figure 9 shows the deoptimization chains for GDN's three sub-kernels: cumsum, fused_fwd, and kkt_solve. Each intermediate kernel gives the concrete before-and-after code and profiling delta used to build a SkillCard. In the cumsum chain, KLineage removes vectorized loads, warp-shuffle scan, and shared-memory transpose in turn; removing the shared-memory transpose increases latency from 0.012 ms to 0.034 ms, which grounds the smem_transpose card in Appendix H, Table 11. Figure 9 also captures ordering dependencies reused online: in fused_fwd, phase decomposition sets up the matrix-computation structure needed before applying MMA and swizzle optimizations.

Case Study 2: Online Transfer to Sparse Attention on Ascend 910B. In Section 4.5, Table 3 and Appendix F, Table 10, Sparse Attention is held out from skill induction. When Codex is given only the five expert AscendC kernel sources as few-shot examples, it fails to produce a valid kernel within the two-hour budget, because adapting monolithic dense-attention code to indexed sparse gather and Ascend Cube/Vector synchronization is error-prone. With KLineage, the agent first writes a correct GM-to-UB baseline and then applies verified AscendC SkillCards from the FMHA and GEMM libraries as their preconditions are met: splitting Cube matrix products from Vector softmax once tile shapes align, adding two-slot ping-pong UB staging and prefetching ahead of compute, and introducing event-ordered cross-stream overlap and workspace reuse across phases. The resulting Sparse Attention kernel is correct and combines 13 optimizations.

Dear Authors,

Thank you for your constructive feedback! I am happy to raise my Presentation, Informativeness score and my rating. I hope AC could accept this paper. Good luck!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.