acceptodds
Under review as a conference paper at ICLR 2027

AsmEvo: Agentic Assembly-Level Optimization of AMD GPU Kernels with Functional Equivalence Verification

Abstract

High-performance ML systems increasingly rely on GPU kernels whose editable source is unavailable, generated, or too distant from final machine code to expose remaining optimizations. Existing LLM kernel optimizers and autotuners mainly operate on CUDA, Triton, HIP, or tensor-program source and validate against reference implementations. We study a stricter setting: optimizing an already compiled AMDGPU code object, using as the behavioral reference under a captured launch contract. We present AsmEvo, an agentic assembly-level optimizer for AMD GPU kernels. Given an AMDGPU code object , AsmEvo reconstructs a reassemblable representation, proposes profile-guided low-level edits with a long-horizon agent, rebuilds an ABI-preserving object with regenerated metadata or a conservative in-place patch, and accepts candidates only after capture-scoped equivalence and paired validation against in a generated persistent evaluator. On MI308X, AsmEvo improves all 30 selected KernelBench kernels, reaching geometric-mean and maximum speedup. On MI300X production workloads, it improves all evaluated AITER binaries and vLLM/SGLang Triton assembly kernels, reaching / and / geometric-mean/maximum speedups, respectively, while preserving functional equivalence.

Primary area:
optimization
Keywords:
Assembly, HSACO, Kernel Agent, Compiled

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SfHeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AsmEvo, an agentic framework for optimizing compiled AMD GPU kernels without access to source code. It recovers editable assembly from GPU binaries, employs profiling-guided LLM agents to explore optimizations, and rebuilds candidates while preserving the original launch ABI. A deterministic verification pipeline checks correctness against the original binary under captured execution conditions and validates performance improvements. Experiments on KernelBench and production-derived kernels demonstrate performance gains.

Critical strengths
  • The paper addresses a meaningful post-compilation optimization problem for AMD GPU kernels when source code is unavailable. Targeting compiled code objects complements existing source-level kernel optimization approaches and has practical relevance to deployed GPU workloads.
  • AsmEvo combines assembly recovery, ABI-preserving rebuilding, agentic optimization, and capture-based correctness and performance validation into a unified framework. The evaluation spans both benchmark and production-derived kernels and demonstrates promising performance improvements.
Critical weaknesses
  • The paper integrates existing techniques for binary recovery, assembly optimization, and agentic search, but does not clearly identify the novel technical insights arising from this integration. A more systematic ablation study is needed to clearly establish the contributions of individual components.

  • Correctness is validated against the original binary under fixed captured execution conditions, which may not generalize to unseen inputs or states. The absence of independent validation on held-out inputs raises concerns about the reliability of the optimized kernels.

Questions for authors
  1. What are the key technical contributions of AsmEvo beyond integrating existing optimization techniques? Can the authors provide more systematic ablation studies to demonstrate the benefits of individual components?

  2. How well do the optimized kernels generalize to unseen inputs or execution states? Can the authors provide correctness results on independent test cases beyond the captured executions?

  3. What specific assembly-level transformations account for the reported speedups? Can the authors provide representative before-and-after assembly examples and profiling evidence to explain their effectiveness?

7 comments
AuthorsOpenReview ↗

Response to Reviewer SfHe (1/5): Technical contributions beyond integration [W1, Q1]

[W1, Q1] Technical contributions

Weakness 1: "The paper integrates existing techniques for binary recovery, assembly optimization, and agentic search, but does not clearly identify the novel technical insights arising from this integration."

Question 1: "What are the key technical contributions of AsmEvo beyond integrating existing optimization techniques?"

Thank you for the comment, and for recognizing the practical value and technical difficulty of source-free optimization. AsmEvo's key contributions go beyond combining existing techniques: they address the new problems that arise when source-free AMDGPU binaries are optimized directly. To our knowledge, AsmEvo is the first system to combine agentic search, ABI- and metadata-preserving AMDGPU code-object rebuilding, and capture-scoped evaluation against the original deployed binary (§2). Our technical contributions are the following three.

First, reassemblable binary recovery. Raw disassembly cannot be reassembled. It lacks the declaration layer, such as kernel descriptors and metadata. In addition, PC-relative relocations are resolved to concrete values. Instruction lengths then change on reassembly, which shifts later addresses and breaks branches. AsmEvo therefore reconstructs the declaration layer directly from the kernel descriptor bytes and the AMDGPU metadata note. It also re-symbolizes PC-relative operands. The recovered assembly must pass a byte-level round-trip check before later edits can build on it (§3.3).

Second, ABI-preserving rebuild. Assembly edits often change register, LDS, and scratch usage. Existing assembly-level methods avoid this by restricting the edit space. CuAsmRL, for example, only reorders SASS instructions. AsmEvo instead recomputes the resource fields in the kernel descriptor and AMDGPU metadata, while the kernarg layout, symbol interface, and launch semantics stay frozen (§3.5). This allows broader edits, such as inserting or deleting instructions and changing register usage. The original host launcher can still invoke the output unchanged.

Third, long-horizon search for assembly. AsmEvo extends the single-lineage evolution of AVO (Chen et al., 2026) to assembly optimization. The LLM only proposes edits. A deterministic controller decides correctness and speedup, using ABBA interleaving, A/A calibration, and the lower bound of a bootstrap confidence interval (§3.7). We also extend the single lineage to a tree-structured verified lineage. It supports multi-start exploration, verified composition, and stall-triggered redirection (§3.8). As a result, the 12-hour search accumulates only verified improvements.

These contributions show in the results. AsmEvo improved all 30 KernelBench kernels. It also improved all 8 production kernels from AITER and vLLM/SGLang (1.34× geometric mean, 2.08× maximum). Claude Code had the same model and the same original binaries and captures, but none of this infrastructure. It improved only 2 of the 8 production kernels (1.05×). With the model held fixed, this gap demonstrates the value of the complete AsmEvo framework, although Table 4 does not isolate the contribution of each design individually.

AuthorsOpenReview ↗

Response to Reviewer SfHe (2/5): Contribution of individual components [W1, Q1]

[W1, Q1] Ablation

Weakness 1: "A more systematic ablation study is needed to clearly establish the contributions of individual components."

Question 1: "Can the authors provide more systematic ablation studies to demonstrate the benefits of individual components?"

On more systematic ablations, Table 4 compares search drivers (we have renamed §4.4.3, formerly "Ablation Study", to "Search-Driver Comparison"). Each of its two comparisons targets specific mechanisms. Full vs. Single tests multi-start exploration together with the larger GPU allocation. Single uses one GPU and one direction, and always starts from the current best. Its geometric mean drops from 1.34× to 1.18×, and its improved kernels drop from 8/8 to 6/8. The speedup of kernel_unified_attention_2d drops from 1.96× to 1.05×. Because Single also has a quarter of the GPUs, this comparison does not separate the two factors. Full vs. No Sup. tests stall handling as one bundle. Both use the same model, tools, and GPU cap. No Sup. evaluated more candidates (644 vs. 583) and spent more on API calls (about USD 2,218 vs. 1,847) than Full. Yet its geometric mean drops to 1.26×, so this part of the gain is not explained by greater resource use.

The verification requirement itself cannot be switched off: removing it would let unverified candidates become the basis of subsequent search, which conflicts with the premise that deterministic checks own correctness. Two lighter variants remain possible and were not run: a single verified chain in place of the verified tree, and team search that keeps the best single candidate per round instead of composing. Composition occurs only in team mode, so its effect is part of the Full vs. Single difference but is not isolated by it.

The saved-log analysis provides complementary evidence for stall handling. Across 24 KernelBench searches, 10 kernels committed edits along supervisor-redirected paths, corresponding to an approximately 15% geometric-mean increase from their pre-trigger versions (Response to Reviewer ErK6, 3/3). This is lineage attribution rather than a controlled ablation.

Separating each component would need a full 4-GPU, 12-hour search per component per kernel. One pass over the 8 production kernels alone needs 40 searches, or 1,920 GPU-hours and about USD 9,000 in API costs. Reliable conclusions would also need several repetitions, which is a large cost in time and money. §4.4.3 and the caption of Table 4 also state this: Single and No Sup. are bundled search-policy changes rather than single-factor ablations.

AuthorsOpenReview ↗

Response to Reviewer SfHe (3/5): Correctness on held-out inputs [W2, Q2]

[W2, Q2] Held-out correctness

Weakness 2: "Correctness is validated against the original binary under fixed captured execution conditions, which may not generalize to unseen inputs or states. The absence of independent validation on held-out inputs raises concerns about the reliability of the optimized kernels."

Question 2: "How well do the optimized kernels generalize to unseen inputs or execution states? Can the authors provide correctness results on independent test cases beyond the captured executions?"

Thank you for this comment, which is important for the reliability of AsmEvo's results. We first note that AsmEvo's correctness does not rely on the captured test data alone. It is also backed by structural safeguards. First, the rebuild freezes the kernarg layout, symbol interface, and launch semantics. Every candidate passes static ABI and resource checks (§3.5, §3.7). The optimized binary thus has the same calling convention as K0K_0, and the original host launcher can invoke it. Second, the equivalence check compares the entire captured memory, not just the outputs. Every input and every untouched range must match K0K_0 byte for byte, and out-of-footprint writes are rejected (§3.4). A candidate therefore cannot pass by changing state outside the outputs. Third, the optimized production binaries pass the AITER unit tests and the vLLM/SGLang framework tests in the native runtime (Appendix F, “Dispatch and End-to-End Validation”). These tests use inputs and runtime state generated by the frameworks, not the captured data.

On top of these safeguards, we added independent held-out tests as the reviewer suggests. We chose 8 kernels: 2 tolerance-checked and 2 byte-exact kernels from each of Level 1 and Level 2. We ran two groups of tests, with 10 cases per kernel in each group. Group (1) keeps the captured shape and launch configuration and samples new input values at random. Each input buffer is drawn from a Gaussian with the mean and standard deviation of its captured data, truncated to its captured value range. Group (2) samples new shapes from a Gaussian centered on the captured shape. The standard deviation of each dimension is 25% of its captured value. Each new shape is re-captured and gets a newly generated harness, again with random input values. For runtime state, both groups use input data and initial output-buffer contents that differ from the original capture. Output buffers are filled with sentinel values before each run, and guard regions follow every memory block, so both missing and out-of-bounds writes are detected. For each case, we also first check that K0K_0 agrees with a PyTorch float64 reference, so that the inputs are valid.

KernelCriterionCaptured shape, new valuesNew shapes, new values
KernelBench Level 1
37_FrobeniusNorm_AllClose10/10, 3.4×10⁻⁵10/10, 1.7×10⁻⁴
88_MinGPTNewGeluAllClose10/10, 1.2×10⁻³10/10, 1.2×10⁻³
36_RMSNorm_ByteExact10/10, bit-identical10/10, bit-identical
98_KLDivLossByteExact10/10, bit-identical10/10, bit-identical
KernelBench Level 2
30_Gemm_GroupNorm_HardtanhAllClose10/10, 3.6×10⁻⁷10/10, 4.8×10⁻⁷
88_Gemm_GroupNorm_Swish_Multiply_SwishAllClose10/10, 3.2×10⁻⁶10/10, 4.1×10⁻⁶
81_Gemm_Swish_Divide_Clamp_Tanh_ClampByteExact10/10, bit-identical10/10, bit-identical
64_Gemm_LogSumExp_LeakyReLU_LeakyReLU_GELU_GELUByteExact10/10, bit-identical10/10, bit-identical

Correctness of the final candidates on unseen input values and shapes. Each cell reports passing cases out of ten and the maximum absolute error against K0K_0 ("bit-identical" when every output matches K0K_0 bit for bit).

All 160 cases pass their Eq. 1 predicate (AllClose or ByteExact, as listed) (table above). There are no missing or out-of-bounds writes. The 4 byte-exact kernels give outputs bitwise identical to K0K_0 in all 80 of their cases. The errors of the 4 tolerance-checked kernels are of the same magnitude as on the original capture. For example, the maximum absolute error of 88_Gemm_GroupNorm_Swish_Multiply_Swish is 3.3×10⁻⁶ on the original capture and 4.1×10⁻⁶ on new shapes. The optimized kernels thus remained equivalent to K0K_0 in all tested cases with unseen input values, shapes, and initial buffer states. These tests remain finite and do not change the formal scope of our guarantee. Consistent with §3.9, each new shape is re-captured and re-checked automatically before the optimized object is applied. The full setup and results are in Appendix D.3 of the revision and are discussed in §4.4.4.

Reviewer_yVomOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AsmEvo, a system for optimizing compiled AMD GPU code objects without source code. It reconstructs a reassemblable AMDGCN representation while preserving ABI-visible properties, and uses a captured real dispatch to generate a persistent evaluator. Candidates are compared against the original binary under the same launch contract and captured state.An LLM agent proposes profile-guided assembly edits, while a deterministic controller manages rebuilding, validation, and candidate commitment. On MI308X, AsmEvo improves all 30 selected KernelBench kernels, reaching 1.41× geometric-mean and 3.53× maximum speedup. On MI300X, it improves all evaluated AITER binaries and vLLM/SGLang Triton kernels, reaching 1.12×/1.33× and 1.49×/2.08× geometric-mean/maximum speedups, respectively, while preserving functional equivalence. The paper also describes search mechanisms such as multi-start exploration, lineage management, and stall-triggered redirection.

Critical strengths

The problem setting is practically relevant and technically difficult.Deployed AMD GPU kernels may be available only as compiled code objects. Optimizing such artifacts requires simultaneously recovering editable assembly, preserving symbols and kernel descriptors, maintaining ABI compatibility, and keeping metadata consistent with resource usage. The paper addresses this complete pipeline rather than treating assembly editing as an isolated code-generation problem.

The separation between LLM proposals and deterministic verification is well designed.The LLM proposes edits and interprets feedback, but a deterministic controller owns rebuilding, static checks, correctness validation, paired ABBA timing, A/A calibration, health checks, and confidence-bound commit gates. The architectural decision to deny the LLM authority over correctness or speedup claims is what makes the loop's empirical results credible, even though the individual measurement techniques are standard.

Critical weaknesses

The evaluation is limited by a filtered benchmark subset. KernelBench covers 30 selected kernels, retained only after a GEAK-style agent produced compilable HIP implementations that beat PyTorch, so the 100% improvement rate and 1.41× geometric mean are conditional on this subset, and eight production kernels (three AITER, five Triton) are too few to support broad generalization.

The agentic search contribution is not isolated. The "Single" and "No Sup." settings in Table 4 are bundled policy changes rather than one-factor ablations, so the results show the full policy works but not which components drive the gains.The equivalence claim is narrower than it sounds. Correctness is only empirical differential equivalence for the captured launch, state, and validation policy, untested on unseen shapes or values, and some KernelBench outputs use cosine comparison rather than strict error bounds.

Questions for authors

1.Have the authors run repeated 12-hour searches with different seeds on a representative subset? If so, how much do final speedups, number of commits, and selected instruction patterns vary across runs?

2 How many KernelBench candidates were available before the 30 reported kernels were selected? Please quantify how the compilability and “HIP-faster-than-PyTorch” filters affect the composition of the evaluated set, and clarify whether selection was performed independently of AsmEvo

3.Please report total model-inference cost, GPU-hours, number of candidate builds and formal evaluations, and wall-clock time per kernel.

4.Can the authors provide one-factor-at-a-time comparisons for multi-start exploration, verified lineage, composition, stall-triggered redirection, and supervisor escalation?

5.What evidence supports applying an optimized object beyond the captured input shape, data values, and runtime state? For candidates using cosine-based floating-point checks, what are the maximum absolute and relative errors, and do the reported speedups remain under stricter numerical predicates?

6 comments
AuthorsOpenReview ↗

Response to Reviewer yVom (1/5): Search stability and benchmark selection [W1, Q1, Q2]

[Q1] Seed variation

Question 1: "Have the authors repeated 12-hour searches with different seeds on a representative subset? If so, how much do final speedups, number of commits, and selected instruction patterns vary across runs?"

We repeated the full 12-hour search with three seeds on one AITER FMHA kernel and one Triton A16W16 GEMM kernel, using the same model, tools, capture, hardware, and search configuration.

For aiter_fmha_bwd_hd128_bf16_causal_a32, all three runs reach 1.05×. They produce 3, 4, and 2 accepted commits, all of which lie on their respective final lineages. Despite different auxiliary edits—barrier elimination, cache hints, address/exec-mask simplification, and s_waitcnt adjustment—all three independently remove redundant BF16 RTNE NaN guards.

For _gemm_a16_w16_kernel, the three runs reach 1.25×, 1.25×, and 1.27×. They produce 6, 2, and 3 accepted commits, with 5, 2, and 3 commits on the final lineages. All three independently remove the nt hint from hot B-tile loads to make effective use of L2, while their other edits differ across load/compute scheduling, LDS pipelining, loop rotation, and cross-iteration prefetching.

Thus, the exact lineage and number of commits vary across seeds, but the main optimization opportunities recur and final performance is stable. Both kernels have moderate gains; kernels with larger gains may vary more.

[W1, Q2] KernelBench selection

Weakness 1: "The evaluation is limited by a filtered benchmark subset. KernelBench covers 30 selected kernels, retained only after a GEAK-style agent produced compilable HIP implementations that beat PyTorch, so the 100% improvement rate and 1.41× geometric mean are conditional on this subset,..."

Question 2: "How many KernelBench candidates were available before the 30 reported kernels were selected? Please quantify how the compilability and "HIP-faster-than-PyTorch" filters affect the composition of the evaluated set, and clarify whether selection was performed independently of AsmEvo."

Thank you for this comment. We agree that the 100% improvement rate and the 1.41× geometric mean are obtained on a filtered set. The paper does not fully describe the selection procedure or its effect, and this is an omission on our part.

There were 200 candidates before selection: all Level 1 and Level 2 KernelBench operators. A GEAK-style agent (GPT-5.0) generated a HIP implementation for each operator and refined it for five rounds. All 200 implementations compiled. The compilability filter therefore removed no operator and does not affect the composition of the evaluated set. The faster-than-PyTorch filter kept 81 operators: 30 from Level 1 (30%) and 51 from Level 2 (51%). Fused Level 2 operators usually leave more room for fusion at the source level, so their HIP versions are more often faster than PyTorch. This step thus raised the Level 2 share among passing operators from 50% to about 63%. It also removed the operators whose HIP versions are slower than PyTorch. Every remaining K0K_0 is therefore a strong baseline, and the evaluated set leans toward kernels that are harder to improve further. Finally, we used Opus 4.8 to select 15 of the 30 Level 1 operators and 15 of the 51 Level 2 operators, based on operator-type diversity and static features such as assembly length. Each level thus makes up half of the evaluated set, regardless of the different pass rates. §4.2 states that the 30 kernels are exactly the HIP-faster-than-PyTorch subset; this omits the last step, and we have corrected it. The 30 kernels cover activations and losses, reductions and normalization, pooling, convolution, matrix multiplication, and fused pipelines (Appendix C).

These K0K_0 were optimized at the source level and are already faster than PyTorch. They are strong baselines with less headroom left. The 119 implementations that were filtered out are slower than PyTorch and usually leave more room for improvement. The filter is therefore unlikely to inflate AsmEvo's results; if anything, it concentrates the evaluation on kernels that are harder to improve further. On this set, AsmEvo still improved all 30 kernels, with a 1.41× geometric mean.

The selection is fully independent of AsmEvo. It was completed before AsmEvo was run. It used only the compilation results, the HIP vs. PyTorch performance comparison, and the static features above. No AsmEvo result was used, and no kernel was added or removed because of an AsmEvo result. We have added the full selection procedure to Appendix C of the revision, and §4.4.1 now states that the results were obtained on this set.

AuthorsOpenReview ↗

Response to Reviewer yVom (2/5): Contribution of individual search components [W2, Q4]

[W2, Q4] Component ablation

Weakness 2: "The agentic search contribution is not isolated. The "Single" and "No Sup." settings in Table 4 are bundled policy changes rather than one-factor ablations, so the results show the full policy works but not which components drive the gains."

Question 4: "Can the authors provide one-factor-at-a-time comparisons for multi-start exploration, verified lineage, composition, stall-triggered redirection, and supervisor escalation?"

Thank you for these comments, which help improve the paper.

The reviewer notes that Single and No Sup. are bundled policy changes, so Table 4 only shows that the full policy works. §4.4.3 already presents Table 4 as a comparison of search drivers, not a single-factor ablation. Its original title, "Ablation Study", is misleading, and we have renamed it "Search-Driver Comparison". Still, Table 4 shows more than that the full policy works. Each of its two comparisons targets specific mechanisms.

Full vs. Single tests multi-start exploration together with the larger GPU allocation. Single uses one GPU and one direction, and always starts from the current best. This corresponds to common single-path iteration. It lowers the geometric mean from 1.34× to 1.18× and the improved kernels from 8/8 to 6/8. The speedup of kernel_unified_attention_2d drops from 1.96× to 1.05×. Single also has a quarter of the GPU allocation (but about half of Full's API cost), so this comparison does not separate the two factors. Full vs. No Sup. tests stall handling. Both use the same model, tools, and GPU cap. No Sup. only disables stall-triggered re-planning, backtracking, and supervisor escalation, as one bundle. It evaluated more candidates and spent more on API calls than Full. Yet its geometric mean dropped from 1.34× to 1.26×. This part of the gain is therefore not explained by greater resource use.

We have not run one-factor comparisons for the five components. The verification requirement itself cannot be removed: only versions that passed the formal gate can become start points, and each worker is reset to a verified state after evaluation (§3.8). Removing it would let unverified candidates become the basis of subsequent search, which conflicts with a premise the reviewer noted positively: deterministic checks own correctness. Within this constraint, two lighter variants are possible: a single verified chain without returning to earlier nodes, in place of the verified tree, and team search that always keeps the best single candidate per round instead of composing. A composition is forwarded only if it passes all correctness gates and beats both parents, and committed only if it also beats the current best (§3.8). Since composition occurs only in team mode, its effect is part of the Full vs. Single difference but is not isolated by it. We did not run these variants because of the cost below, and we state this limitation in §4.4.3.

Separating the five components one by one would need a full 4-GPU, 12-hour search per component per kernel. One pass over the 8 production kernels alone needs 40 searches, or 1,920 GPU-hours and about USD 9,000 in API costs. On the 30 KernelBench kernels, it needs 150 searches and 7,200 GPU-hours. Reliable conclusions would also need several repetitions, which requires substantial resources.

AuthorsOpenReview ↗

Response to Reviewer yVom (3/5): Correctness beyond the captured inputs [W2, Q5]

[W2, Q5] Held-out correctness

Weakness 2: "The equivalence claim is narrower than it sounds. Correctness is only empirical differential equivalence for the captured launch, state, and validation policy, untested on unseen shapes or values,..."

Question 5: "What evidence supports applying an optimized object beyond the captured input shape, data values, and runtime state?"

On applying an optimized object beyond the captured input shape, data values, and runtime state, we first note that AsmEvo's correctness does not rely on the captured test data alone. It is also backed by structural safeguards. First, the rebuild freezes the kernarg layout, symbol interface, and launch semantics. Every candidate passes static ABI and resource checks (§3.5, §3.7). The optimized binary thus has the same calling convention as K0K_0. Second, the equivalence check compares the entire captured memory, not just the outputs. Every input and every untouched range must match K0K_0 byte for byte, and out-of-footprint writes are rejected (§3.4). A candidate therefore cannot pass by changing state outside the outputs. Third, the optimized production binaries pass the AITER unit tests and the vLLM/SGLang framework tests in the native runtime (Appendix F.5). These tests use inputs and runtime state generated by the frameworks, not the captured data.

Beyond these safeguards, we added tests on unseen values and unseen shapes. To cover both predicate types and both levels, we chose 8 kernels: 2 tolerance-checked and 2 byte-exact kernels from each of Level 1 and Level 2. We ran two groups of tests, with 10 cases per kernel in each group. Group (1) keeps the captured shape and launch configuration and samples new input values at random. Group (2) samples new shapes from a Gaussian centered on the captured shape. The standard deviation of each dimension is 25% of its captured value. We keep only shapes for which the original launcher still dispatches the same kernel with the same block size. Each new shape is re-captured and gets a newly generated harness, as in §3.9. In both groups, input values come from a Gaussian fitted to the mean and standard deviation of the captured data, truncated to the captured value range. For each case, we first check that K0K_0 agrees with a PyTorch float64 reference, so that the shape and inputs are valid. Before each run, we fill the output buffers with sentinel values and place guard regions after every memory block. Both missing writes and out-of-bounds writes are therefore detected. All 160 cases pass their Eq. 1 predicate (AllClose or ByteExact, as listed), with errors of the same magnitude as on the original capture (table below). For the same kernel and block configuration, the optimized kernels remained equivalent to K0K_0 in all tested cases with unseen shapes and values. The full setup and results are in Appendix D.3 of the revision and are discussed in §4.4.4.

KernelCriterionCaptured shape, new valuesNew shapes, new values
KernelBench Level 1
37_FrobeniusNorm_AllClose10/10, 3.4×10⁻⁵10/10, 1.7×10⁻⁴
88_MinGPTNewGeluAllClose10/10, 1.2×10⁻³10/10, 1.2×10⁻³
36_RMSNorm_ByteExact10/10, bit-identical10/10, bit-identical
98_KLDivLossByteExact10/10, bit-identical10/10, bit-identical
KernelBench Level 2
30_Gemm_GroupNorm_HardtanhAllClose10/10, 3.6×10⁻⁷10/10, 4.8×10⁻⁷
88_Gemm_GroupNorm_Swish_Multiply_SwishAllClose10/10, 3.2×10⁻⁶10/10, 4.1×10⁻⁶
81_Gemm_Swish_Divide_Clamp_Tanh_ClampByteExact10/10, bit-identical10/10, bit-identical
64_Gemm_LogSumExp_LeakyReLU_LeakyReLU_GELU_GELUByteExact10/10, bit-identical10/10, bit-identical

Correctness of the final candidates on unseen input values and shapes. Each cell reports passing cases out of ten and the maximum absolute error against K0K_0 ("bit-identical" when every output matches K0K_0 bit for bit).

For runtime state, both groups use input data and initial output-buffer contents that differ from the original capture. These tests are still a finite sample, so we keep the correctness guarantee scoped to the captured launch (§3.9). A new shape is re-captured and re-checked, and this process is automated.

Reviewer_ErK6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AsmEvo recovers assembly from compiled AMDGPU kernels for LLM-guided editing. It rebuilds compatible binaries and checks them against the original in a fixed replay environment. The search can return to successful versions or explore another starting point when progress stalls.

Critical strengths
  • The rebuilding tools preserve the ABI while keeping resource declarations consistent with the edited code's metadata. Native-runtime checks on vendor and JIT kernels support compatibility with the original dispatch paths
  • Reciprocal placement reduces bias in paired timings, with A/A calibration checking measurement stability. After independent rebuilding, final candidates are timed directly against the original.
Critical weaknesses
  • Seven KernelBench cases accept floating outputs at cosine similarity ≥0.9999 even when absolute error is large; twice the reference output would pass (Appendix D). This affects the 3.53× FrobeniusNorm and strongest Level 2 results, with a similar policy used for production FP8 MoE. Rechecking these candidates with justified absolute and relative tolerances would clarify which speedups preserve numerical agreement.
  • Full uses four GPUs versus one for Single and changes the search policy, while Code lacks the same tools. The Opus 4.8/Opus 5 comparison keeps the full framework fixed, leaving the performance of a simpler loop with a stronger model unresolved. Full’s gains over No Sup. are based on one 12-hour search per result (§4.1, Table 4). Repeated comparisons using the same strong model and tools under matched GPU/API budgets would help isolate the contribution of the search mechanism.
Questions for authors
  • What were the actual GPU-hours and API costs for each variant in Table 4? How much additional computation did Full use relative to Single and Code?
  • According to the saved search logs, which improvements followed a supervisor-triggered change of direction? How much did those edits contribute to the final speedup?
6 comments
AuthorsOpenReview ↗

Response to Reviewer ErK6 (1/3): Numerical agreement under stricter tolerances

[W1] Cosine-checked kernels

Weakness 1: "Seven KernelBench cases accept floating outputs at cosine similarity ≥0.9999 even when absolute error is large; twice the reference output would pass (Appendix D). This affects the 3.53× FrobeniusNorm and strongest Level 2 results, with a similar policy used for production FP8 MoE. Rechecking these candidates with justified absolute and relative tolerances would clarify which speedups preserve numerical agreement."

Thank you for raising this point. We agree that cosine alone can accept a scaled reference output. In our harness, cosine applies only to declared floating-point output tensors. Every input, untouched range, integer output, and guard canary must match K0K_0 byte for byte (Appendix D), so out-of-footprint writes and corrupted state are rejected.

We directly rechecked the 7 final candidates on the same captures with AllClose(a=10⁻⁵, r=10⁻²) from Eq. 1. All 7 pass, with a largest relative error of 8.4×10⁻³. Their reported speedups therefore preserve numerical agreement under these stricter element-wise bounds. Four of these kernels also pass AllClose on 20 held-out cases each, using new input values and shapes (Response to Reviewer yVom, 3/5).

For the production kernel aiter_fmoe_fp8_blockscale_subgu256, BF16 atomic accumulation makes K0K_0 itself non-bitwise reproducible. Over 40 runs per object, optimized-vs.-K0K_0 differences (median maximum absolute error 3; relative L2 error 0.39%–0.50%) are at the same level as K0K_0's own run-to-run variation (3; 0.36%–0.50%). The edit changes no arithmetic instruction. Together with the run-to-run analysis, this supports numerical agreement; the measured speedup remains 1.33×. These results will appear in Appendix D.1–D.2 and §4.4.4. Future runs will use AllClose together with cosine for floating-point outputs.

KernelSpeedupMax abs. errorMax rel. error
KernelBench Level 1
37_FrobeniusNorm_3.53×9.2×10⁻⁵7.9×10⁻⁷
88_MinGPTNewGelu1.69×1.2×10⁻³8.4×10⁻³
KernelBench Level 2
37_Matmul_Swish_Sum_GroupNorm1.94×3.1×10⁻⁶6.1×10⁻⁵
88_Gemm_GroupNorm_Swish_Multiply_Swish1.33×3.3×10⁻⁶4.6×10⁻⁵
38_ConvTranspose3d_AvgPool_Clamp_Softmax_Multiply1.29×2.3×10⁻¹⁰2.9×10⁻⁷
51_Gemm_Subtract_GlobalAvgPool_LogSumExp_GELU_ResidualAdd1.20×00
30_Gemm_GroupNorm_Hardtanh1.19×3.6×10⁻⁷3.9×10⁻⁶

Element-wise numerical agreement between K0K_0 and the final candidate for the seven Tolerance-checked KernelBench kernels on the same capture. All seven pass AllClose(a=10⁻⁵, r=10⁻²).

ComparisonPairsMax abs. (median [range])RMSERelative L2Within 2 BF16 ULPs
K0K_0 vs. K0K_07803 [2, 4]0.222–0.3100.36–0.50%94.1%
Optimized vs. optimized7803 [2, 4]0.243–0.3000.39–0.49%93.9%
K0K_0 vs. optimized1,6003 [2, 6]0.242–0.3120.39–0.50%93.7%

Run-to-run output differences of aiter_fmoe_fp8_blockscale_subgu256 (1.33×) on the same capture, over 40 runs each of K0K_0 and the optimized object.

AuthorsOpenReview ↗

Response to Reviewer ErK6 (2/3): Fairness of the search-driver comparison [W2, Q1]

[W2, Q1] Resource use

Weakness 2: "Full uses four GPUs versus one for Single and changes the search policy, while Code lacks the same tools."

Question 1: "What were the actual GPU-hours and API costs for each variant in Table 4? How much additional computation did Full use relative to Single and Code?"

Thank you for the detailed reading of §4.1 and Table 4. Table 4 compares search drivers rather than providing a single-factor ablation. Full performs parallel, multi-start search: four workers explore different directions from multiple verified start points. Single uses one worker and one direction, always restarting from the current best, which represents best-so-far single-path iteration. The intended comparison is therefore multi-start parallel search versus a single-direction search from the best current version.

Full has 4× Single's GPU allocation, but about 1.95× its API cost and 2.3× its candidate evaluations; formal evaluations are similar (32 vs. 29). Full improves 8/8 kernels at 1.34× geometric mean, versus 6/8 at 1.18× for Single. Because we did not run Single with matched GPU-hours, this result reflects the combined effect of the search policy and larger parallel allocation; it does not isolate either factor.

Code lacks AsmEvo's tools by design. Since no directly comparable agent targets source-free AMDGPU code objects (§2), we gave Claude Code the same model, original HSACO binaries, captures, and GPU/time caps. This comparison asks whether a general-purpose coding agent can turn the same budget into long-horizon optimization of source-free binary kernels, without AsmEvo's recovery, rebuild, gating, and search control.

The table reports resources reconstructed from the saved logs for all 8 production kernels. GPU allocation is the cap, not measured utilization. Observed run time sums each kernel's first-to-last model-call span; API cost is the client estimate from logged token usage, not an external bill. The revision adds this accounting as Table 5 and Appendix B.3.

Although Code received the same 4-GPU, 12-hour cap as Full, each session terminated on its own after only 0.2–0.5 hours (2.6 hours total; USD 29). It improved only 2/8 kernels, with a 1.05× geometric mean. In our runs, the general-purpose loop therefore could not sustain long-horizon optimization of HSACO binaries or make effective use of the available budget. By contrast, AsmEvo sustained the 12-hour searches and improved all 8 kernels at 1.34× geometric mean. The key difference is not the nominal cap, but whether the system can use it for continued, verified optimization.

VariantGPU cap / kernelGPU allocationObserved run timeAPI costCandidates evaluatedFormal evalsCommits
Full4 GPU × 12 h384 GPU-h100.3 hUSD 1,8475833227
Single1 GPU × 12 h96 GPU-h88.8 hUSD 9472542923
No Sup.4 GPU × 12 h384 GPU-h108.0 hUSD 2,2186443024
Opus 54 GPU × 12 h384 GPU-h107.8 hUSD 2,6423043836
Code4 GPU × 12 h384 GPU-h2.6 hUSD 29–––

Resources used by the variants in Table 4, totaled over the eight production kernels. Observed run time is the first-to-last model-call span per kernel; API cost is the client-side estimate from token usage (Appendix B.3).

[W2] Stronger model

Weakness 2: "The Opus 4.8/Opus 5 comparison keeps the full framework fixed, leaving the performance of a simpler loop with a stronger model unresolved."

The fixed-model comparison above shows the framework-level gap. With AsmEvo fixed, upgrading from Opus 4.8 to Opus 5 further raises the geometric mean from 1.34× to 1.37×. AsmEvo therefore also benefits from a stronger model: framework and model improvements are complementary. We did not run a simpler loop with Opus 5; we will state this limitation in the revised §4.4.3.

AuthorsOpenReview ↗

Response to Reviewer ErK6 (3/3): Robustness of the search-mechanism gains [W2, Q2]

[W2] Repeated searches

Weakness 2: "Full's gains over No Sup. are based on one 12-hour search per result (§4.1, Table 4). Repeated comparisons using the same strong model and tools under matched GPU/API budgets would help isolate the contribution of the search mechanism."

We agree that Full vs. No Sup. uses one 12-hour search per kernel. The two variants have the same model, tools, and GPU cap; No Sup. evaluates more candidates (644 vs. 583) and costs more (USD 2,218 vs. 1,847), so Full's gain is not explained by greater resource use. Full leads on 6/8 kernels. Excluding fused_moe_kernel_gptq_awq, the largest-gap kernel, Full still leads in geometric mean (1.26× vs. 1.23×). Repeating all 8 kernels once would cost 768 GPU-hours and about USD 4,000, so we could not run several repetitions during rebuttal. The Reproducibility Statement already notes that each speedup comes from one 12-hour search.

[Q2] Supervisor redirections

Question 2: "According to the saved search logs, which improvements followed a supervisor-triggered change of direction? How much did those edits contribute to the final speedup?"

This analysis was missing from the paper. On KernelBench, 24 searches triggered the supervisor 60 times. Ten kernels then committed edits along the redirected path, increasing their speedups by about 15% in geometric mean. For 98_KLDivLoss, a trigger after a 2.2-hour stall redirected the search to DPP and control-flow optimizations. Commits v5–v8 raised speedup from 1.15× to 1.36×; a later trigger led to v9 and a final 1.39×.

Among the 8 production kernels, 4 searches triggered the supervisor 10 times. Two produced accepted commits afterward: FMHA backward (1.05×) and FMHA forward (1.03×). For FMHA forward, No Sup. gives 1.00×; for FMHA backward, it also reaches 1.05×, so that gain does not require the supervisor.

These logs are consistent with productive redirection after stalls, but constitute lineage attribution from single runs rather than a controlled ablation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.