- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents AsmEvo, an agentic framework for optimizing compiled AMD GPU kernels without access to source code. It recovers editable assembly from GPU binaries, employs profiling-guided LLM agents to explore optimizations, and rebuilds candidates while preserving the original launch ABI. A deterministic verification pipeline checks correctness against the original binary under captured execution conditions and validates performance improvements. Experiments on KernelBench and production-derived kernels demonstrate performance gains.
Critical strengths
- The paper addresses a meaningful post-compilation optimization problem for AMD GPU kernels when source code is unavailable. Targeting compiled code objects complements existing source-level kernel optimization approaches and has practical relevance to deployed GPU workloads.
- AsmEvo combines assembly recovery, ABI-preserving rebuilding, agentic optimization, and capture-based correctness and performance validation into a unified framework. The evaluation spans both benchmark and production-derived kernels and demonstrates promising performance improvements.
Critical weaknesses
-
The paper integrates existing techniques for binary recovery, assembly optimization, and agentic search, but does not clearly identify the novel technical insights arising from this integration. A more systematic ablation study is needed to clearly establish the contributions of individual components.
-
Correctness is validated against the original binary under fixed captured execution conditions, which may not generalize to unseen inputs or states. The absence of independent validation on held-out inputs raises concerns about the reliability of the optimized kernels.
Questions for authors
-
What are the key technical contributions of AsmEvo beyond integrating existing optimization techniques? Can the authors provide more systematic ablation studies to demonstrate the benefits of individual components?
-
How well do the optimized kernels generalize to unseen inputs or execution states? Can the authors provide correctness results on independent test cases beyond the captured executions?
-
What specific assembly-level transformations account for the reported speedups? Can the authors provide representative before-and-after assembly examples and profiling evidence to explain their effectiveness?
7 comments
Response to Reviewer SfHe (2/5): Contribution of individual components [W1, Q1]
[W1, Q1] Ablation
Weakness 1: "A more systematic ablation study is needed to clearly establish the contributions of individual components."
Question 1: "Can the authors provide more systematic ablation studies to demonstrate the benefits of individual components?"
On more systematic ablations, Table 4 compares search drivers (we have renamed §4.4.3, formerly "Ablation Study", to "Search-Driver Comparison"). Each of its two comparisons targets specific mechanisms. Full vs. Single tests multi-start exploration together with the larger GPU allocation. Single uses one GPU and one direction, and always starts from the current best. Its geometric mean drops from 1.34× to 1.18×, and its improved kernels drop from 8/8 to 6/8. The speedup of kernel_unified_attention_2d drops from 1.96× to 1.05×. Because Single also has a quarter of the GPUs, this comparison does not separate the two factors. Full vs. No Sup. tests stall handling as one bundle. Both use the same model, tools, and GPU cap. No Sup. evaluated more candidates (644 vs. 583) and spent more on API calls (about USD 2,218 vs. 1,847) than Full. Yet its geometric mean drops to 1.26×, so this part of the gain is not explained by greater resource use.
The verification requirement itself cannot be switched off: removing it would let unverified candidates become the basis of subsequent search, which conflicts with the premise that deterministic checks own correctness. Two lighter variants remain possible and were not run: a single verified chain in place of the verified tree, and team search that keeps the best single candidate per round instead of composing. Composition occurs only in team mode, so its effect is part of the Full vs. Single difference but is not isolated by it.
The saved-log analysis provides complementary evidence for stall handling. Across 24 KernelBench searches, 10 kernels committed edits along supervisor-redirected paths, corresponding to an approximately 15% geometric-mean increase from their pre-trigger versions (Response to Reviewer ErK6, 3/3). This is lineage attribution rather than a controlled ablation.
Separating each component would need a full 4-GPU, 12-hour search per component per kernel. One pass over the 8 production kernels alone needs 40 searches, or 1,920 GPU-hours and about USD 9,000 in API costs. Reliable conclusions would also need several repetitions, which is a large cost in time and money. §4.4.3 and the caption of Table 4 also state this: Single and No Sup. are bundled search-policy changes rather than single-factor ablations.
Response to Reviewer SfHe (3/5): Correctness on held-out inputs [W2, Q2]
[W2, Q2] Held-out correctness
Weakness 2: "Correctness is validated against the original binary under fixed captured execution conditions, which may not generalize to unseen inputs or states. The absence of independent validation on held-out inputs raises concerns about the reliability of the optimized kernels."
Question 2: "How well do the optimized kernels generalize to unseen inputs or execution states? Can the authors provide correctness results on independent test cases beyond the captured executions?"
Thank you for this comment, which is important for the reliability of AsmEvo's results. We first note that AsmEvo's correctness does not rely on the captured test data alone. It is also backed by structural safeguards. First, the rebuild freezes the kernarg layout, symbol interface, and launch semantics. Every candidate passes static ABI and resource checks (§3.5, §3.7). The optimized binary thus has the same calling convention as , and the original host launcher can invoke it. Second, the equivalence check compares the entire captured memory, not just the outputs. Every input and every untouched range must match byte for byte, and out-of-footprint writes are rejected (§3.4). A candidate therefore cannot pass by changing state outside the outputs. Third, the optimized production binaries pass the AITER unit tests and the vLLM/SGLang framework tests in the native runtime (Appendix F, “Dispatch and End-to-End Validation”). These tests use inputs and runtime state generated by the frameworks, not the captured data.
On top of these safeguards, we added independent held-out tests as the reviewer suggests. We chose 8 kernels: 2 tolerance-checked and 2 byte-exact kernels from each of Level 1 and Level 2. We ran two groups of tests, with 10 cases per kernel in each group. Group (1) keeps the captured shape and launch configuration and samples new input values at random. Each input buffer is drawn from a Gaussian with the mean and standard deviation of its captured data, truncated to its captured value range. Group (2) samples new shapes from a Gaussian centered on the captured shape. The standard deviation of each dimension is 25% of its captured value. Each new shape is re-captured and gets a newly generated harness, again with random input values. For runtime state, both groups use input data and initial output-buffer contents that differ from the original capture. Output buffers are filled with sentinel values before each run, and guard regions follow every memory block, so both missing and out-of-bounds writes are detected. For each case, we also first check that agrees with a PyTorch float64 reference, so that the inputs are valid.
| Kernel | Criterion | Captured shape, new values | New shapes, new values |
|---|---|---|---|
| KernelBench Level 1 | |||
37_FrobeniusNorm_ | AllClose | 10/10, 3.4×10⁻⁵ | 10/10, 1.7×10⁻⁴ |
88_MinGPTNewGelu | AllClose | 10/10, 1.2×10⁻³ | 10/10, 1.2×10⁻³ |
36_RMSNorm_ | ByteExact | 10/10, bit-identical | 10/10, bit-identical |
98_KLDivLoss | ByteExact | 10/10, bit-identical | 10/10, bit-identical |
| KernelBench Level 2 | |||
30_Gemm_GroupNorm_Hardtanh | AllClose | 10/10, 3.6×10⁻⁷ | 10/10, 4.8×10⁻⁷ |
88_Gemm_GroupNorm_Swish_Multiply_Swish | AllClose | 10/10, 3.2×10⁻⁶ | 10/10, 4.1×10⁻⁶ |
81_Gemm_Swish_Divide_Clamp_Tanh_Clamp | ByteExact | 10/10, bit-identical | 10/10, bit-identical |
64_Gemm_LogSumExp_LeakyReLU_LeakyReLU_GELU_GELU | ByteExact | 10/10, bit-identical | 10/10, bit-identical |
Correctness of the final candidates on unseen input values and shapes. Each cell reports passing cases out of ten and the maximum absolute error against ("bit-identical" when every output matches bit for bit).
All 160 cases pass their Eq. 1 predicate (AllClose or ByteExact, as listed) (table above). There are no missing or out-of-bounds writes. The 4 byte-exact kernels give outputs bitwise identical to in all 80 of their cases. The errors of the 4 tolerance-checked kernels are of the same magnitude as on the original capture. For example, the maximum absolute error of 88_Gemm_GroupNorm_Swish_Multiply_Swish is 3.3×10⁻⁶ on the original capture and 4.1×10⁻⁶ on new shapes. The optimized kernels thus remained equivalent to in all tested cases with unseen input values, shapes, and initial buffer states. These tests remain finite and do not change the formal scope of our guarantee. Consistent with §3.9, each new shape is re-captured and re-checked automatically before the optimized object is applied. The full setup and results are in Appendix D.3 of the revision and are discussed in §4.4.4.
Response to Reviewer SfHe (1/5): Technical contributions beyond integration [W1, Q1]
[W1, Q1] Technical contributions
Thank you for the comment, and for recognizing the practical value and technical difficulty of source-free optimization. AsmEvo's key contributions go beyond combining existing techniques: they address the new problems that arise when source-free AMDGPU binaries are optimized directly. To our knowledge, AsmEvo is the first system to combine agentic search, ABI- and metadata-preserving AMDGPU code-object rebuilding, and capture-scoped evaluation against the original deployed binary (§2). Our technical contributions are the following three.
First, reassemblable binary recovery. Raw disassembly cannot be reassembled. It lacks the declaration layer, such as kernel descriptors and metadata. In addition, PC-relative relocations are resolved to concrete values. Instruction lengths then change on reassembly, which shifts later addresses and breaks branches. AsmEvo therefore reconstructs the declaration layer directly from the kernel descriptor bytes and the AMDGPU metadata note. It also re-symbolizes PC-relative operands. The recovered assembly must pass a byte-level round-trip check before later edits can build on it (§3.3).
Second, ABI-preserving rebuild. Assembly edits often change register, LDS, and scratch usage. Existing assembly-level methods avoid this by restricting the edit space. CuAsmRL, for example, only reorders SASS instructions. AsmEvo instead recomputes the resource fields in the kernel descriptor and AMDGPU metadata, while the kernarg layout, symbol interface, and launch semantics stay frozen (§3.5). This allows broader edits, such as inserting or deleting instructions and changing register usage. The original host launcher can still invoke the output unchanged.
Third, long-horizon search for assembly. AsmEvo extends the single-lineage evolution of AVO (Chen et al., 2026) to assembly optimization. The LLM only proposes edits. A deterministic controller decides correctness and speedup, using ABBA interleaving, A/A calibration, and the lower bound of a bootstrap confidence interval (§3.7). We also extend the single lineage to a tree-structured verified lineage. It supports multi-start exploration, verified composition, and stall-triggered redirection (§3.8). As a result, the 12-hour search accumulates only verified improvements.
These contributions show in the results. AsmEvo improved all 30 KernelBench kernels. It also improved all 8 production kernels from AITER and vLLM/SGLang (1.34× geometric mean, 2.08× maximum). Claude Code had the same model and the same original binaries and captures, but none of this infrastructure. It improved only 2 of the 8 production kernels (1.05×). With the model held fixed, this gap demonstrates the value of the complete AsmEvo framework, although Table 4 does not isolate the contribution of each design individually.