acceptodds
Under review as a conference paper at ICLR 2027

Evok: Evidence-Centric Multi-Agent Framework for GPU Kernel Generation

Abstract

High-performance GPU kernels are critical for improving the efficiency of modern deep-learning workloads, but their development still relies heavily on expert tuning. While recent LLM-based approaches show promise for automating kernel generation, they still struggle to achieve stable correctness and high performance across diverse operators and GPU platforms. This limitation stems from candidate-centric generation, which leaves reusable semantic patterns, implementation choices, and measured failures implicit in prior optimization histories rather than preserving them as reusable search evidence. To address this limitation, organizes prior and runtime evidence as persistent optimization state to guide LLM-based kernel search. It introduces two coupled designs: 1) evidence-guided staged optimization, which decomposes generation into route planning, candidate realization, and local admission so that reusable optimization opportunities can guide exploration under target-local validation; and 2) evidence-aware multi-agent collaboration, which enables specialized agents to accumulate, refine, and exchange validated optimization evidence across successive kernel searches. Experiments on KernelBench across Tesla T4, RTX 4090, and RTX PRO 6000 show that achieves 3.6–8.6 level-wise mean speedup over PyTorch eager, outperforming direct-LLM and agentic baselines.

Primary area:
optimization
Keywords:
GPU kernel generation, large language models, multi-agent, kernel optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_XKPJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The work proposes Evok, an evidence-centric multi-agent framework for GPU kernel optimization. The key idea is to preserve prior optimization experiences as structured evidence. It features evidence-guided staged optimization, which decomposes generation into route planning, candidate realization, and local admission. In this way, the reusable optimization opportunities can guide exploration under target-local validation. Another point is the multi-agent collaboration that enables specialized agents to accumulate, refine, and exchange validated optimization evidence. Experiments on KernelBench across three GPU platforms report speedups over PyTorch and improvements over existing LLM-based kernel generation systems, including EGG and CudaForge.

Critical strengths
  1. This paper tackles GPU kernel optimization which is important. Evidence-based method is motivated.

  2. The framework proposes a way to structure and exchange successful optimization experiences.

  3. The empirical results are promising compared with other approaches.

Critical weaknesses
  1. Limited methodological novelty compared to existing agentic kernel optimization frameworks. EGG is closely similar, and this is not the first multi-agent system. As far as I know, https://arxiv.org/pdf/2509.07506 this paper seems to be the first multi-agent system, and K-search https://sky.cs.berkeley.edu/project/k-search/ and AccelOpt https://arxiv.org/pdf/2511.15915 also use memory / evidence based approach. Also the papers that they mentioned and cited should be considered as well. I would appreciate if the authors could read more about the work and clarify the distinction.

  2. The fairness of the experimental comparison is a concern. Do all methods in comparison receive the same token / money budgets? Matched search/inference budget can make the comparison more convincing.

Questions for authors

Please see the weakness parts.

Reviewer_GAUUOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Evok is a framework for GPU kernel generation that stores semantic transformations, implementation blueprints, and hardware-specific validation as reusable evidence. Three agents perform route planning, implementation, and verification, incorporating both successful and failed optimization attempts.

Critical strengths

Well-motivated evidence representation. Explicitly scoping positive and negative evidence while requiring fresh validation on the target GPU addresses a meaningful challenge: reusing optimization experience without assuming that performance transfers across hardware.

Critical weaknesses
  1. Insufficient validation against reward hacking. Prior work has documented reward hacking in GPU kernel generation, including exploitation of evaluation-state leakage [1] and shortcuts that exploit narrow test distributions to inflate measured speedups [2]. Evaluation on fresh, diverse valid inputs and explicit checks for timing and state manipulation would strengthen the central claims.
  2. The claimed contribution is not sufficiently isolated. Evok combines deterministic rewrites, implementation reuse, library paths, and runtime optimizations. The ablations do not clearly isolate structured evidence or multi-agent collaboration under otherwise identical resources.

[1] Baronio, C., et al. Kevin: Multi-Turn RL for Generating CUDA Kernels. arXiv preprint arXiv:2507.11948, 2025. [2] Zhang, Y., et al. KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch? arXiv preprint arXiv:2607.16241, 2026.

Questions for authors
  1. With identical templates, implementation assets, and search budgets, how much improvement comes specifically from typed evidence and multi-agent coordination?
  2. What concrete checks prevent reward hacking, including cached outputs, test-distribution shortcuts, and timing manipulation? Do the reported correctness and speedup results hold under an independent evaluation with fresh inputs and controlled execution state?
Reviewer_bjUZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Evok, an LLM-based system that generates GPU kernels for PyTorch programs and keeps the outcomes of earlier optimization runs as structured records that guide later searches. Each record has three parts: a semantic route with its preconditions (e.g., collapsing a constant result, simplifying a singleton dimension, aggregating before materializing a tensor), an implementation blueprint, and a validation outcome on a specific GPU (unverified for seed records; for measured records, a success or a failure tagged with the task, shape, dtype, backend and hardware it applies to). Three agents use the records in stages. An Evidence Agent retrieves records and selects a route that passes a deterministic guard; a Realization Agent produces and tunes candidate kernels by rewriting, by adapting a stored blueprint, or by LLM generation; and a Verification Agent compiles, tests and times the candidates against PyTorch eager and writes the outcome back. On the 250 KernelBench Level 1–3 tasks on Tesla T4, RTX 4090 and RTX PRO 6000, with GPT-5.5 as the model, Evok passes all correctness checks, is faster than eager on 98–100% of tasks, and reports level-wise mean speedups of 3.5–8.6× over eager, against 1.1–2.7× for torch.compile and 1.6–3.2× for EGG; the other baselines are direct generation with three LLMs, AutoTriton and CudaForge. Ablations on which evidence is retrieved (Figure 3) and on ledger size (Table 3) show that more evidence raises Fast1 on RTX 4090 from 48.0% to 100.0% and lowers LLM calls per task from 7.2 to 4.2, and Table 2 attributes the speedups to route types.

Critical strengths

The record design is clear and sensible. Keeping apart whether a rewrite is legal, how it is implemented, and whether it worked on a given GPU lets routes and blueprints move across GPUs while timing results do not: a slow or unstable result only down-ranks candidates within its recorded scope, and every reused route must be compiled, tested and timed again on the target GPU before it counts.

The reported results are strong and broad: 250 tasks on three GPUs, compiler baselines including max-autotune, and agentic baselines run with the same GPT-5.5 model. The ledger-size study (Table 3) and the mechanism table (Table 2) help show where the speedups come from.

Critical weaknesses
  1. The novelty is limited, and the experiments do not test what is new.

Prior work. Staged agents (EGG) and hardware feedback (CudaForge) already appear in the paper's own baselines, and reusing past successes and failures is established (e.g., ExpeL, AAAI 2024; AccelOpt, MLSys 2026). The paper should also say how its records differ from the skills or memories of recent kernel preprints (KernelBlaster, KernelSkill/KernelClaw, KLineage, KOPE), even though these need not be compared against.

Ablations. No single agent with the same ledger and tools tests whether three agents are needed, and no untyped memory (e.g., text notes) at the same budget tests whether the ⟨S, B, V_h⟩ structure matters; Figure 3 varies only which evidence is retrieved, against a direct-generation reference.

Case study. The Linear–Sigmoid–Sum case (Figure 2) is a standard row-wise fusion on a 128×10 input, and the paper does not show which records were retrieved or what happens without them.

  1. The gain over the baselines is not shown under equal information, and the metric and cost reporting do not support the headline claims.

Ledger. With an empty ledger, Evok gets 48.0% Fast1 and 2.1× on RTX 4090 (Table 3), below EGG's 78–100% Fast1 per level and 2.2× mean speedup (Table 1), so the advantage rests on a ledger whose exact provenance and size are not given and with which no baseline is run. The ledger may hold same-operator records from other GPUs (Section 4.1), there is no task-disjoint split, and the one result without same-operator records (96.0%, Section 4.5) cites an appendix missing from the PDF.

Metric. Speedup is an arithmetic mean over correct tasks relative to PyTorch eager, which a few large ratios can dominate: on RTX 4090 Level 2, direct generation with ChatGPT 5.5 reports 3.7×, above EGG and max-autotune, while beating eager on only 54% of tasks. There is no geometric mean, median or run-to-run variation, and part of the gain comes from removing computation that is redundant in the benchmark programs (Table 2).

Cost. Only LLM calls per task are reported, and not for the agentic baselines; tokens, wall-clock time, GPU time and the cost of the runs that built the ledger are not reported, and no general-purpose coding agent (e.g., Claude Code or Codex) is run with the same tools and budget.

Questions for authors

Ledger (clarification and experiment). Where do the ledger's seeds and historical traces come from, and what does Table 1 look like when a ledger built only from tasks disjoint from the evaluation tasks is given to Evok and EGG alike?

Ablations at a matched budget (experiment). How do three controls perform with the same evaluator and budget: one agent with the same ledger and tools, the same system with records stored as plain text notes, and a general-purpose coding agent?

Metrics and cost (clarification). What are the geometric-mean and median speedups, and the tokens, wall-clock time and GPU time per task (including the runs that built the ledger), for Evok and the baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.