SWE-KERNEL-BENCH: A Live Benchmark for GPU Kernel Generation

We introduce SWE-KERNEL-BENCH, a live GPU kernel optimization benchmark that is periodically refreshed with new tasks derived from performance improvements merged into widely used open-source repositories. Its automated pipeline converts merged pull requests into tasks, each pairing an unoptimized source implementation with the corresponding maintainer-reviewed and shipped gold implementation. The benchmark evaluates whether coding agents can attain this performance target while preserving correctness. The first release contains 161 tasks from 10 production repositories and spans CUDA and Triton. Evaluating the benchmark with eight contemporary agentic systems shows that the best-performing system produces correct kernels for 91% of tasks, yet matches or exceeds gold performance on only 54%, revealing substantial headroom in real-world GPU kernel optimization.