acceptodds
Under review as a conference paper at ICLR 2027

Distill Before You Measure: Dual Pruning for Evaluation-Efficient GPU Kernel Optimization

Abstract

GPU kernel optimization requires repeated compilation, correctness checking, and performance measurement. When candidate generation outpaces target-device evaluation, device feedback becomes the search bottleneck. We propose teacher-distilled dual pruning, a two-stage framework for optimization under limited evaluation budgets. Offline, a strong language model analyzes public implementations, patches, and failure cases, distilling optimization actions, applicability conditions, coupled changes, and source-level checks into linked knowledge records. Online, these records guide two pruning layers. Search-space pruning excludes routes incompatible with task and hardware conditions and guides search and generation within the retained space. Candidate-set pruning combines failure knowledge with tool results to inspect generated programs, block confirmed violations, and guide repair and selection before measurement. The framework requires no training or fine-tuning of the online model and supports open- and closed-source models. On 100 KernelBench Level 2 tasks on NVIDIA RTX 4090, it finds faster implementations on 77% of tasks with at most three candidate evaluations per task. The evaluated baselines first reach the same coverage with five to nine evaluations, corresponding to a 40.0%–66.7% reduction in candidate-evaluation budget. Experiments on NVIDIA H100 and with GLM-5.3-Flash further demonstrate applicability across hardware platforms and online models.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
GPU kernel optimization, large language models, teacher-distilled knowledge, dual pruning, evaluation-efficient search, hardware-aware optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_x14hOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies LLM-driven GPU kernel optimization when only a few target-device evaluations are allowed per task. Offline, a strong teacher model distills public code, patches and failure cases into records that link an optimization action to its applicability conditions, coupled code changes and source checks (405 cards with 587 options for KernelBench Level 2); a second model checked selected records against their sources, the authors revised the records, and developer-maintained constructors and checks are versioned with them. Online, a fixed model uses the records in two layers. Search-space pruning removes design routes whose conditions are false for the task and hardware, and the model generates candidates within the remaining routes. Candidate-set pruning inspects, repairs, blocks and ranks the generated programs before measurement, and a persistent pool keeps unmeasured programs for later evaluations. On all 100 KernelBench Level 2 tasks (RTX 4090, non-thinking DeepSeek-V4.1-Flash, an initial pool of up to eight candidates, at most three evaluations per task), the method reaches Fast1 coverage of 46%, 72% and 77% after one, two and three evaluations, and strict Fast1.05 coverage of 45% after three. Five baselines (Best-of-N, Iterative, OpenEvolve, and local ports of STARK and EGG) need five to nine evaluations to reach 77%, which the paper reports as 40.0–66.7% fewer evaluations; at that coverage the method has used 94.8M online tokens versus 1.6–7.2M, and 12.6 versus 28.5–58.0 GPU-worker hours. The paper also reports an H100 run, a 24-task run with GLM-5.3-Flash, knowledge ablations on 24 tasks, and a Level 1 pool-size study.

Critical strengths
  1. The problem is well posed and the protocol is careful. Device evaluations are treated as the scarce resource: every attempted evaluation, including failures, consumes budget; all tasks stay in the denominator; and tokens and GPU time are reported separately. Under this protocol the method reaches Fast1 coverage earlier than every baseline on RTX 4090 and H100 (77% versus 63% after three evaluations on RTX 4090), and on RTX 4090 it also leads on the strict 1.05× criterion (45 versus 39 tasks). Where target hardware is scarce or remote, this is a useful result.

  2. The experimental record is extensive and self-critical. The appendices audit baseline results above 10× that pass the numerical checks but implement the wrong formula (task 42), re-measure selected sources in fresh processes alongside reference-equivalent controls, classify every winning program by the path it actually executes (Table 7), and include several knowledge studies. This makes the claims checkable; most of the concerns below come from these appendices.

Critical weaknesses

1. The novelty is limited, and the gains are not shown to come from the distilled knowledge. The first layer is essentially a rule filter over applicability conditions, and the second is code review, repair and ranking before measurement; both are familiar parts of agentic kernel optimization. The contribution therefore rests on the distilled knowledge, but the experiments do not isolate its effect:

  • The removal ablations also remove constructors, specialized checks and automatic patches (Table 10), and are compared with an earlier full-system run on a different model service, with one run per arm.
  • When tools and construction support are held fixed in the same batch (Table 34), adding the knowledge changes the Fast1 and strict counts after three evaluations by at most one of 24 tasks.
  • All 77 RTX 4090 winners keep a library convolution or matrix-multiplication body plus a custom tail kernel (Table 7), the split torch.compile makes by default, and a one-sentence hint to do this matches the full system's Fast1, though not its strict count. The task 82 case shows that the relevant records were in the prompt, not that they were needed.

2. The evaluation does not show an advantage over strong alternatives or a favorable total cost.

  • Metric. The headline numbers use Fast1, which reference-equivalent code can pass (the paper's controls measure up to 1.014×). Under the strict 1.05× criterion, the method leads on RTX 4090 but not on H100 or with GLM-5.3-Flash.
  • Baselines. The baselines are local ports, read at the first three evaluations of ten-evaluation runs, and at matched coverage they use 13–61× fewer tokens than the method. There is no token-matched baseline and no general-purpose coding agent (e.g., Claude Code or Codex) with the same evaluator and budget.
  • Cost. The method saves GPU evaluations by using about 40× more tokens than Iterative, and neither the end-to-end time per task nor the offline cost of building the knowledge is reported, so whether the trade pays off is unclear.
Questions for authors
  1. Attribution (experiment). In one batch and on one model service, with more than one run per arm, how does FULL compare with GENERIC PREP and with a removal arm that keeps the constructors, checks and patches and drops only the records?
  2. Baselines on equal terms (experiment). How do a token-matched baseline without teacher records (e.g., the same model generating eight candidates and reviewing and ranking them before each evaluation) and a general-purpose coding agent perform, both with the same evaluator and three evaluations?
  3. Cost (clarification). What did building the knowledge cost (teacher and reviewer tokens, author and developer hours), and what is the end-to-end time per task, including model latency, for the method and the baselines?
4 comments

We are grateful for the time you devoted to the review. Your questions showed where the shared record needed a more concrete explanation and guided our additional controls.

For ease of reading, we recap our notation. FULL denotes our complete method. B caps GPU evaluations per task and search. Fast₁ counts tasks with a correct implementation faster than the reference. Stable Fast₁ adds timing stability. Strict successes pass stability checks and exceed 1.05× in every seed and reference bracket.

Q1. What does the shared knowledge record contribute?

Thank you for asking us to make the shared record more concrete. It links a design choice to checks on its implementation. For a row of length N reduced in partitions of width b, the number of partial results, ceil(N/b), must agree across the launch grid, scratch allocation and merge bounds. A mean still divides by N. During design, the record helps choose a valid route. During inspection, it identifies the coupled source edits to check. A tool-confirmed violation can block evaluation. A concern raised by the model can prompt inspection or repair. Appendices L and N document these dependencies and checks.

Table 25. Source checks for a partitioned row reduction

QuantityValueWhere to check it
Partial resultsceil(N/b)Launch grid, scratch allocation, and merge bounds
Mean divisorNFinal reduction and normalization

Q2. What changes when only the delivery of teacher records varies?

We appreciate your request to isolate record delivery. The four configurations shared the same official DeepSeek service, tools, constructors, checks, patches, and evaluator. We used the 24 representative KernelBench Level 2 tasks selected for the submitted paper. Across three independent B3 searches per task, FULL achieved 46/72 strict successes, compared with 34/72 for generation-only delivery, 30/72 for assessment-only delivery, and 31/72 for generic preparation. FULL led generic preparation by 6, 6, and 3 strict successes across the repeats. Assessment-only delivery led on stable correctness, with 69 successes to FULL’s 66.

When the system retrieved plain summaries of 33 public source files, it achieved 26 strict successes, compared with 45 for FULL, across 69 completed matched pairs. In the separate same-source excerpt study (Table 40), the historical full-delivery run reached 19 Fast₁ tasks using 22.76M reported tokens. With excerpts, the system reached 13 tasks using 27.37M tokens.

Table 26. Four configurations with shared tools and support

Knowledge deliveryRepeat 1Repeat 2Repeat 3All 72Strict <10×
FULL23 / 22 / 1722 / 21 / 1621 / 17 / 1366 / 60 / 4646
Generation + repair21 / 15 / 920 / 15 / 1023 / 20 / 1564 / 50 / 3434
Assessment22 / 20 / 1123 / 18 / 924 / 18 / 1069 / 56 / 3029
Generic preparation22 / 17 / 1123 / 17 / 1020 / 17 / 1065 / 51 / 3131

Triplets are stable correct / stable Fast₁ / strict. Each repeat has 24 tasks. The last column applies the submission’s speed screen to strict results. The task-cluster bootstrap interval for FULL minus generic strict coverage is 8.3–34.7 percentage points (95%).

Table 27. Submitted knowledge and source comparisons at B3

StudyConfigurationTasksCorrectFast₁StrictTokens (M)
Stage deliveryGEN_ONLY241916919.169
Stage deliveryASSESS_ONLY24211798.512
Stage deliveryGENERIC_PREP24201786.638
CapabilitiesPrior FULL-C2423191321.909
CapabilitiesNO_POS242011813.770
CapabilitiesNO_NEG2422181012.708
CapabilitiesNEITHER24211677.555
CapabilitiesCoarse route hint24241965.978
Same sourcePrior FULL-E2420191122.763
Same sourceSource excerpts2419131027.368
Raw sourceFULL main 16-task subset1613137—
Raw sourceGeneric raw-source search161291—

Rows are comparable within the named study. FULL-C and FULL-E are the corresponding historical references from the submission.

Table 28. Plain summaries in place of the distilled records

CohortPairsFULLPlain summaryFULL tokensSummary tokens
All observed7266 / 60 / 4670 / 54 / 2763,966,65637,846,932
Complete B3 pairs6963 / 59 / 4567 / 52 / 2661,620,02137,491,074
Repeat 12423 / 22 / 1722 / 15 / 521,286,08913,373,953
Repeat 22422 / 21 / 1624 / 17 / 821,394,55013,838,406
Repeat 32421 / 17 / 1324 / 22 / 1421,286,01710,634,573

Triplets are stable correct / stable Fast₁ / strict. The summaries cover 33 public sources. The one-time summarization cost is 320,308 reported tokens.

Q3. How much does a library body with a custom tail explain?

We appreciate your question about how much the library body explains. All 77 faster RTX 4090 implementations used a library body and a custom tail. To assess the contribution of that design, we evaluated torch.compile with default Inductor on the full reference for each of the 24 tasks. The compiler used one fixed program per task and zero model calls. It achieved stable correctness on 24 tasks, stable Fast₁ on 15, and strict acceleration on 12. FULL’s first B3 search per task yielded 23, 22, and 17, respectively. A one-sentence hint matched the historical FULL run’s 19/24 Fast₁ tasks, with 6 vs. 13 strict successes. All comparisons timed the full forward pass.

On task 82, the selected implementation moved tanh after the window maximum. Because tanh is monotone, the rewrite cut 16 tanh evaluations per full window to one. The full forward pass achieved a 2.126× speedup in the original measurement and 2.129–2.130× in fresh processes. The source shows which operation changed. The matched comparisons above quantify the contribution of record delivery across tasks.

Table 29. FULL and the fixed compiler on the 24 tasks

ConfigurationGPU slotsStable correctStable Fast₁StrictTokensHTTP
FULL7223221721,286,089518
torch.compile2424151200

Each search uses B3 and one run per task. The compiler evaluates the original full task once per task with default Inductor and zero model calls.

Table 30. Single route hint compared with historical FULL

StudyConfigurationTasksCorrectFast₁StrictTokens (M)
CapabilitiesPrior FULL-C2423191321.909
CapabilitiesCoarse route hint24241965.978

Table 31. Source transformation in Level 2 task 82

ConfigurationMeasured speedupExecuted work
Raw-source search1.003762×Separate tanh, scaling, bias and pooling
FULL2.125992×Window max before monotone tanh, 16 calls → 1
FULL fresh-process repeats2.1298 / 2.1291 / 2.1299×Three remeasurements

Q4. How do the threshold, hardware, and online model affect coverage?

Thank you for asking us to separate threshold effects from hardware and model changes. On the 100 RTX 4090 tasks at B3, FULL led the best baseline 77 vs. 63 on Fast₁ and 45 vs. 39 under the strict criterion. Unfiltered counts were 77 vs. 65 and 45 vs. 40. The 12 reference-equivalent controls measured between 0.9931× and 1.0136×, below the strict threshold. On H100, FULL reached 70/100 Fast₁ at B2 vs. 52/100 for the best baseline and 75/100 vs. 69/100 at B3. Baselines reaching 75 needed B4–B7, so FULL used 25.0%–57.1% fewer slots per task. With GLM, FULL led Fast₁ 23/24 vs. 20/24 at B3. EGG led strict coverage on H100 (36 vs. 34), and Iterative led with GLM (15 vs. 14).

Table 32. B3 coverage with and without the 10× screen

MethodFast₁ <10× / allStrict <10× / all
FULL77 / 7745 / 45
Best-of-N52 / 5313 / 14
Iterative63 / 6529 / 31
OpenEvolve60 / 6028 / 28
STARK-port57 / 6034 / 38
EGG-port56 / 5739 / 40

Table 33. All 100 Level 2 tasks on H100

MethodFast₁ B1Fast₁ B2Fast₁ B3Correct B3Stable B3Strict B3Match 75 Fast₁
FULL477075898834B3
Best-of-N305161757223B7
Iterative285269959323B4
STARK-port204963858433B6
EGG-port385258706836—

Each method uses DeepSeek-V4.1-Flash. Fast₁ target crossings use observed prefixes. The B3 reduction is 25.0%–57.1% for baselines reaching 75 tasks.

Table 34. GLM-5.3-Flash on the same 24 Level 2 tasks

MethodB3 C / F / SB3 closedMax BMax B closedMax B F / STokens (M)Match 23 Fast₁
FULL23 / 23 / 1423B32323 / 1418.569B3
Best-of-N16 / 14 / 1123B102324 / 193.205B8
Iterative19 / 19 / 1523B102123 / 183.799B8
OpenEvolve20 / 18 / 1224B102423 / 184.268B9
STARK-port21 / 20 / 1423B102123 / 184.958B7
EGG-port15 / 13 / 924B102320 / 174.313—

C / F / S denotes Correct / Fast₁ / Strict. Closed counts are completed prefixes out of 24. Max-B tokens count reported online usage.

Table 35. GLM resources at 23 Fast₁ tasks

MethodBGPU receiptsWorker hTokens (M)HTTP
FULL3692.7418.569437
Best-of-N81775.352.572223
Iterative81757.913.049230
OpenEvolve92059.263.736256
STARK-port71516.163.063313

Q5. How do stronger baselines perform under the same GPU budget?

We appreciate your concern that the stopping budget can affect search behavior. The submitted B3 baselines use the first three evaluations from B10 runs of our shared-evaluator ports. Following your suggestion, we ran OpenEvolve directly at B3. The standard version achieved 5/24 strict successes. A variant explicitly told the budget achieved 2/24, compared with 17/24 for FULL’s first repeat. In the preparation-budget control, OpenEvolve used 71.84M reported tokens vs. 63.97M for FULL across 72 paired B3 searches. OpenEvolve achieved 45 vs. 60 stable Fast₁ successes and 22 vs. 46 strict successes. Both used DeepSeek-V4.1-Flash.

At B3, GPT-6 Astra in OpenEvolve achieved strict speedups on 17/24 tasks and stable correctness on 24/24. DeepSeek in FULL reached 17/24 and 23/24, respectively. Applying public prices to reported token use gives estimated API costs of USD 7.69–USD 8.35 for Astra vs. USD 3.13 for FULL. FULL’s estimate corresponds to 1.31–1.35 Astra evaluations per task.

Running Astra inside dual pruning would let us compare strict coverage and API cost with Astra–OpenEvolve while holding the online model and GPU budget fixed. To test a stronger teacher independently, we could ask it to distill the same public sources and evaluate its records with DeepSeek online. The general-purpose coding agent you suggest could use the same evaluator with three GPU evaluations per task.

Table 36. Direct search and compiler controls on 24 tasks

ConfigurationGPU slotsStable correctStable Fast₁StrictTokensHTTP
FULL7223221721,286,089518
DeepSeek light722317810,136,690502
DeepSeek deep72221689,985,002493
Standard OpenEvolve7218135320,96372
OpenEvolve informed of B37214102331,86172
torch.compile2424151200

Each search uses B3 and one run per task. The compiler evaluates the original full task once per task with default Inductor and zero model calls.

Table 37. Matched preparation budgets across paired searches

RepeatPairsFULLOpenEvolveFULL tokensOE tokens
Pooled7266 / 60 / 4659 / 45 / 2263,966,65671,841,885
Complete pairs6963 / 57 / 4457 / 43 / 2161,719,10669,304,809

Same DeepSeek-V4.1-Flash model, tasks and B3 budget. Triplets are stable correct / stable Fast₁ / strict. HTTP attempts total 1,530 and 3,468, with 87 and 4 attempts lacking usage, respectively.

Table 38. Astra and DeepSeek at the same B3 evaluation budget

ConfigurationCorrect / stableAll F / S<10× F / STokensAPI cost
Astra + OpenEvolve B324 / 2423 / 1722 / 16597,185USD 7.69–USD 8.35
DeepSeek + FULL B323 / 2322 / 1722 / 1721,286,089USD 3.13

Same 24 tasks, one search per task. F / S denotes stable Fast₁ / strict. The equal-spend comparison uses FULL’s B3 API cost. One Astra and 34 FULL requests lack usage.

Table 39. Initial candidate pool on 24 Level 1 tasks at B3

PoolCorrectStableFast₁StrictAM / GMTokens (M)HTTP
42020971.270 / 1.03819.943435
824241571.273 / 1.10126.581608
1624231971.328 / 1.18937.251788

Nine operator families. AM and GM are arithmetic and geometric means over correct tasks. Submitted Table 4.

Q6. What did online search and offline knowledge construction cost?

Thank you for asking for a fuller accounting of online search and knowledge construction. At matched Fast₁ coverage on the 100-task benchmark, FULL used 12.64 GPU-worker hours and 94.763M reported online tokens. The baselines used 28.51–58.02 hours and 1.565M–7.225M tokens. In the paired B3 study, the median accumulated API and worker times per search were 186 s and 206 s for FULL and 268 s and 148 s for expanded-compute OpenEvolve. End-to-end latency also depends on queueing and concurrency.

Our historical Level 2 records document the sources and revisions. We recognize the value of a fuller cost record. Our records for that build regrettably lack aggregate teacher and reviewer token counts and hands-on hours. To give a measured breakdown, we tracked extraction and audit use in the new 33-source study. Initial extraction used 0.699M reported tokens and auditing used 0.659M more. Extending the Level 2 workflow to Level 3 took about eight calendar days of Codex-assisted development, including integration and testing.

Table 40. Recorded resources to reach 77 Fast₁ tasks

MethodBGPU receiptsWorker hTokens (M)HTTP attempts
FULL330012.6494.7632,031
Best-of-N988732.511.565900
Iterative549829.832.387500
OpenEvolve989358.026.630900
STARK-port879333.327.2251,398
EGG-port769428.515.0961,087

Worker time is accumulated over evaluation receipts. Tokens are the reported online usage in the submission.

Table 41. Median accumulated time per paired B3 search

PhaseFULLOpenEvolve
Median API time / search186 s268 s
Median worker time / search206 s148 s

Table 42. Offline build effort on the same public sources

StageSourcesInput tokensOutput tokensTotal tokensHTTP
Initial extraction33530,319169,071699,39068
Additional audit33539,753119,294659,04752
Audited library total331,070,072288,3651,358,437120

The selected Level 2 catalog contains 405 cards and 587 options. Records document its sources and revisions. The table measures the new matched extraction study.

Thank you for helping us clarify the contribution and improve the comparisons. We will include the supporting analyses and tables in the revision and would be grateful to know whether they answer your concerns.

Reviewer_tc5rOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a two-stage pruning process for LLM-generated GPU kernels as a cheaper alternative for execution-based verification. Prior to the kernel generation, in an offline phase, a stronger teacher model reviews some public documents on kernel generation and creates a knowledge bank. Next, in the online phase, informed by the knowledge bank, they first search over a pruned design space, and then, guided by the tool and failure knowledge, they search over the candidate set. Their experiments on KernelBench show that they can reach a given performance with fewer candidates per task.

Critical strengths

The topic of LLM-aided Kernel search is very timely and important, and the paper highlights the execution-based verification as a bottleneck for scaling this approach. The paper is clearly written and does a good job of explaining the method for using LLMs to prune the candidate set. The ablations are well-designed and informative.

Critical weaknesses

The main motivation of the paper is that GPU evaluation is the bottleneck, but it's not clear whether this is actually the case. The proposed method uses significantly more online model compute (~95M tokens vs. 1.6–7.2M for the baselines), while the offline teacher cost is not quantified. I'd like to see a comparison of the end-to-end wall-clock time and compute cost, especially how long each GPU evaluation takes compared to model generation and reasoning. They show that they can be more evaluation efficient up to three GPU evaluations, however for the method to be considered as an alternative for the execution-based verification, they need to compare the scaling behavior of their method. In fact, from Fig. 1 and 3, it is unclear whether the method continues to scale or saturates early. Ideally, they should show the gains on harder tasks such as KernelBench level 3 too. While the authors have controlled for the timing noise, their main results are only with one search run per task, which doesn’t account for the sampling noise of runs. I’d suggest repeating the experiments with multiple seeds and with higher speedup thresholds.

Questions for authors

Have you compared against using the stronger teacher model directly during online optimization? Can you better isolate where the gains are coming from in the pipeline? For example, how much is due to generating better candidates versus selecting better ones? I’d suggest running an oracle selection experiment over a fixed candidate pool to clarify how much room for improvement remains in each component. What do you think about using symbolic-based Kernel verifiers for the kernel selection, such as [1]? With the advancements in this direction, do you still think the execution-based verification would remain a heavy bottleneck of the process? I’m curious to know if you have done ablations on scaling the offline knowledge construction? Both in terms of teacher compute and and the sources.

[1] https://arxiv.org/abs/2511.12638

5 comments

We sincerely thank you for your careful review. Your questions showed where the submission needed clearer evidence on cost, scale and verification and guided our follow-up analyses.

For ease of reading, we recap our notation. FULL denotes our complete method. B caps GPU evaluations per task and search. Fast₁ counts tasks with a correct implementation faster than the reference. Stable Fast₁ adds timing stability. Strict successes pass stability checks and exceed 1.05× in every seed and reference bracket.

Q1. What does a smaller GPU evaluation budget save?

Thank you for asking about practical savings. With limited access to the target GPU, we validate and time credited implementations there. On RTX 4090, FULL reached 77/100 Fast₁ at B3. Baselines reached it at B5–B9, so FULL used 40.0–66.7% fewer slots per task. Its 300 evaluations took 12.64 worker hours (about 152 seconds each for compilation, validation and timing). Baselines used 498–893 evaluations and 28.51–58.02 worker hours. Reported online use was 94.763M tokens for FULL and 1.565M–7.225M for baselines (submission Table 2).

The 72 paired B3 searches used the 24 representative KernelBench Level 2 tasks selected for the submitted paper. Median accumulated API time per search was 186 s for FULL and 268 s for expanded-compute OpenEvolve. Median GPU-worker time was 206 s and 148 s, respectively. End-to-end latency also depends on scheduling, endpoint queues and overlap.

Table 5. First observed budget at matched coverage

MethodFast₁ 46Fast₁ 72Fast₁ 77Strict 45
FULLB1B2B3B3
Best-of-NB3B6B9—
IterativeB2B4B5B9
OpenEvolveB2B7B9B10
STARK-portB3B5B8B5
EGG-portB2B6B7B5

Table 6. Recorded resources to reach 77 Fast₁ tasks

MethodBGPU receiptsWorker hTokens (M)HTTP attempts
FULL330012.6494.7632,031
Best-of-N988732.511.565900
Iterative549829.832.387500
OpenEvolve989358.026.630900
STARK-port879333.327.2251,398
EGG-port769428.515.0961,087

Worker time is accumulated over evaluation receipts. Tokens are the reported online usage in the submission.

Table 7. Median accumulated time per paired B3 search

PhaseFULLOpenEvolve
Median API time / search186 s268 s
Median worker time / search206 s148 s

Table 8. RTX 4090 coverage by evaluation budget

BFULLBoNIterativeOpenEvolveSTARKEGG
B171 / 46 / 3241 / 22 / 556 / 30 / 1447 / 20 / 929 / 16 / 1144 / 32 / 21
B286 / 72 / 4162 / 41 / 1090 / 55 / 2375 / 50 / 2066 / 40 / 2664 / 47 / 30
B390 / 77 / 4573 / 52 / 1395 / 63 / 2979 / 60 / 2878 / 57 / 3472 / 56 / 39
B4—77 / 59 / 1596 / 74 / 3181 / 67 / 3486 / 68 / 4279 / 62 / 41
B5—80 / 67 / 2497 / 79 / 3382 / 69 / 3490 / 74 / 4988 / 71 / 45
B6—83 / 73 / 2699 / 85 / 3585 / 71 / 3891 / 76 / 5091 / 75 / 47
B7—85 / 75 / 2899 / 87 / 3888 / 75 / 4093 / 76 / 5092 / 77 / 48
B8—85 / 75 / 2899 / 91 / 4389 / 76 / 4294 / 78 / 5294 / 80 / 50
B9—87 / 79 / 31100 / 93 / 4690 / 78 / 4395 / 81 / 5695 / 80 / 52
B10—89 / 79 / 34100 / 94 / 4990 / 78 / 4595 / 81 / 5696 / 84 / 53

Each cell is Correct / Fast₁ / Strict out of 100. The submission excludes ≥10× observations from speed counts. FULL was evaluated through B3.

Q2. How does offline analysis effort affect the resulting library?

We appreciate your focus on library construction. On the same 33 public sources, initial DeepSeek extraction used 0.699M reported tokens and the audit used 0.659M, totaling 1.358M. At one B3 search per task, both libraries reached 8/24 strict speedups. Teacher effort varied at fixed corpus size. Adapting the Level 2 workflow to Level 3 took about eight calendar days of Codex-assisted development for source analysis, integration and testing.

Table 9. Offline build effort on the same public sources

StageSourcesInput tokensOutput tokensTotal tokensHTTP
Initial extraction33530,319169,071699,39068
Additional audit33539,753119,294659,04752
Audited library total331,070,072288,3651,358,437120

The selected Level 2 catalog contains 405 cards and 587 options. Records document its sources and revisions. The table measures the new matched extraction study.

Table 10. Online results of the two knowledge libraries

ConfigurationGPU slotsStable correctStable Fast₁StrictTokensHTTP
DeepSeek light722317810,136,690502
DeepSeek deep72221689,985,002493

Q3. How does the method scale to B10 and to Level 3?

We appreciate your request to test larger budgets and complete models. We extended the same 72 FULL searches on 24 Level 2 tasks from B3 to B10. The number of searches with a strict speedup rose from 46 at B3 to 49 at B5 and 54 at B10. Stable Fast₁ counts were 60, 62, and 70 at those budgets. B3 accounted for 85% of the strict successes observed by B10.

Level 3 on complete models. We evaluated all 50 tasks on RTX 4090 with DeepSeek-V4.1-Flash. FULL targeted eight initial candidates and used at most three evaluations per search. Across B1, B2 and B3, its Correct coverage rose from 48 to 49 to 50, Fast₁ from 22 to 30 to 32, and strict acceleration from 1 to 6 to 8. The best baselines after one evaluation reached 26 correct and 14 Fast₁ tasks. At B3, the corresponding best baseline counts were 44 and 30. EGG led B3 strict coverage (9 vs. 8). FULL covered more tasks than EGG on correctness (50 vs. 32) and Fast₁ (32 vs. 18). FULL also obtained 49 timing-stable correct and 31 stable Fast₁ results.

Iterative first matched 48/50 correct at B5, versus B1 for FULL, requiring 80% fewer evaluation slots for FULL at that target. Best-of-N and Iterative first matched 32/50 Fast₁ at B4, versus B3 for FULL, a 25% lower slot budget. On the 49 tasks for which all six methods completed B3, FULL led the strongest baseline 49 to 44 on correctness and 32 to 30 on Fast₁.

FULL obtained correct implementations across all 12 architecture groups and Fast₁ results in 11 groups. Coverage includes 7/9 mobile or grouped CNNs, 4/5 vision or window models, 3/3 causal-attention tasks and 5/10 recurrent models. Both state-space tasks passed correctness checks. The knowledge for these full models tracks repeated work, producer–consumer dependencies and observable state. Across search, recovery and fixed-source checks, FULL recorded 166 positions and 76.120M online tokens. The five baselines completed 249/250 B10 searches.

For complete models, the 167 knowledge cards and 285 options describe the work to save, the implementation, the changes that must be made together, and the semantics to verify. They connect a proposed action to a region, its producer and consumer, and any observable state. For example, whole-sequence and chunked projections are alternatives for the same recurrent work. A confirmed source defect blocks evaluation until the source is repaired and rechecked.

Validation on recurrent task 33 used the same candidate sources. Two implementations passed all 12 initial, carry and reset checks and Sanitizer. The retained implementation ran at 0.9163×, adding correctness credit. The candidate with a tanh sign error failed all 12 state checks.

Table 11. Level 2 searches extended through B10

BudgetSearchesStable correctStable Fast₁Strict
B372666046
B572686249
B1072717054

24 tasks × three independent searches. All later budgets retain the original B3 candidates and outcomes.

Table 12. Level 3 coverage at each GPU budget

BFULLBoNIterativeOpenEvolveSTARKEGG
B148 / 22 / 125 / 14 / 225 / 7 / 323 / 9 / 226 / 10 / 417 / 6 / 2
B249 / 30 / 634 / 22 / 538 / 16 / 333 / 12 / 334 / 13 / 623 / 10 / 5
B350 / 32 / 842 / 30 / 644 / 25 / 439 / 17 / 338 / 18 / 632 / 18 / 9
B4—46 / 37 / 747 / 32 / 540 / 24 / 341 / 24 / 834 / 20 / 9
B5—46 / 39 / 948 / 34 / 642 / 26 / 442 / 26 / 936 / 22 / 9
B6—46 / 39 / 949 / 36 / 643 / 27 / 543 / 29 / 1040 / 24 / 9
B7—46 / 39 / 949 / 36 / 745 / 29 / 544 / 30 / 1244 / 26 / 10
B8—46 / 39 / 1049 / 38 / 1045 / 29 / 645 / 33 / 1344 / 26 / 10
B9—46 / 39 / 1249 / 39 / 1145 / 30 / 646 / 33 / 1346 / 28 / 11
B10—47 / 42 / 1549 / 39 / 1346 / 31 / 747 / 34 / 1348 / 31 / 13

Each cell is Correct / Fast₁ / Strict out of 50. Speed counts apply the ≥10× screen. At B3, FULL has 49 timing-stable correct tasks and 31 stable Fast₁ tasks, alongside 50 correct and 32 Fast₁ tasks on the main criteria.

Table 13. Level 3 budgets to reach the same coverage

TargetFULLBoNIterativeOpenEvolveSTARKEGG
Correct 48/50B1—B5——B10
Correct 49/50B2—B6———
Correct 50/50B3—————
Fast₁ 22/50B1B2B3B4B4B5
Fast₁ 30/50B2B3B4B9B7B10
Fast₁ 32/50B3B4B4—B8—
Strict 8/50B3B5B8—B4B3

The earliest baseline reaches 48 correct tasks at B5 (80% fewer slots for FULL at B1), 49 correct at B6 (66.7% fewer at B2), and 32 Fast₁ tasks at B4 (25% fewer at B3). A dash indicates no observed match within the evaluated budget.

Q3 continued. How does the method scale to B10 and to Level 3?

We appreciate the need to compare the methods on the same complete-model tasks.

Table 14. Level 3 tasks with completed prefixes for all six methods

MethodB1 · 50 tasksB3 · 49 tasks
FULL48 / 22 / 149 / 32 / 8
Best-of-N25 / 14 / 242 / 30 / 6
Iterative25 / 7 / 344 / 25 / 4
OpenEvolve23 / 9 / 239 / 17 / 3
STARK-port26 / 10 / 438 / 18 / 6
EGG-port17 / 6 / 232 / 18 / 9

Triplets are Correct / Fast₁ / Strict. On the 49 common B3 tasks, FULL leads the best baseline 49 to 44 on correctness and 32 to 30 on Fast₁. EGG leads strict coverage 9 to 8.

Table 15. Level 3 coverage by model architecture

ArchitecturenFULL C / F / SBoN Fast₁Iterative Fast₁OpenEvolve Fast₁STARK Fast₁EGG Fast₁
MLP33 / 1 / 023111
Sequential CNN44 / 3 / 031222
Residual CNN33 / 2 / 112211
Dense connectivity44 / 3 / 132322
Multibranch CNN44 / 1 / 013110
Mobile / grouped CNN99 / 7 / 355253
Vision / window blocks55 / 4 / 032100
Recurrent models1010 / 5 / 176224
Causal attention33 / 3 / 210232
Encoder–decoder11 / 1 / 010111
VLAD aggregation22 / 2 / 021001
State-space models22 / 0 / 010001
All tasks5050 / 32 / 83025171818

Groups partition all 50 tasks by architecture. FULL obtains correct results in all 12 groups and Fast₁ successes in 11. The two state-space tasks are correct and have central speedups at or below 1×.

Table 16. Level 3 recorded GPU positions and online token use

MethodBudgetClosed prefixesPositionsTokensUnknown usage
FULLB34916676,120,43217
Best-of-NB10505002,222,3630
IterativeB10505004,768,1170
OpenEvolveB10505005,120,0210
STARK-portB10505007,132,2160
EGG-portB10494976,073,6310

FULL’s 166 positions include the original 145, 12 recovery positions and nine fixed-source diagnostic positions. Baselines complete 249/250 B10 trajectories and record 25,316,348 tokens. EGG task 4 times out during compilation at slot eight and retains its earlier measured result. The 2,497 recorded baseline positions and the failed slot account for 2,498 budget positions.

Q4. Do the gains persist across repeats and higher speedup thresholds?

Thank you for asking about sensitivity to the speed threshold and repeated searches. Across three independent B3 searches per Level 2 task, FULL found strict speedups on 17, 16, and 13 of 24 tasks, compared with 11, 10, and 10 for generic preparation. On the original 100-task RTX 4090 run, FULL reached 45 strict speedups at B3 and the best baseline reached 39. At thresholds of 1.10×, 1.25×, 1.50×, and 2.00× applied to the central speedup estimate, FULL covered 38, 27, 12, and 2 tasks. The baseline maxima were 37, 25, 14, and 3. All six selected strict implementations also cleared the strict threshold in each of three fresh-process remeasurements.

Table 17. Strict successes in the three independent searches

RepeatFULLGeneric preparation
11711
21610
31310

Each repeat uses the same 24 tasks.

Table 18. Sensitivity to performance thresholds on RTX 4090

MethodFast₁ <10× / allStrict <10× / all1.05×1.10×1.25×1.50×2.00×
FULL77 / 7745 / 45473827122
Best-of-N52 / 5313 / 141510731
Iterative63 / 6529 / 31302515113
OpenEvolve60 / 6028 / 2830251662
STARK-port57 / 6034 / 38363223132
EGG-port56 / 5739 / 40413725143

B3, 100 tasks. The five threshold columns use the screened central ratio. Strict separately requires timing stability and all seed/reference comparisons above 1.05×.

Table 19. Independent verification of fixed implementations

MethodSourcesCorrect 3/3Fast₁ 3/3Strict 3/3
FULL101086
Best-of-N5530
Iterative9974
OpenEvolve8852
STARK-port8843
EGG-port8876
Reference equivalent121210

48 frozen results and 12 controls, three fresh processes each, 180 evaluations and zero API calls. The 36 control timings span 0.993068×–1.013605×.

Reviewer_FGduOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes teacher-distilled dual pruning for GPU kernel optimization under limited device-evaluation budgets. Offline knowledge extracted from public implementations and failures guides design-space pruning and candidate inspection, repair, and selection, while a persistent pool retains unmeasured alternatives. On 100 KernelBench Level 2 tasks on RTX 4090, the method finds faster implementations for 77% of tasks with three evaluations per task, compared with five to nine for the baselines. Additional experiments investigate cross-hardware and cross-model applicability and the contributions of knowledge and supporting capabilities.

Critical strengths

1.A meaningful formulation with substantial early-coverage improvements. When candidate generation capability is relatively abundant, evaluation on the target device can become the bottleneck. On the RTX 4090, the proposed method achieves a Fast1 coverage of 77% after three evaluations, compared with 63% for the strongest baseline under the same budget. The method also leads on the stricter metric that additionally requires timing stability: 45% coverage versus a baseline maximum of 39%. These results support the practical value of improving candidate quality before measurement.

2.Concrete connections between optimization knowledge and executable behavior. The knowledge records contain applicability conditions and implementation details that must be modified jointly, rather than merely listing optimization suggestions. Through source-code examples such as checking dependency relations in reduction operations and verifying whether a fused helper function is actually invoked, the method acquires clear technical substance. Distinguishing between "tool-confirmed violations" and "model-raised concerns" is also a sensible design choice.

Critical weaknesses

1.The experiments do not sufficiently isolate the contribution of dual pruning. The capability-removal experiments change several components at once: NO_POS removes the specialized constructor and route-selection support, while NO_NEG removes the dedicated inspection and patching functionality (Appendix D.3). In addition, FULL-C comes from an earlier run using the official model service, whereas the capability-removal groups use a compatible service. These comparisons therefore measure the effect of the whole set of capabilities rather than the individual roles of the two pruning layers. In the more carefully controlled knowledge-delivery experiment, the variants that use knowledge only in the generation phase, only in the evaluation phase, and the generic-preparation group yield 9, 9, and 8 strictly sped-up tasks respectively, but the full method—which uses knowledge in both phases—is missing from the same batch. The additional benefit of combining the two phases therefore remains unverified.

2.The contribution beyond substantially increased preparation remains uncertain. At the same 77% coverage, the proposed method reports an online token usage of 94.763 million, versus 1.565 million to 7.225 million for the baselines (Table 2). The paper transparently reports this trade-off, and for its stated research goal, using an identical device evaluation budget is reasonable. Still, a generic-preparation baseline with a comparable model compute budget is needed to determine how much of the gain comes from the proposed knowledge organization and pruning mechanism and how much from the additional candidate generation, inspection, and repair. Moreover, because each task is searched only once, the magnitude of variance in these differences is unknown.

Questions for authors

1.Can you provide a same-batch comparison of full dual-stage knowledge delivery, generation-only delivery, assessment-only delivery, and generic preparation, using identical tools, construction support, model service, and preparation allowances?

2.Under a comparable online model-computation budget, how well does generic candidate generation, inspection, and repair perform? This would help distinguish the benefit of the proposed mechanism from the benefit of additional preparation.

3.How variable are the principal coverage differences across independent search runs? Timing repeats of fixed sources address measurement stability, but do not establish the robustness of the search policy.

1 comment

We sincerely thank you for your careful review. The submission could have shown each stage’s contribution more clearly. Your questions guided our same-batch ablation.

For ease of reading, we recap our notation. FULL denotes our complete method. B caps GPU evaluations per task and search. Fast₁ counts tasks with a correct implementation faster than the reference. Stable Fast₁ adds timing stability. Strict successes pass stability checks and exceed 1.05× in every seed and reference bracket.

Q1. Does delivering teacher records at both stages help?

We appreciate your request to isolate each stage. The four configurations ran together on the 24 representative KernelBench Level 2 tasks selected for the submitted paper, with three independent searches per task and configuration. They shared the official DeepSeek service, constructors, tools, checks, patches, evaluator and preparation limits. Each search targeted eight initial candidates with three GPU evaluation slots. We varied which stages received teacher records. Submission Table 10 studies removing both records and their tools.

In 72 searches, joint delivery yielded 12 more strict successes than generation-only, 16 more than assessment-only and 15 more than generic preparation. The submission’s ≥10× screen lowers the assessment-only strict count from 30 to 29.

Table 1. Same batch knowledge delivery at B3

Record deliveryStable correct /72Stable Fast₁ /72Strict /72Strict per repeat (each /24)
FULL66604617 / 16 / 13
Generation and repair only6450349 / 10 / 15
Assessment only69563011 / 9 / 10
Generic preparation65513111 / 10 / 10

Q2. How does generic preparation perform at comparable token use?

We appreciate your point about comparable token budgets. We set OpenEvolve’s token targets from paired FULL runs and used extra requests for generation, inspection and ranking without teacher records. Across 72 B3 pairs, OpenEvolve reported 71.84M tokens to FULL’s 63.97M (+12.3%, per-search ratio 1.00–1.16). FULL achieved 60 vs. 45 stable Fast₁ and 46 vs. 22 strict successes. Both used DeepSeek-V4.1-Flash. Repeated refinements of similar designs with extra tokens yielded limited gains, suggesting a risk of local optima. This motivated pruning the decision space with distilled knowledge to improve search efficiency.

Table 2. FULL versus OpenEvolve with matched preparation

RepeatPairsFULLOpenEvolveFULL tokensOE tokens
12423 / 22 / 1720 / 14 / 721,286,08923,711,964
22422 / 21 / 1620 / 16 / 1021,394,55024,097,332
32421 / 17 / 1319 / 15 / 521,286,01724,032,589
Pooled7266 / 60 / 4659 / 45 / 2263,966,65671,841,885
Complete pairs6963 / 57 / 4457 / 43 / 2161,719,10669,304,809

Same DeepSeek-V4.1-Flash model, tasks and B3 budget. Triplets are stable correct / stable Fast₁ / strict. HTTP attempts total 1,530 and 3,468, with 87 and 4 attempts lacking usage, respectively.

Q3. How much did the result vary across independent searches?

Thank you for emphasizing repeated searches. FULL led generic preparation by 6, 6, and 3 strict successes, with a pooled advantage of 20.8 percentage points (95% bootstrap interval 8.3–34.7 points). Generation-only delivery led in the third repeat. Resampling kept each of the 24 tasks’ three searches together. FULL’s strict counts against expanded-compute OpenEvolve were 17 vs. 7, 16 vs. 10, and 13 vs. 5.

Table 3. Four configurations with shared tools and support

Knowledge deliveryRepeat 1Repeat 2Repeat 3All 72Strict <10×
FULL23 / 22 / 1722 / 21 / 1621 / 17 / 1366 / 60 / 4646
Generation + repair21 / 15 / 920 / 15 / 1023 / 20 / 1564 / 50 / 3434
Assessment22 / 20 / 1123 / 18 / 924 / 18 / 1069 / 56 / 3029
Generic preparation22 / 17 / 1123 / 17 / 1020 / 17 / 1065 / 51 / 3131

Triplets are stable correct / stable Fast₁ / strict. Each repeat has 24 tasks. The last column applies the submission’s speed screen to strict results. The task-cluster bootstrap interval for FULL minus generic strict coverage is 8.3–34.7 percentage points (95%).

Table 4. Paired differences across the 24 task clusters

MeasureFULL − OpenEvolve95% interval
Correct difference (pp)+9.72[-4.17, 25.00]
Stable Fast₁ difference (pp)+20.83[6.94, 36.11]
Strict difference (pp)+33.33[18.06, 48.61]

Differences and intervals are in percentage points. Bootstrap resampling keeps each task’s three searches together.

Thank you for guiding these analyses. We will include Tables 1–4 in the revision and would be grateful to know whether the evidence addresses your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.