acceptodds
Under review as a conference paper at ICLR 2027

CUDACoach: Guiding CUDA Kernel Optimization with Diverse Strategies Learned from Verified Code

Abstract

Optimizing CUDA kernels is critical for high-performance deep learning systems. Recently, multi-role iterative frameworks powered by large language models have advanced this optimization process by establishing an automated “planning-generation-execution” pipeline, which effectively enhances kernel correctness and performance. However, improving strategy planning within these frameworks requires addressing two challenges: (a) Off-the-shelf planners repeat generic strategies across kernels with different performance profiles, limiting the search for faster implementations. (b) Supervised training to improve these recommendations requires reliable strategy demonstrations, but successful kernel implementations alone do not provide explicit modification guidance. We introduce CUDACoach, a framework for learning repair and optimization strategies to guide a frozen coder. (a) To obtain reliable demonstrations for supervised initialization, we propose verified strategy mining, which reconstructs strategies from successful kernel edits and validates them through implementation by the code model. (b) To counter repetitive optimization strategies, we use a two-level diversity reward during reinforcement learning to encourage distinct technique combinations among candidates and downweight techniques frequently used across training. We evaluate CUDACoach on all 250 KernelBench tasks with three downstream coders. With Qwen3-Coder-Plus as the coder, our method achieves 60.4% correctness and produces correct kernels faster than PyTorch on 19.2% of tasks, compared with 44.8% and 13.2% for single-stage iterative refinement and 42.4% and 13.6% when the coder also serves as the planner. Ablations confirm the benefits of diversity-reward training and the multi-kernel context for kernel correctness and optimization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
CUDA kernel optimization, Large language models, Reinforcement learning, Verified strategy mining, Diversity reward

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KAqgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CUDACoach trains a Qwen3-8B planner to guide a frozen CUDA coder through kernel repair and optimization. It constructs supervised examples by recovering strategies from successful teacher edits, then retaining a strategy only if the coder can implement it successfully without seeing the teacher's improved kernel. Subsequent GRPO training combines execution and format rewards with a diversity reward that uses historical technique frequencies and within-state log-determinant diversity. At inference, the planner can compare multiple correct kernels and their profiles, while strategy clustering limits redundant implementations. Experiments cover all 250 KernelBench tasks with three downstream coders. The largest improvement is with Qwen3-Coder-Plus: correctness rises from 44.8% to 60.4% and fast1 from 13.2% to 19.2% relative to single-stage iteration.

Critical strengths
  1. Verified strategy mining is a concrete and useful supervision design. The frozen coder must realize the recovered strategy without access to the teacher's improved kernel, and its implementation must pass execution checks. This is stronger evidence of actionable guidance than accepting a plausible retrospective explanation alone. The detailed prompts and reward specification also make the approach tractable.

  2. The learned planner has strong results in a useful matched-harness comparison. With Qwen3-Coder-Plus, it improves correctness from 42.4% to 60.4% over self-planned compositional iteration, with fast1 increasing from 13.6% to 19.2% (Table 1). Against single-stage iteration, the gains correspond to 39 additional correct tasks and 15 additional faster-than-reference tasks. Evaluation across all three KernelBench levels and multiple coders is informative, even though benefits vary by coder.

Critical weaknesses
  1. The mechanism-specific evidence does not yet isolate the proposed contributions. Most directly, the correctness gain attributed to multi-kernel context is difficult to understand with the stated algorithm. Section 3.4 and Appendix use that context only when a correct-kernel pool already exists; an initially incorrect task undergoes repair with an empty pool until its first success, after which historical-best retention preserves correctness. Changing only the optimization context therefore has no apparent causal path to task correctness when repair trajectories are held fixed. Yet Table 2 reports 60.4% versus 52.8% correctness, and Section 4.4 reports a seven-task gap already at iteration 1, before multiple correct parents can be available. More broadly, the paper does not compare SFT-only training with RL, verified with unverified reconstructed strategies, or historical weighting with local diversity alone. Because planner-coder separation (Zhu et al., 2026) and determinant-based diversity (Chen et al., 2026b) have precedents, these controls matter for establishing the distinctive contribution.

  2. The acceleration claims need stronger measurement and uncertainty controls. Appendix selects the archived maximum measured speedup while explicitly allowing reference timings to vary across evaluations; the selected kernel can therefore differ from the minimum-latency kernel. It does not state an independent final remeasurement against a common reference. The diversity reward adds only four fast1 tasks and two fast2 tasks (Table 2), while the Qwen3.8-Flash fast1 gain is one task and its correctness falls by 16 tasks (Table 1). These facts do not negate the substantial Qwen3-Coder-Plus correctness gain, but they limit confidence in the claimed optimization benefits and their generality.

Questions for authors
  1. What differs between the full and no-multi-kernel arms before the first correct implementation is found? Please explain the iteration-1 correctness gap and show a comparison that shares identical repair prompts, generations, and outcomes until first success, then varies only the optimization context. If independent sampling explains the gap, please separate that variation from the claimed context benefit.

  2. How much do execution verification and the two diversity levels add beyond the SFT checkpoint and execution-reward RL? Targeted, comparably budgeted controls for verified versus unverified strategies and global versus local diversity would materially affect my assessment. Please also clarify the contribution of the auxiliary analyses.

6 comments
AuthorsOpenReview ↗

Response to the Concerns of Reviewer [Part 1]

Q1: Correctness differences before the first successful repair and the contribution of multi-kernel context.

A1: We thank the reviewer for identifying the need to separate repair from optimization. Multi-kernel context is used after a correct implementation becomes available. Its intended contribution is to improve the search for faster implementations. When repair generations and outcomes are shared, this mechanism does not change whether the first correct implementation is found. Historical-best retention then preserves that correct implementation during optimization.

(1) Why the original correctness curves differ. The original comparison did not enforce identical sampled repair trajectories. Although both arms started from the same initial kernels and used the same repair procedure, their generated repair strategies and implementations could differ. The aggregate correctness difference of 60.4% versus 52.8% therefore should not be interpreted as the isolated effect of optimization context.

We traced the seven-task gap at iteration 1 to a net difference of six successful repairs and one task counted inconsistently in the plotted results. The first repair step succeeded on 24 initially incorrect tasks in one arm and 18 in the other. We will align the treatment of the remaining task in the revised curve. The early gap reflects differing repair outcomes and counting, rather than a benefit from comparing multiple correct kernels.

(2) Evaluation from a shared repair trajectory. To address the reviewer's requested comparison, we preserve the same repair prompts, generated strategies, candidate implementations, and evaluation outcomes until the first successful repair. Both optimization branches then start from the same correct kernel, feedback, and profiling information. We reuse the recorded single-context continuation and run multi-context continuations from this shared state. Single-context optimization uses the best correct parent, while multi-context optimization can use multiple correct parents as they become available.

This evaluation includes all 40 initially incorrect tasks that were successfully repaired in the recorded single-context trajectory and had optimization iterations remaining.

We obtain the following results. The geometric mean measures speedup over each task's shared first correct implementation. The fast₁ and fast₂ counts use the same fixed PyTorch reference timings for both configurations.

MetricSingle contextMulti-kernel context
Correct tasks40/4040/40
fast₁10/4013/40
fast₂6/407/40
Geometric-mean speedup over the shared starting kernel4.75×5.28×

The multi-context configuration yields three additional tasks faster than PyTorch and a higher geometric-mean improvement over the shared starting kernels.

(3) Optimization of initially correct tasks. We also report a subset analysis of the paper's results on all 91 tasks with the same correct initial implementation in both configurations. These tasks enter optimization without requiring an initial repair.

MetricSingle contextCUDACoach
Correct tasks91/9191/91
fast₁28/9136/91
fast₂15/9116/91
Geometric-mean speedup over the shared initial kernel3.46×3.96×

On these initially correct tasks, CUDACoach yields eight additional tasks faster than PyTorch and a higher geometric-mean improvement over the shared initial kernels. Together, the two analyses cover optimization from an initially correct implementation and from a shared successful repair. We will revise the discussion to distinguish these optimization results from the variation in repair outcomes before the first correct implementation is found.

AuthorsOpenReview ↗

Response to the Concerns of Reviewer [Part 2]

Q2: Contributions of execution verification, reinforcement learning, and historical frequency weighting.

A2: We thank the reviewer for requesting a clearer account of the training components. Strategy verification tests whether a reconstructed strategy can guide a successful implementation, while RL trains the planner using feedback from the implementations produced by the frozen coder. Historical frequency weighting adds information about technique repetition across training states to the local diversity objective.

(1) Execution-verified supervision and reinforcement learning. A successful teacher edit does not by itself establish that a reconstructed strategy provides actionable guidance. We therefore ask the frozen coder to apply each strategy to the original kernel without seeing the teacher's improved implementation. Repair strategies must pass compilation and numerical checks, while optimization strategies must additionally reduce latency relative to the parent kernel.

Role of the auxiliary analyses. The 4,700 CASS kernel analyses provide supervision on algorithms, memory access, thread mapping, synchronization, and performance bottlenecks. They are mixed with verified repair and optimization strategies during SFT, combining understanding of the current implementation with learning actionable changes from execution feedback.

All configurations below use Qwen3-Coder-Plus on the 250 KernelBench tasks.

Planner configurationCorrect tasksfast₁fast₂
SFT on unverified strategies1063520
SFT on verified strategies1143720
RL without the diversity reward1334421
Full CUDACoach1514823

In the table above, verified-strategy SFT covers eight additional correct tasks and two additional fast₁ tasks, with fast₂ unchanged. Because verification excludes strategies whose downstream implementations fail execution checks, the retained strategy supervision consists of guidance that the coder has successfully implemented. We will add these SFT comparison results to the revised manuscript.

RL without the diversity reward improves over SFT on all three metrics. Compared with this RL baseline, full CUDACoach yields 18 additional correct tasks, four additional fast₁ tasks, and two additional fast₂ tasks.

(2) Historical frequency weighting beyond local diversity. The reward ablation above removes both diversity components, so we additionally examine saved local log-det and historical-weighted configurations.

Local log-det evaluates complementarity only among strategies for the same state. Local diversity can remain high even when a planner repeatedly recombines the same small set of techniques across tasks. Historical frequency weighting reduces common techniques' weights in the Gram matrix, adding information across states while retaining complementarity within each state.

We compare the recorded training configurations at updates 10, 20, and 30 on the same 199 optimization states. At each checkpoint, four strategies per state give 796 strategies per configuration. Each strategy receives four coder implementations, and its outcome is determined by the best execution reward. These are strategy-level training-validation rates. Correctness requires a correct implementation. Acceleration success additionally requires speedup greater than 1.1 times the execution-reward baseline.

Training updateLocal correctnessWeighted correctnessLocal acceleration successWeighted acceleration success
1074.87%77.14%56.78%57.66%
2080.53%81.41%61.31%62.69%
3080.78%83.42%62.69%64.32%

The historical-weighted configuration has higher correctness and acceleration success at all three reported checkpoints. At update 30, its correctness is 2.64 percentage points higher. The acceleration-success gap grows from 0.88 to 1.38 to 1.63 percentage points over updates 10, 20, and 30.

AuthorsOpenReview ↗

Response to the Concerns of Reviewer [Part 3]

Q3: Consistency of the diversity-reward gains and reliability of speedup measurement.

A3: We examined whether the diversity-reward gains persist when the no-diversity baseline is repeated. The full result reported in the paper exceeds the original baseline and both additional baseline runs in correctness, fast₁, and fast₂. We first clarify the measurement protocol, then report these comparisons.

Consistent reference timing. The appendix inadvertently retained wording from an earlier implementation. CUDACoach uses a fixed PyTorch reference latency for each task. Among candidates that pass verification, maximizing speedup is therefore equivalent to minimizing kernel latency. Recomputing the speedups using a common fixed reference per task leaves the fast₁ and fast₂ counts for CUDACoach and all three single-stage baselines unchanged. We will correct the appendix to state this protocol clearly.

Repeated no-diversity baseline. We compare the original baseline result with two additional inference runs on the same 250-task benchmark. The additional runs use the same trained checkpoint, shared initial kernels, coder, and evaluation settings. Each run has limits of 10 iterations and 40 candidate slots per task. Means and sample standard deviations summarize the three baseline outcomes.

MetricOriginal baselineAdditional run 1Additional run 2Baseline mean ± SDFull CUDACoach, paper result
Correct tasks133137137135.67 ± 2.31151
fast₁44414242.33 ± 1.5348
fast₂21171818.67 ± 2.0823

Comparing the reported full result with these three baseline outcomes gives gains of 14–18 correct tasks, 4–7 fast₁ tasks, and 2–6 fast₂ tasks. The advantage is therefore present with each of the three baseline outcomes. We will add the additional results and summary statistics to the revised manuscript.

Reviewer_LSC1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents CUDACoach, a framework that trains a planner to guide a frozen coder in CUDA kernel repair and optimization. It addresses the tendency of off-the-shelf planners to repeatedly recommend similar optimization strategies across different kernels. The method constructs supervision through Verified Strategy Mining: strategies are reconstructed from successful code modifications and retained when they guide a coder to produce another successful modification. The planner is initialized through supervised fine-tuning and further trained using execution-based rewards and a two-level diversity reward. At inference time, the framework combines information from multiple candidate kernels. Evaluation on KernelBench examines performance across three downstream coders.

Critical strengths
  1. Important problem. Repetitive, generic optimization recommendations can limit the effectiveness of planner–coder workflows. Learning to provide more specific and useful guidance is a meaningful direction for automated GPU programming.

  2. Detailed evaluation and documentation. The paper evaluates multiple downstream coders and includes component ablations, iterative optimization analyses, and a case study. The appendices provide substantial detail on data construction, rewards, and inference procedures.

Critical weaknesses
  1. The contribution beyond teacher distillation is insufficiently isolated. Successful code modifications and auxiliary kernel analyses are generated by Claude Opus 4.5, after which strategies are reconstructed and filtered using Qwen3-Coder-Plus. Consequently, an important alternative explanation is that the gains primarily arise from transferring strong-model knowledge into the smaller planner. An untrained Qwen3-8B planner or directly use Claude-opus-4.5 as planner baseline are missing.

  2. Greater strategy diversity does not necessarily establish better bottleneck awareness. Similar kernels may reasonably benefit from the same optimization techniques, and differences in DRAM utilization alone do not establish that different technique categories are required. The proposed reward encourages historically uncommon techniques and distinct combinations, but a strategy can receive a diversity bonus when its downstream implementation is correct without being faster. Thus, increased technique diversity may not always indicate more appropriate optimization decisions.

  3. Performance gains are uneven across coders, and their robustness is unclear. Compared with single-stage iteration, Qwen3-Coder-Plus shows substantial correctness gains, but the results are less consistent for the other coders. With Qwen3.8-Flash, correctness decreases from 86.8% to 80.4%, while fast₁ increases only from 40.4% to 40.8%. With Kevin-32B, fast₁ decreases from 18.8% to 18.4%. The diversity-reward ablation improves fast₁ by four tasks and fast₂ by two tasks, without uncertainty estimates for these differences. These results leave open whether the method reliably improves optimization quality or primarily improves repair for particular coders.

  4. The efficiency comparison does not account for total cost. Fewer coder generations do not necessarily imply lower overall cost, since CUDACoach additionally invokes the planner and technique annotator, profiles multiple kernels, and processes larger contexts. Appendix D explicitly acknowledges that the reported counts compare code generations rather than total computation. End-to-end latency, token usage, and API cost should be reported under matched budgets.

  5. Insufficient justification of profiling inputs. The planner receives only 24 NCU metrics, but the paper does not explain how they were selected, or evaluate whether this subset provides sufficient diagnostic information. NCU can provide hundreds of various metrics.

  6. limited coverage of modern GPU optimization techniques. The motivating analysis primarily relies on DRAM utilization. Although the detailed LayerNorm case focuses mentioned occupancy, Layernorm is a super easy task for agents. The paper does not demonstrate effective reasoning examples about real critical modern optimization techniques, e.g., Tensor Core utilization, warp specialization, TMA-based pipelines, or shared-memory layouts. Consequently, the evaluation leaves unclear whether the planner can diagnose and address the broader range of bottlenecks encountered in modern GPU kernels.

Questions for authors
  1. Please add untrained Qwen3-8B and Claude Opus 4.5 as planner baselines, keeping the downstream coder, inference procedure, and coder-generation budget fixed. How do they compare with the trained planner in correctness, speedup, and cost?

  2. Tables 2–3 show that the diversity reward increases strategy diversity and improves downstream results. Do these gains arise from more accurate bottleneck identification or just broader exploration of optimization techniques?

  3. Please report results across multiple independent runs for Table 1 and the diversity-reward ablation, including uncertainty estimates. Are the small fast₁/fast₂ improvements statistically reliable?

  4. Please report total token usage and API cost, including planning, annotation, and profiling. Does CUDACoach retain its advantage over single-stage iteration under a matched total-cost budget?

  5. Please provide the exact 24 NCU metrics and explain their selection criteria. What evidence shows that this subset supports diagnosing bottlenecks beyond the DRAM-focused examples, particularly those real challenging tasks for agents. Such as flash-attention, MoE inference/training kernels, FP8/FP4 instructions, which involve Tensor Cores, warp specialization, and TMA-based pipelines.

10 comments
AuthorsOpenReview ↗

Response to the Concerns of Reviewer [Part 1]

Q1: Contribution beyond teacher distillation and comparisons with alternative planners.

A1: The gains from RL beyond SFT exceed those from SFT over the Qwen3-8B base model. We compare the base model, the SFT checkpoint, and both RL variants using Qwen3-Coder-Plus as the frozen coder on all 250 KernelBench tasks.

Both RL variants start from the same SFT checkpoint, which already incorporates supervision derived from teacher-generated code modifications and auxiliary kernel analyses. During RL, only the planner is updated using downstream execution feedback. The full configuration additionally uses the diversity reward.

PlannerCorrect tasksfast₁
Qwen3-8B10930
SFT-only planner11437
RL without the diversity reward13344
Full CUDACoach planner15148

SFT adds 5 correct tasks and 7 fast₁ tasks over Qwen3-8B. Full RL then adds 37 correct tasks and 11 fast₁ tasks over SFT. These larger gains after SFT support a substantial contribution from RL beyond the supervised transfer of teacher knowledge. We will add the Qwen3-8B and SFT comparisons to the revised manuscript.

We could not complete the Claude Opus 4.5 planner comparison because our available endpoint does not currently provide access to this model.

AuthorsOpenReview ↗

Response to the Concerns of Reviewer [Part 2]

Q2: Does diversity improve bottleneck identification or exploration, and why reward correct implementations without immediate speedup?

A2: Diversity RL encourages broader exploration to discover strategies suited to each kernel and reduce premature concentration on generic templates. The planner uses code and profiling information to propose alternatives, and execution feedback rewards those that improve performance. Similar kernels may still benefit from the same techniques, and historical weighting does not exclude useful common techniques.

With the diversity reward, the most common technique-set frequency decreases from 16.9% to 6.7% in Table 3, while performance improves from 44 to 48 fast₁ tasks and 21 to 23 fast₂ tasks in Table 2. We will clarify that the DRAM comparison illustrates strategy repetition rather than independently establishing which techniques a kernel requires.

The reward design balances exploration with execution-based improvement. Correct implementations without immediate speedup remain eligible for a diversity bonus so that alternative approaches can be further refined or combined. However, they receive zero optimization task reward unless they exceed 1.1× speedup over the input kernel. Incorrect implementations receive no diversity bonus.

To assess this tradeoff, we compared correct strategy pairs from the same state with equal format rewards across all 148 training updates. In each pair, one implementation exceeded the 1.1× threshold and the other did not. Adding the diversity bonus reversed their reward ordering in only 62 of 11,330 pairs, or 0.55%. At inference, we select the lowest-latency implementation that passes verification.

AuthorsOpenReview ↗

Response to the Concerns of Reviewer [Part 3]

Q3: Robustness of optimization gains across runs and downstream coders.

A3: CUDACoach has the highest reported mean on all three metrics, and the task-level analyses show optimization benefits beyond repair. We summarize the reported results for each method on the 250-task KernelBench benchmark.

(1) Aggregate comparisons. The following comparison fixes Qwen3-Coder-Plus as the coder. Entries give the mean and sample standard deviation across the three independent runs.

MethodCorrectness (%)fast₁ (%)fast₂ (%)
Single-stage iteration45.60 ± 0.8013.07 ± 0.617.73 ± 0.61
Without diversity reward54.27 ± 0.9216.93 ± 0.617.47 ± 0.83
Single-kernel context53.87 ± 1.5116.00 ± 1.748.67 ± 0.83
CUDACoach59.87 ± 1.6718.80 ± 0.409.33 ± 0.61

Relative to single-stage iteration, the mean gains are 14.27 percentage points in correctness, 5.73 points in fast₁, and 1.60 points in fast₂. Compared with the variant without the diversity reward, mean fast₁ and fast₂ are both 1.87 percentage points higher.

Exploratory significance analysis.

Comparison with CUDACoachCorrectness p-valuefast₁ p-valuefast₂ p-value
Single-stage iteration0.00110.00040.0327
Without diversity reward0.01320.01590.0396
Single-kernel context0.01010.10190.3314

Two-sided Welch tests show higher correctness, fast₁, and fast₂ for CUDACoach than for single-stage iteration and the variant without the diversity reward, with all corresponding p-values below 0.05. Against the single-kernel-context variant, the correctness difference reaches p = 0.0101, while the fast₁ and fast₂ differences do not reach statistical significance.

(2) Optimization beyond repair. We also examine all 91 tasks with the same correct initial implementation in the single-kernel-context and full configurations. These tasks enter optimization directly. On this subset, CUDACoach increases fast₁ from 28 to 36 tasks and fast₂ from 15 to 16 tasks. The geometric-mean speedup over the shared initial implementations increases from 3.46× to 3.96×. These gains demonstrate improvements to already-correct kernels, beyond increasing repair coverage.

(3) Transfer across downstream coders. With Kevin-32B, Table 1 shows increases from 142 to 153 correct tasks and from 19 to 23 fast₂ tasks, although fast₁ changes from 47 to 46. The benefits therefore extend to another coder, with gains varying across metrics.

For Qwen3.8-Flash, the two attention cases in A5 demonstrate successful planner-guided optimization. We also observed cases where the coder did not fully implement the planner's instructions. In NetVLAD, the planner required forward() to invoke custom CUDA kernels, but the coder retained framework operators in that execution path, causing the candidate to fail the implementation checks.

For matrix multiplication with both inputs transposed, the planner requested float4 loads, but Qwen3.8-Flash used scalar loads. Manually adding only the requested vectorized loading reduced median latency from 23.4 ms to 19.2 ms, a 1.22× speedup over the original candidate, across three repetitions on the same GPU. Both versions passed all five correctness checks in each repetition.

These cases show that downstream implementation fidelity can limit the benefit of useful planning guidance. We will add the aggregate comparisons and task-level analyses to the revised manuscript.

Reviewer_ZarjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper trains an 8B planner to produce repair and optimization strategies for a frozen CUDA coder. It reconstructs strategies from successful kernel edits and retains those that guide a successful modification when implemented without access to the improved kernel. Supervised initialization is followed by GRPO with execution rewards and a diversity reward combining historical technique frequencies with within-state log-determinant diversity. At inference, strategies draw on multiple correct kernels and are clustered before implementation. Evaluation covers all 250 KernelBench tasks and three coders, with comparisons against fixed-parent sampling, direct iteration, and self-planned compositional iteration. Gains vary by coder and metric.

Critical strengths
  1. The paper provides a concrete training pipeline for a natural-language planner that guides a frozen coder in both repair and optimization. Its supervision is checked by having the coder implement a recovered strategy without seeing the improved teacher kernel (Section 3.2). This makes the instruction’s ability to guide a successful edit part of data selection and provides a practical interface between planning and implementation.

  2. The evaluation includes a controlled planner comparison that keeps the downstream coder and compositional inference procedure fixed (Table 1). The improved results support the practical value of the trained planner. Table 2 provides additional evidence for the complete diversity reward and multi-kernel context, and evaluation with three coders shows how the planner’s benefits vary across settings.

Critical weaknesses
  1. The empirical comparison with relevant published optimization methods is incomplete. Table 1 includes sampling and iteration baselines, a self-planner, and Kevin-32B; these establish useful controlled gains. QiMeng-Kernel (Zhu et al., AAAI 2026), which also learns strategies for a separate coder, and CUDA-L1 (Li et al., ICLR 2026), which evaluates CUDA optimization on KernelBench, are absent from the performance comparison. Section 2.1 explains methodological differences from QiMeng-Kernel, whose main results target Triton. A meaningful comparison requires accounting for target language, workloads, hardware, and search resources, or explaining concrete comparability or reproduction constraints. Without this context, it is difficult to judge the practical advance over existing optimizers.

  2. The additional value of historical technique-frequency weighting is not established. The two-level reward combines this weighting with a within-group diversity objective adapted from DQO (Chen et al., ICLR 2026). Tables 2–3 and Appendix F support the complete diversity reward, but removing the entire diversity term does not isolate historical weighting’s benefit over within-group diversity alone. The evidence supports the combined reward and the planner, but leaves the additional contribution of historical weighting unresolved.

Questions for authors
  1. What evidence positions CUDACoach relative to published methods such as CUDA-L1 and QiMeng-Kernel? Please clarify which comparisons are feasible under comparable workloads, target language, hardware, and search resources, and explain any concrete barriers.

  2. What limitation of within-group diversity rewards does historical technique-frequency weighting address? What evidence supports its additional value within the proposed two-level design?

1 comment
AuthorsOpenReview ↗

Gentle Reminder for Feedback

Dear Reviewer Zarj,

We greatly appreciate the time and effort you have dedicated to reviewing our paper. We would be very grateful if you could find some time to look over the rebuttal. We are pleased to provide any further clarification.

Best regards,

The Authors of CUDACoach

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Evidence and Clarifications

Dear Area Chair,

Thank you for overseeing our submission. The reviewers recognized the value of verified strategy mining and the strong controlled planner comparison. Our rebuttal adds evidence addressing the main questions about contribution attribution and optimization benefits:

  • The gains extend beyond teacher-supervised initialization. (LSC1, KAqg) Under the same iterative search procedure and coder-generation budget, RL improves over the SFT checkpoint from 114 to 151 correct tasks and from 37 to 48 fast₁ tasks. Verified-strategy SFT also solves eight more tasks than unverified-strategy SFT. These controls demonstrate additional value from verification and execution-based learning while acknowledging SFT’s foundational role.

  • Diversity improves actual kernel performance, beyond repair and threshold counts. (LSC1, KAqg) Across three independent inference runs, CUDACoach achieves 18.80 ± 0.40% fast₁, compared with 16.93 ± 0.61% without diversity and 13.07 ± 0.61% for single-stage iteration. On 89 initially correct tasks with paired results, diversity reduces latency on 68 tasks, including 17 with at least 2× improvement over the no-diversity implementations. Sixty latency improvements leave both fast₁ and fast₂ classifications unchanged, explaining why threshold counts understate the optimization gains.

  • The multi-kernel contribution is now evaluated separately from repair, and the timing protocol is clarified. (KAqg) We acknowledge that the original correctness gap did not isolate optimization context. In the requested comparison sharing identical repair trajectories until first success, both configurations retain 40/40 correctness, while multi-kernel context increases fast₁ from 10 to 13 and geometric-mean speedup over the shared starting kernels from 4.75× to 5.28×. We also clarified the fixed-reference timing protocol and identified the outdated appendix wording requiring correction.

  • Practical evidence now includes detailed cost accounting and more complex workloads. (LSC1) We report 32.1% lower estimated API expenditure, with local planner occupancy and profiling time disclosed separately. Two attention case studies demonstrate verified 1.31× and 1.51× improvements over their best parents, supported by profiling evidence and concrete layout and synchronization changes.

  • We added targeted evidence for historical weighting and clarified comparisons with prior systems. (Zarj, KAqg) Historical weighting improves both correctness and acceleration success over local diversity alone at all three evaluated checkpoints on the same 199 optimization-validation states. We also audited CUDA-L1’s released H100 implementations under our evaluation requirements and documented the specific artifact gaps preventing a matched reproduction of QiMeng-Kernel.

Reviewer LSC1 explicitly indicated that our responses to three of their five questions looked satisfactory. Our subsequent replies directly addressed the remaining concerns about optimization magnitude and the interpretation of teacher supervision.

Together, these additions strengthen the paper’s contribution: learning actionable, diverse strategies that improve a frozen coder’s repair and optimization capabilities. We respectfully ask you to consider this expanded evidence in your assessment.

Best regards,
The Authors of CUDACoach

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.