- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CUDAPERF, a reinforcement learning framework for CUDA program generation that augments outcome-based verifiable rewards with structure-aware rewards derived from CUDA programs. Specifically, it trains an offline pairwise ranker over performance-relevant structural features, such as memory coalescing, arithmetic intensity, occupancy, control divergence, synchronization, and atomic operations, and combines the resulting structural score with execution-based signals for correctness and speedup during online multi-turn RL. Experiments on C→CUDA and PyTorch→CUDA translation show that CUDAPERF improves the performance of Qwen models, particularly Qwen-3-32B, and outperforms several existing baseline models in correctness and execution speed.
Critical strengths
-
Strong practical motivation. Automatically generating high-performance CUDA kernels is valuable for accelerating scientific computing and machine-learning workloads. Improving LLM-based CUDA generation therefore addresses a meaningful systems problem with clear practical relevance.
-
Addresses the sparsity and coarseness of outcome-based rewards. Existing RL-based code-generation methods often rely primarily on signals such as compilation success and test-case correctness, which can be sparse and provide limited guidance for improving performance. CUDAPERF augments these signals with CUDA-specific structural features, including memory coalescing, arithmetic intensity, occupancy, control divergence, synchronization, and atomic operations. These features provide a more informative learning signal, helping guide the policy toward generating CUDA programs that are both functionally correct and computationally efficient.
-
Provides evidence that structural and verifiable rewards are complementary. The ablation study compares structural-only, verifiable-only, and joint rewards across the CUDAPERF, BabelTower, and KernelBench datasets. The joint reward consistently achieves the strongest correctness and speedup results, indicating that structural and verifiable rewards provide complementary benefits.
Critical weaknesses
-
Lacks direct comparisons with other RL frameworks and reward designs. The main results in Table 1 compare CUDAPERF primarily with different models, such as OpenAI O4-mini and Qwen2.5-Coder-32B. However, the paper does not compare CUDAPERF with alternative RL frameworks or reward designs under the same base policy model and training data. This makes it difficult to determine whether the reported gains are specifically attributable to the proposed structural RL framework.
-
Uncontrolled comparison with baselines. CUDAPERF is obtained by fine-tuning Qwen-3-32B, while the proposed CUDAPERF dataset already shows Qwen-3-32B outperforming all other reported baselines in both correctness and speedup. Therefore, CUDAPERF’s superior performance on this dataset is partly expected from inheriting the capabilities of a stronger base model. More importantly, the comparison with QiMeng-MuPa is not controlled at the backbone level. As reported in Appendix A.16, the authors use the translator-Llama-3-8B model for QiMeng-MuPa, which is a fine-tuned version of Meta-Llama-3-8B, whereas CUDAPERF is trained from Qwen-3-32B. The substantial difference in model scale and backbone makes it difficult to attribute the performance gap to the proposed RL framework.
-
The test-time multi-turn protocol is insufficiently specified. Section 2.2.2 describes a multi-turn refinement process during online RL training, in which compilation errors, failed test cases, speedup measurements, and structural features are fed back into subsequent prompts. However, the paper does not specify what information is provided at test time, or whether the baseline models receive the same feedback and refinement opportunities. This makes it difficult to determine whether the reported gains are attributable to the proposed reward design or to differences in the test-time interaction protocol.
-
The generalization of the structural ranker is not sufficiently validated. The ranker learns a mapping from a CUDA program’s structural feature vector to a scalar score, where the induced ordering is expected to match the candidates’ measured speedup. However, the paper does not evaluate whether this mapping generalizes to unseen source programs, newly generated CUDA candidates, different workloads, or different hardware settings. The reported training loss is insufficient to establish such generalization, since the ranker may simply learn dataset-specific correlations between the handcrafted features and speedup.
-
The paper overlooks recent work on partial-success verifiable rewards. CUDAPERF uses the test-case pass rate directly in its correctness penalty, i.e., for partially correct programs. Recent work (Beyond Binary: https://arxiv.org/pdf/2601.03525) has shown that naive pass-rate rewards, despite providing denser supervision than binary outcome rewards, may suffer from cardinality bias: frequently passed or easy test cases can dominate the learning signal, while progress on difficult frontier tests receives relatively little credit. The authors should discuss this issue and clarify whether it applies to their CUDAPERF.
Questions for authors
- Can the authors compare CUDAPERF with alternative RL frameworks?One possible experiment would be to train Qwen-3-32B using CUDAPERF and an existing RL framework/reward configuration under the same training data, and then compare both the training cost (e.g., total GPU hours) and final performance using the metrics reported in Table 1.
- Can the authors compare CUDAPERF with QiMeng-MuPa under a controlled setting? Since the paper evaluates QiMeng-MuPa using the translator-Llama-3-8B model, one possible experiment would be to train CUDAPERF starting from the same Meta-Llama-3-8B backbone and compare it with translator-Llama-3-8B under the same data and evaluation protocol. Alternatively, if feasible, the authors could implement QiMeng-MuPa using Qwen-3-32B and compare it directly with CUDAPERF based on Qwen-3-32B. Such comparisons would help disentangle the effects of the underlying model backbone from those of the proposed training framework.
- Can the authors clarify the exact test-time multi-turn protocol for both CUDAPERF and all baseline models? In particular, please specify the prompt contents, feedback provided at each refinement step, number of refinement turns, etc.
- Can the authors evaluate the structural ranker on source programs and CUDA candidates held out entirely from training, and report a simple metric such as pairwise ranking accuracy or Spearman correlation with measured speedup?
- Can the authors discuss recent work on pass-rate-based reward design? In particular, they should discuss methods that address the potential bias and limitations of using a naive test-case pass rate as a reward.
Response to Reviewer Tob3
We appreciate the reviewer's detailed suggestions and acknowledgment of the importance of structural rewards.
1. Controlled RL Comparison: Table 3 already provides a controlled experiment using the same Qwen-3-32B backbone, comparing structural-only, verifiable-only, and combined rewards. Across all three datasets, the combined reward consistently achieves the highest correctness and speedup. Appendix A.15 also reports training costs.
2. Backbone Fairness: We acknowledge that QiMeng-MuPa uses a different backbone, as explicitly documented in Appendix A.16. Our central evidence for the structural reward is therefore the controlled ablation in Table 3, rather than cross-backbone comparisons.
3. Multi-turn Evaluation: Section 2.2.2 describes the refinement feedback, including compilation errors, failed test cases, speedup measurements, and structural properties. Appendix A.17 already compares single-turn and multi-turn training reward progression, showing lower reward saturation for single-turn training.
4. Ranker Generalization: We agree that training loss alone is insufficient to establish generalization. We will report pairwise ranking accuracy and Spearman correlation on entirely held-out programs and candidates in the revised submission.
5. Partial-success Rewards: We appreciate the reference to Beyond Binary. We acknowledge that pass-rate-based rewards may suffer from cardinality bias and will discuss this limitation and investigate frontier-aware reward weighting in the camera-ready version.