acceptodds
Under review as a conference paper at ICLR 2027

CORAL: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization

Abstract

GPU kernel optimization requires generating correct implementations, identifying effective optimization techniques, and applying them to suitable workloads. We present CORAL, a reinforcement learning framework that couples skill discovery with skill exploitation within a single shared language model through a dynami- cally evolving skill library. CORAL jointly trains three agents sharing one LLM backbone: a Skill Selection Agent that retrieves relevant techniques via BM25 and LLM reranking, a Policy Agent that generates multi-turn Triton kernels conditioned on selected skills, and a Skill Summary Agent that distills successful rollouts into reusable skills. Candidate skills are added only after execution-based verification confirms first-turn speedups on their source tasks. The agents are initialized via a structured SFT cold start on diversity-filtered data and jointly optimized with multi-turn REINFORCE and per-agent advantage estimation. On KernelBench, CORAL-14B achieves 37.2%, 70.6%, and 32.2% on Level 1, 2, and 3 under the Fast threshold using the last output of three-turn rollouts, outperforming our repro- duced Dr. Kernel-14B baseline. Ablation studies show benefits from inference-time skills, joint training, learned selection, and diversity filtering, with trade-offs across task levels and speedup thresholds.

Primary area:
reinforcement learning
Keywords:
Co-Evolving, Skill, RL

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_xmf4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at how a model can learn from past GPU kernel optimization attempts and reuse those techniques on new tasks. The authors propose CORAL, where the same language model takes on three roles: skill selection, kernel generation, and skill summarization. The three roles are trained together through reinforcement learning. The model extracts new skills from successful optimization runs, verifies them through actual execution, and then adds them to a skill library for later tasks. The experiments are mainly on KernelBench. Under three-turn evaluation using the last output, CORAL performs better than the authors' reproduced Dr. Kernel baseline. The ablations also suggest that skill injection and joint training help, though the gains depend on model size, task difficulty, and evaluation protocol.

Critical strengths

1. The problem is practical, and the framework is fairly complete.

GPU kernel optimization is not just about generating correct code. It also involves learning from optimization techniques that have worked before. CORAL brings skill selection, kernel generation, and experience summarization into one reinforcement learning framework, and uses actual execution results to check whether a skill works. The idea of accumulating experience is not entirely new, but I think training the three roles together is still valuable.

2. The results are good, and the paper includes quite a few ablations.

Under three-turn evaluation using the last output, CORAL-14B improves Level 2 Fast₁ from 58.5% to 70.6%, and Level 3 Fast₁ from 22.1% to 32.2%. When the RL losses for Selection and Summary are removed, Level 3 Fast₁ drops to 20.0%, which suggests joint training does help. Also, with fourteen turns and best-output selection, CORAL-8B improves Level 2 Fast₁.₂ from 39.4% to 55.4%. So the method is not only effective under three-turn evaluation.

Critical weaknesses

The performance gains depend quite a bit on the number of search turns and the evaluation protocol.

CORAL-14B shows a clear improvement on Level 3 with three turns and last-output evaluation. But with fourteen turns and best-output selection, Fast₁ is only 68.5% versus 67.5%, and Fast₁.₂ is 22.2% versus 22.0%. These are almost the same as Dr. Kernel-14B. So I think CORAL's advantage may be more about doing better when the search budget is small, rather than finding substantially better results even after many search turns. The authors do report these numbers, but I think they should make this distinction clearer when discussing the main performance gains.

Questions for authors

Q1. Are the training budgets fair across the ablations?

Could the authors explain whether full CORAL and Ablations 3 and 5 use the same or roughly comparable training data, task sampling, number of policy rollouts, and extra compute for skill verification? Some settings differ in more than just removing a loss term. The training data and the way skills are used may also change. So I'd like to know how much of the improvement comes from joint training itself, and how much might be affected by extra data or compute.

Q2. Is CORAL mainly better at finding good kernels earlier?

Under fourteen-turn best-output evaluation, CORAL-14B and Dr. Kernel-14B perform quite similarly on Level 3. That makes me wonder whether CORAL's improvement in the three-turn setting mainly comes from finding useful optimizations within the first few turns. If the authors could show performance by turn, such as the speedup of the best kernel found up to each turn, it would help explain the method's advantage more clearly.

8 comments

Thanks for your review. We have addressed the concerns as follows:

  1. The performance gains depend quite a bit on the number of search turns and the evaluation protocol.

We agree that the gap between CORAL and Dr. Kernel will be smaller with more search turns. But in 19/24 cases, the CORAL is still better than Dr. Kernel.

  1. Are the training budgets fair across the ablations?

There is some difference in the compute budget across the ablations. However, we just choose the best turn for each ablation as shown in Figure 3; more compute is useless.

  1. Is CORAL mainly better at finding good kernels earlier?

CORAL's recursive evolution method is quite like some autoresearch methods. Based on their experience, there is 2 assertions:

  1. The Agent with skill can improve performance fast at the beginning.
  2. The Agent with skill may improve performance slowly but steadily at the end.

The gap may not be that big, but it's still there, so such a method is still useful.

AuthorsOpenReview ↗

A Kind Reminder for Reading the Response

Dear Reviewer xmf4,

Thank you for your comments.

As the rebuttal period is ending soon, we would greatly appreciate it if you could take a final look at our responses and see whether they address your concerns.  Thank you for your time and consideration. 

The Authors

Thank you for the response. However, my main concerns remain. It would be helpful to provide experimental results to further clarify these questions.

Reviewer_d4xvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets on "Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization". Folllow existing methods of co-evolving skills and policy RL training, this paper introduces "CORAL", a unified framework than uses one policy for skills selection, kernel generation and skills summarization. After cold start SFT, the policy is trained by TRLOO with combined objective, including loss for selecion, policy and summary. On KernelBench, CORAL gets improvements compared with its base model and surpass baselines such as close-source/open-source models and Dr. Kernel.

Critical strengths
  1. The paper introduces skill/policy co-evolution to kernel generation research, which is novel and meaningful. I agree that the skills should match the policy's capability of kernel generation and skills should be examined before deployment.

  2. The paper provides comprehensive experiments and ablation studies, which I think are convincing and meaningful.

Critical weaknesses
  1. I concern about the stability of using only one policy for joint RL on a combined objective. To the best of my knowledge, the joint training is usually unstable. And in the eq.6, CORAL introduces 2 new hyper-parameters, which requests much tuning works. Moreover, the paper does not explain why the hyperparameter is good or do ablation studies on the wselectw_{select} and wsummaryw_{summary}.

  2. I am not sure if the experiment setting is fair for baselines like Dr. Kernel. In inference, CORAL's model are provided with skills, while Dr. Kernel not. And ablation study ("Ablation 1") shows that CORAL is not better than Dr. Kernel when the skills are removed.

  3. I concerns about "expand to k+1 parallel schemes". It looks like the CORAL need k+1 times cost in rollout, which is too expensive.

  4. (Minor) the paper writing need to be improved. For example, in lines from 151 to 161, what kind of kernels are CORAL generating? It seems that authors mixed Trition and CUDA.

Questions for authors
  1. Could you please provide more information about the weakness1, especially explantions for choice of hyperparameters and more ablation studies?

  2. Why do you choose to use one policy for a joint RL, instead of just train a policy while freezing the model for selection and summary?

2 comments

Thanks for your review. We have addressed the concerns as follows:

  1. The stability of using only one policy for joint RL on a combined objective

As shown in Figure 3, the stability of the joint training is good.

  1. The choice of hyperparameters

Because of the cost of training, we just randomly choose the hyperparameters based on experience. This is not a deliberate choice, but it's effective.

  1. The experiment setting is fair for baselines like Dr. Kernel.

We believe skill will become a part of the "checkpoint" and can be reused. If we use Dr. Kernel to generate the kernel in real life, it will not use the skills, but if we use CORAL, the skill will be reused. As a result, we think the experimental setting is fair.

  1. The expand to k+1 parallel schemes

A lot of GRPO uses n = 16 to parallel the rollout, although we need to expand to k+1 parallel schemes, n can be smaller. In this paper, we use k=3 and n=4, so the computation cost is not that high.

  1. What kind of kernels are CORAL generating?

We are generating the Triton kernels. Apologies for the confusion.

  1. Why do you choose to use one policy for a joint RL, instead of just training a policy while freezing the model for selection and summary?

We want to use one model to handle all the three processes, so we choose this way. We also have an ablation study on using only the policy loss in the RL, which is worse than joint training.

AuthorsOpenReview ↗

A Kind Reminder for Reading the Response

Dear Reviewer d4xv,

Thank you for your comments.

As the rebuttal period is ending soon, we would greatly appreciate it if you could take a final look at our responses and see whether they address your concerns.  Thank you for your time and consideration. 

The Authors

Reviewer_1SXuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents CORAL, a reinforcement-learning framework for Triton GPU-kernel optimization. Experiments on KernelBench compare CORAL-8B and CORAL-14B with general LLMs, open-weight models, and reproduced Dr. Kernel baselines. The main three-turn evaluation reports 37.2/70.6/32.2 Fast-1 at 14B for Levels 1/2/3. The paper studies five ablations, 14-turn last-turn/best-turn scaling behavior, approximately matched token-cost comparisons, and two qualitative transfer case studies.

Critical strengths
  1. The end-to-end formulation of skill discovery and exploitation is consequential and timely. Treating selection, policy use, and summarization as jointly optimized components addresses a real limitation in agentic kernel optimization: the usefulness of a technique changes as the policy improves. The inclusion of a null skill scheme during training is a useful device for learning whether skills help on each task.

  2. The evaluation is broader than a single headline metric. The paper reports multiple KernelBench levels and speedup thresholds, five ablations, 8B and 14B models, last-turn versus best-turn behavior, an approximately matched token-cost comparison, training-cost overhead, and two transfer case studies. This breadth makes the paper more informative than a simple leaderboard delta and exposes several honest trade-offs.

Critical weaknesses
  1. The method admits skills based only on source-task speedup, and the showcased success relies partly on recognizing a task-specific identity that makes most computation unnecessary. The paper does not report how often such shortcuts occur, how much of the aggregate gain they explain.

  2. The paper’s statistical and methodological support is too thin for its headline claims. The 46% relative improvement on Level 3 relies on a reproduced baseline that differs greatly from the published result, on a single training run, and on sampling std that does not address seed/run variance.

Questions for authors
  1. Please report how often generated solutions bypass the reference computation through semantic or constant-output shortcuts, and what CORAL’s performance is when such solutions are separated from conventional custom-kernel optimizations.

  2. What is the size and composition of the final learned skill library for each model and ablation? How many unique skills are admitted, how many are eventually retrieved, and how often are retrieved skills irrelevant or harmful?

2 comments

Thanks for your review, we have addressed the concerns as follows:

  1. The method admits skills based only on source-task speedup, and the showcased success relies partly on recognizing a task-specific identity that makes most computation unnecessary. The paper does not report how often such shortcuts occur, how much of the aggregate gain they explain.

The author is confused about why this is a weakness. There is a lot of mathematical shortcuts in the kernel optimization, summarizing such shortcuts as skills and reuse in the future is a good feature. We do not think this is a weakness. The author can assure that there is no overlap between the training and testing data. It just shows the generalization ability of such skills.

  1. The paper’s statistical and methodological support is too thin for its headline claims. The 46% relative improvement on Level 3 relies on a reproduced baseline that differs greatly from the published result, on a single training run, and on sampling std that does not address seed/run variance.

The reproduced baseline differs greatly from the published result

This may be because we use different hardware to reproduce the result. We do not have an H100 to run the Dr. Kernel baseline, so the result may be different.

On a single training run

Because of the cost of the training, we only train the model once (including the ablation study). This is a common practice in the RL community.

Sampling std that does not address seed/run variance

We do have 8 runs in the evaluation, and the std is calculated over the 8 runs.

  1. Please report how often generated solutions bypass the reference computation through semantic or constant-output shortcuts, and what CORAL’s performance is when such solutions are separated from conventional custom-kernel optimizations.

We need to clarify that finding such shortcuts is a common practice in kernel optimization. Since the amont is too big, we use deepseek-v4.1-flash to scan all the generated code to find the shortcuts and found 19 cases among 28000 (250 tasks x 8 runs x 14 turns) generated code.

  1. What is the size and composition of the final learned skill library for each model and ablation? How many unique skills are admitted, how many are eventually retrieved, and how often are retrieved skills irrelevant or harmful?

The size and composition of the final learned skill library

Here is the size of the final learned skill library for each model and ablation:

ModelSize
CORAL-14B960
Ablation 10
Ablation 2152
Ablation 3346
Ablation 4677
Ablation 50

But we need to clarify that increasing the size of the skill library does not necessarily mean the final performance will be better as we present in Figure 3.

The number of unique skills eventually retrieved

We use all saved skills in the skill library to evaluate the performance, for each level, all ablations use 35-50 distinct skills across each level.

how often are retrieved skills irrelevant or harmful?

It's hard to tell the "harmful" part. But the result between CORAL-14B and Ablation 5 can prove that most of the retrieved skills are useful.

AuthorsOpenReview ↗

A Kind Reminder for Reading the Response

Dear Reviewer 1SXu,

Thank you for your comments.

As the rebuttal period is ending soon, we would greatly appreciate it if you could take a final look at our responses and see whether they address your concerns.  Thank you for your time and consideration. 

The Authors

Reviewer_PCteOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CORAL studies GPU kernel optimization through a shared language model that performs skill selection, kernel generation, and skill summarization. The three roles are jointly trained with reinforcement learning, while successful optimization experiences are distilled into an external skill library after execution-based verification.

Critical strengths
  1. The paper addresses a practical problem. GPU kernel optimization requires both correctness and speed. Reusing successful optimization experience across tasks is a useful direction for reducing repeated trial and error.

  2. The framework forms a clear learning loop. Skill selection, kernel generation, and skill summarization are connected through execution feedback. This provides a concrete way to turn individual optimization successes into guidance for later tasks.

  3. The short-rollout gains are meaningful. CORAL-14B improves Fast₁ from 30.4%/58.5%/22.1% to 37.2%/70.6%/32.2% across the three KernelBench levels compared with the reproduced Dr. Kernel baseline. Improvements across multiple levels support the practical value of the complete system.

Critical weaknesses
  1. The novelty appears incremental. SkillRL already studies skill–policy co-evolution, and Skill1 already jointly trains skill selection, utilization, and distillation within one policy. CORAL mainly adds kernel-specific rewards and verification. The paper needs to explain more clearly what methodological advance these changes provide beyond adapting existing ideas to GPU optimization. What is the main methodological difference from Skill1, beyond the task and reward definitions? Can the author provide a direct comparison or a more precise explanation?

  2. The experiments do not clearly establish the value of the added mechanisms. There is no frozen-library control to show that continued skill evolution is necessary. Selection and summarization losses are also removed together, leaving their individual contributions unclear. The results show that the full system works, but do not sufficiently explain why its additional components are needed. Could the author provide more experiments or an explanation of this?

  3. The effect of randomness is not sufficiently separated from the claimed gains. Skill admission relies on newly sampled code, so an apparent improvement may partly reflect sampling luck rather than the candidate skill. I am not sure whether the author checked for this.

Questions for authors

see weakness

4 comments

Thanks for your review, we have addressed the concerns as follows:

  1. The novelty appears incremental.

Thanks for your question. We have to clairfy that Skill1 is concurrent work with CORAL, so it may have some ideas overlap. However, we believe that the contributions of CORAL are not trivial.

We would like to explain the main differences between CORAL and SkillRL/Skill1 in detail.

  • The Skill1 use the estimated score as the distill reward (summarization reward in CORAL), which is not very accurate. We use the execution-based verification to get the more accurate reward.

  • The Skill1 can only deal with single skill injection RL, however, CORAL can deal with multi-skill injection RL, which is more flexible and practical.

  • The skill will only be summarized when the trigger condition is met, which is more flexible and practical.

  • Since the agent is lack of selection and summarization ability (mostly in long-tail tasks), we need to do cold start before RL training. We do find the cold start process have some key techniques, for example, the skills that are too generic and easily learnable should be filtered out, etc. We have mentioned this in section 3.5. SkillR1's task is relatively easy and probably have been trained on its base model, so it may not need to do cold start. But it's important for the long-tail tasks.

  • We also design the kernel-specific rewards and verification to further improve the performance.

  1. The experiments do not clearly establish the value of the added mechanisms.

There is no frozen-library control

In our settings the skill library is empty at the beginning of RL, so we the skill library can not be frozen.

Selection and summarization losses are also removed together, leaving their individual contributions unclear.

Yes, because of the cost of the training, we only test the ablation study on remove or keep both selection and summarization losses. We do think this is a reasonable suggestion but we can's afford to do more experiments, and we believe such jointly training is our main contribution. Maybe the result from Skill1 can partly prove the contribution of the selection and summarization losses.

The results show that the full system works, but do not sufficiently explain why its additional components are needed. Could the author provide more experiments or an explanation of this?

We believe our results can clearly explain the value of the added mechanisms. For example,

Ablation 1 shows the importance of skill injection.

Ablation 2 shows the importance of the process in cold start.

Ablation 3 shows the importance of the training summarization and selection loss.

Ablation 4 shows the importance of the llm selection.

Ablation 5 shows the importance of the skill in RL training.

Appendix G shows the importance of the evolution of the skill library.

  1. Skill admission relies on newly sampled code, so an apparent improvement may partly reflect sampling luck rather than the candidate skill. I am not sure whether the author checked for this.

We do agree that the kernel result may have some randomness, but we have mentioned this in section 3.4. The skill will be saved only when the speedup is bigger than 1.2 * r1r_1, which is a relatively high threshold. We use this to partiallyavoid the random noise.

AuthorsOpenReview ↗

A Kind Reminder for Reading the Response

Dear Reviewer PCte,

Thank you for your comments.

As the rebuttal period is ending soon, we would greatly appreciate it if you could take a final look at our responses and see whether they address your concerns.  Thank you for your time and consideration. 

The Authors

I raise my score to 3, but I think the author could use a more intuitive paragraph to show ablation results.

Thanks for your review, we will update our paper in the next version!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.