acceptodds
Under review as a conference paper at ICLR 2027

Less Is More: Learning What to Profile for GPU Kernel Optimization

Abstract

Large language models have recently shown strong potential for automatically optimizing GPU kernels, yet their effectiveness depends critically on the hardware information available to the kernel optimization model. Existing systems typically rely on predefined profiling strategies, leaving what to profile largely outside the optimization process. We show that effective profiling is not simply about collecting more information: increasing the number of profiling metrics does not monotonically improve kernel optimization, while the most useful profiling information varies across kernels. Our theoretical analysis further shows that, under such heterogeneous profiling utility, kernel-conditioned selection can outperform any single fixed profiling configuration. Motivated by this observation, we propose AutoProfiler, a feedback-driven framework for learning a deterministic profiling tree from downstream optimization outcomes. The tree is iteratively refined through REWRITE and SPLIT operations, with an LLM generating candidate modifications and empirical evaluation determining their acceptance. Once constructed, the tree quickly gathers the required profiling information for any kernel, allowing the LLM rewriter to be invoked only once. Experiments on KernelBench show that our approach improves downstream kernel optimization over fixed profiling configurations under constrained profiling budgets. These results suggest that hardware profiling should evolve from static metric collection toward profiling reasoning, making profiling itself a new dimension for improving kernel optimization.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
GPU Kernel Optimization, Hardware Profiling, Adaptive Metric Selection, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LxhvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces AutoProfiler, a framework for dynamically selecting hardware profiling metrics to guide LLM-based GPU kernel optimization. This is motivated by the observation that collecting more profiling information does not necessarily improve optimization quality and that useful metrics can vary across kernels. AutoProfiler learns a deterministic profiling tree through two LLM-guided operations: REWRITE, which introduces additional profiling metrics, and SPLIT, which partitions kernels into different profiling regimes. Candidate modifications are evaluated using downstream kernel optimization feedback, while inference-time profiling decisions require no additional LLM calls. Experiments on KernelBench report an average speedup of 1.7$\times$, outperforming several agentic baselines, with reduced LLM invocation overhead. The paper argues that learning kernel-specific profiling strategies can improve the efficiency and effectiveness of LLM-driven GPU kernel optimization.

Critical strengths

AutoProfiler introduces a feedback-driven method for learning a deterministic profiling policy, eliminating the need for additional LLM calls during profiling decisions at inference time. The separation between policy construction and deployment is appealing from a systems perspective. Furthermore, the evaluation trains the profiling policy on a separate dataset and reports improvements on KernelBench, providing encouraging evidence that the learned policy can generalize to unseen workloads.

Critical weaknesses

The evaluation does not convincingly establish how much of the reported improvement can be attributed to the proposed profiling policy. In Table 1, AutoProfiler is compared against agentic frameworks with different optimization strategies and numbers of LLM calls. Although the authors standardize the backbone model and optimization rounds, these comparisons do not isolate the contribution of adaptive profiling. The more relevant controlled comparison in Table 2 shows only a modest improvement over static profiling (1.379$\times$ vs. 1.314$\times$) on 48 kernels, without confidence intervals. Furthermore, Table 1 reports speedup only over correctly solved problems, making comparisons potentially misleading when correctness rates differ. Finally, while Figure 3 reports overall optimization time, the paper does not separately quantify profiling overhead or account for the offline cost of constructing the profiling policy. These limitations make it difficult to assess the actual performance and efficiency benefits of the proposed approach.

Questions for authors

As LLMs become increasingly capable of understanding GPU kernel performance, stronger models may already be able to identify relevant profiling metrics or dynamically decide what to profile without requiring a separately trained profiling policy. Have the authors considered comparing AutoProfiler against an LLM that directly selects profiling metrics based on the kernel code and previously collected hardware information? Such a comparison would help establish whether the learned profiling tree provides meaningful benefits beyond the capabilities of modern LLMs, particularly when accounting for the cost of constructing the policy.

1 comment
AuthorsOpenReview ↗

Response to the Weaknesses and Questions

Thank you for your detailed comments on the experimental evaluation. We have conducted additional controlled experiments and analyses to clarify the contribution of adaptive profiling, the comparability of evaluation results, and the associated profiling costs. We address these concerns below.

(1) Isolating the contribution of adaptive profiling.

To directly assess the contribution of our profiling policy, we conducted an additional controlled experiment using KernelAgent, which employs a fixed static profiling configuration. Specifically, we replaced only its static profiling component with the profiling tree learned by AutoProfiler, while keeping the rest of the optimization framework unchanged. Under static profiling, KernelAgent achieves a speedup of 1.410×. With AutoProfiler's adaptive profiling, the speedup increases to 1.587×, corresponding to a 12.6% relative improvement.

This controlled comparison provides direct evidence that the learned profiling policy improves downstream kernel optimization, beyond the effects of different agentic optimization frameworks.


(2) Response to the Comparability of Speedup Results under Different Correctness Rates

We follow the common practice in LLM-based kernel optimization of reporting speedup only for kernels that pass correctness verification, as performance improvements are meaningful only for functionally correct implementations. To provide a more comprehensive evaluation, we therefore report both correctness and speedup to provide a comprehensive assessment of each method. Notably, AutoProfiler achieves the highest correctness and speedup among all compared methods, demonstrating that its performance gains do not come at the expense of correctness.


(3) Response to Profiling Overhead and Offline Policy Construction Cost

Offline Profiling Policy Construction Cost. We quantify the offline cost of constructing the profiling tree below. Tree construction takes 28.2 hours of wall-clock time and consumes 2.98M tokens. This is a one-time offline cost: the resulting tree is frozen and reused across kernel optimization tasks, without requiring reconstruction for individual kernels.

Cost ComponentTime / Resource
Total wall-clock time28.2 h
LLM execution time16.1 h
NCU profiling5.2 h
GPU validation8.0 h
Token consumption2.98M

Runtime Profiling Overhead and Optimization Efficiency. AutoProfiler incurs 21.5 s of profiling time per task, comparable to 23.8 s with static profiling. Thus, its efficiency advantage does not come from reducing profiling overhead, but from providing more informative profiling signals for downstream LLM-based optimization. These signals reduce unnecessary LLM calls and shorten optimization iterations. As shown in Figure 3, AutoProfiler achieves higher speedups than the baseline under a comparable cumulative wall-clock budget, demonstrating the optimization benefits of its adaptive profiling policy.


(4) Response to Whether Direct LLM Reasoning Can Replace a Learned Profiling Policy

To investigate whether a capable LLM can directly determine effective profiling metrics without a separately learned policy, we conducted an additional experiment using GPT-5.6-Sol as a profiling decision-maker. At each round, the LLM selects at most six profiling metrics based on the kernel code and profiling results collected in previous rounds, iteratively proposing additional metrics over three rounds. The metrics selected across all three rounds are then collected and used for kernel optimization under the same optimization procedure as AutoProfiler. We evaluate both approaches on 50 kernels. AutoProfiler achieves a speedup of 1.642×, whereas direct LLM-based profiling reaches only 1.441×. This result indicates that directly reasoning about profiling metric selection with a strong LLM does not match the effectiveness of the learned profiling tree, highlighting the value of learning a dedicated profiling policy for downstream kernel optimization.

Reviewer_cXLSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AutoProfiler, a framework that learns kernel-specific hardware profiling policies to guide LLM-based GPU kernel optimization. Instead of collecting a fixed set of profiling metrics, AutoProfiler constructs a deterministic profiling tree through REWRITE and SPLIT operations, guided by downstream optimization feedback. The learned tree adaptively selects profiling metrics for each kernel before invoking the LLM rewriter. Experiments on KernelBench demonstrate improvements in optimization performance and correctness over several existing approaches.

Critical strengths
  1. The paper identifies an important observation that more profiling information does not necessarily lead to better kernel optimization.
  2. The tree-based profiling policy and feedback-driven REWRITE and SPLIT operations provide a systematic approach to adaptive metric selection.
  3. The experiments demonstrate improvements in both speedup and correctness. The comparisons with static profiling and the ablation studies provide useful evidence for the effectiveness of the proposed approach.
Critical weaknesses
  1. Proposition 1 establishes that an oracle per-kernel selection policy is no worse than the best fixed configuration. However, this result follows directly from the definitions and does not establish the effectiveness of the learned profiling tree. The theoretical motivation could be better connected to the proposed algorithm.
  2. Although the method reduces LLM calls during inference, the paper does not clearly quantify the actual profiling latency or the offline cost of constructing the profiling tree. These costs are important for assessing the practical efficiency of AutoProfiler.
Questions for authors
  1. Could the authors clarify how the theoretical result supports the effectiveness of the learned profiling tree, rather than only the oracle selection policy?
  2. How does the actual NCU profiling time of AutoProfiler compare with fixed profiling configurations? Could the authors briefly discuss how the offline tree construction cost is amortized?
1 comment
AuthorsOpenReview ↗

Response to the Weaknesses and Questions

W1 & Q1: Response to the Role of Proposition 1 in Motivating the Learned Profiling Tree

Thank you for highlighting the need to better connect Proposition 1 with our algorithm. Our intention is to formalize the potential advantage of kernel-specific adaptive profiling over a shared static configuration, thereby motivating the design of a learned profiling policy that selects metrics according to kernel characteristics. The Oracle experiment quantifies this potential advantage, while our learned profiling tree aims to realize it through optimization-driven metric selection. Its effectiveness is further validated by downstream optimization experiments, where the learned tree consistently outperforms static profiling. We will revise the paper to make this motivation and its connection to our algorithm more explicit.


W2 & Q2: Response to Runtime Profiling Latency and Offline Tree Construction Cost

Runtime Profiling Latency. AutoProfiler incurs 21.5 s of profiling time per task, comparable to 23.8 s with static profiling. Thus, its efficiency advantage does not stem from lower profiling overhead, but from providing more informative profiling signals for downstream LLM-based optimization. These signals reduce unnecessary LLM calls and shorten optimization iterations. As shown in Figure 3, AutoProfiler achieves higher speedups than the baseline under a comparable cumulative wall-clock budget.

Offline Tree Construction Cost. Constructing the profiling tree takes 28.2 hours of wall-clock time and consumes 2.98M tokens. This is a one-time offline cost: the resulting tree is frozen and reused across kernel optimization tasks, so individual kernels do not incur this construction cost. The cost is therefore amortized over subsequent optimization tasks.

Reviewer_Y3mJOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates what hardware information we should feed to an LLM when asking it to optimize GPU kernels. Existing kernel agents use a fixed set of profiler metrics, feeding the same information into the prompt for all tasks.The authors argue that more metrics isn’t always better and optimizing different kernels need different info. They propose AutoProfiler, a decision tree that starts from two basic metrics (compute usage and memory throughput), then decides which extra metrics to collect for each kernel before calling the LLM. The tree is built offline on a synthetic kernel set. During the real kernel optimization after building the tree, an LLM proposes changes (e.g., add more metrics at a leaf, or split a leaf on a threshold), and a change is kept only if it improves results on the training kernels. On KernelBench with GPT-5.6 Sol on an RTX 4090 GPU, they report better speedup and correctness than CudaForge, KernelMem, KernelAgent and AutoKernel. In addition, they provide analysis on metric count, performance with other LLMs and multi-round use.

Critical strengths
  • The question itself is sensible. Asking what to profile, instead of just whether to profile, is a finer and more reasonable angle for kernel agents.
  • During deployment (test time), the approach is cheap, easy to read, and interpretable: a fixed decision tree picks the metrics, no LLM is needed for that part, and the rewriter (LLM) is called once.
Critical weaknesses

Weakness #1. The technical contribution is thin.

On a high level, this paper takes the old idea of a diagnostic flowchart for performance problems (tools like PerfExpert used hand-written trees for this long ago, and the top-level split here is the standard roofline taxonomy) and uses it as a scaffold for a GPU kernel agent. The way the tree is grown is basically greedy decision-tree learning: propose a split or add some metrics, keep the change if the score goes up. The LLM only proposes candidates. The one new thing is using downstream optimization reward as the score, which is a small twist on a known recipe.

Weakness #2. The theory doesn’t add anything, and the oracle experiment appears problematic

The theory part adds no value. Proposition 1 just says that picking the best profiling config for each kernel can’t be worse than using one config for all kernels. That’s a one-line fact (the average of the per-kernel best is at least the best average), it holds for any choice problem, and it says nothing specific about profiling or kernels. Presenting it as theoretical analysis oversells it. The oracle experiment that follows isn’t needed to support the proposition either, since the proposition is obviously true no matter what. Its only real use is to show roughly how much there is to gain from per-kernel configs, and even that is unreliable.

In particular, the oracle comparison isn't apples to apples. The static number is one run per kernel, while the oracle is the best of four runs (one for each config) per kernel, and since the rewriter (LLM) is random, the best of four runs will look better even if the configs make no difference. So part or all of the gain may just be luck. A fair check would be to compare against the best of four runs with the same config, or at least report how much a single config varies between repeated runs.

Weakness #3. The experiments don’t support the claims.

CudaForge, KernelAgent and AutoKernel are built to iterate over many rounds, but they’re all cut to one round here, which takes away what makes them work. It’s also unclear whether the baselines get the same correctness-verified Triton starting point that AutoProfiler uses, which could explain a lot of the correctness gap.

AutoProfiler’s own speedup changes a lot between the main table, the ablation, the 48-kernel analysis, the cross-LLM table and the multi-round figure, and the static baseline has different values in different tables, with no clear explanation of the subsets. In the 48-kernel table, the tree closes only a small part of the gap between the static baseline and the oracle, which doesn’t match the claim that it recovers a substantial portion. The “less is more” result comes from adding randomly sampled metrics, so it shows that random extra metrics hurt, not that more relevant info hurts.

The paper is motivated by profiling being expensive, but it never measures profiling time, never says what the profiling budget is, and doesn’t report what it costs to build the tree. More importantly, the cross-LLM table has no fixed-metric baseline, so it doesn’t show the tree helps those models more than a fixed metric set would.

Questions for authors
  1. How do the baselines do when they’re allowed their normal number of iterations? Do they get the same verified Triton seed as AutoProfiler?
  2. Can you report speedup on the same set of problems for all methods, and explain why AutoProfiler’s speedup differs so much across tables?
  3. What is the actual profiling budget, how much profiling time does the tree save compared with fixed configs, and how much did it cost to build the tree?
  4. In the cross-LLM experiment, how does a fixed metric set do with each backbone?
4 comments
AuthorsOpenReview ↗

Response to the Critical Weaknesses

Response to Weakness One

Thank you for your comments, which give us an opportunity to clarify the key distinctions and contributions of our work. Specifically, our contributions lie in two aspects: (1) identifying the counterintuitive phenomenon that more profiling can degrade LLM-based kernel optimization, and (2) learning an adaptive profiling policy driven by downstream optimization gains rather than predefined diagnostic heuristics or classification criteria.

1. We identify a previously overlooked problem: more profiling can lead to worse kernel optimization. Generating high-performance GPU kernels traditionally requires substantial human expertise, and LLM-based kernel optimization has emerged as a promising alternative. While existing approaches commonly employ profiling techniques, including Roofline-based analysis, we identify a counterintuitive phenomenon: collecting more profiling metrics does not necessarily improve kernel optimization and can even degrade performance. We term this phenomenon Profiling Metric Overload and propose an adaptive profiling framework to address it.

2.Our tree construction is driven by optimization utility rather than classification accuracy. Conventional decision trees select splits based on criteria such as information gain or Gini impurity, aiming to improve sample partitioning. In contrast, our goal is not to classify kernels, but to identify which profiling metrics actually contribute to downstream LLM-based optimization. Metrics that effectively distinguish kernel characteristics do not necessarily lead to better optimization decisions. Therefore, conventional split criteria are not directly aligned with our objective. Instead, we use downstream optimization rewards to guide tree expansion, learning a profiling policy that prioritizes metrics based on their actual optimization utility.


Response to Weakness Two

1.Clarification of the role of Proposition 1. Thank you for highlighting the need to clarify the role of Proposition 1. Our intention is to formalize the potential advantage of kernel-specific adaptive profiling over a shared static configuration, thereby motivating the exploration of adaptive profiling strategies. The subsequent Oracle experiment aims to empirically quantify this potential in practical LLM-based kernel optimization. We will revise the presentation of Proposition 1 to make its purpose and scope clearer.

2.A fair comparison confirms the Oracle advantage. We appreciate your concern regarding the unequal sampling budgets in the original Oracle experiment. To address this, we conducted an additional experiment on the full KernelBench Level 1 benchmark (100 tasks), comparing Oracle against Static best-of-4, where Static is also allowed four independent runs and the best result is selected for each kernel. Static improves from 1.664× to 1.740×, but remains below Oracle's 1.958×. This result indicates that the Oracle advantage cannot be explained solely by repeated sampling, providing stronger empirical evidence for the potential benefit of kernel-specific profiling configurations.

Profiling ConfigurationSpeedup
Static (Single Run)1.664×
Static (Best-of-4)1.740×
Oracle1.958×

Response to the Less-is-More Experimental Concern

Thank you for pointing out this. To further investigate whether adding relevant metrics can also degrade optimization, we conducted additional experiments on the full KernelBench Level 1 benchmark (100 tasks), keeping the rewriting process fixed and varying only the profiling metrics. We consider two augmentation strategies: (1) Nearest-Leaf Augmentation, which adds metrics from the nearest neighboring leaf in the profiling tree; and (2) LLM-Guided Augmentation, which adds metrics recommended by the LLM for each task. Both strategies augment AutoProfiler's original profiling configuration. As shown below, AutoProfiler achieves 1.810× speedup, compared with 1.692× and 1.531× under the two augmentation strategies, respectively.

The results show that even adding profiling metrics selected through task-aware or tree-guided strategies can degrade downstream optimization performance. This further supports our observation that more profiling information does not necessarily lead to better kernel optimization, extending the original finding beyond randomly sampled metrics.

Profiling ConfigurationAdditional Metric Selection StrategySpeedup
AutoProfilerOriginal adaptive metrics1.810×
Nearest-Leaf AugmentationMetrics from the nearest neighboring leaf1.692×
LLM-Guided AugmentationMetrics recommended by the LLM1.531×
AuthorsOpenReview ↗

Response to the Questions

Response to the Question 1

Multi-round optimization is already evaluated in Section 5.3.4 and Figure 3. We recursively apply the frozen profiling tree over consecutive optimization rounds, using the kernel produced at each round as the input for the next. Figure 3 compares the optimization trajectories in terms of cumulative wall-clock time. AutoProfiler improves from 1.629× in the first round to 2.112× after seven rounds, while CudaForge reaches 1.846× after four rounds within a comparable cumulative time. Notably, AutoProfiler requires substantially less time per iteration by avoiding repeated expensive profiling and LLM-based profiling decisions, achieving a more favorable optimization-cost trade-off.


Response to the Question 2

We thank the reviewer for raising this concern. To further assess the robustness of our findings, we conducted an additional evaluation on the full KernelBench Level 1 benchmark (100 tasks), comparing Static, AutoProfiler, and Oracle under the same evaluation setting. As shown in the following table, AutoProfiler achieves a 12.3% improvement over Static, while Oracle reaches 1.958×. These results reinforce the findings from our original evaluation on representative kernels: kernel-specific adaptive profiling consistently outperforms a shared static configuration, while further gains are attainable through per-kernel optimal profiling configurations. This conclusion also holds when extending the evaluation to the full 100-task Level 1 benchmark, further supporting the robustness of our findings.

Profiling ConfigurationSpeedup
Static1.612×
AutoProfiler1.810×
Oracle1.958×

Response to the Question 3

Offline Tree Construction Cost. We report the cost of constructing the profiling tree below. Building the tree takes 28.2 hours of wall-clock time and consumes 2.98M tokens. This is a one-time offline cost: the resulting tree is frozen and reused during kernel optimization, rather than reconstructed for individual tasks.

Cost ComponentTime / Resource
Total wall-clock time28.2 h
LLM execution time16.1 h
NCU profiling5.2 h (276 routes)
GPU validation8.0 h
Token consumption2.98M

Runtime Profiling and Optimization Efficiency. AutoProfiler incurs 21.5 s of profiling time per task, comparable to 23.8 s with static profiling. Its efficiency advantage therefore does not stem from lower profiling overhead, but from providing more informative profiling signals for downstream LLM-based optimization. These signals reduce unnecessary LLM calls and shorten optimization iterations. As demonstrated in Figure 3, AutoProfiler achieves higher speedups than baseline under a comparable cumulative wall-clock budget, highlighting the benefit of more effective profiling guidance for downstream optimization.


Response to the Question 4

Thank you for this valuable suggestion. Given the experimental scale involving four backbone LLMs and eight benchmark tasks, this comparison requires substantial evaluation efforts. We are actively conducting additional experiments to evaluate fixed profiling metrics across different backbones and will report the results as soon as they are available.

Thanks authors for the detailed response and the additional experiments. The Static best-of-4 comparison and metric augmentation experiments partially address my concerns.There are still a few issues remain to be addressed. For example:

  • the technical novelty is still limited, and the theoretical results are not significant
  • the fairness of baseline comparisons can be further strengthened, particularly regarding initialization and iteration budgets That being said, , I am willing to increase my recommendation

Thank you for being willing to increase your recommendation. We appreciate your recognition of the additional experiments.

Regarding the technical novelty, we understand your concern that the decision tree itself is not a fundamentally new algorithm, and that the theoretical results are primarily supportive rather than a major theoretical contribution. Our intention, however, is not to introduce a new tree-learning algorithm, but to identify a practical and effective solution to an overlooked problem in LLM-based kernel generation: how to make profiling information more useful for downstream optimization decisions.

We believe the tree-based design is particularly suitable for this purpose for two reasons. First, it provides an efficient and deterministic decision process, avoiding the overhead of repeatedly invoking an LLM for profiling interpretation. Second, and more importantly, it is model- and architecture-agnostic: it does not introduce an additional Transformer architecture or require modifying the underlying kernel generation model. This allows our profiling mechanism to serve as a lightweight, plug-and-play component that can be integrated with different kernel generation methods, most of which are based on Transformer models or agentic systems.

From this perspective, we see the main contribution as introducing a complementary optimization paradigm: rather than improving kernel generation solely through better models, prompts, or search strategies, we show that better profiling-driven decisions can improve the effectiveness of existing kernel generation approaches. We hope this perspective helps clarify the practical and methodological novelty of our work.

Due to limited computational resources, these experiments are still ongoing, and we will continue to supplement the results. We appreciate this valuable suggestion, which will help us further strengthen the empirical evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.