acceptodds
Under review as a conference paper at ICLR 2027

TileTune: An Efficient End-to-End Autotuning System for Tile-Based GPU Kernels

Abstract

Tile-based domain-specific languages simplify GPU kernel development, but achieving high performance still requires tuning parameters such as tile sizes and pipeline depths. Existing tuning approaches face a tradeoff between efficiency and generality: search-based methods are broadly applicable but incur compilation and benchmarking costs for each evaluated configuration, while heuristic-based methods reduce evaluation work but often require kernel-specific customization. Moreover, the explicit tile structure that this new generation of languages exposes remains largely unexploited, as existing tuners were designed for loop-level schedules or fixed kernel templates. We present TileTune, an end-to-end autotuning system that provides efficient, out-of-the-box tuning for tile-based GPU kernels, which balances generality, efficiency, and tile-awareness. At the algorithmic level, we propose a lightweight cost model using static program information rather than user-written, kernel-specific selection rules. It uses a shared analysis of explicit tile operations and resource requirements to rank and prune candidates while retaining empirically good selections. At the system level, we build a parallel task-processing runtime that reduces the cost of evaluating survivors through efficient dispatching, grouped compilation, and overlapping compilation and benchmarking. Across a range of kernels and hardware platforms, TileTune achieves end-to-end tuning speedup over the TileLang search-based baseline. Its shortlist retains the full-search optimum in all evaluated A100, H200 and B300 workloads.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
GPU kernels, DL compiler, machine learning system, AI infrastructure, autotuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ihn6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces TileTune, a system designed to reduce the time needed to find high-performing GPU kernel configurations using TileLang. It first uses static information from tile programs to prune candidate configurations before empirical evaluation. The remaining candidates are then evaluated using grouped compilation and compilation–benchmarking overlap, with the fastest valid kernel selected based on measured latency. The experiments compare end-to-end tuning time, pruning quality, and selection cost. Across 25 workloads on H200, TileTune achieves a 2.16× speedup over the four-GPU TileLang Autotune baseline. With a shortlist budget of α = 0.5, it retains the full-search optima across all evaluated workloads on A100, H200, and B300. Its selection cost is higher than Roller's but lower than XGBoost's, while both baselines retain the optima less consistently.

Critical strengths

(+) By leveraging static tile-program information for candidate pruning and streamlining the evaluation of retained configurations, TileTune reduces end-to-end autotuning time while preserving the full-search optima across the evaluated workloads.

Critical weaknesses

(-) Limited evidence for generality and pruning robustness. The evaluation mainly focuses on GEMM and attention-related workloads, with pruning tested only at a 50% shortlist budget. This makes it difficult to assess whether TileTune generalizes to kernels with different computational patterns or remains effective under more aggressive pruning. In addition, the ablation focuses on system-level optimizations without isolating the contributions of the work and underfill views. It therefore remains unclear whether combining these views through rank multiplication improves pruning quality over using either view alone.

(-) Overemphasis on comparisons with a baseline using fewer GPUs. The paper highlights speedups of 4.02–5.35× against the one-GPU TileLang baseline, while TileTune uses four GPUs. Although a resource-matched comparison is included, the aggregate speedup on H200 drops from 4.02× to 2.16×. Emphasizing the one-GPU results in the abstract and Figure 5 may give an exaggerated impression of the efficiency gains under equal GPU resources.

Questions for authors

Q1. Pruning budget sensitivity. How would TileTune perform under a smaller shortlist budget, such as α = 0.3? Would it still retain the full-search optima, and how would this affect the selected-kernel performance?

Q2. Generality across kernel types. Would TileTune remain effective on other types of tile-based kernels, such as reduction-heavy or data-movement-oriented kernels, without requiring kernel-specific modifications?

2 comments

Thank you for your rigorous review. We value your feedback on the evaluation of our work. We would like to clarify that while our paper includes a pruning model, the core contribution of TileTune is to rethink the autotune system in the era of tiled-based DSLs (As in Global Comment 1). We identify that the unique structure of tiled DSL programs enables a more efficient, system-level orchestration of compilation and benchmarking. Consequently, our cost model is designed to be simple and general, serving as a heuristic pruner to narrow the search space rather than a high-precision performance estimator. Nevertheless, we agree that the robustness and generality of the pruner are critical, and we appreciate your suggestions for deeper evaluation. Below, we address your concerns point by point.

W1. Ablation of work and underfill views. We thank the reviewer for this insightful suggestion. We have performed an ablation study to isolate the individual contributions of the "work view" and "underfill view." As shown in the table below (using H200 as a representative case),

α\alphaTileTune (work x underfill)Work view onlyUnderfill view only
0.525/2525/2525/25
0.425/2524/2524/25
0.323/2520/2514/25

The data confirms that the work view provides the necessary baseline accuracy (based on a classic roofline model), while the underfill view acts as a targeted correction for wave quantization effects.

W2. Overemphasis on comparisons with a baseline using fewer GPUs. We have updated our manuscript so that the speedups over the 4-GPU baseline are reported as our primary results (See Global Comment 2). We are sorry for the confusion caused by our earlier presentation.

Q1. Pruning budget sensitivity. As shown in the table in Global Comment 3, Even at α=0.3\alpha=0.3, TileTune stays essentially lossless (98.8%–99.5% geomean across all GPUs). At this tighter budget the optimum is no longer retained on every workload (23/25), yet the kernels actually selected stay near-optimal. Our primary objective is to develop a near-lossless pruner. Therefore, we adopt a conservative setting of α=0.5\alpha=0.5 to ensure the highest possible fidelity in performance.

Q2. Generality across kernel types. We agree that our evaluation did not separate the regimes. The pruner prices tile-level reuse, so it works best when the tile choice changes traffic that dominates latency. Reduction and data-movement kernels sit at the other end of this spectrum. Even there, the loss stays small, because a 50% shortlist already contains a near-optimal configuration. On B300:

kerneltypepoolGeomean (latency)
split-K row-sumreduction3780.98
transposedata-movement960.97
repeat_kvdata-movement481.0
stencil convconvolution741.0

We agree with the reviewer's concern about the generality of the cost model, and we believe its root cause is that the model's validity depends on which configuration inputs it actually covers. For instance, whether rasterization (thread-block swizzling) is enabled for a GEMM is an input we do not currently model at all, so two candidates that differ only in that setting are indistinguishable to the pruner. In the camera-ready version we will therefore constrain the pruner's inputs explicitly. This keeps the pruner within its intended role — an intermediate optimizer that discards clearly slow candidates — rather than an accurate GPU cost model.

Dear Reviewer ihn6,

Thank you again for your time and suggestions. As the discussion ends on November 18 (AoE), we would appreciate knowing whether our rebuttal has addressed your concerns. If so, we would be grateful if you would consider a possible score increase at your discretion. We are happy to clarify any remaining points before the deadline.

Best regards, The authors

Reviewer_CsU9OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents TileTune, an autotuning system for TileLang that aims to reduce the cost of searching kernel configurations. It combines a static cost model to prune candidates with a runtime that groups compilation tasks and overlaps CPU compilation with GPU benchmarking.

The authors evaluate TileTune on five kernel families across A100, H200, and B300 GPUs. They report 4.02–5.35× speedups over the default single-GPU TileLang autotuner. With a 50% shortlist budget, TileTune retains the full-search optimum for all evaluated workloads.

The results are encouraging, and the system addresses a practical problem. My main concern is whether the current evaluation is sufficient to support the broader claims about generality and tuning efficiency.

Critical strengths

Practical problem. Exhaustive autotuning can be expensive because each configuration requires compilation, correctness checking, and benchmarking. Reducing this overhead is useful for kernel development.

Simple and interpretable approach. The pruning model uses information available from the tile program, including memory traffic, computational work, launch-grid size, and pipeline depth. It does not require training data or kernel-specific ranking rules.

Good end-to-end results. The reported speedups are significant across three GPU generations. The fact that the shortlisted candidates include the full-search optimum in all tested workloads is also encouraging.

Reasonable system design. The paper addresses both candidate selection and evaluation overhead. The ablation study helps clarify the contribution of each optimization.

Critical weaknesses

Generality is not fully demonstrated. The implementation is limited to TileLang and its CUDA backend. In addition, three of the five kernel families are GEMM variants, and the KDA experiment covers only part of the operator. The current evaluation does not establish whether the approach works equally well for more diverse tile programs or other DSLs.

The cost model may miss important performance factors. The model does not explicitly account for cache behavior, synchronization, register pressure, occupancy, or warp scheduling. These factors can change the ranking of configurations, particularly for attention kernels. The paper acknowledges this limitation, but provides little analysis of when and how often it affects the results.

The shortlist budget needs further investigation. The reported 100% optimum retention is based on retaining 50% of the candidates. This is useful, but still leaves half of the search space to compile and benchmark. Results at smaller budgets would help establish how much tuning cost can be saved without sacrificing kernel quality.

The baseline comparison could be stronger. The paper compares against exhaustive TileLang autotuning and uses Roller and XGBoost for the pruning evaluation. However, it remains unclear how TileTune compares with stronger tile-aware analytical methods or learning-based approaches that use tile-level features. The cost of collecting XGBoost training data also makes the comparison dependent on the assumed workload and model-reuse scenario.

Some system assumptions limit the conclusions. The performance of grouped compilation and pipelined evaluation depends on group size, worker count, compiler compatibility, and available GPUs. These parameters are manually configured. More evidence is needed to understand how robust the gains are under different workloads and deployment conditions.

Questions for authors

How does optimum retention change when the shortlist budget is reduced to 25%, 10%, or lower? What is the corresponding end-to-end tuning time?

Can the authors analyze cases where the static ranking disagrees with measured performance? Which factors, such as register pressure, occupancy, or cache behavior, account for most of these errors?

How much of the implementation is specific to TileLang? What would be required to support Triton or other tile-based DSLs?

Are the baseline comparisons conducted with comparable CPU resources, GPU resources, and compilation-cache conditions? How would the results change if the XGBoost model could reuse training data across workloads?

How sensitive are the results to compilation-group size and the number of compiler and benchmarking workers? Could these parameters be selected automatically?

1 comment

Thank you for your constructive review and positive feedback. We address your five questions below.

Q1. Ablation of α\alpha. As shown in the table in Global Comment 3, reducing α\alpha to 0.250.25 provides a compelling trade-off: it maintains nearly lossless performance (98.3%–99.5% geomean across all GPUs) while significantly accelerating the tuning process (3.69x speedup over the 4-GPU baseline). However, our primary objective is to develop a near-lossless pruner. Therefore, we adopt a conservative setting of α=0.5\alpha=0.5 to ensure the highest possible fidelity in performance.

Q2. Error analysis of static ranking. We acknowledge that our static cost model, while effective for candidate pruning, simplifies complex GPU architectural factors like register pressure. However, the key strength of TileTune is that it does not rely on a perfect ranking. Instead, it uses the static model to produce a sufficiently large and high-quality "shortlist." Even when the model's top-1 prediction is not the true optimum, the exhaustive search over the shortlist (which contains the true optimum in all our experiments) effectively mitigates these ranking errors. According to Global Comment 3, we observe performance degradation when reducing α\alpha to 0.1. Our analysis reveals that discrepancies typically arise in kernels where performance is bottlenecked by register-induced occupancy limits rather than pure memory traffic or compute intensity. When α=0.5\alpha = 0.5, the global optimum is consistently retained, demonstrating that our shortlist budget is sufficiently wide to buffer against these ranking inaccuracies.

Q3. Integrating with other DSL. TileTune’s design is modular, separating the search strategy (pruner + pipelined evaluation) from the DSL-specific frontend. The pruning logic currently relies on a set of general tile-level features (e.g., memory traffic, computational intensity, grid dimensions, and pipeline depth) that are common to most tile-based compilers. To support other DSLs like Triton, the primary task would be to implement a parser that parses the target DSL’s Intermediate Representation (IR, e.g. for Triton it’s TTGIR) and adapts to our system.

Q4. Baseline comparisons. We thank the reviewer for this concern. We confirm that all baseline experiments were conducted under identical environmental conditions, including equivalent CPU/GPU resources and synchronized compilation-cache states.

Regarding the comparison with XGBoost, we recognize that it requires a training phase. In our evaluation, the "tuning time" for the XGBoost-based pruning includes the overhead of training data collection and model training on a representative subset of workloads. Despite this overhead, TileTune maintains a significant advantage in end-to-end tuning time. Furthermore, while XGBoost can potentially reuse training data across kernels, its performance is highly sensitive to the diversity of the training set. In contrast, TileTune’s static, rule-based cost model requires zero training data and generalizes instantly to unseen kernel types without the cold-start cost.

Q5. Ablation of compilation group size and number of workers. Our ablation study on compilation group size and number of workers (Appendix. A) demonstrates that TileTune’s performance gains are robust across a range of configurations. Specifically, the optimal compilation-group size is primarily CPU-dependent and can be easily tuned once for a given platform., while the number of workers scales the throughput of the pipelined evaluation.

Reviewer_kYgvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper concerns autotuning for tile based domain specific languages. It observes that SOTA autotuning techniques that are search based are general but inefficient. Heuristics based techniques are efficient but require custom information. Neither exploits information provided by tile based languages. The paper proposes a new autotuning technique, TileTune, that is tile-aware and uses straightforward analytical cost models at the tile granularity to efficiently prune candidate configurations. It also proposes a pipelined profiling and compilation workflow that further reduces the end to end autotuning cost. It compares against the out-of-the-box autotuner of TileLang and shows that TileTune is both faster and exposes the optimal configuration. Unfortunately, most of the results are provided for a 1 GPU version of TileLang's autotuner vs. 4 GPU version of TileTune.

Critical strengths
  • The paper addresses an important problem.
  • TileTune is faster and finds the best configuration (but see caveat below).
Critical weaknesses
  1. The ideas are not particularly novel, which by itself is not problematic for me as long as they solve an important problem.
  2. The comparisons of TileTune vs. TileLang's autotuner baseline are often given with 4 GPUs for the former and 1 for the latter. The exception is for the H200 in Section 5.1, where we see that the speedup comes down significantly from 1 to 4 GPUs (4x to 2x). The paper does not provide the 4GPU number for A100 and B300. It just seems odd to keep describing benefits relative to 1 GPU baseline while using 4 GPUs for TileTune.
Questions for authors
  1. Any response to allay the "ideas not novel" comment is welcome. For example, the idea could be simple, but it will help to know if the implementation was complex.
  2. Please address why numbers with 1 GPU for TileLang are quoted in most places (instead of the resource-equivalent 4 GPU). In particular, what are the speedups and configuration candidates for A100 and B300?
3 comments

Thank you for the positive evaluation and for recognizing the practical importance and effectiveness of TileTune. Below, we address your two questions directly.

Q1. Novelty. Our contribution/novelty is discussed in Global Comment 1. In practice, the end-to-end system implementation is quite complex—such as integrating static analysis with TileLang/TVM TIR and adapting to the TVM-FFI compilation and caching pipeline—but we intentionally omitted these low-level engineering details from the main text to leave room for the core narrative, and we will provide useful implementation details in the appendix of the revised version.

Q2. 4-GPU baseline. We have updated our manuscript so that the speedups over the 4-GPU baseline are reported as our primary results (See Global Comment 2). We are sorry for the confusion caused by our earlier presentation.

Reviewer_kYgvOpenReview ↗

Response to authors

Thank you for your response. It addresses my major concern of comparison against 1 GPU. I also appreciate your discussion about novelty considerations, though the comment remains. I will keep my score as weak accept.

Thank you for confirming that our rebuttal addresses your major concern about the single-GPU comparison. We appreciate your acknowledgment of the novelty discussion and will further strengthen the positioning of our contribution in the revision. Thank you for keeping your score as weak accept.

Comments on the paper

1 comment

Dear ACs and reviewers,

We have highlighted in red the updated parts of our revised manuscript, mainly the revisions addressing the 4-GPU baseline issue that you raised, to make them easier to review.

Best regards, The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.