- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses how to choose a fast GPU kernel from the many implementations available in CUTLASS without exhaustively benchmarking every option. The main idea describe each candidate through its expected hardware behavior, such as data reuse, memory demands, and GPU utilization, rather than relying only on configuration settings. A learned model then uses these descriptions to rank candidates and select an implementation with potentially good performance. The main contribution is a set of statically computed features that estimate each kernel's data reuse, memory traffic, resource usage, and pipeline behavior. These features augment the kernel configuration parameters, allowing a learned model to rank candidates without executing them. Experiments show that the approach reduces mean selection regret, the throughput lost relative to the fastest candidate, from 17.3% with NVIDIA’s heuristic to 6.2%, and supports adaptation to different numerical precisions and fused operations.
Critical strengths
- Searching the large CUTLASS kernel space requires substantial overhead for both compilation and profiling. The proposed approach uses statically computed hardware features with standard learning models, making it relatively straightforward to implement while avoiding candidate kernel benchmarking at selection time. The evaluation shows very clear gain over SOTA approaches.
Critical weaknesses
The paper claims that hardware-aware features enable data-efficient transfer across numerical precisions. However, the transfer experiments only fine-tune pretrained BF16 selectors without comparing against models trained from scratch using the same target-domain data. This makes it difficult to determine whether the reported performance improvements arise from transferable knowledge or simply from the available target-domain supervision. Moreover, data efficiency is evaluated in terms of the fraction of GEMM shapes rather than the actual number of kernel measurements, even though each shape may involve thousands of measured configurations. A comparison against from-scratch training under identical measurement budgets would provide stronger evidence for the claimed transfer efficiency.
Questions for authors
How does fine-tuning a BF16-pretrained selector compare with training the same model from scratch using exactly the same target-domain measurements? This comparison is important for determining whether the reported data efficiency comes from knowledge transferred across precisions or simply from the available target-domain training data.
We thank the reviewer for their feedback and for recognizing the practical relevance of our selection approach and the performance improvements over existing approaches.
How does BF16 pretraining compare with training from scratch under identical target-domain budgets?
Our transfer experiments are intended to demonstrate that the hardware-aware representation remains effective across numerical precisions, with strong performance even when fine-tuning uses a limited fraction of target-domain shapes. To quantify the additional benefit of pretrained parameters independently of the representation, we are currently preparing and running the requested experiments; we will submit a revision in the next few days including the requested from-scratch comparison under identical target-domain measurement budgets. We will also report the exact number of target-domain kernel measurements used at each training fraction, ensuring that data efficiency is evaluated in terms of measurement cost and not just shape coverage.