- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
KernelWeaver addresses a practical limitation of current kernel agents: optimizing one kernel for a few representative inputs often leads to poor performance on other shapes seen in deployment. The paper proposes generating candidates for hardware-derived shape regions, selecting a small portfolio based on marginal performance gain under deployment costs, and dispatching requests through a shape-aware router. Online feedback is then used to identify poorly served shapes and trigger further refinement. The evaluation covers nine operators on an RTX 4090 and an A800. The paper reports geometric-mean speedups of 1.59x and 1.44x over PyTorch eager, respectively, while deploying 28 kernels selected from a pool of 96 candidates.
Critical strengths
1.The paper identifies a real and underexplored issue in kernel-agent evaluation. Reporting performance on a few representative shapes can hide severe regressions elsewhere, and the motivation experiments make this point clearly.
2.Reframing the problem as portfolio construction is sensible. A kernel that is not the best on average can still be useful if it covers shapes where the current portfolio performs poorly.
3.The method considers deployment concerns beyond kernel latency, including compilation cost, code size, and routing overhead. This is more realistic than evaluating each shape independently.
4.The experimental section is reasonably transparent about some trade-offs. For example, the paper reports that portfolio selection slightly hurts Softmax performance, and that several held-out cases remain below eager performance.
Critical weaknesses
1.The core shape-partitioning procedure is not specified precisely enough to reproduce. The paper lists hardware checks involving shared memory, Tensor Core granularity, occupancy, and cache behavior, but does not explain how these checks become concrete shape boundaries. It is also unclear how tile candidates are chosen, how overlapping regions are resolved, or how “extreme-shape families” are defined.
2.The cost-aware selection objective is under-justified. Compilation time, code size, and routing-table complexity are normalized and summed without weights, but their relative importance depends heavily on the deployment setting. The paper does not provide the actual budget values, the normalization details, or sensitivity experiments for alternative cost weights.
3.The claimed compilation-cost advantage is incomplete. The reported 13.53 seconds only covers compilation of the final deployed portfolio. It excludes the cost of generating, compiling, validating, and profiling the full candidate pool, as well as the cost of LLM inference. Since KernelWeaver profiles every candidate across the target shape set, the total optimization cost may be substantially larger than the reported deployment compilation time.
4.The evidence for online refinement is weak. The ablation is conducted with fixed-shape offline timings, so it does not demonstrate that the feedback loop works under a real deployment trace. The paper does not explain how long feedback must be collected, how low-gain cases are detected online, how much refinement costs, or how regressions are prevented before updated kernels are deployed.
5.The baseline comparison is not broad enough. The paper compares against PyTorch eager, torch.compile, Starter Kernel, and AutoKernel, but omits direct comparisons with dynamic-shape optimization systems such as DietCode and Roller, or with a strong Triton autotuning baseline under the same search budget. PyTorch eager is a relatively weak reference for several of these operators.
6.The evaluation lacks variance and tail-performance analysis. Kernel generation is stochastic, yet results are reported from what appears to be a single generation run. There are no confidence intervals, repeated runs, or p95/p99 latency results. The held-out evaluation also lacks a clear sampling protocol and exact shape lists.
Questions for authors
1.Can the authors provide a complete algorithm or pseudocode for shape partitioning? How are tile configurations generated, how are hardware checks converted into boundaries, and how are overlapping regions handled?
2.What are the exact deployment budgets and normalization formulas used in portfolio selection? How do results change when the relative weights of compilation time, code size, and routing complexity vary?
3.Can the authors report the full end-to-end optimization cost, including LLM generation, candidate compilation, correctness checking, profiling, routing-table construction, and final deployment? At what request volume does this cost amortize against torch.compile?
4.Why are DietCode, Roller, or a budget-matched Triton autotuning baseline not included? Is the torch.compile compilation cost measured over exactly the same shape coverage as KernelWeaver?
5.What is the concrete protocol for online refinement? What workload trace is used, how is a low-gain shape identified, how many requests are needed before refinement, and how are correctness and performance regressions controlled before deployment?
6.How are held-out shapes sampled and kept separate from prompt design and method development? Please provide the exact shapes, dtypes, layouts, fallback rate, worst-case slowdown, and tail latency results.
7.Can the authors report results over multiple independent kernel-generation runs and repeated timing measurements? This is especially important because shape-aware candidate generation accounts for most of the reported gain.