acceptodds
Withdrawn from ICLR 2027

EvoCompile: Bridging the Compilation Effectiveness Gap through Experience-Adaptive Agentic Optimization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern deep learning compilers provide powerful optimization capabilities, yet effectively utilizing them across diverse training workloads still requires workload-specific decisions. Different workloads may expose insufficient computation to the compiler, contain compiler-unfriendly program structures, or leave additional optimization opportunities unexplored. We characterize this problem as the compilation-effectiveness gap, defined as the performance gap between general compiler usage and workload-specific optimization. We present EvoCompile, an experience-adaptive agentic optimization framework that reduces this gap by learning how workloads should be presented to existing compilers. EvoCompile accumulates reusable optimization experience and applies targeted interventions across compilation, program, and kernel levels, refined through execution feedback. We evaluate EvoCompile on 119 workloads from HuggingFace, TIMM, and TorchBench, together with large-scale training and cross-backend scenarios. EvoCompile consistently improves compilation effectiveness while preserving correctness, achieving a 1.89× geometric-mean speedup on TorchBench. Code is available at https://anonymous.4open.science/r/EvoCompile-8C20/

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Deep Learning Compilation, Compiler Optimization, Agentic Optimization, Experience-Adaptive Systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jAn4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes EvoCompile, an LLM agent that improves how deep learning compilers such as torch.compile are used on a given training workload, without modifying compiler internals. The agent profiles the workload, looks up matching optimization schemes in an experience memory, and turns them into compilation-, program- and kernel-level interventions. Execution feedback refines each recipe, and both successful and failed attempts update the memory. EvoCompile is compared with the PyTorch benchmark's default compilation workflow, built-in compiler modes and the Cursor coding agent on HuggingFace, TIMM and TorchBench.

Critical strengths
  1. EvoCompile works without modifying compiler internals. It changes compilation scopes, applies fullgraph to suitable regions and registers new kernels through torch.library, so it can be used with existing compilers. The paper applies it to Inductor, OpenXLA and Thunder.

  2. The agent's changes are constrained. Instead of freely deciding what to modify, the Optimizer fills in the operations of a validated scheme, and every recipe is evaluated against the frozen vanilla baseline. This guards against edits that miss the compilation bottleneck or break the workload's correctness.

Critical weaknesses
  1. Without the experience memory, EvoCompile performs close to naive compile (Table 4), so the results depend on what the memory contains. Sec. 3.3.3 says the memory is constructed from previously observed cases, but not which cases these were. Since the paper asks whether experience can be reused on new workloads (Sec. 1), it would help to state whether the evaluated workloads were among them.

  2. The improvement over the default compilation workflow is a few percent (Table 1). EvoCompile gets 10 optimization iterations on each workload, while each compiler baseline is a single fixed configuration. Compilation-level interventions alone reach the full HuggingFace result (Table 4). A simple search over these interventions with the same budget and no LLM would be a useful baseline.

Questions for authors
  1. Did the reported runs start from the experience memory in the supplementary material? It includes cases with recipes for evaluated workloads, for example XLNetLMHeadModel and demucs. If so, was each workload's own case in the memory when that workload was optimized?

  2. Sec. 1 says later workloads benefit from the outcomes of earlier ones. In what order were the 119 workloads processed? How do the results on one suite change when the memory is built only from the other suites?

  3. With the same 10 optimization iterations, what does a simple search over compilation-level interventions achieve without the LLM, for example random search? What does EvoCompile's search cost per workload, in LLM calls and time?

  4. In Table 2, EvoCompile matches TorchTitan on Llama3-8B at both sequence lengths. The gains appear on the debug variants of about 0.32B and 33M parameters. What limits EvoCompile on the full-size model?

  5. How was the Cursor baseline set up, for example its starting point, prompt, iteration budget and access to execution feedback?

Reviewer_Vyj1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoCompile is an agentic framework for improving the performance of deep learning workloads by adapting how they are presented to existing compilers. It collects compilation and profiling evidence, retrieves optimization schemes from an experience memory, and applies interventions at three levels: compilation configuration and boundaries, program rewriting, and kernel replacement or generation. Execution feedback guides repair, further exploration, and updates to the experience memory.

The paper evaluates 119 workloads from HuggingFace, TIMM, and TorchBench, with additional cross-backend, cross-hardware, and training case studies. Relative to eager execution, EvoCompile reports geometric-mean speedups of 2.14×, 2.01×, and 1.89× across the three suites, compared with 2.03×, 1.99×, and 1.77× for the default benchmark compilation workflow.

Critical strengths
  • The paper addresses a practical optimization problem beyond individual kernel generation. Compilation boundaries, graph breaks, program structure, and optimizer coverage can affect end-to-end performance. Coordinating interventions across these levels is useful, and the framework integrates them without requiring changes to compiler internals.

  • The evaluation includes realistic workloads and relevant compiler baselines. Testing 119 workloads against the official benchmark compilation workflow and built-in optimization modes is more informative than comparing only with eager execution or naive compilation. The component ablations and additional backend and hardware experiments provide encouraging evidence that the framework can identify complementary optimization opportunities.

Critical weaknesses
  • The improvements over compilation baselines are modest and lack statistical validation. EvoCompile achieves speedups of 2.14× vs. 2.03× on HuggingFace, 2.01× vs. 1.99× on TIMM, and 1.89× vs. 1.77× on TorchBench compared with default compilation, which is not very impressive. The paper reports no confidence intervals, significance tests, or variability across independent runs, making it difficult to assess whether these gains—particularly on TIMM—reliably exceed measurement noise. Optimization costs are also unreported, leaving their practical value unclear.

  • Comparisons with relevant agentic optimization approaches are limited. The only evaluated agent baseline is Cursor; the paper neither discusses nor compares with AlphaEvolve or related evolutionary program-optimization frameworks. Although these systems may require adaptation to the compilation setting, a representative agentic search baseline using the same tools and optimization budget would help establish the distinct benefit of EvoCompile’s experience-guided design.

Questions for authors
  • How were the initial optimization schemes, predicate tree, rewrite templates, and kernel catalog created? Were any developed using the evaluated workloads? Is memory updated across evaluation tasks, and how sensitive are results to workload order?
  • What are the optimization wall time, number of LLM calls, token usage, and compilation/evaluation costs per workload?
Reviewer_3bqhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes EvoCompile, a framework for improving the effectiveness of existing deep learning compilers through workload-specific optimization. It collects compilation and runtime information, retrieves relevant optimization experiences, and applies interventions at compilation, program, and kernel levels. Execution feedback is then used to refine optimization strategies and update the experience memory. Experiments on 119 workloads demonstrate improvements over standard compilation configurations, with additional evaluations on large-scale training and different compiler backends.

Critical strengths
  1. The paper identifies an important gap between general compiler usage and workload-specific optimization, addressing a meaningful problem in deep learning systems.
  2. The combination of experience reuse, multi-level interventions, and feedback-driven refinement provides a systematic approach to compiler optimization.
  3. The experiments cover 119 workloads across three benchmark suites, with additional cross-backend evaluations and ablation studies demonstrating the effectiveness of the proposed framework.
Critical weaknesses
  1. The paper does not fully explain how the initial experience memory is constructed and how experiences are separated across evaluation workloads. More details would help readers understand the generalization capability of the framework. This appears to be a clarification issue.
  2. The reported performance improvements focus on execution latency after optimization. Since EvoCompile involves multiple optimization iterations, additional discussion of LLM calls, compilation costs, and optimization overhead would help assess its practical efficiency.
Questions for authors
  1. How is the initial memory constructed, and how do the authors ensure that experiences from evaluation workloads do not affect the fairness of the comparison?
  2. Does the reported performance improvement include the costs of LLM calls and compilation during optimization? If not, could the authors briefly discuss when these costs can be amortized?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.