acceptodds
Under review as a conference paper at ICLR 2027

From Hotspots to Edit Locations: Guiding Repository-Level Code Optimization

Abstract

Modern automatic code optimization approaches rely on an LLM to optimize code. Given the cost of invoking an LLM and its limited context length, in large repositories, we must choose where to optimize within a limited budget. Profiling identifies costly functions, called hotspots, but optimizing hotspots alone may not be sufficient: we may also need to optimize their (direct or indirect) callers to reduce the calls to hotspots. However, the callers may be too many to examine within a limited budget. We introduce CHICO, a localization approach that combines profiling information with optimization opportunities detection. Given a hotspot function and its callers, our approach uses a lightweight static analysis to detect potential opportunities to reduce the number of calls to the hotspot in the callers, and consider only these callers for the LLM to optimize. We implement CHICO as a prototype that integrates this localization approach with SemOpt, a state-of-the-art optimization approach for a single function. We evaluate the system on GSO and SWE-fficiency-Lite against four baselines using four language models. Our approach successfully optimizes 10.29–123.53% more cases than the baselines on GSO and 12.50–239.13% on SWE-fficiency-Lite.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, automated code optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mFf5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CHICO, a repository-level code optimization system that combines profiling with lightweight static analysis to select hotspot and caller candidates. Caller candidates receive reuse or reorganization suggestions, which guide a SemOpt-based optimizer. The evaluation compares complete systems on GSO and SWE-fficiency-Lite across four language models and includes a controlled comparison of localization inputs.

Critical strengths
  1. The opportunity-based caller-selection design is worth investigating. Combining loop and call-input checks with execution cost and opportunity type provides a concrete way to select callers for inspection. Supplying both a starting location and an optimization direction is a useful design choice (Sections 2.2–2.3).
  2. The complete-system results are encouraging. Table 2 reports consistent improvements in optimization metrics across the evaluated models and benchmarks. For example, GSO Opt@1 with Sol increases from 46.08% for POLO to 63.73% for CHICO. Table 3 also provides evidence that the combined localization input helps the optimizer.
Critical weaknesses
  1. The localization ablation leaves the source of the gains unclear. Table 3 keeps the optimizer, strategy library, validation, and control flow fixed, which is useful. However, it replaces a single Profile-Top1 target with up to three opportunities, adding caller candidates, optimization directions, and supporting evidence together. This establishes the value of the combined input but does not separate opportunity-aware selection from broader candidate coverage or additional guidance. Since opportunity-based caller selection is the central methodological contribution, this uncertainty materially limits how much the experiment establishes about that contribution.
  2. The contribution beyond closely related localization work is not sufficiently established. EffiHolmes already studies the gap between profiling hotspots and fix locations using execution paths and guided reasoning. CHICO cites it but distinguishes the two mainly by their inputs: a reported performance issue versus a performance test. That distinction is meaningful, and a direct benchmark comparison may not be appropriate. However, I would like a clearer account of the localization capability or efficiency trade-off that CHICO adds. CHICO reports downstream optimization success without directly analyzing the selected locations. Together with the limited comparison in Section 4, this leaves the methodological advance less clear than the complete-system improvement. This affects my assessment of originality and significance.
Questions for authors
  1. Clarification of contribution attribution. Do you have comparisons or existing run-level analyses that separate the benefit of opportunity-aware selection from providing additional caller candidates and optimization guidance? In particular, does the advantage persist against cost-ranked hotspot/caller candidates under comparable candidate and optimization budgets?
  2. Clarification of the ranking criterion and benchmark coverage. Have you examined the relationship between call multiplicity, average per-call cost, and successful caller-side edits in the two benchmarks? What evidence supports the current cumulative-cost and opportunity-type selection rule as a way to prioritize useful candidates? Frequent calls need not be redundant, so this question concerns avoidable work, not call count alone.
  3. Clarification of the relationship to EffiHolmes. Beyond input requirements, what localization capability or efficiency trade-off distinguishes CHICO, and which existing results or concrete cases support that distinction? Evidence need not take the form of agreement with a single expert patch, since multiple edit locations may yield valid optimizations.
Reviewer_HdymOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CHICO, a localization approach for repository-level code optimization with LLMs. Its key idea is that a hotspot function may have little room for improvement, while its callers can be optimized to reduce the number of calls to it. CHICO combines profiling with lightweight static analysis to find such callers, for example hotspot calls inside loops, and passes them together with the hotspots to a SemOpt-based LLM agent. On GSO and SWE-fficiency-Lite with four LLMs, CHICO outperforms Codex, PEACE, Optimo, and POLO on all optimization metrics.

Critical strengths
  1. Clear and practical motivation. The observation that a hotspot may have little room for improvement, while its caller can avoid repeated calls to it, is intuitive and well illustrated by the motivating example.
  2. Relatively solid experimental setup. The evaluation covers four LLMs and four baselines on two repository-level benchmarks. Scoring is conservative: every task stays in the denominator, and empty or failed patches receive no credit. The Reference setting, which gives baselines the edit locations from expert patches, also helps calibrate how much localization matters.
Critical weaknesses
  1. The experiments do not support the core claim.
    1. The contribution of localization is not convincingly established. The paper argues that optimizing callers is more effective than optimizing hotspots alone, but it never compares hotspot-only and caller-only localization. The hotspot-only mode described in Section 2.2 is not evaluated. In the only ablation (Table 3), localization adds just 5 tasks on SWE-fficiency-Lite ≥Expert, and the larger GSO gain comes from a single model with no significance test.
    2. The comparison with Optimo and PEACE largely reflects CHICO's optimizer and strategy library. The "w/o localization" variant receives the same Profile-Top1 target as these baselines, yet already far exceeds them (with Terra, SWE ≥Expert 58 vs. 24/31). This suggests the margin comes mainly from CHICO's optimizer and its 433-strategy library mined from GitHub performance commits. The paper also does not specify how commits overlapping with the benchmarks were excluded from this library.

Limited novelty. The method largely adapts established software engineering techniques to an LLM coding agent: profiling to find hotspots, static analysis (CodeQL/Semgrep) to detect loop-invariant calls, and SemOpt as the optimizer. Each component is existing work, and diagnosing inefficient loops beyond what profilers report was already studied in [1], which is not cited. The contribution is mainly an engineering integration rather than a new research insight. [1] L. Song and S. Lu. Performance Diagnosis for Inefficient Loops. ICSE 2017.

Questions for authors

Can you compare hotspot-only, caller-only, and combined localization using the same optimizer and the same budget? In successful patches, how often were callers actually edited? This would directly affect my score. Were commits from the benchmark repositories, especially the expert patches, excluded from the strategy library? How does CHICO perform without the library?

Reviewer_4NTuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents CHICO, which guides repository-level code optimization by identifying both costly functions and opportunities to reduce repeated work in their callers. Its Python implementation combines profiling and static analysis with a SemOpt-based agent, retaining the best validated patch across up to five sessions. Across four models on GSO and SWE-fficiency-Lite, CHICO outperforms four baselines on the reported optimization metrics. The evaluation also compares against PEACE and Optimo with expert-derived target functions and includes an ablation that replaces CHICO's localization with the top profiled function.

Critical strengths
  1. The paper changes what localization looks for, from where profiled cost accumulates to where repeated work can be removed. Unlike PEACE and Optimo (target supplied) or POLO (cost propagated over the call graph), CHICO checks whether a caller invokes the hotspot inside a loop and whether the call's inputs depend on the loop variable (Section 2.2, Figure 1).

  2. CHICO beats the baselines even when they are told where to edit. In the reference setting, the target-based baselines start from the function the expert patch changed. CHICO receives no such location information, yet it still scores higher on optimization metric, illustrated in Table 2.

Critical weaknesses
  1. My main concern is that the ablation does not isolate CHICO's main idea. Section 1 argues that callers can be better edit sites than hotspots, but that adding every caller is ineffective because the list of functions grows too large. CHICO therefore keeps only callers that call a hotspot inside a loop, which the paper calls optimization opportunity detection. Table 3 compares CHICO with a single variant that uses the top profiled function instead. That swap changes the profiler, the number of candidates, the callers and the hints all at once. So Table 3 cannot tell how much of the gain comes from opportunity detection.

  2. The strongest baseline is a reimplementation that is hard to judge. POLO is the best baseline in Table 2, and all of the paper's smallest margins are against it. The original POLO targets C and C++, profiles with Callgrind, ranks functions with an iterative PageRank-style score, and uses a separate decision agent that picks among several samples. Section 3.1.2 only says the workflow was reimplemented for Python with py-spy, and no appendix explains how these parts were carried over. It is therefore unclear how faithful the POLO numbers are.

Questions for authors
  1. How many of the final patches came from a caller opportunity, how many from direct hotspot optimization, and how many from open exploration? Appendix D says the controller records the round and decision behind each validated patch, so this breakdown may already be available.

  2. How was POLO carried over to Python? In particular, does the reimplementation keep the iterative Func Rank score, the separate decision agent and the selection among several samples, and which model and budget did it run with?

  3. Do Codex, PEACE and Optimo get the same loop of up to five iterations with correctness and timing feedback that CHICO uses, or do they return a single patch? Section 3.1.2 describes a feedback loop only for POLO.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.