acceptodds
Under review as a conference paper at ICLR 2027

CuraLoc: Repository-Level Fault Localization via Planned Code Inspection

Abstract

A bug report describes what went wrong, but rarely where the code should change. Existing repository-level localization methods often let language models iteratively decide what code to inspect next, requiring repeated model calls and substantial token usage. We investigate whether these inspection decisions can instead be planned before source retrieval. In this research, we introduce CuraLoc, which first selects candidate files and represents their symbols using compact behavioral and structural descriptions. In a single planning decision, the model jointly selects starting symbols, chooses code relationships to inspect from each, and forms hypotheses about likely edits. Deterministic retrieval collects the requested source for a final ranking informed by those hypotheses, concentrating model reasoning on choosing and comparing code rather than directing each retrieval step. We demonstrate the efficacy of our proposed approach on the SWE-Bench Lite and Verified datasets. On SWE Bench Verified with GPT-5.2, CuraLoc raises Func@1 from LocAgent's 53.8% to 59.3% while using 90.3% fewer recorded tokens, and also outperforms RepoSearcher (56.2%) and Agentless (39.6%) on Func@1. On SWE-bench Lite, CuraLoc achieves a high 83.8% Func@5 accuracy improving on Agentless by more than 15% in relative accuracy. Additionally, CuraLoc supports different model backbones, with strong localization accuracy demonstrated using both GPT-5.2 and GPT-6 Astra. These results provide evidence that repository-level code inspection can be planned effectively without iterative model-controlled navigation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Repository-level fault localization, Large language Models, Context curation, Evidence selection, code understanding, graph-guided retreival

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7wRmOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CuraLoc, a method for identifying edit locations for repository-level bug fixes. It consists of three stages: selecting candidate files; jointly planning starting symbols and retrieval actions from structural information and behavioral summaries; and deterministically retrieving source code for final ranking. Experiments on SWE-bench Lite and Verified demonstrate the effectiveness of the approach.

Critical strengths

The paper is clearly written. The proposed method is easy to understand, well motivated, and logically structured.

The reported improvements in localization accuracy provide empirical support for the effectiveness of the overall approach.

Critical weaknesses

CuraLoc commits to an initial retrieval plan without an explicit mechanism for revising it based on subsequently inspected source code. If the initial selection and planned retrieval fail to cover the correct edit locations, the final ranking stage may be unable to recover them. The failure analysis in lines 1031–1042, which shows that most failures involve incomplete coverage of reference edit symbols. This raises the possibility that some token savings come from restricting exploration at the cost of missing relevant locations.

In lines 95–102, the paper identifies issue-conditioned selection of code relationships as a key distinction between CuraLoc and RepoGraph. However, the existing ablations do not directly isolate the contribution of this design. Comparing neighborhood expansion with no expansion establishes the usefulness of additional code context, but does not demonstrate that having the model select relationship types is more effective than using a fixed expansion strategy.

Questions for authors

How could CuraLoc detect and recover from an initial plan that misses the correct edit locations? Could the authors compare the current fixed-plan approach with a variant allowing one feedback-driven replanning or additional retrieval step under the same overall token budget, reporting both localization accuracy and recovery of initially missed targets?

Could the authors isolate the contribution of issue-conditioned relationship selection by fixing the candidate pool, starting symbols, final ranking procedure, and retrieval token budget, while comparing model-selected actions against fixed-relation or relation-agnostic expansion strategies? How much of the improvement remains attributable specifically to the model’s choice of relationships?

3 comments

We thank the reviewer for the positive assessment and the precise questions. We ran both requested experiments on SWE-bench Lite (274 tasks, GPT-5.2) and report them below, with the corresponding changes in the revision. All arms share one fresh control run; its scores (Func@1 65.3, Func@10 87.2) fall within the paper's three-run Lite results (64.4 ± 1.6, 87.1 ± 0.2).

W1 / Q1. Recovering from an incomplete initial plan

We agree that CuraLoc does not revise its plan after reading code: we chose a fixed schedule to bound cost. The reviewer is correct that it can therefore miss locations the plan does not cover. Two existing results and the requested experiment bear on this.

  • Planned expansion already recovers locations missed by the starting symbols. In our component ablation (paper Table "Single-run SWE Bench Lite comparisons"), removing neighborhood expansion lowers Func@10 from 87.6 to 83.9: 11 tasks gained and 1 lost. Every gained task adds reference edit symbols that were absent from the starting selection.
  • The token savings do not come from lower coverage. On SWE-bench Verified, CuraLoc and LocAgent reach the same Func@10 (79.6 vs 79.1), yet CuraLoc uses 90% fewer tokens. CuraLoc's misses are concentrated in multi-file tasks, where iterative exploration covers more files.

Feedback-driven variant. We replaced the one-step final stage with a feedback loop: the model inspects candidate files one tier at a time and moves on when it judges the fix is not in the current tier. It uses up to 4 calls: one for each of three tiers of the 10 candidate files (files 1–3, 4–7 and 8–10) and one for the final ranking. It reuses the control's plan and starting symbols.

Final stageFunc@1Func@5Func@10Final-stage tokensCalls
Planned, one step (CuraLoc)65.385.087.221.7K1
Feedback-driven62.074.178.812.3K2.1

Final-stage tokens are the tokens used to read the retrieved code and rank it.

Feedback recovers few missed targets. On 29 tasks, the planned read set omits at least one gold symbol. On these, the feedback variant ranks all gold symbols in its top 10 on 3 tasks, against 2 for the control. Overall it does worse: it stops after declaring the fix found on 94% of tasks, and at Func@5 it loses 34 tasks while gaining 4. Planned reading already includes every gold symbol on 89% of tasks (245/274). In the revision, we will add this experiment to the ablation section and discuss the fixed-schedule trade-off.

W2 / Q2. Isolating issue-conditioned relation selection

We agree the original ablation compares expansion with no expansion, not model-selected relations with a fixed strategy.

Following the reviewer's design, every variant reuses the control's candidate pool, starting symbols and final ranking procedure, and reads the same number of symbols per task. Only the expansion rule changes:

Expansion ruleRead set has all gold (%)Func@1Func@5Func@10Final-stage tokens
Model-selected relations (CuraLoc)89.465.385.087.221.7K
All relations, fixed interleaving88.365.383.986.921.5K
Plan relevance, no graph86.965.383.685.820.4K
Starting symbols only (smaller budget)83.963.983.285.46.7K

At the same reading budget, model-selected relations match or exceed every fixed rule. Differences against the fixed all-relations and relevance-only rules are small and not significant on 274 tasks (coverage: 7 vs 4 and 11 vs 4 tasks), while removing expansion significantly lowers coverage (15 tasks lost, 0 gained; p < 0.001). Issue-conditioned selection thus reaches the coverage of the strongest fixed rule without hand-choosing the relation set. In the revision, we will add this comparison to the ablation section.

Thank you for the response.

Regarding W1 & Q1, the described variant still reads code in batches within a predefined scope and decides whether to continue. It does not reconsider or revise the retrieval plan based on the code inspected. Moreover, the reported token usage is 12.3K versus 21.7K, which does not establish a comparison under approximately the same overall token budget, as requested. The performance decline may therefore also be related to the difference in token usage.

Regarding W2 & Q2, the experimental results show that, compared with the strongest fixed strategy, Func@1 is identical, while Func@10 improves by only 0.3 percentage points, corresponding to a net gain of approximately one successful task out of 274. These results therefore do not establish that allowing the model to select relationship types is more effective than using a fixed expansion strategy.

For these reasons, I maintain my current score. However, my overall assessment lies between weak reject and weak accept, and I hope the AC will take this into account when making the decision.

Thank you for the careful follow-up. We ran the requested comparison on both benchmarks.

W1 / Q1. Replanning after reading code, at a comparable budget

The reviewer is right that our earlier variant did not revise the plan and used fewer tokens. The new variant replaces CuraLoc's final stage with two calls. The model first reads the source of the planned starting symbols, then rewrites the retrieval plan: for each starting symbol it sees the relation the plan chose and the neighbors available under every relation (up to 12 each), plus the symbol names of the top three files, and may keep, drop or switch relations or request any listed symbol, in priority order. The requested code is retrieved and ranked with the unchanged ranking template; the first call's assessment is added to the diagnosis. Stage 1, Stage 2 and the starting symbols are shared with CuraLoc.

Func@1Func@5Func@10Final-stage tokensSymbols read
SWE-bench Verified (454)
CuraLoc (planned)59.978.081.122.2K42.5
Read, replan, rank59.078.081.120.7K17.2
SWE-bench Lite (274)
CuraLoc (planned)65.385.087.221.7K43.0
Read, replan, rank63.983.986.520.5K16.9

GPT-5.2. Verified uses a fresh CuraLoc run; Lite uses the control from our first response. Over three Lite replanning runs, Func@1 is 64.1 ± 0.2 and Func@10 86.1 ± 0.4; every run is below CuraLoc on every metric, and within CuraLoc's three-run range in the paper (Func@1 64.4 ± 1.6).

  • CuraLoc is at parity with replanning, while reading about 2.5 times as many symbols. No difference is significant in either direction (p ≥ 0.34, exact McNemar): on Verified, Func@5 and Func@10 are identical; on Lite, CuraLoc is numerically higher on every metric. We find no measurable gain from this replanning step.
  • The model did replan. It retrieved code outside the original plan on 36% of tasks on both benchmarks.
  • Recovery of initially missed targets is offset by new misses. On the 79 Verified tasks whose planned read set misses a gold symbol, replanning reads all gold symbols on 4 tasks and has all of them in its top 10 on 7 (CuraLoc: 5); on 11 other tasks it drops gold symbols the plan had read. On Lite: 2 of 29 recovered, 7 dropped.
  • Budget. Total final-stage tokens are comparable (Verified: 10.08M for CuraLoc, 9.41M for replanning). We could not match the budget on every task: both replanning calls include the issue and the starting symbols' code, and on 275 of 454 Verified tasks this alone exceeds CuraLoc's final-stage budget (median 16.2K vs 13.3K). We therefore report both cases. On the 161 tasks where replanning stayed within CuraLoc's budget, accuracy is the same (Func@1 60.2 for both, Func@10 80.7 vs 80.1). On the 293 tasks where it used more (16.1K vs 11.6K), it was not more accurate (Func@1 58.4 vs 59.7, Func@10 81.2 vs 81.6).

Planning once is therefore sufficient for this design question: revising the plan after reading code did not improve accuracy, while CuraLoc's cost stays fixed in advance. This tests one replanning design. In the revision, we will replace the earlier feedback experiment with this one on both benchmarks.

W2 / Q2. Model-selected relations vs fixed expansion

We agree the earlier comparison did not establish an advantage. We extended it to each single relation, with the same candidate pool, starting symbols, ranking procedure and number of symbols read (Lite, 274 tasks):

Expansion ruleRead set has all gold (%)Func@1Func@5Func@10
Model-selected relations (CuraLoc)89.465.385.087.2
All relations, fixed interleaving88.365.383.986.9
Class siblings first88.764.683.286.9
Callers first87.265.083.686.5
Callees first86.965.083.685.8
Overrides first86.964.283.985.8
Relevance only, no graph86.965.383.685.8

Each single-relation rule reads its relation's neighbors first, then fills by Stage 2 relevance; the relation supplies 97% of added symbols for siblings but only 4–19% for the other three.

No fixed rule scores higher than CuraLoc on any measure; against the best fixed rule per column, CuraLoc's margins are 0.7 points in coverage and 0, 1.1 and 0.3 points at Func@1, @5 and @10, none significant. Planned expansion itself matters: reading only the starting symbols loses gold coverage on 15 tasks and gains none (p < 0.001). Which relation is followed matters less: these results do not establish an incremental benefit of issue-conditioned selection over fixed interleaving of all relations. In the revision, we will report this table and state the claim at this strength: model-selected relations are at parity with the strongest fixed rules.

Reviewer_bfKmOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a hierarchical pipeline for fault localization. The framework aims to leverage LLMs to first select files based on repository structure, then identify symbols and related code using behavioral and structural descriptions, and finally rank edit locations based on retrieved source code, rather than iteratively searching for fault locations.

The authors evaluate CuraLoc on SWE-bench Lite and Verified against LocAgent, RepoSearcher, and Agentless, using GPT-5.2 and GPT-6 Astra. On SWE-bench Verified with GPT-5.2, CuraLoc improves Func@1 from LocAgent's 53.8% to 59.3% while using 90.3% fewer recorded tokens. Compared with RepoSearcher, which achieves 56.2% Func@1, CuraLoc consumes 74.3% more tokens.

Critical strengths
  • The hierarchical pipeline enables fault localization without iterative code search, potentially reducing the overhead associated with repeated repository exploration.
Critical weaknesses
  • The core idea of this paper lacks novelty. CuraLoc appears to mainly incorporate additional information into the repository sketch.

  • The evaluation of CuraLoc does not convincingly support its claimed accuracy–token gains and trade-offs. Compared with RepoSearcher, CuraLoc achieves only a 3.1 percentage-point improvement in Func@1 while consuming 74.3% more tokens with GPT-5.2, and only a 0.4 percentage-point improvement while consuming 32% more tokens with GPT-6 Astra.

Questions for authors
  • What distinguishes CuraLoc from existing fault localization techniques?

  • What accounts for CuraLoc's effectiveness, given that its advantage over RepoSearcher becomes smaller with the more powerful GPT-6 Astra?

  • Could data contamination contribute to CuraLoc's performance, given that its advantage over RepoSearcher is smaller with GPT-6 Astra than with GPT-5.2?

  • Which model is used to generate the behavioral summaries? Are these summaries generated for every function, given that the paper mentions they can be cached?

5 comments

We thank the reviewer for their questions and constructive feedback. We answer each point below and note the corresponding changes in the revision.

W1 / Q1. Novelty: "CuraLoc mainly incorporates additional information into the repository sketch"

CuraLoc's novelty lies in how it uses the repository description, not in what the description contains. CuraLoc decides what to read before reading any source: one LLM step over compact code descriptions selects the symbols and relations to inspect, retrieval is then deterministic, and a final step ranks the retrieved code. The number of LLM stages is fixed at three (126K tokens per task). Agentic localizers instead read code, decide where to look next, and repeat; LocAgent averages about 1.3M tokens per task.

MethodCore ideaIterative navigation
AgentlessHierarchical localization: files → classes/functions → edit linesNo
LocAgentLLM agent navigates a repository graph with search toolsYes
RepoSearcherLLM agent explores the repository with lightweight search toolsYes
OrcaLocaLLM agent with priority-scheduled actions, action decomposition and context pruningYes
SHERLOCHypothesis-driven exploration: a reasoning LLM forms and tests fault hypotheses with compact repository tools, returning line ranges with diagnostic findingsYes
RepoGraphRetrieves related code through repository graph relations (a retrieval module for agents)No
Generic LLM planningSeparates planning from tool executionVaries
CuraLoc (ours)One planning step selects the symbols and graph relations to inspect; deterministic retrieval; final rankingNo

The ablations show that this planning step, not the added information, drives the gains (paper Table "Single-run SWE Bench Lite comparisons"):

  • Removing the code summaries lowers Func@1 only from 67.2 to 64.6.
  • Replacing planned selection with file-rank and source-order selection, using the same information, lowers Func@1 to 42.3. The selected evidence then contains all reference edit symbols on 54.0% of tasks instead of 90.5%.

Supporting results:

  • Same-backbone comparison: higher accuracy at lower cost. On SWE-bench Verified, CuraLoc improves Func@5 by 3–25% over all five baselines and uses 54–90% fewer tokens than the agentic methods (LocAgent, OrcaLoca, SHERLOC). We rerun six localizers with their default settings on GPT-5.2 and GPT-6 Astra, so only the method differs; prior methods report results on different and often older models, which mixes the effect of the method with the effect of the model.
  • Repair parity at much lower cost. With the same repair module and model, CuraLoc's locations match LocAgent's repair rate on SWE-bench Verified while localization uses 90% fewer tokens (W2).

In the revision, we will make explicit in the method section that the gain comes from planning what to read, supported by these ablations.

W2. Accuracy–token tradeoff against RepoSearcher

CuraLoc's additional tokens over RepoSearcher, the strongest low-cost baseline, buy a reliable gain with practical effect:

  • The gain is reliable. On Verified with GPT-5.2 (3 runs, paired bootstrap 95% CIs), CuraLoc exceeds RepoSearcher on every function metric:
CuraLoc − RepoSearcherFunc@1Func@5Func@10MRR
Difference+3.2 pts [+1.0, +5.4]+2.7 pts [+0.7, +4.8]+5.1 pts [+3.1, +7.3]+0.044 [+0.024, +0.065]
  • What the gain means in practice. CuraLoc places the fix first on 31 issues where RepoSearcher does not, against 15 the other way (majority of runs). This carries into repair: when a repair model receives only each localizer's top-ranked location, CuraLoc's locations resolve 4.2 points more Verified issues than those of the agentic localizer LocAgent (49.1% vs 44.9%, p = 0.018) with 88% fewer end-to-end tokens.
  • Tokens with GPT-5.2. CuraLoc uses 125.7K tokens per task in total, against RepoSearcher's 72.1K. Of CuraLoc's total, 43.1K are code summaries from the smaller GPT-5.4-mini; counting GPT-5.2 tokens only, the model RepoSearcher uses, CuraLoc uses 82.6K. Summaries are reusable across issues (Q4): with reuse within a run, summary tokens drop to 29.1K and the total to 110.2K per task on Verified, at the same accuracy (Func@1 59.9; one run).
  • Tokens with GPT-6. The review's 32% figure compares CuraLoc's total (138.9K per task) with RepoSearcher's (105.0K), counting the GPT-5.4-mini summary tokens the same as GPT-6 tokens. On GPT-6 tokens alone, CuraLoc uses 15% fewer than RepoSearcher (89.2K vs 105.0K). CuraLoc's cost is also bounded by design: from GPT-5.2 to GPT-6 it rose 10% in total (125.7K → 138.9K) and 8% on the GPT model alone (82.6K → 89.2K), while RepoSearcher's rose 46% (72.1K → 105.0K).

In the revision, we will add the paired confidence intervals, tokens by model with the measured effect of summary reuse, and the repair evaluation.

(continued in next comment)

(continued)

Q2. What accounts for CuraLoc's effectiveness, given its smaller advantage over RepoSearcher with GPT-6?

The ablations attribute the gains to planned selection (W1) and to a final ranking over the retrieved source, which raises Func@1 from 55.8 to 67.2 compared with ranking from the plan alone. Graph expansion adds locations the starting symbols miss (Func@10 83.9 → 87.6).

With GPT-6, all methods approach the ceiling. CuraLoc and RepoSearcher tie on Func@1 (304 vs 303 tasks) and Func@5, and CuraLoc matches or exceeds every baseline on every function metric while using fewer GPT-6 tokens (W2).

Q3. Could data contamination contribute to CuraLoc's performance?

We agree contamination is a fair question. Three observations suggest it does not explain the differences between methods:

  • All methods share the backbone, tasks and issue text, so memorized knowledge is equally available to each, and paired comparisons cancel it.
  • CuraLoc was developed only on SWE-bench Lite; on the 370 Verified tasks outside Lite, its advantage holds.
  • A smaller gap under GPT-6 is what higher baseline accuracy predicts, not a sign of contamination.

Q4. Which model generates the behavioral summaries? Is every function summarized?

As stated in the paper, GPT-5.4-mini generates the summaries in both the GPT-5.2 and GPT-6 runs.

Not every function is summarized, by design: describing the whole repository would add cost and irrelevant code. CuraLoc describes only:

  • all symbols in the top 3 candidate files;
  • symbols in files ranked 4–10 linked to them by calls, callers, inheritance or overrides;
  • all symbols in files ranked 4–10 that import, or are imported by, a top-3 file.

Each description comes from the cheapest source: the symbol's docstring (40% of described symbols on SWE-bench Lite), a cached summary of the same code (34%, with reuse within a run), or a new summary (26%). Summaries describe code, not the issue, and are keyed by a hash of the source. Our reported runs disabled reuse, and all summary tokens are counted in our totals. In the revision, we will also name the summary model in the method section.

Thank you for the detailed response and additional experimental results. However, my main concerns remain:

  1. Novelty. I understand the stated distinction: single-step planning followed by deterministic retrieval and final ranking, rather than iterative exploration. However, it remains unclear why this constitutes a substantive methodological advance over existing approaches. The ablations against file-rank and source-order selection support the effectiveness of the selection strategy, but do not resolve the concern about its technical novelty.
  2. Accuracy–efficiency trade-off. The confidence intervals support accuracy improvements over RepoSearcher with GPT-5.2. While the per-model token breakdown is useful, a model-aware cost comparison is still needed to assess the overall trade-off. Moreover, the downstream repair evaluation compares CuraLoc against LocAgent rather than RepoSearcher, leaving the practical benefit over the relevant low-cost baseline unestablished.
  3. Diminishing accuracy gains with stronger models. The ceiling explanation does not resolve the question of added value: under GPT-6, CuraLoc and RepoSearcher achieve nearly identical accuracy. Even if both approach the model’s effective ceiling, CuraLoc’s proposed mechanism still needs to demonstrate a meaningful efficiency advantage or another practical benefit over RepoSearcher in this setting.

Overall, the rebuttal strengthens the evidence for effectiveness but does not resolve my concerns about substantive novelty and the overall accuracy–efficiency advantage. I therefore maintain my original rating.

Reviewer_uGCZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CuraLoc, a repository-level fault localization approach that replaces iterative LLM-driven code exploration with a three-stage workflow involving candidate file selection, planned symbol-level inspection, and final ranking of potential edit locations. By leveraging compact behavioral and structural code descriptions, CuraLoc jointly determines which symbols and code relationships to inspect before retrieving source code through deterministic operations. Experiments on SWE-bench Lite and Verified demonstrate improved localization accuracy over baselines.

Critical strengths

The paper addresses an important problem in repository-level fault localization: reducing the overhead of iterative LLM-driven code exploration while maintaining localization accuracy.

Results are promising. CuraLoc achieves higher localization accuracy than the evaluated baselines on SWE-bench Verified, with substantial token savings under GPT-5.2.

The paper includes ablation studies, token consumption analysis, statistical significance testing, and evaluations with different model backbones.

Critical weaknesses

W1. More justification is needed to address the concern on limited methodological novelty. The proposed approach largely combines existing techniques, including hierarchical file/function localization (Agentless), graph-guided code retrieval (LocAgent and RepoGraph), and hypothesis-driven localization (RGFL and SHERLOC). The primary distinction appears to be planning retrieval actions upfront rather than iteratively. However, this resembles existing planning-based agent architectures, and the paper does not sufficiently demonstrate the unique benefits of this design over comparable iterative approaches under controlled settings.

W2. Evaluation soundness concern, especially localization metrics and ground truth. The evaluation relies on Func@k and File@k, which measure whether predicted locations match the reference developer patches. However, real-world bugs may be fixed by alternative multiple buggy locations, and the reference patch does not necessarily represent the only correct solution. Consequently, these metrics may penalize valid alternative locations and do not necessarily reflect actual fault localization effectiveness. Furthermore, Func@1 inherently cannot succeed on multi-symbol tasks, yet such tasks remain in the evaluation denominator.

W3. Missing downstream repair evaluation. Although the paper motivates CuraLoc as a technique for improving automated program repair, it only evaluates localization accuracy rather than actual issue-resolution rates. It remains unclear whether the reported improvements in Func@k translate into better end-to-end repair performance. This limitation is particularly concerning given the ambiguity of reference edit locations. Existing localization methods, such as LocAgent, RGFL, and SHERLOC, have already investigated their impact on downstream repair.

W4. Insufficient evaluation of multi-location bugs. Although the method can return multiple edit locations, the evaluation provides limited analysis of its effectiveness on bugs requiring changes across multiple functions or files. The reported ablation involves only 35 multi-symbol tasks, and the method's one-hop structural expansion may struggle with bugs involving distant or disconnected locations. More comprehensive evaluation across different numbers and distributions of edit locations would strengthen the paper.

W5. Incomplete baseline comparisons. The paper compares against Agentless, LocAgent, and RepoSearcher but omits several recent and closely related approaches, including OrcaLoca, RGFL, and SHERLOC. Comparisons with these methods are important for establishing both the novelty and empirical advantages of CuraLoc.

W6. Limited generalizability and inconsistent efficiency gains. The evaluation is restricted to Python tasks from SWE-bench Lite and Verified. Furthermore, the token-efficiency advantage over LocAgent is not consistent across model backbones: under GPT-6 Astra, CuraLoc consumes more total tokens than LocAgent when summary generation is included, while providing only a marginal Func@1 improvement over RepoSearcher.

Questions for authors

What are the fundamental methodological differences between CuraLoc and existing hierarchical, graph-guided, and hypothesis-driven localization approaches?

Given that a bug may have multiple valid repair locations, how do the authors justify using reference-patch-based Func@k and File@k as indicators of localization correctness? Could alternative valid edit locations affect the conclusions? Can the authors integrate CuraLoc with an existing repair agent and evaluate whether its localization improvements translate into higher issue-resolution rates or reduced end-to-end costs?

How does CuraLoc perform on issues requiring modifications across multiple functions or files, particularly when the edit locations are structurally distant or disconnected?

Why does CuraLoc's token-efficiency advantage over LocAgent disappear under GPT-6 Astra when all tokens are counted? How generalizable are the reported benefits to other model families, repositories, and programming languages?

4 comments

We thank the reviewer for their questions and constructive feedback. We answer each point below and note the corresponding changes in the revision.

W1 / Q1. Methodological novelty

CuraLoc decides what to inspect before reading any code: it plans which evidence to gather, retrieves it, then reasons over it. Inspection is issue-specific without being iterative, and the number of LLM stages is fixed at three regardless of exploration depth. Agentic methods instead interleave search and reasoning, so each step depends on the code read in the previous one.

MethodCore ideaIterative navigation
AgentlessHierarchical localization: files → classes/functions → edit linesNo
LocAgentLLM agent navigates a repository graph with search toolsYes
RepoSearcherLLM agent explores the repository with lightweight search toolsYes
OrcaLocaLLM agent with priority-scheduled actions, action decomposition and context pruningYes
SHERLOCHypothesis-driven exploration: a reasoning LLM forms and tests fault hypotheses with compact repository tools, returning line ranges with diagnostic findingsYes
RepoGraphRetrieves related code through repository graph relations (a retrieval module for agents)No
Generic LLM planningSeparates planning from tool executionVaries
CuraLoc (ours)One planning step selects the symbols and graph relations to inspect; deterministic retrieval; final rankingNo

Our main contributions:

  • Planned code inspection. CuraLoc plans code inspection before retrieving any source: one LLM step over compact code descriptions selects the symbols and relations to inspect, and retrieval is deterministic, so localization stays cheap (126K tokens per task). Agentic localizers instead read code, decide where to look next, and repeat, which costs LocAgent about 1.3M tokens per task.
  • Same-backbone comparison: higher accuracy at lower cost. On SWE-bench Verified, CuraLoc improves Func@5 by 3–25% over all five baselines and uses 54–90% fewer tokens than the agentic methods (LocAgent, OrcaLoca, SHERLOC). We rerun six localizers with their default settings on GPT-5.2 and GPT-6 Astra, so only the method differs; prior methods report results on different and often older models, which mixes the effect of the method with the effect of the model.
  • Repair parity at much lower cost. With the same repair module and model, CuraLoc's locations match LocAgent's repair rate on SWE-bench Verified while localization uses 90% fewer tokens, and resolve more issues when repair uses only the top-ranked location (W3).
  • Failure analysis. A cross-method breakdown shows where planned inspection helps and where it does not.

Results on SWE-bench Verified (454 tasks, GPT-5.2, same scorer for all methods; mean ± sample standard deviation across runs):

MethodRunsFunc@1Func@3Func@5Func@10MRRFile@1Tokens/task
CuraLoc359.3 ± 0.972.9 ± 0.477.2 ± 0.679.6 ± 1.10.840 ± 0.00675.8 ± 0.3125.7K ± 0.8K
OrcaLoca1 (runs 2–3 pending)59.371.171.671.60.83076.7365.5K
RepoSearcher356.2 ± 0.469.5 ± 0.674.4 ± 1.074.4 ± 1.00.796 ± 0.00475.8 ± 0.872.1K ± 2.9K
LocAgent353.8 ± 0.569.7 ± 0.874.8 ± 0.379.1 ± 0.30.791 ± 0.00373.7 ± 0.61,292.7K ± 4.4K
SHERLOC349.6 ± 1.766.6 ± 0.670.1 ± 0.775.2 ± 0.80.737 ± 0.01376.9 ± 1.8273.4K ± 13.3K
Agentless339.6 ± 1.057.0 ± 1.761.6 ± 1.963.1 ± 1.60.613 ± 0.02270.0 ± 1.264.6K ± 0.2K

OrcaLoca's default configuration returns at most 3 locations, which caps its Func@5 and Func@10. OrcaLoca runs 2–3 are pending.

In the revision, we will add the method comparison table to the method section.

W2 / Q2a. Reference-patch metrics and alternative valid locations

We agree that a reference patch is not the only valid fix. Func@k and File@k are the standard measures in this line of work (Agentless, LocAgent, OrcaLoca, SHERLOC). The same rule applies to every method: an alternative valid location can cost any method credit, so the comparison stays consistent. Downstream repair (W3), judged by the repository's tests rather than by reference locations, shows CuraLoc matching LocAgent at a fraction of the localization cost.

On Func@1 and multi-symbol tasks: localizing multi-symbol and multi-file edits is outside the scope of this paper (see W4). Func@1 is 0 for every method on multi-symbol tasks, so including them lowers all scores equally and does not change the ranking. On the 335 single-symbol tasks, Func@1 is: CuraLoc 80.4, OrcaLoca 80.3, RepoSearcher 76.1, LocAgent 72.9, SHERLOC 67.2, Agentless 53.7. In the revision, we will report single-symbol Func@1 separately and list multi-location localization as future work.

(continued in next comment)

(continued)

W3 / Q2b. Downstream repair

We agree that repair is the end goal. We pass each localizer's ranked functions, in rank order, to the same Agentless-style repair module (GPT-5.2) and score patches with the official SWE-bench harness on all 454 SWE-bench Verified tasks; only the locations differ. The module receives either each localizer's top 5 or only its top-ranked function, and generates 42 candidate patches per task. Using only the top-ranked function shows whether ranking quality carries through to repair, and cuts repair tokens by 40% (see our general comment on downstream repair). Majority vote submits the patch proposed most often; Pass@1 submits the first, greedy patch; Pass@5 counts a task as resolved if any of the first five patches resolves it. A patch resolves a task if the repository's tests pass.

Repair inputLocalizerMajority votePass@1Pass@5Tokens/task (localization + repair)
Top 5CuraLoc53.1%48.0%59.9%181K
Top 5LocAgent53.1%50.0%59.5%1,349K
Top 1CuraLoc49.1%46.9%–159K
Top 1LocAgent44.9%42.7%–1,330K

With five locations, CuraLoc matches LocAgent (paired McNemar: p = 1.00, 0.36 and 0.90) with 87% fewer end-to-end tokens. With only the top-ranked location, CuraLoc's locations resolve 4.2 points more issues (p = 0.018 for majority vote, 0.040 for Pass@1) with 88% fewer tokens, because CuraLoc ranks the fix first more often. In the revision, we will add this repair evaluation, with both settings, to the results section.

W4 / Q3. Multi-location bugs

CuraLoc targets errors whose fix lies in a single function, the same setting as the methods we compare with (Agentless, LocAgent, OrcaLoca, SHERLOC). Localizing errors that span several functions or files is out of scope here and remains an open, challenging problem. Our contribution is making localization accurate and cheap by planning code inspection rather than navigating iteratively. In the revision, we will state this scope explicitly and list multi-location localization as future work.

W5. Missing baselines

We added OrcaLoca and SHERLOC, rerun on GPT-5.2 with their default settings (W1 table). CuraLoc ties OrcaLoca on Func@1, leads it on Func@3–10 and MRR, and uses 66% fewer tokens. CuraLoc leads SHERLOC on all function-level metrics with 54% fewer tokens. In the revision, the main results table includes both, with mean ± standard deviation across runs.

W6 / Q4. GPT-6 token efficiency and generalizability

Tokens. With GPT-6 (paper Table "Single-run GPT-6 Astra"), CuraLoc's 138.9K tokens per task consist of 89.2K GPT-6 tokens and 49.7K for code summaries generated by the much smaller GPT-5.4-mini.

  • Counting GPT-6 tokens only, CuraLoc uses fewer than LocAgent (89.2K vs 95.3K) and RepoSearcher (105.0K), which use GPT-6 alone, while matching or exceeding their accuracy.
  • The summary tokens make the total higher, but they come from a cheaper model.
  • Summaries describe code, not the issue, so they can be cached and reused across issues in the same repository. Our reported runs disabled this reuse and regenerated summaries for every task. With reuse within a run, 57% of model-written summaries come from cache (SWE-bench Lite). On SWE-bench Verified, reuse within a run lowers summary tokens to 29.1K and the total to 110.2K per task, at the same accuracy (Func@1 59.9; one run).
  • CuraLoc's cost is also predictable: bounded by design at 126K tokens per task with GPT-5.2 and 139K with GPT-6 (+10%). Over the same change, RepoSearcher's cost rose 46% (72.1K → 105.0K), and LocAgent's varied about 14× (1,293K → 95.3K), since its agent loop has no fixed length.

Accuracy under GPT-6. CuraLoc's absolute lead shrinks with GPT-6 because GPT-6 makes fewer mistakes overall, leaving less room for any method to improve. A fairer measure is how many of LocAgent's mistakes CuraLoc fixes:

  • With GPT-5.2, CuraLoc fixes about one in eight of LocAgent's top-1 errors (12%), lowering the error rate from 46.2% to 40.7%.
  • With GPT-6, CuraLoc again fixes about one in ten (10%), lowering it from 36.8% to 33.0%.

CuraLoc's advantage over LocAgent is therefore largely stable across models.

Other languages and model families. Our evaluation covers Python (SWE-bench). CuraLoc needs only a symbol graph and code descriptions, which can be built for other languages, but we have not evaluated them. Evaluating additional model families is likewise future work. In the revision, we will report GPT-6 tokens by model with the measured effect of summary reuse, and state both limitations in the Limitations section.

Thanks for the new experiments and clarification.

But my concern on the novelty can still remain, especially, in summary table in W1 / Q1. Methodological novelty, it actually makes me a bit confused why such a workflow (symbol inspect+ retrieval +ranking) can achieve improvements given there is no big technical contribution compared to other techniques. The improvements over some baselines are not that significant, which may enhance the concern.

Thank you for the follow-up. We see CuraLoc's improvement as coming from what the model is shown at each step. Our finding is that, with issue-specific evidence curated for each call, three primary-model calls can do the work that agentic localizers spread across open-ended search. The ablations trace the gain to that evidence. Replacing issue-based selection with source order lowers SWE-bench Lite Func@1 from 67.2 to 42.3, and ranking without reading the selected code gives 55.8. Revising the plan after reading code adds nothing at comparable token cost (SWE-bench Verified Func@1 59.9 vs 59.0). With this evidence, CuraLoc's three calls exceed LocAgent's iterative search (Func@1 59.3 vs 53.8; +5.5 points, paired bootstrap 95% CI [2.8, 8.3]) with 90% fewer tokens. Against RepoSearcher's tool-based search, CuraLoc is more accurate with GPT-5.2 (Func@1 59.3 vs 56.2; +3.2 points, 95% CI [1.0, 5.4]) and matches it with GPT-6 at 5% lower dollar cost. Against OrcaLoca, CuraLoc ties at Func@1 (59.3) and leads at Func@5 (77.2 vs 71.6) and Func@10 (79.6 vs 71.6) with about one-third of the tokens. These ranking gains carry over to repair. When repair receives only each localizer's top-ranked function, CuraLoc's locations resolve 49.1% of Verified issues (majority vote over 42 patches), against 44.9% for LocAgent (p = 0.018) and 47.8% for RepoSearcher. Moving from five locations to one costs CuraLoc 4.0 points against 8.2 for LocAgent, so CuraLoc's single top location is often enough, and sending one location cuts repair tokens by 40%. CuraLoc thus matches or exceeds iterative search in both localization and repair, with three primary-model decisions and bounded cost.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment: Final Downstream Repair Results

CuraLoc's ranking advantage and cost savings carry through to repair. When repair relies on each localizer's top-ranked location, CuraLoc's locations resolve 4.2 points more SWE-bench Verified issues than LocAgent's (p = 0.018) with 88% fewer end-to-end tokens. When repair sees each localizer's top 5, both resolve the same share of issues, and CuraLoc still uses 87% fewer tokens.

Repair inputLocalizerMajority votePass@1Tokens/task (localization + repair)
Top 1CuraLoc49.1%46.9%159K
Top 1LocAgent44.9%42.7%1,330K
Top 5CuraLoc53.1%48.0%181K
Top 5LocAgent53.1%50.0%1,349K

All 454 SWE-bench Verified tasks, GPT-5.2. The same Agentless-style repair module receives each localizer's top-ranked function (top 1) or its top-5 ranked functions in rank order (top 5), and generates 42 candidate patches per task. Majority vote submits the most frequent patch; Pass@1 submits the first, greedy patch. Patches are scored with the official SWE-bench harness.

  1. CuraLoc's top choice is often enough to fix the issue. Giving repair a single location is the cheaper setting, and it works best when the first choice is accurate. In this setting, CuraLoc's locations resolve 39 issues that LocAgent's do not, against 20 the other way (majority vote, p = 0.018), and 48 vs 29 at Pass@1 (p = 0.040). With five locations, both resolve the same share (53.1%).
  2. CuraLoc keeps most of its repair rate with a single location. Moving from five locations to one lowers CuraLoc's majority-vote rate by 4.0 points, against 8.2 for LocAgent.
  3. The savings compound. One location cuts repair tokens by 40% (54.8K → 32.9K per task). With CuraLoc's lower localization cost (126.6K vs 1,297.9K per task), CuraLoc with top-1 repair is the cheapest of these pipelines, at 159K tokens per task.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.