- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CuraLoc, a method for identifying edit locations for repository-level bug fixes. It consists of three stages: selecting candidate files; jointly planning starting symbols and retrieval actions from structural information and behavioral summaries; and deterministically retrieving source code for final ranking. Experiments on SWE-bench Lite and Verified demonstrate the effectiveness of the approach.
Critical strengths
The paper is clearly written. The proposed method is easy to understand, well motivated, and logically structured.
The reported improvements in localization accuracy provide empirical support for the effectiveness of the overall approach.
Critical weaknesses
CuraLoc commits to an initial retrieval plan without an explicit mechanism for revising it based on subsequently inspected source code. If the initial selection and planned retrieval fail to cover the correct edit locations, the final ranking stage may be unable to recover them. The failure analysis in lines 1031–1042, which shows that most failures involve incomplete coverage of reference edit symbols. This raises the possibility that some token savings come from restricting exploration at the cost of missing relevant locations.
In lines 95–102, the paper identifies issue-conditioned selection of code relationships as a key distinction between CuraLoc and RepoGraph. However, the existing ablations do not directly isolate the contribution of this design. Comparing neighborhood expansion with no expansion establishes the usefulness of additional code context, but does not demonstrate that having the model select relationship types is more effective than using a fixed expansion strategy.
Questions for authors
How could CuraLoc detect and recover from an initial plan that misses the correct edit locations? Could the authors compare the current fixed-plan approach with a variant allowing one feedback-driven replanning or additional retrieval step under the same overall token budget, reporting both localization accuracy and recovery of initially missed targets?
Could the authors isolate the contribution of issue-conditioned relationship selection by fixing the candidate pool, starting symbols, final ranking procedure, and retrieval token budget, while comparing model-selected actions against fixed-relation or relation-agnostic expansion strategies? How much of the improvement remains attributable specifically to the model’s choice of relationships?
3 comments
Thank you for the response.
Regarding W1 & Q1, the described variant still reads code in batches within a predefined scope and decides whether to continue. It does not reconsider or revise the retrieval plan based on the code inspected. Moreover, the reported token usage is 12.3K versus 21.7K, which does not establish a comparison under approximately the same overall token budget, as requested. The performance decline may therefore also be related to the difference in token usage.
Regarding W2 & Q2, the experimental results show that, compared with the strongest fixed strategy, Func@1 is identical, while Func@10 improves by only 0.3 percentage points, corresponding to a net gain of approximately one successful task out of 274. These results therefore do not establish that allowing the model to select relationship types is more effective than using a fixed expansion strategy.
For these reasons, I maintain my current score. However, my overall assessment lies between weak reject and weak accept, and I hope the AC will take this into account when making the decision.
Thank you for the careful follow-up. We ran the requested comparison on both benchmarks.
W1 / Q1. Replanning after reading code, at a comparable budget
The reviewer is right that our earlier variant did not revise the plan and used fewer tokens. The new variant replaces CuraLoc's final stage with two calls. The model first reads the source of the planned starting symbols, then rewrites the retrieval plan: for each starting symbol it sees the relation the plan chose and the neighbors available under every relation (up to 12 each), plus the symbol names of the top three files, and may keep, drop or switch relations or request any listed symbol, in priority order. The requested code is retrieved and ranked with the unchanged ranking template; the first call's assessment is added to the diagnosis. Stage 1, Stage 2 and the starting symbols are shared with CuraLoc.
| Func@1 | Func@5 | Func@10 | Final-stage tokens | Symbols read | |
|---|---|---|---|---|---|
| SWE-bench Verified (454) | |||||
| CuraLoc (planned) | 59.9 | 78.0 | 81.1 | 22.2K | 42.5 |
| Read, replan, rank | 59.0 | 78.0 | 81.1 | 20.7K | 17.2 |
| SWE-bench Lite (274) | |||||
| CuraLoc (planned) | 65.3 | 85.0 | 87.2 | 21.7K | 43.0 |
| Read, replan, rank | 63.9 | 83.9 | 86.5 | 20.5K | 16.9 |
GPT-5.2. Verified uses a fresh CuraLoc run; Lite uses the control from our first response. Over three Lite replanning runs, Func@1 is 64.1 ± 0.2 and Func@10 86.1 ± 0.4; every run is below CuraLoc on every metric, and within CuraLoc's three-run range in the paper (Func@1 64.4 ± 1.6).
- CuraLoc is at parity with replanning, while reading about 2.5 times as many symbols. No difference is significant in either direction (p ≥ 0.34, exact McNemar): on Verified, Func@5 and Func@10 are identical; on Lite, CuraLoc is numerically higher on every metric. We find no measurable gain from this replanning step.
- The model did replan. It retrieved code outside the original plan on 36% of tasks on both benchmarks.
- Recovery of initially missed targets is offset by new misses. On the 79 Verified tasks whose planned read set misses a gold symbol, replanning reads all gold symbols on 4 tasks and has all of them in its top 10 on 7 (CuraLoc: 5); on 11 other tasks it drops gold symbols the plan had read. On Lite: 2 of 29 recovered, 7 dropped.
- Budget. Total final-stage tokens are comparable (Verified: 10.08M for CuraLoc, 9.41M for replanning). We could not match the budget on every task: both replanning calls include the issue and the starting symbols' code, and on 275 of 454 Verified tasks this alone exceeds CuraLoc's final-stage budget (median 16.2K vs 13.3K). We therefore report both cases. On the 161 tasks where replanning stayed within CuraLoc's budget, accuracy is the same (Func@1 60.2 for both, Func@10 80.7 vs 80.1). On the 293 tasks where it used more (16.1K vs 11.6K), it was not more accurate (Func@1 58.4 vs 59.7, Func@10 81.2 vs 81.6).
Planning once is therefore sufficient for this design question: revising the plan after reading code did not improve accuracy, while CuraLoc's cost stays fixed in advance. This tests one replanning design. In the revision, we will replace the earlier feedback experiment with this one on both benchmarks.
W2 / Q2. Model-selected relations vs fixed expansion
We agree the earlier comparison did not establish an advantage. We extended it to each single relation, with the same candidate pool, starting symbols, ranking procedure and number of symbols read (Lite, 274 tasks):
| Expansion rule | Read set has all gold (%) | Func@1 | Func@5 | Func@10 |
|---|---|---|---|---|
| Model-selected relations (CuraLoc) | 89.4 | 65.3 | 85.0 | 87.2 |
| All relations, fixed interleaving | 88.3 | 65.3 | 83.9 | 86.9 |
| Class siblings first | 88.7 | 64.6 | 83.2 | 86.9 |
| Callers first | 87.2 | 65.0 | 83.6 | 86.5 |
| Callees first | 86.9 | 65.0 | 83.6 | 85.8 |
| Overrides first | 86.9 | 64.2 | 83.9 | 85.8 |
| Relevance only, no graph | 86.9 | 65.3 | 83.6 | 85.8 |
Each single-relation rule reads its relation's neighbors first, then fills by Stage 2 relevance; the relation supplies 97% of added symbols for siblings but only 4–19% for the other three.
No fixed rule scores higher than CuraLoc on any measure; against the best fixed rule per column, CuraLoc's margins are 0.7 points in coverage and 0, 1.1 and 0.3 points at Func@1, @5 and @10, none significant. Planned expansion itself matters: reading only the starting symbols loses gold coverage on 15 tasks and gains none (p < 0.001). Which relation is followed matters less: these results do not establish an incremental benefit of issue-conditioned selection over fixed interleaving of all relations. In the revision, we will report this table and state the claim at this strength: model-selected relations are at parity with the strongest fixed rules.
We thank the reviewer for the positive assessment and the precise questions. We ran both requested experiments on SWE-bench Lite (274 tasks, GPT-5.2) and report them below, with the corresponding changes in the revision. All arms share one fresh control run; its scores (Func@1 65.3, Func@10 87.2) fall within the paper's three-run Lite results (64.4 ± 1.6, 87.1 ± 0.2).
W1 / Q1. Recovering from an incomplete initial plan
We agree that CuraLoc does not revise its plan after reading code: we chose a fixed schedule to bound cost. The reviewer is correct that it can therefore miss locations the plan does not cover. Two existing results and the requested experiment bear on this.
Feedback-driven variant. We replaced the one-step final stage with a feedback loop: the model inspects candidate files one tier at a time and moves on when it judges the fix is not in the current tier. It uses up to 4 calls: one for each of three tiers of the 10 candidate files (files 1–3, 4–7 and 8–10) and one for the final ranking. It reuses the control's plan and starting symbols.
Final-stage tokens are the tokens used to read the retrieved code and rank it.
Feedback recovers few missed targets. On 29 tasks, the planned read set omits at least one gold symbol. On these, the feedback variant ranks all gold symbols in its top 10 on 3 tasks, against 2 for the control. Overall it does worse: it stops after declaring the fix found on 94% of tasks, and at Func@5 it loses 34 tasks while gaining 4. Planned reading already includes every gold symbol on 89% of tasks (245/274). In the revision, we will add this experiment to the ablation section and discuss the fixed-schedule trade-off.
W2 / Q2. Isolating issue-conditioned relation selection
We agree the original ablation compares expansion with no expansion, not model-selected relations with a fixed strategy.
Following the reviewer's design, every variant reuses the control's candidate pool, starting symbols and final ranking procedure, and reads the same number of symbols per task. Only the expansion rule changes:
At the same reading budget, model-selected relations match or exceed every fixed rule. Differences against the fixed all-relations and relevance-only rules are small and not significant on 274 tasks (coverage: 7 vs 4 and 11 vs 4 tasks), while removing expansion significantly lowers coverage (15 tasks lost, 0 gained; p < 0.001). Issue-conditioned selection thus reaches the coverage of the strongest fixed rule without hand-choosing the relation set. In the revision, we will add this comparison to the ablation section.