acceptodds
Under review as a conference paper at ICLR 2027

Small Agents with Semantic Search: Efficient Multilingual Code Localization

Abstract

Locating relevant files from natural-language requests is a core subtask for agents operating over code repositories. We investigate whether this task can be delegated to compact, specialized models to enable on-device search while reducing the token usage, latency, and inference cost of larger agents. We show that semantic search improves file localization, with gains in accuracy, cross-language transfer, and inference efficiency. To study this setting, we introduce a training framework for file-localization agents built around \colgrep, a local semantic search tool based on late-interaction retrieval models. Our recipe combines weighted supervised fine-tuning on teacher trajectories, assigning turn-level credit based on retrieval outcomes, with reinforcement learning on localization quality. We train three model families with fewer than two billion parameters to formulate search queries, inspect retrieved content, and identify relevant files. On localization tasks derived from SWE-bench Lite and Multi-SWE-bench Flash, \colgrep-equipped agents substantially improve over their base models and outperform corresponding \grep-based agents. In addition to improving localization accuracy, \colgrep reduces mean end-to-end trajectory latency by 44.1% on CPU while using 29.1% fewer tokens, and enables better generalization to programming languages unseen during fine-tuning. These results suggest that compact, tool-specialized localization agents can provide an efficient interface between natural-language requests and large codebases. We release all models, software, data, and training pipelines.

Primary area:
reinforcement learning
Keywords:
Agents, LLM, Search, Code, RL, Retrieval

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_G22aOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work explores training small <2B param models for agentic code localization via distillation with SFT and then RL. The RL recipe follows the setting and recipe closely of past methods like CodeScout [1]. But the main difference in this work is using a semantic search tool like colgrep and seeing how that compares with lexical tools like grep.

[1] Sutawika et al., CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents, arXiv 2026.

Critical strengths
  • The work tackles an important practical question about code localization which to my knowledge past works do not explore in depth (does using a semantic search tool help beyond just lexical search tools like grep?)
  • The results show that training with the colgrep tool helps performance over grep only approaches, helps with efficiency and generalization to other languages.
  • The constructed training data could be a valuable resource for the community.
Critical weaknesses
  • The paper lacks evaluation of downstream performance to understand whether the localization performed actually helps a SWE agent perform its task (e.g., issue resolution better)
  • Methodologically, the paper largely follows similar past papers like CodeScout [1], but with just the small tweak of using colgrep on top of just bash/grep.
  • The latency discussion seems to ignore initial indexing cost required for colgrep?
Questions for authors

Please see weaknesses above.

6 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer G22a

We thank the reviewer for their time and constructive feedback. Below, we address each of the raised concerns in turn.

W1: We refer the reviewer to our General Rebuttal, where we report additional end-to-end experiments on SWE-bench Lite showing improved issue-resolution success when delegating localization to our smaller agent.

W2: We acknowledge that our training recipe follows standard practice, similar to CodeScout’s setup: SFT distills a stronger teacher’s search behavior and teaches tool-use formatting, while RL improves localization through direct interaction. However, our study goes beyond adding colgrep to an existing agent. We separately train semantic and lexical policies using demonstrations generated with their respective tools, showing that the semantic advantage persists after both receive SFT and RL at this model size. Our agent also outperforms direct semantic retrieval even with oracle selection among the top-10 files (Table 2). These comparisons establish benefits from both semantic access and the learned search policy. We further show reduced dependence on multilingual policy training and lower CPU/GPU trajectory latency. Starting from the same Qwen3-1.7B base model, our agent substantially outperforms CodeScout’s reported 1.7B result on SWE-bench Lite in file F1 (67.29 versus 56.57), while matching its 4B result (67.03) and supporting multilingual localization.

W3: In section 3.1, we state that repository indexes are built once before training and evaluation, and Appendix A.5 explicitly excludes indexing from latency measurements. In practice, the index is reused across searches for the same repository snapshot, amortizing its cost over subsequent queries. Our results therefore measure search latency over prepared indexes.

We once again thank the reviewer for their constructive feedback.

AuthorsOpenReview ↗

Reminder for Reviewer G22a

Dear Reviewer,

We appreciate your time, engagement, and valuable review. As the discussion period concludes, we hope that our responses have addressed your concerns and clarified the points raised. Please let us know if there are any remaining questions or issues we can further address.

Best,

The Authors

Dear Reviewer,

Please read the authors’ rebuttal and join the discussion. Kindly clarify whether your main concerns have been addressed and raise any follow-up questions promptly so the authors have time to respond.

Thank you for your time and effort!

Reviewer_wezdOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper trains small agents for code-file localization using colgrep, an existing semantic search tool. The training recipe combines SFT weighted by retrieval outcomes with RL rewards for localization quality and shorter interactions. On localization tasks adapted from SWE-bench Lite and Multi-SWE-bench Flash, semantic-search agents achieve higher file F1 than trained grep agents while using fewer tokens and reducing latency. The paper also studies transfer across programming languages.

Critical strengths

The paper provides useful comparisons between search interfaces. Both grep and colgrep agents receive training with their respective tools, and semantic search improves file F1 on both benchmarks. The authors also measure CPU and GPU latency and separate model inference from search overhead. These experiments provide concrete evidence that semantic search can help small localization agents in the tested setting.

Critical weaknesses

1.The method is incremental, and its advantage over existing systems remains unclear. The main components use established retrieval and RL methods, with task-specific SFT weights and a turn penalty. Comparisons focus on variants within the authors’ framework, without directly evaluating stronger existing code-localization or agentic retrieval systems. The evaluation is also narrow: both benchmarks test file localization using one harness and retriever. Cross-language transfer is useful, but remains within the same task. Even within coding, the paper does not show whether better localization improves downstream repair quality or cost. This leaves the practical benefit of delegating search to a small agent only partially demonstrated.

2.The cost analysis does not fully establish an inference-efficiency advantage. Latency and token counts are useful, but depend on hardware, caching, and implementation choices. The paper does not compare end-to-end inference FLOPs or provide consistent compute accounting across methods, including model prefill and decoding, query encoding, and retrieval scoring. Measurements also assume prebuilt indexes loaded into memory, without reporting index size, memory requirements, or construction and update costs. The results support lower online latency under the tested configuration, but do not establish the overall inference-cost trade-off against competing methods at comparable localization quality.

Questions for authors
  1. Could you compare against stronger existing code-localization or agentic retrieval systems under comparable conditions? This would help establish the method’s competitiveness beyond the current harness.

  2. Could you report inference FLOPs or a consistent compute estimate across methods, alongside latency, memory use, and index overhead? Please distinguish model computation from retrieval costs and clarify when preprocessing costs can reasonably be amortized.

  3. Does using this localization agent improve the total cost or success rate of a coding agent that subsequently repairs the issue? Such a comparison would directly test the proposed division of work between small and large models.

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer wezd

We thank the reviewer for their time and constructive feedback. Below, we address each of the raised concerns in turn.

Critical Weaknesses

WI: While we acknowledge that our training recipe builds on established SFT and RL methods, our contribution lies primarily in the empirical understanding of how semantic search changes the capabilities and efficiency of compact localization policies. We separately train semantic and lexical agents using demonstrations generated with their respective tools, showing that the semantic advantage persists after both receive SFT and RL. We further demonstrate reduced dependence on multilingual supervision: across three model families, restricting policy SFT to Python reduces multilingual file F1 by an average of only 2.13 points, versus 14.90 for lexical search. The trained policy also outperforms direct semantic retrieval even with oracle selection from the top-10 files (67.34 versus 59.67 file F1 on SWE-bench Lite), establishing benefits beyond the retriever alone. To contextualize these findings beyond our own baselines, we provide comparisons with published localization results below (see Questions for Authors section). We agree that evaluation within one harness and retriever limits the generality of our conclusions. Regarding downstream utility, please refer to our General Rebuttal as it reports additional end-to-end repair experiments with Haiku 5.5 and GPT-5-mini.

W2: We report estimated model inference FLOPs alongside latency, memory use, and index overhead, distinguishing model computation from CPU retrieval and preprocessing costs.

Metricgrep harnesscolgrep harness
SWE-bench Lite file F163.75%67.29%
Multi-SWE-bench file F150.16%54.00%
GPU latency / sample2.15 s1.92 s
LLM inference time / sample1.89 s1.02 s
CPU search time / sample0.25 s0.87 s
Trajectory tokens / sample7,8575,574
Processed tokens / sample30,30012,929
Estimated model compute / sample104.27 TFLOPs44.49 TFLOPs
Peak GPU footprint20.58 GiB20.14 GiB
Peak tool RAM113 MiB318 MiB

Using a consistent token-based compute estimate, ColGREP reduces model computation from 104.27 to 44.49 TFLOPs per sample, while improving file F1 on both benchmarks. Although CPU retrieval takes longer, reduced LLM inference time brings overall latency from 2.15 to 1.92 seconds, with comparable GPU memory use.

MetricSWE-bench LiteMulti-SWE-bench
Mean GPU index build time69.32 s15.58 s
Mean fresh index size231.23 MiB44.23 MiB

Index construction takes 4–120 seconds, depending on repository size, and is reported separately from per-sample inference and retrieval. It is a one-time cost per indexed repository version, amortized with every subsequent search using that index. This amortization is appropriate for repeated searches over the same repository; the initial build remains part of the cost for a fresh repository.


Questions For Authors

Q1: On SWE-bench Lite, our agent built on top of Qwen3-1.7B achieves higher reported file F1 than CodeScout-1.7B and similar performance to CodeScout-4B, demonstrating competitiveness beyond our harness.

SystemBase ModelFile F1 ↑Precision ↑Recall ↑
LocAgentClaude-3.5-Sonnet31.3918.8394.16
OpenHands-BashQwen3-32B (Thinking)58.9854.2671.53
CodeScoutQwen3-1.7B56.5756.5756.57
CodeScoutQwen3-4B67.0366.6167.88
OpenHands-BashClaude-Sonnet-4.581.8780.1785.40
OursQwen3-1.7B67.3465.2971.60

Q2: We refer to our previous comment addressing W2.

Q3: We refer the reviewer to our General Rebuttal, where we report additional end-to-end repair experiments directly evaluating the impact of delegating localization to a smaller agent on repair success and total token usage.

We once again thank the reviewer for their constructive feedback.

AuthorsOpenReview ↗

Reminder for Reviewer wezd

Dear Reviewer,

Thank you for your engagement and thoughtful review. As the discussion period comes to a close, we hope that we have addressed your concerns. If there are any remaining questions or points that require further clarification, please let us know.

Best,

The Authors

Dear Reviewer,

Please read the authors’ rebuttal and join the discussion. Kindly clarify whether your main concerns have been addressed and raise any follow-up questions promptly so the authors have time to respond.

Thank you for your time and effort!

AuthorsOpenReview ↗

Reminder for Reviewer wezd

Dear Reviewer,

Since the Author/Reviewer discussion period concludes on Nov. 18, we would be happy to address any remaining concerns or questions.

Best, The Authors

Reviewer_QZZ9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies small agents for repository-level file localization. These agents receive an issue description, formulate queries for the pre-trained semantic-search tool colgrep, inspect results, and submit relevant files. Training combines teacher demonstrations, turn-weighted SFT, and RL on localization quality. The main contribution is an empirical comparison of semantic-search-equipped and lexical-search-only agents below 2B parameters. The experiments show improvements in file-level F1, transfer across programming languages, and localization trajectory efficiency.

Critical strengths

Useful and concrete research question. Understanding how much search capability can be delegated to a small local model is relevant to building efficient coding agents.

The lexical baseline receives task-specific training. The comparison goes beyond giving a trained semantic-search agent an advantage over an untrained baseline. File-F1 improvements hold across three model families after SFT and persist for Qwen3-1.7B after RL.

The cross-language result is informative. With training sample counts matched, Python-only SFT reduces multilingual benchmark F1 much less for colgrep agents than for grep agents (§5.5). This suggests that a multilingual retriever can reduce the need for multilingual policy fine-tuning.

Useful ablations and measured efficiency. The paper examines direct retrieval, single-turn versus multi-turn operation, SFT weighting, and RL rewards. It measures latency rather than relying only on token counts, and acknowledges important limitations.

Critical weaknesses

1 - The contribution is incremental and not sufficiently differentiated from prior work. Small trained localization agents, SFT followed by RL, and iterative semantic retrieval all have close precedents in CodeScout, ToolTrain, SweRank+, and Orion. The authors acknowledge these works, but the discussion does not clearly establish what substantial new capability or understanding this combination provides beyond them. The semantic-versus-lexical comparison and transfer findings are useful. However, much of the proposed system combines existing components, so stronger empirical insight is needed to establish a sufficient contribution.

2 - The practical value of the localization trade-off is not established. The motivation is to delegate search from a larger coding agent, but the experiments do not evaluate whether this preserves repair success. File F1 is a reasonable localization metric, but its relationship to downstream utility is not demonstrated. For example, on SWE-bench Lite the small agent exceeds deepseek's file F1 while having substantially lower recall, 72.67 versus 90.73 (Table 1). Missing a critical file could prevent a fix or require renewed searching. Conversely, additional irrelevant files could mislead an editor. The experiments do not establish which trade-off is preferable. The substantial class/function-F1 disadvantage relative to the trained grep agent adds to this uncertainty.

3 - The efficiency results do not establish the benefit of replacing the larger agent’s search. Larger models are compared on localization quality, whereas the headline token and latency savings compare small colgrep and small grep agents. These are useful but different comparisons. Together, they do not demonstrate lower total coding cost at comparable repair success. The latency measurements also exclude indexing and use prepared indexes on server hardware, limiting the evidence for consumer-device deployment.

4 - Some narrower findings need stronger robustness evidence. Five evaluation seeds are reported without uncertainty estimates or demonstrated replication across independent training runs. Some weighted-SFT gains are small. All subsequent RL runs start from weighted SFT, leaving its advantage over ordinary SFT followed by RL unclear for readers. Additionally, the RL and latency evidence concerns one model, despite the broader three-family SFT evaluation.

Questions for authors

Do you have evidence connecting the reported localization improvements to downstream repair success? In particular, can you characterize cases where higher precision comes with missed relevant files, and whether a downstream editor can recover without losing the efficiency benefit?

Can you report localization tokens and latency for the larger-model baselines in Table 1 under the same measurement protocol? This would help distinguish the benefit of semantic access from the benefit of using a small specialist.

How variable are the final interface gains and the smaller weighted-SFT gains? Please clarify evaluation variation, independent training replication, and whether ordinary-SFT then RL was compared with weighted-SFT then RL.

6 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer QZZ9 (Part 1)

We thank the reviewer for their time and constructive feedback. Below, we address each of the raised concerns in turn.

Critical Weaknesses

W1: Our work differs from the cited studies in both its setting and the empirical insights it provides. CodeScout and ToolTrain train localization agents with lexical search and structural lookup tools; SweRank+ trains the retriever and reranker rather than the search policy; and Orion studies semantic-search policies for document retrieval. Our paper and analysis demonstrate reduced dependence on multilingual supervision: across three model families, restricting policy SFT to Python reduces multilingual file F1 by an average of only 2.13 points, versus 14.90 for lexical search. The trained agent also outperforms direct semantic retrieval with oracle selection from the top-10 files (67.34 versus 59.67 F1 on SWE-bench Lite), demonstrating gains beyond the retriever alone. Compared with our separately trained 1.7B grep baseline, file F1 increases from 63.75 to 67.29. This suggests that, at this model size, policies benefit more from a strong semantic tool such as colgrep, which reduces their search burden by matching and ranking candidate code while the policy describes the desired behavior. With grep, the policy must infer repository-specific terms and iteratively refine lexical queries. Reduced LLM inference also outweighs retrieval overhead, lowering trajectory latency by 44.1% on CPU and 10.6% on GPU over precomputed indexes. These findings clarify what semantic access enables beyond the established training recipe. Furthermore, starting from the same Qwen3-1.7B base model, our agent substantially outperforms CodeScout’s reported 1.7B result on SWE-bench Lite in file F1 (67.29 versus 56.57), while matching its 4B result (67.03) and supporting multilingual localization.

W2: We acknowledge that our localization metrics alone do not establish whether delegation preserves repair success. As explained in section 3.3, we prioritize file F1 to balance coverage and precision when selecting files for downstream inspection. DeepSeek’s higher recall (90.73 versus 72.67) comes with lower precision (53.26 versus 64.77), so it does not automatically imply better repair context. Irrelevant context increases inspection cost and may impair model performance, consistent with the findings of [Hong et al. (2025), Context Rot: How Increasing Input Tokens Impacts LLM Performance]. This motivates our metric choice, while leaving the downstream trade-off to empirical evaluation. Relative to the trained grep agent, colgrep improves both file recall (70.60 to 72.67) and precision (60.64 to 64.77), so its file-level gains do not come from sacrificing coverage. While the disadvantage in class/function F1 is clearly discussed in the paper, our intended output is a relevant file set, which the larger editor can inspect to identify the exact symbols requiring modification. Lower symbol-level F1 therefore does not necessarily prevent successful repair, although it may require additional inspection.

W3: Regarding the benefits of delegating localization to a smaller agent, we refer the reviewer to our General Rebuttal, where we report additional end-to-end repair experiments evaluating both repair success and total token usage.

W4: We agree that the current experiments do not conclusively demonstrate that weighted SFT results in better post-RL performance than ordinary SFT. We opted for weighted SFT to preserve useful actions within imperfect demonstrations while reducing imitation of unproductive actions. This approach uses existing training annotations and leaves the fine-tuning compute budget unchanged, and we did find a small but consistent advantage in using it over ordinary SFT. We will revise the wording to describe these results as suggestive evidence for a practical training choice rather than established superiority. As pointed out, all reported RL experiments start from weighted SFT. Thus, the comparative benefit of this initialization after RL remains untested and could be investigated in future studies.

AuthorsOpenReview ↗

Rebuttal to Reviewer QZZ9 (Part 2)

Questions For Authors

Q1: As stated in the Introduction, our motivation is to reduce repository-search overhead by providing the coding agent with a focused initial context. Identifying relevant files early may also benefit repair by preserving context for reasoning and patch generation. Please see our General Rebuttal for additional end-to-end experiments evaluating downstream repair success and token savings.

Q2: We supplement the Qwen3-32B comparison in Table 1 with localization latency and trajectory token usage. With both models using ColGREP, these preliminary measurements show ~3× lower latency for our 1.7B specialist, alongside higher F1 on SWE-bench Lite and similar F1 on Multi-SWE-bench.

ModelAvg latencyAvg trajectory tokensSWE-bench Lite file F1Multi-SWE-bench file F1
Ours (Qwen3-1.7B)1.92 s5,57467.34%53.37%
Qwen3-32B6.53 s4,26463.50%54.02%

Q3: We selected weighted SFT for its modest but consistent file-F1 gains across all tested model families. We did not compare ordinary SFT + RL against weighted SFT + RL.

AuthorsOpenReview ↗

Reminder for Reviewer QZZ9

Dear Reviewer,

We appreciate your time, engagement, and valuable review. As the discussion period concludes, we hope that our responses have addressed your concerns and clarified the points raised. Please let us know if there are any remaining questions or issues we can further address.

Best,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Rebuttal

We would like to thank the reviewers for their comments, particularly for emphasizing the importance of evaluating the impact of delegating localization to a smaller agent. We agree that localization metrics alone cannot determine whether this division of labor improves issue resolution or reduces the resources required by the coding agent.

To address this, we conducted additional end-to-end repair experiments on SWE-bench Lite. We evaluated coding agents based on Anthropic’s Haiku-5.5 and OpenAI’s GPT5-mini. In the Agent setting, we used our Qwen3-1.7B student to generate a list of relevant files that we then provided to the main repair agent alongside the input text. In contrast, the baseline calls the repair agent directly via the API and allows it to conduct search independently using its harness.

Coding agentMetricBaselineAgentΔ\Delta
Haiku 5.5Resolved (%)63.7%66.7%+3.0 pp
Haiku 5.5Tokens (M, total)24.61M23.02M-6.5%
GPT5-miniResolved (%)48.7%50%+1.3 pp
GPT5-miniTokens (M, total)98.5M98.8M+0.30%

In this experiment, delegation yields a net increase of resolved issues. This provides direct evidence of downstream utility beyond file-localization F1. While the gains are modest and token savings are not always consistent, one plausible explanation is that its established public repositories are likely included in the training data of the coding agents. This allows the agents to rely partly on their prior knowledge of repository structure and identifiers, which we observed in the above experiment traces. This reduces the amount of exploration a specialist can replace, as end-to-end repair includes editing and testing, which can offset savings concentrated in localization.

To avoid this issue, we conducted complementary experiments evaluating delegation to a ColGrep based search agent built on MiniCPM5-2B following our recipe, orchestrated by Sonnet 5.5, on 52 code-search questions across 13 repositories created in 2026, thus targeting repository exploration while reducing the likelihood of prior exposure. Delegation reduced orchestrator API tokens by 34.3% (95% CI 28 to 40), while answer accuracy was 80.8%, versus 83.3% with grep (−2.6 percentage points; 95% CI −10.9 to +5.1). Further, we explored combining direct search and delegation and thus matched the observed 83.3% accuracy with 21.2% fewer API tokens. Together, these experiments demonstrate the practical value of delegating repository exploration to a small team of specialists: the SWE-bench Lite run reveals higher observed repair success and modest resource savings, while the targeted search benchmark shows significant reductions in orchestrator token usage. The extent of the benefit depends on the workload and integration strategy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.