acceptodds
Under review as a conference paper at ICLR 2027

CodeRetrX: Efficient and Scalable Function Retrieval with Hierarchical Code Search

Abstract

Function-level code search, the task of locating the functions in a large codebase that implement a given behavior (e.g., password authentication), is fundamental to program comprehension and underpins vulnerability discovery, large-scale migration, and data compliance. Emerging LLM-based tools (e.g., AI coding assistants) explore repositories iteratively, spending tokens at every step to decide what to read next, yet still miss most of the relevant functions. We present CodeRetrX, a framework for near-exhaustive, low-cost function retrieval at scale. It is built on hierarchical code search: fragment-level embeddings first locate query-relevant fragments inside every function, an LLM-guided gate then retains only the functions that contain such fragments, and full LLM evaluation is applied to those candidates alone. This design bounds token usage while retaining fragment-level semantics, enabling precise retrieval across large codebases without exhaustive traversal. On CodeRetrBench, our new benchmark of 18 multi-language repositories, CodeRetrX attains 88.16% recall, the highest among the compared methods, using only 26.17% of the tokens of exhaustive LLM traversal. Against embedding-then-LLM baselines, it raises recall from 61.43% to 88.16% (1.44x) at 78.83% of their cost. Its recall is more than 25x that of industrial coding assistants (Cursor, Claude Code) and more than 6x that of LocAgent, a state-of-the-art localization agent. Deployed as the retrieval stage of our vulnerability-discovery agent, CodeRetrX has contributed to 23 security advisories (21 CVEs, 20 of High or Critical severity) and two paid bug bounties in Web3 infrastructure, one rated High.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Repository-Level Code Search, Function Retrieval, Hierarchical Retrieval, Code Embeddings, Large Language Models, Program Analysis, Software Engineering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dL2SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper claims to find a code search method that mitigates the recall-cost dilemma. Given a NL query, the proposed method (CODERETRX) uses a parser to break a repository into fragments and uses embedding models to retrieve the potentially relevant fragments and their corresponding functions. These functions are then provided to an LLM, which determines if the code snippet is relevant to the provided natural language query. The experimental results showed that the proposed method achieved high recall with lower cost.

Critical strengths
  • Proposed a method that combines embedding and generative models for repository code search.

  • Compares the "cost" of different methods based on a practical unit (USD)

Critical weaknesses
  • Very small benchmark size. The repository contains only 18 repositories. After checking the anonymous repo provided at the top of the paper, the number of queries is also very limited. It is unclear whether the recall improvement is due to improved methodology or overfitting on the small benchmark.

  • The evaluated embedding and generative model(s) are limited and outdated. The main experiment is based on OpenAI's text-embedding-3-large (released Jan 2024) and gpt-oss-120b (released Aug 2025). I would recommend testing newer embedding/generative models to demonstrate the generalizability of CODERETRX (please see questions).

  • It is unclear how cost (in USD) in calculated in the experiments, making the "recall-cost dilemma" somewhat confusing. (See questions)

Questions for authors

Benchmarking

  • The authors proposed a task on repository-level retrieval, but there are a large number of available repo-level repositories (e.g., LocBench from LocAgent cited in the paper, and RepoBench [1]). Evaluation results on existing benchmarks can help prove the method's generalizability and make it easier to compare with other methodologies.

[1] https://proceedings.iclr.cc/paper_files/paper/2024/file/d191ba4c8923ed8fd8935b7c98658b5f-Paper-Conference.pdf

  • The current size of the benchmark (CODERETRBENCH) is very limited. Please consider extending it or including other benchmarks for evaluation.

Application Scenario

  • The use case of CODERETRX is confusing. The paper discusses vulnerability detection as an important downstream task. However, if the task requires high accuracy or is serious, why would cost be a big issue? Additionally, the authors claim, "Exhaustive LLM traversal is complete but unaffordable." What about using smaller/cheaper LLMs for traversal?

Experiment

  • The embedding model evaluated in the paper (text-embedding-3-large) is 2.5 years old. Based on my experience, there are newer (and potentially better) code retrieval choices on the market (e.g. Voyage-code-3 and newer Voyage-code-4, both are blackbox models; There are also a few competitive open source models from COIR leader board [2]). Similarly, there are also a lot of other choices for the generative model. There are tons of strong open-source/black-box releases since Aug 2025.

[2] https://huggingface.co/spaces/mteb/leaderboard

  • It is unclear how "cost" is calculated in the paper, especially when the generative model being utilized is an open-sourced one. Is it simply based on # of tokens being embeded, and # tokens input/output for the generative model? If so, did you consider a reasonable KV-cache technique?
Reviewer_m6F9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on high-recall function retrieval in large code repositories while reducing LLM inference costs. It first introduces CodeRetrBench, a new benchmark containing 18 repositories across six programming language groups and three retrieval task families. The paper then proposes CodeRetrX, a hierarchical code search framework that uses fragment-level embeddings and LLM-based filtering to identify candidate functions, followed by full-function verification. Experiments on CodeRetrBench show improved recall and lower costs compared with several retrieval baselines. The authors also demonstrate its application to real-world vulnerability discovery.

Critical strengths
  1. The problem of retrieving all relevant functions from large code repositories is practical and important.

  2. The proposed framework improves both recall and efficiency. The experiments cover multiple programming languages, repository sizes, and baseline methods.

Critical weaknesses
  1. The paper is hard to follow. Some key concepts and methodological choices lack clear explanations and motivations. For example, Section 3.2 relies on examples provided in Appendix B. These examples are important for understanding the motivation and should be included in the main paper. Similarly, several essential details are deferred to the appendix, reducing the self-containedness and readability of the paper.

  2. The ablation studies do not clearly demonstrate the individual contributions or necessity of the proposed components. In particular, it remains unclear whether the improvements mainly come from fine-grained code representations or the proposed hierarchical filtering strategy. More targeted ablation studies and comparisons with simpler fragment-level retrieval methods are needed to justify the design choices and establish the methodological contributions.

  3. The evaluation is limited, and the proposed benchmark is insufficiently justified. The experiments rely entirely on the authors' CodeRetrBench, without evaluation on established code retrieval benchmarks or downstream tasks to demonstrate effectiveness. The baseline comparisons also do not include several common retrieval techniques, such as query rewriting, intent identification, and hybrid retrieval. Moreover, the LLM-generated ground truth lacks independent validation. The real-world case studies also do not clearly demonstrate whether improved retrieval performance benefits downstream tasks compared with existing retrieval methods. These limitations raise concerns about the reliability and generalizability of the results.

Questions for authors
  1. How were the agent-based baselines, particularly Cursor and Claude Code, configured and prompted for the retrieval tasks? Since these agents are generally designed to locate a small number of relevant code locations rather than exhaustively retrieve all matching functions, could the authors clarify whether they were explicitly instructed to perform exhaustive retrieval and how their search budgets and stopping conditions were controlled?
Reviewer_WJJzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper tackles the problem of high-recall function-level code search. For this, the paper introduces a new benchmark CodeRetrBench, which across repositories of different sizes and query topics, has relevant judgements (that are athough obtained from an LLM). The proposed CodeRetrX is a coarse-to-fine pipeline which identifies the relevant line within function unit with a higher recall than existing embedding based approaches.

Critical strengths

The idea intuitively makes sense since using just one embedding per function that current approaches use leads to matching based on relevant lines that can have false similarity. The gains on the proposed CodeRetrBench seem consistent across different languages over prior approaches. The paper also gives a comprehensive cost comparison against the baseline embedding methods.

Critical weaknesses

The paper is missing the LLM Re-ranker baseline, especially the SweRankLLM which considerably improves recall over SweRankEmbed.

The paper should also benchmark the method on other localization benchmarks from SweRank, rather than only on the CodeRetrBench introduced in this paper. There are also other relevant multi-language benchmark / approaches like SweRank+ [1]

The whole coarse-to-fine cascaded ranking approach is well explored and prior approaches like Agentless [2] have tackled hierarchical localization (file -> function -> line)

Moreover, the paper doesn't have any empirical evidence of this increased recall improving a downstream task i.e. leveraging the proposed approach for an end-to-end task like bug fixing

[1] SweRank+: Multilingual, Multi-Turn Code Ranking for Software Issue Localization [2] Agentless: Demystifying LLM-based Software Engineering Agents

Questions for authors

Do the embedding baselines also use the LLM extracted search terms or do they use the query directly.

The reference set is gpt-oss-120b's verdict on each function, and CodeRetrX's final step uses the same model making the same full-body judgment. How much would the numbers change from using a different model for this?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.