acceptodds
Under review as a conference paper at ICLR 2027

SweRank+: Multilingual, Multi-Turn Code Ranking for Software Issue Localization

Abstract

Maintaining large-scale, multilingual codebases hinges on accurately localizing issues, which requires mapping natural-language error descriptions to the relevant functions that need to be modified. However, existing ranking approaches are often Python-centric and perform a single-pass search over the codebase. This work introduces SweRank+, a framework that couples SweRankMulti, a cross-lingual code ranking tool, with SweRankAgent, an agentic search setup, for iterative, multi-turn reasoning over the code repository. SweRankMulti comprises a code embedding retriever and a listwise LLM reranker, and is trained using a carefully curated large-scale issue localization dataset spanning multiple popular programming languages. SweRankAgent adopts an agentic search loop that moves beyond single-shot localization with a memory buffer to reason and accumulate relevant localization candidates over multiple turns. Our experiments on issue localization benchmarks spanning various languages demonstrate new state-of-the-art performance with SweRankMulti, while SweRankAgent further improves localization over single-pass ranking.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Software Issue Localization, Automated Code Repair, Multi-Language code models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RL1QOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SweRank+ extends SweRank to multilingual function-level issue localization. It introduces SweLocMulti, with 155,663 training instances across ten programming languages, and trains retrievers and rerankers using the existing SweRank objectives. It also adds a GPT-5-driven agent that reformulates queries using retrieved code and aggregates candidates across turns. Evaluation on five benchmarks shows improvements from multilingual training and further gains from the agent.

Critical strengths
  • Extending to ten languages addresses a practical gap in Python-heavy localization research. The dataset and resulting models are the paper’s strongest contribution.

  • Broad empirical validation on multilingual and Python benchmarks.

Critical weaknesses
  • Limited methodological novelty. The main contributions are the multilingual dataset and trained models, while the data pipeline and training objectives largely follow SweRank. The SweRankAgent also combines familiar components, and its distinction from prior iterative repository retrieval [1] needs clearer discussion.

  • Missing details and deeper experiments. The agent’s search budget and memory management are underspecified. There are also no component ablations, parameter sensitivity experiments, or comparisons across agent LLMs to explain where the gains come from.

[1] Fengji Zhang et al. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. EMNLP 2023.

Questions for authors
  1. What were the maximum and average search rounds, token usage, and inference cost? Were these budgets matched to the six-query Reformulate baseline?

  2. What is new in the SweRankAgent beyond adapting existing iterative retrieval pipelines to localization? Could you support this with component ablations, sensitivity experiments, and comparisons using different agent LLMs?

Reviewer_ZxaNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SWERank+, an issue localization framework that includes two components: SWERankMulti and SWERankAgent. SWERankMulti is trained on SWELocMulti, a multilingual dataset curated by the authors, and employs an embedding model as a retriever for initial candidates and an LLM as a reranker. SWERankAgent uses a multi-turn search paradigm: it iteratively reformulates the query, searches for candidates, and aggregates retrieved candidates to refine localization. Evaluation demonstrates that their approach outperforms baseline approaches.

Critical strengths

The paper's insight that multilingual retrievers are needed for issue localization makes sense, and the curated dataset is appreciated.

Critical weaknesses
  • W1: The paper’s technical novelty is minor: they train SWERankMulti with existing methods. For the embedding model, they finetune with InfoNCE contrastive loss; for the LLM reranker, they follow SWERank’s approach. For the agent workflow, the multi-turn search paradigm makes sense, but still somewhat trivial.

  • W2: The evaluation is limited. The paper does include evaluation demonstrating the effectiveness of different components in SWERank+. However, they lack:

    • Comparisons of SWERankEmbed with state-of-the-art embedding models, e.g., voyage-code-4 [1];
    • Fixed embedding models compared with more rerankers with recent LLMs. For instance, the flash version of the latest series of Qwen[2], GLM[3], DeepSeek[4], etc.
    • Comparison with baseline agents with more recent backbone models. Although they run with LocAgent or OpenHands, they only use Claude 3.5 as the backbone in Table 2. I won’t be surprised if the baseline’s performance improves with recent models, even smaller ones [2,3,4].

[1] https://blog.voyageai.com/2026/08/13/voyage-code-4/

[2] https://huggingface.co/Qwen/Qwen3.8-Flash-Next

[3] https://huggingface.co/zai-org/GLM-5.3-Flash

[4] https://www.deepseek.com/en/news/deepseek-v4-1-flash/

  • W3: The naming in the paper is not appealing. The authors introduced SWERank+, SWERankMulti, SWERankAgent, SWELocMulti, SWERankEmbedMulti, SWERankLLMMulti, along with several referenced works also begin with "SWE". I understand that all of the terms are relevant to software engineering, and it's a kind of naming convention, but having so many "SWEXX" terms reduces the clarity of the paper.
Questions for authors

Please better highlight the technical novelty and improve the evaluation (see weaknesses).

Reviewer_BSyZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper has two main contributions: (1) SweRankMulti which is a multi-lingual code embedder and reranker, which the paper trains on a specially curated multi-lingual dataset; (2) SweRankAgent whose main difference over the baseline in the paper is that it performs multi-turn search where the baseline performs a single-turn search or query-reformulation based multi-turn search. Experiments show that SweRankMulti performs better than baseline models and single-language fine-tuning on multilingual evaluations, and SweRankAgent outperforms baselines on agentic tasks.

Critical strengths

Extending model training and evaluations to multi-lingual settings is valuable. If the training corpus will be released, it could be potentially useful for future work.

Critical weaknesses
  1. The results on SweRankMulti are not interesting. Fine-tuning on multi-lingual corpora and observing improvements in performance on multi-lingual tasks is a largely known idea, so the novelty of the method and the finding is very limited.
  2. The motivation behind focusing on only the narrow multi-turn retrieval aspect of agentic workflows is unclear. Actively used coding agents such as claude code, opencode, codex, etc. perform multi-turn agentic search already. So its unclear to me what this paper is trying to highlight. Further, on this note, comparisons of SweRankAgent against these agents is missing.
  3. Throughout the paper, very old LLMs, from roughly two years ago, are being evaluated. LLM development is very fast, and so the relevance of findings on such old models is questionable.
Questions for authors
  1. How does SweRankAgent compare to agents such as claude code?
  2. Do the benefits of the proposed retrieval framework enhanced agentic coding with more recent LLMs as well?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.