acceptodds
Withdrawn from ICLR 2027

LangSelect: Cost-Aware Target-Language Routing for LLM Code Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Code generators usually write in a programming language fixed before decoding. For function-level tasks that several languages can solve and the same unit tests can check, we study target-language routing under a verifier with LANGSELECT, a loop in which a policy orders candidate languages, a failed attempt falls back to the next language, and the code tokens of every attempt are counted. We build MultiLang-Bench, 3,000 tasks with verified solutions in eight languages. We show that the final pass rate depends only on which languages are tried and that, under a mild independence assumption, adjacent languages should be ordered by cost per success conditional on earlier failures. With GPT-5, a fixed language order emits 50.3% fewer code tokens than default-prompt Python on 450 held-out tasks (final pass rate 92.9% vs. 92.0%) and 48.2% fewer on a second, 400-task mix, where its pass rate is 2.8 points lower. A supervised selector and a LinUCB bandit that recover about 87% of the headroom in replay emit 3.7% and at least 13.9% more tokens than Python-only in live generation. A chain that ends in Python keeps the expected pass rate of Python-only; recomposed from logs, it saves 39% on the data used to choose it and 13% (95% CI−18 to 38%) in a fresh run on 60 of these tasks. Against a terse Python prompt, a cross-campaign estimate leaves the fixed order 11–15% and this chain no saving. With DeepSeek-Coder-6.7B, bandit routing emits 2.3 times as many tokens as Python-only. For GPT-5, generation time increases under routing, and reasoning tokens are not observed.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM code generation, programming language routing, cost-aware inference, code generation efficiency, multilingual code generation, contextual bandits, verification, inference-time routing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6YhnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

LangSelect chooses the programming language requested from a code-generating LLM and tries another language if the output fails verification. The paper introduces MultiLang-Bench, with 3000 function-level tasks and verified solutions in 8 languages, and compares fixed orders, supervised selectors, and bandit policies. The evaluation shows that reference-solution replay favors learned selectors, but live generation does not. In its setting, using a fixed order substantially reduces emitted code tokens relative to default-prompt Python, with results depending on task difficulty, prompt style, and generator; actual billing savings are not established.

Critical strengths
  • Choosing the output language for LLM code generation is an important problem (however, what I expect seems not appearing in the paper).
Critical weaknesses
  • The evaluation does not capture the important language-selection problem. Language choice can affect implementation quality and execution performance, for example in CUDA/Triton/TileLang kernel generation and optimization. The paper instead studies emitted code length on standalone functions. The task scale and difficulty do not establish usefulness in demanding code-generation settings, and unit-test correctness alone does not capture their performance or integration requirements. Model coverage is also limited to GPT-5 and DeepSeek-Coder-6.7B-Instruct, with only 80 tasks for the latter. Since the reported savings depend strongly on the generator's verbosity and language competence, these experiments are insufficient to establish a broader conclusion about language selection. The evaluation also does not match what the title suggested.
  • The contribution is unclear relative to prior work [1, 2] and a simple baseline. It is unclear what routing adds over simply asking for concise Python. The learned selectors do worse than a fixed language order in live generation. The fixed order saves code tokens, but a terse Python prompt alone saves 41.6%, and the paper does not compare the two in the same experiment. It also leaves out reasoning tokens and the input cost of retries. I therefore cannot tell whether switching languages actually makes code generation cheaper, or mainly changes how verbose the model is. The classical ordering rule does not add much novelty either.
Questions for authors
  • Is there an evaluation on more demanding tasks where language choice affects execution performance or implementation quality, such as kernel generation or optimization? How does the routing objective account for those outcomes rather than only emitted code length?
  • Does the fixed-order result hold across several capable code generators under the same task and prompt-style controls? Which languages remain preferable when the generator's verbosity and language competence change?
  • Is there a same-campaign comparison of the fixed routing policy against terse Python, with corrected verifiers, matched correctness, and total provider-reported usage? If so, how much of the reported saving remains? Please also clarify the contribution relative to intermediate- and bridging-language selection. [1] Xun Deng et al. Assessing Code Generation with Intermediate Languages, 2024. https://arxiv.org/abs/2407.05411 [2] Micheline Bénédicte Moumoula et al. Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation, 2025. https://arxiv.org/abs/2509.19918
Reviewer_5HkiOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates target-language routing (LangSelect) for function-level, unit-test-verified code generation where any language in a portfolio L = {C, Clojure, Go, Java, Julia, Python, Ruby, Rust} is acceptable. A routing policy orders up to B=3 candidate languages, executes each attempt against unit tests and a language-fidelity check, stops at the first passing attempt, and counts all emitted code tokens across failed and successful attempts (C_t = F_t + W_t + S_t). Proposition 1 shows that the final pass rate depends only on the set of attempted languages (so ending a chain with Python preserves Python's pass rate) and that adjacent languages should be ordered by conditional cost per success c_bar_i^(k) / rho_i^(k) under conditional cost-outcome independence. The authors introduce MultiLang-Bench (3,000 tasks with verified solutions in all 8 languages) and evaluate static fixed orders (Ruby->Clojure->Julia as run; Ruby->Clojure->Python recomposed), a CodeBERT+metadata supervised classifier, and a warm-started Neural-LinUCB bandit across reference-solution replay and six live campaigns (MAIN, REP, HARD, SUP, STYLE, OPEN) using GPT-5 and DeepSeek-Coder-6.7B.

Critical strengths

None

Critical weaknesses

1: The targeted task: selecting proper language for single-turn function generation, used to be an interesting topic in 2023-2025 era, but has already obsolete in the multi-turn agentic coding era of 2026. This task is already saturated, especially for the simple coding functions used in this research.

2: The paper only offers a heuristic of Ruby-C-Python generation order for GPT-5 that is not generalisable anywhere. The protocol only takes advantage of the observed weakness of the default-prompt of GPT-5, a model that was obsoleted one year ago, and does not generalise to any other models.

Questions for authors

I have no questions because I don't think the fundamental directional flaw of this research could be improved by any questions.

Reviewer_RTzpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LANGSELECT, a framework that selects programming languages and fallback orders to reduce token usage in LLM code generation. The authors introduce MultiLang-Bench, which contains 3,000 tasks in eight programming languages. Experiments show that a fixed language order can reduce code tokens by about 50% compared with default Python on GPT-5. The paper also studies how failed attempts, fallback strategies, and prompt styles affect these savings.

Critical strengths

S1: The paper studies code generation efficiency from an interesting angle by considering the choice of programming language. It also provides a theoretical analysis of how language ordering affects success rates and token costs.

S2: The experiments are fairly comprehensive, covering different routing methods, ablation studies, prompt styles, and different models. The authors also discuss negative results and limitations.

Critical weaknesses

W1: The task setting is quite specific, mainly focusing on function-level problems that can be solved in multiple languages and verified by unit tests. In real software projects, the programming language is usually fixed by the existing codebase and deployment environment, so the practical value of this approach seems limited.

W2: The paper only measures generated code tokens, without considering hidden reasoning tokens, input tokens, or the overhead of repeated calls. In fact, the fixed-order strategy reduces code tokens but takes much longer on average, so it is unclear whether the method actually reduces inference costs or improves efficiency.

W3: The Python-only baseline is not strong enough, as the main experiments compare language routing against Python with a default prompt rather than an optimized one. The authors' own experiments show that a terse Python prompt alone reduces code tokens by 41.6%, suggesting that the additional benefit of language routing may be much smaller than the main results suggest.

W4: Although the fixed-order strategy reduces token usage, it also lowers the final pass rate in some experiments, including drops of 2.8 and 16 percentage points on REP and McEval, respectively. Moreover, the paper mainly compares language routing, which allows up to three attempts, against a single Python attempt, without considering Python retries under the same budget.

Questions for authors

Q1: Could the authors clarify which practical software development scenarios could benefit from LANGSELECT beyond standalone, language-independent functions? How would this approach work when the programming language is already fixed by an existing project?

Q2: Could the authors evaluate actual inference costs, including input tokens, hidden reasoning tokens, and repeated calls? Given the longer generation time of the fixed-order strategy, does the method still provide an advantage in overall cost or latency?

Q3: Could the authors directly compare LANGSELECT against a Python-only baseline with a terse prompt under the same experimental conditions? Would language routing still provide significant additional savings if similar prompt optimizations were applied to all languages?

Q4: Could the authors include a Python-only baseline that allows multiple generation and verification attempts under the same token budget or maximum number of attempts? Does cross-language routing still offer a clear advantage when comparing methods at similar final pass rates?

Comments on the paper

1 comment

Dear Reviewers and Area Chairs,

We sincerely thank you for your time, thoughtful evaluations, and constructive feedback throughout the review process.

Your comments have helped us identify important limitations and opportunities to strengthen our work. We appreciate the careful consideration given to our submission.

We will take your suggestions into account, further improve the manuscript, and prepare a stronger version for submission to a future conference.

Thank you once again for your valuable feedback and contributions to improving our research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.