- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
LangSelect chooses the programming language requested from a code-generating LLM and tries another language if the output fails verification. The paper introduces MultiLang-Bench, with 3000 function-level tasks and verified solutions in 8 languages, and compares fixed orders, supervised selectors, and bandit policies. The evaluation shows that reference-solution replay favors learned selectors, but live generation does not. In its setting, using a fixed order substantially reduces emitted code tokens relative to default-prompt Python, with results depending on task difficulty, prompt style, and generator; actual billing savings are not established.
Critical strengths
- Choosing the output language for LLM code generation is an important problem (however, what I expect seems not appearing in the paper).
Critical weaknesses
- The evaluation does not capture the important language-selection problem. Language choice can affect implementation quality and execution performance, for example in CUDA/Triton/TileLang kernel generation and optimization. The paper instead studies emitted code length on standalone functions. The task scale and difficulty do not establish usefulness in demanding code-generation settings, and unit-test correctness alone does not capture their performance or integration requirements. Model coverage is also limited to GPT-5 and DeepSeek-Coder-6.7B-Instruct, with only 80 tasks for the latter. Since the reported savings depend strongly on the generator's verbosity and language competence, these experiments are insufficient to establish a broader conclusion about language selection. The evaluation also does not match what the title suggested.
- The contribution is unclear relative to prior work [1, 2] and a simple baseline. It is unclear what routing adds over simply asking for concise Python. The learned selectors do worse than a fixed language order in live generation. The fixed order saves code tokens, but a terse Python prompt alone saves 41.6%, and the paper does not compare the two in the same experiment. It also leaves out reasoning tokens and the input cost of retries. I therefore cannot tell whether switching languages actually makes code generation cheaper, or mainly changes how verbose the model is. The classical ordering rule does not add much novelty either.
Questions for authors
- Is there an evaluation on more demanding tasks where language choice affects execution performance or implementation quality, such as kernel generation or optimization? How does the routing objective account for those outcomes rather than only emitted code length?
- Does the fixed-order result hold across several capable code generators under the same task and prompt-style controls? Which languages remain preferable when the generator's verbosity and language competence change?
- Is there a same-campaign comparison of the fixed routing policy against terse Python, with corrected verifiers, matched correctness, and total provider-reported usage? If so, how much of the reported saving remains? Please also clarify the contribution relative to intermediate- and bridging-language selection. [1] Xun Deng et al. Assessing Code Generation with Intermediate Languages, 2024. https://arxiv.org/abs/2407.05411 [2] Micheline Bénédicte Moumoula et al. Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation, 2025. https://arxiv.org/abs/2509.19918
Dear Reviewers and Area Chairs,
We sincerely thank you for your time, thoughtful evaluations, and constructive feedback throughout the review process.
Your comments have helped us identify important limitations and opportunities to strengthen our work. We appreciate the careful consideration given to our submission.
We will take your suggestions into account, further improve the manuscript, and prepare a stronger version for submission to a future conference.
Thank you once again for your valuable feedback and contributions to improving our research.