acceptodds
Under review as a conference paper at ICLR 2027

Think Multilingual, Not Harder: A Framework for Analyzing and Teaching Code-Switched Reasoning

Abstract

Recent developments in reasoning capabilities have enabled large language models to solve increasingly complex mathematical, symbolic, and logical tasks. Interestingly, while reasoning models are often trained to generate monolingual text, these models have also been observed to code-switch (i.e., mix languages). Prior works have either viewed code-switching as an undesirable error, attempted to control code-switching through modifications to input prompts or the output decoding process, or focus on narrow subsets of languages, domains, tasks, and models. We address these gaps by introducing the first linguistically and behaviorally motivated fine-tuning framework for identifying beneficial code-switched reasoning behaviors in large language models and teaching these models to code-switch more effectively for reasoning. We create the Code-Switched Reasoning (CoRe) corpus, consisting of (1) 7k reasoning traces from 15 models, 18 languages, 10 scripts, and diverse reasoning domains, providing insights into potentially helpful code-switching behaviors, and (2) 40 carefully curated datasets for training and evaluating six interventions for improving code-switching in reasoning across three models and seven languages, totaling 120 fine-tuning conditions. Across 80k+ reasoning traces from both language/culture-agnostic and -specific evaluations, English-dominated reasoning, semantically accurate code-switching, and a more even mix of languages are positively associated with correct answers, whereas dense switching is associated with incorrect answers. Moreover, we are able to elicit positive behaviors through fine-tuning tasks that do not directly demonstrate code-switching. Our work suggests that small but well-curated datasets can change how reasoning models code-switch, allowing us to reap the benefits of reasoning in the many languages that lack large-scale reasoning data.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multilingual, code-switching, reasoning, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ouHmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how large language models (LLMs) code-switch during chain-of-thought reasoning, what types of code-switching are associated with better reasoning performance, and how to induce beneficial code-switching via supervised fine-tuning (SFT). The authors introduce the Code-Switched Reasoning (CoRe) corpus with 7k reasoning traces from 15 models across 18 languages, develop a taxonomy of code-switching behaviors, and evaluate six SFT interventions across 120 fine-tuning conditions. The results suggest that English-dominated reasoning, semantically accurate code-switching, and balanced language mixing are linked to better performance, while frequent switching is linked to lower accuracy. The authors also find that translation-based SFT can encourage beneficial code-switching behaviors, suggesting that small but well-curated datasets can change how reasoning models code-switch.

Critical strengths
  • The paper provides a systematic analysis of code-switched reasoning across diverse models, languages, and tasks. The proposed taxonomy helps better understand which code-switching behaviors are associated with better reasoning performance.
  • The paper explores several fine-tuning strategies and finds that translation-based training can encourage code-switching behaviors associated with correct answers, even without explicit code-switched reasoning demonstrations. This provides useful insights into improving multilingual reasoning, especially for low-resource languages.
Critical weaknesses
  • While the paper shows that SFT can encourage code-switching behaviors associated with higher accuracy, it does not demonstrate consistent improvements in actual reasoning performance. As shown in Table 4, some SFT interventions even reduce accuracy on certain languages and benchmarks. This raises concerns about whether encouraging these behaviors can reliably improve reasoning performance.
  • The analysis relies heavily on automatic language identification and LLM-based evaluations of code-switching accuracy and fluency. Although the authors provide some validation, it remains unclear how reliable these measurements are across different languages, especially low-resource ones. This could affect the paper's main conclusions about which code-switching behaviors are beneficial.
Questions for authors
  • Figure 4 shows that translation-based SFT encourages code-switching behaviors associated with higher accuracy, but Table 4 shows mixed results in actual reasoning accuracy. Could the authors explain why these behavioral improvements do not consistently translate into better task performance?
  • Could the authors compare at least one representative training-time baseline (e.g., XCOT [1]) to help clarify how the proposed SFT interventions compare with existing methods?

[1] Chai et al. “XCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning.” AAAI 2025.

5 comments
AuthorsOpenReview ↗

Author Response to Reviewer ouHm

We thank Reviewer ouHm for their thoughtful review of how our work (1) conducts “systematic analysis of code-switched reasoning across diverse models, languages, and tasks,” (2) “helps better understand which code-switching behaviors are associated with better reasoning performance,” and (3) “provides useful insights into improving multilingual reasoning, especially for low-resource languages.” Below, we address their fixable concerns about the paper.

  • “While the paper shows that SFT can encourage code-switching behaviors associated with higher accuracy, it does not demonstrate consistent improvements in actual reasoning performance.”: We showed through our regressions that certain code-switching behaviors are significantly positively associated with reasoning performance (an observational/correlational claim supported in sections 3.2 and 4.2) and that our SFT interventions can encourage these helpful behaviors (a causal claim, given our controlled experimental setup; section 4.2). Figure 4 explicitly shows that fine-tuning for open-domain machine translation encourages four different helpful code-switching aspects for reasoning. Additionally, we have updated section 4.2 with figures to more clearly illustrate gains in reasoning task accuracy from SFT on our open-domain machine translation task. In brief, we achieve accuracy improvements of up to 8%.
  • “As shown in Table 4, some SFT interventions even reduce accuracy on certain languages and benchmarks. This raises concerns about whether encouraging these [code-switching] behaviors can reliably improve reasoning performance.”: While we make some conjectures about which SFT tasks may improve reasoning performance in section 4.1, we do not expect models to respond to interventions uniformly. We conduct the experiments in section 4 of the original paper to test the effects of different interventions. The fact that some interventions cause a performance drop is surprising and informative. For example, one might assume that fine-tuning on English reasoning demonstrations would improve reasoning performance, since most models are strongest in English. Instead, our results demonstrate that fine-tuning on a simple open-domain MT task benefits low-resource languages more than fine-tuning on English reasoning. Since parallel MT data is more readily available in low-resource languages than reasoning demonstrations, future approaches should consider using translation-based SFT tasks for improving reasoning in low-resource settings. See the updated section 4.2 for details.
  • “The analysis relies heavily on automatic language identification and LLM-based evaluations of code-switching accuracy and fluency. Although the authors provide some validation, it remains unclear how reliable these measurements are across different languages, especially low-resource ones.”: Our efforts to validate our pipelines for language identification and measurement of code-switching characteristics are in section 3.1 and its linked appendix. While we would like to conduct additional human evaluations to further validate our measurements, recruiting bilingual English-speaking evaluators for the low-resource languages in our paper is difficult. For example, a search for “Fluent bilingual English and Igbo speakers” on Prolific yields no results.
  • Experiment to validate the code-switching fluency judge: As a next-best proxy for human evaluation, we apply our fluency judge to the SwitchLingua dataset of naturalistic, human-evaluated code-switched utterances. We correlate our judge’s scores with SwitchLingua’s groundtruth scores. SwitchLingua covers many of the languages in our paper (Arabic, Mandarin, French, German, Hindi, Italian, Japanese, Korean, Russian, and Spanish). [Updated 11/12/26] As described in section 3.1 of the updated uploaded paper, we obtain a moderately strong Spearman’s correlation of r = 0.45 (p < 0.001) between our scores and the SwitchLingua fluency scores, confirming that our judge captures significant signals of fluency (see appendix A.23 for details).
  • Reliability of automatic language identification (LID): In the original paper, we reported an average word-level LID accuracy of 0.933 and instance-level LID accuracy of 0.942 on our in-domain validation set (appendices A.8 and A.13). We have updated appendices A.8 and A.13 with more information about the performance of the LID tools.
AuthorsOpenReview ↗

Author Response to Reviewer ouHm (Continued)

  • “Figure 4 shows that translation-based SFT encourages code-switching behaviors associated with higher accuracy, but Table 4 shows mixed results in actual reasoning accuracy. Could the authors explain why these behavioral improvements do not consistently translate into better task performance?”
    • The higher-resource languages in our study (like Hindi) are less likely to benefit from additional fine-tuning than the low-resource languages in our study (like Igbo), since these languages leave little room for improvement to begin with (e.g., Phi-4-reasoning already achieves 90% accuracy on Global PIQA in Hindi without fine-tuning).
    • Models with more officially supported languages and better per-language performance (Qwen 3 8B and DeepSeek R1 Distill Llama 8B) respond better to the SFT than models with limited baseline multilingualism (Phi-4-reasoning). The ability to observe distinct effects on different models and languages is precisely our motivation for including more/less multilingual models and low/high-resource languages.
    • Understanding which SFT interventions and experimental conditions increase reasoning task accuracy is exactly the motivation for (1) our experiments in section 4 in the original paper and (2) our choice to explicitly include the language and benchmark being trained/evaluated on as factors in the mixed effects models for Figures 2 and 3, to isolate the relationship between code-switching and reasoning performance.
    • The updated paper includes our revisions to section 4.2 to more clearly scope our claims about which interventions are helpful, and under what conditions.
  • “Could the authors compare at least one representative training-time baseline (e.g., XCOT [1]) to help clarify how the proposed SFT interventions compare with existing methods?”:
    • The aims of xCoT fundamentally differ from ours, preventing an apples-to-apples comparison. xCoT teaches the model to respond to prompts in all languages using English; we teach our student models to use the same language as the prompt for their final answers, preserving readability for the user. xCoT trains on all languages jointly; we train language-specific models, due to the “curse of multilinguality” (see Longpre et al.’s 2026 ICLR paper “Adaptive Transfer Scaling Laws for Multilingual Pretraining” for a recent study). xCoT introduces code-switching into the prompt/user query; we focus on code-switching in the model reasoning. xCoT trains and evaluates on math reasoning (GSM8K, MGSM, MSVAMP); as discussed in our Related Work, we train and evaluate on more diverse domains/tasks. The SFT intervention conditions presented in our paper as written provide fairer, more controlled comparisons than an xCoT baseline would.
    • The SFT interventions listed in section 4.1 of the original paper include several baselines reflecting existing reasoning SFT methods. For example, “native-language reasoning . . . is a monolingual fine-tuning baseline task” which reflects existing methods that train on demonstrations of reasoning in the target language (e.g., “Breaking Language Barriers in Multilingual Mathematical Reasoning,” Chen et al., Findings 2024). We have updated the paper to more clearly explain how each of the SFT interventions in section 4.1 relate to approaches from other work.
AuthorsOpenReview ↗

Fluency judge validation experiment results

We updated the response to the reviewer with the following result:

Experiment to validate the code-switching fluency judge: As a next-best proxy for human evaluation, we apply our fluency judge to the SwitchLingua dataset of naturalistic, human-evaluated code-switched utterances. We correlate our judge’s scores with SwitchLingua’s groundtruth scores. SwitchLingua covers many of the languages in our paper (Arabic, Mandarin, French, German, Hindi, Italian, Japanese, Korean, Russian, and Spanish). [Updated 11/12/26] As described in section 3.1 of the updated uploaded paper, we obtain a moderately strong Spearman’s correlation of r = 0.45 (p < 0.001) between our scores and the SwitchLingua fluency scores, confirming that our judge captures significant signals of fluency (see appendix A.23 for details).

Reviewer_yJXfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This work relates LLMs' reasoning to a phenomenon called "code-switching," in which humans change languages during communication. The authors assume that code-switching enhances reasoning performance, especially for low-resource languages. First, several existing metrics are accumulated and classified based on brainstorming ideas generated by Gemini to characterize the reasoning process in terms of code-switching (e.g., semantic accuracy of the code-switched language and the degree of language mixing). The authors found that semantic accuracy of the switched language and switch timing have higher associations with task performance among the examined metrics. The authors also examined several tasks that are assumed to improve code-switching metrics correlated with reasoning performance, and found that SFT on a translation task increases semantic accuracy during reasoning in a code-switched language.

Critical strengths
  1. The authors have conducted experiments with diverse models, datasets, and languages for both the observational experiments (Section 3) and the SFT experiments (Section 4).
  2. The authors design the candidate SFT tasks based on the results of the correlation analysis, which is a natural flow.
Critical weaknesses
  1. The authors only observed correlations between code-switching characteristics and reasoning performance. It is still unclear whether there is a causal relationship between the two.
  2. There is a lack of experiments to confirm whether the proposed SFT task actually improves the reasoning performance evaluated in Section 3. One cannot determine whether the proposed SFT actually improves reasoning performance.
  3. The authors propose a taxonomy that was brainstormed by Gemini. However, this does not provide a clear interpretation of reasoning characteristics from the viewpoint of reasoning performance (e.g., code-switching accuracy improves performance while fluency worsens it). I wonder why this classification is reported despite having no clear relationship to the empirical findings.
Questions for authors

Have the authors conducted any interventions on the accuracy of the code-switched language? Does changing the semantic accuracy of the code-switched language (e.g., by rewriting the reasoning process using stronger models) alter task accuracy?

6 comments
AuthorsOpenReview ↗

Response to Reviewer yJXf

We thank Reviewer yJXf for their thoughtful review, including their positive comments that our paper follows a “natural flow” and that we “have conducted experiments with diverse models, datasets, and languages for both the observational experiments (Section 3) and the SFT experiments (Section 4).”

Below, we address Reviewer yJXf’s questions and concerns. We also highlight the specific sections of the revised paper version addressing reviewer concerns in blue in the newly uploaded file.

  • “The authors only observed correlations between code-switching characteristics and reasoning performance. It is still unclear whether there is a causal relationship between the two.”: Our causal claims that SFT modifies code-switched reasoning behavior are supported by the controlled experiments in Section 4 (we do not make any causal claims in Section 3). As to whether code-switching characteristics have a causal relationship with reasoning performance, in our mixed-effects model (as stated in Section 3.1) “the controls and random intercepts keep estimated effects of code-switching from being trivially explained by differences in overall model performance, language resource level, or question difficulty.”
  • “There is a lack of experiments to confirm whether the proposed SFT task actually improves the reasoning performance evaluated in Section 3. One cannot determine whether the proposed SFT actually improves reasoning performance.”: Section 3 confirms observational positive effects of code-switching across many models/datasets/languages, while Section 4 tests for causal effects by varying the SFT setup itself. Reporting effects through code-switching metrics and regression coefficients provides greater insight than reporting raw accuracies alone. The regressions in Figures 2 and 3 allow us to separately model the effects of factors such as the model and the prompt language from the effects of code-switching behaviors. To further clarify the effect of our fine-tuning interventions on reasoning task performance, we have updated our Figure 1 overview to summarize accuracy gains from fine-tuning and updated Section 4.2. In brief, we achieve reasoning task accuracy improvements of up to 8%. As stated by another reviewer (ouHm), “the paper shows that SFT can encourage code-switching behaviors associated with higher accuracy” through its two main studies–Section 3 (observational, “Understanding Code-Switched Reasoning Behaviors”) and Section 4 (experimental, “Producing Code-Switching Behaviors Using Linguistically and Behaviorally Motivated Data Curation”).
  • “The authors propose a taxonomy that was brainstormed by Gemini. However, this does not provide a clear interpretation of reasoning characteristics from the viewpoint of reasoning performance (e.g., code-switching accuracy improves performance while fluency worsens it). I wonder why this classification is reported despite having no clear relationship to the empirical findings.”: The initial motivation for developing the code-switched reasoning behavior taxonomy in Section 3 was to inform the design of our SFT experiments in Section 4. We agree that the relationship between the taxonomy and our empirical findings can be clarified. We have updated Section 4 to more clearly explain the motivating conceptual links between the taxonomy and our SFT experiments. Additionally, we have sharpened section 3.2 with clearer insights and analysis grounded in our taxonomy. In brief, we find that (1) contrary to prior findings, models often code-switch between unrelated/dissimilar languages, (2) code-switching often serves to translate prompt content into a higher-resource language, and (3) code-switching fluency and accuracy can have divergent values and effects on reasoning task accuracy for the same language (see updated paper for more details).
  • “Have the authors conducted any interventions on the accuracy of the code-switched language? Does changing the semantic accuracy of the code-switched language (e.g., by rewriting the reasoning process using stronger models) alter task accuracy?”: While that is an interesting experiment, it would require separate methodology for rewriting the reasoning and would confound our ability to understand whether it is the code-switching, difference in teacher models, or specific rewriting method that caused changes in performance. If Reviewer yJXf’s rewriting suggestion in particular refers specifically to a test-time intervention altering the reasoning as it is being generated, then we believe that this represents a valid, but fundamentally orthogonal direction to our SFT approach, as discussed in our Related Work section. In contrast, our experiments as they stand control well for these confounds by holding the teacher model fixed.

Thanks for the additional explanation of the taxonomy and difficulties of the experiment I have suggested. I also understand that the authors don’t make causal claims in Section 3.

The causal relationship claimed in Section 4 (before the revision) is between the SFT task and code-switching characteristics, not between SFT and reasoning performance. The authors’ comment does not address the point I raised. I recognize that the motivation of this work is to improve reasoning performance in low-resource languages. Given this objective, focusing on analyzing the causal relationship between SFT design and reasoning accuracy is more important than analyzing SFT and code-switching characteristics, for which no causal relationship with reasoning performance has been confirmed. At a minimum, the analysis should investigate whether changes in code-switching characteristics mediate the effect of SFT on reasoning performance.

Furthermore, Figure 5 (and Figure 1) cherry-picks the results of Table 4; the authors pick only specific (language * model) pairs that improved reasoning accuracy in the figures, while there are many performance deteriorations by MT task SFT reported in Table 4 (e.g., Global MMLU of Phi-4-reasoning on Yoruba is 0.31[.27,.35]->0.26[.22,.30], Global PIQA native of Qwen3-8B on Yoruba is 0.48[.38,.58]->0.32[.24,.42]). This causes misunderstandings for the readers.

I therefore changed my rating to 1.

AuthorsOpenReview ↗

Mediation Analysis and Additional Responses to Reviewer yJXf

We thank Reviewer yJXf for their response, especially the suggestion to perform mediation analysis. We respectfully disagree with parts of their characterization of our work:

  • “I recognize that the motivation of this work is to improve reasoning performance in low-resource languages. Given this objective, focusing on analyzing the causal relationship between SFT design and reasoning accuracy is more important than analyzing SFT and code-switching characteristics, for which no causal relationship with reasoning performance has been confirmed.”
    • We disagree. Analyzing the causal relationship between SFT and code-switching characteristics is just as important as analyzing the causal relationship between SFT and reasoning accuracy.
    • Improving reasoning performance in low-resource languages is not the only motivation for this work.
    • Understanding how to systematically modify code-switching behaviors is an additional core motivation for our work, not just obtaining raw accuracy gains (see examples of framing language):
      • In the abstract: “We create the Code-Switched Reasoning (CORE) corpus . . . providing insights into potentially helpful code-switching behaviors.”
      • In the conclusion: “We address the gap in work on how LLMs code-switch during reasoning, types of code-switching helpful for reasoning, and fine-tuning interventions to teach code-switched reasoning, by introducing a framework, data, code, and models for code-switched reasoning.”
  • “The analysis should investigate whether changes in code-switching characteristics mediate the effect of SFT on reasoning performance.”
    • We have updated the paper draft with mediation analysis (Figure 5 and appendix A.29): “The Average Causal Mediation Effects (ACME) on reasoning task accuracy (a measure of how much gains in accuracy are predicted by code-switching behaviors, separate from gains from SFT tasks alone) are significant and positive for open-domain MT and reasoning prompt translation interventions.” (Figure 5(b))
    • Our mediation analysis also supports the conclusion that “fine-tuning on open-domain machine translation can . . . boost overall reasoning task performance in low-resource languages." (Figure 5(d))
  • “Figure 5 (and Figure 1) cherry-picks the results of Table 4; the authors pick only specific (language * model) pairs that improved reasoning accuracy in the figures.”
    • In the prior draft, Figures 1 (panel 2), 2, 3, 4, 6 and Table 4 as presented did not cherry-pick results, but rather contained all results over all conditions in our analyses and experiments.
    • The updated Figure 5 also contains all results over all conditions in our analyses and experiments.
    • The results overview in Figure 1 is abbreviated due to space constraints.
    • Our response to another reviewer (ouHm) with similar concerns also applies here:
      • “We do not expect models to respond to interventions uniformly. We conduct the experiments in section 4 of the original paper to test the effects of different interventions. The fact that some interventions cause a performance drop is surprising and informative. For example, one might assume that fine-tuning on English reasoning demonstrations would improve reasoning performance, since most models are strongest in English. Instead, our results demonstrate that fine-tuning on a simple open-domain MT task benefits low-resource languages more than fine-tuning on English reasoning. Since parallel MT data is more readily available in low-resource languages than reasoning demonstrations, future approaches should consider using translation-based SFT tasks for improving reasoning in low-resource settings.”
      • “The higher-resource languages in our study (like Hindi) are less likely to benefit from additional fine-tuning than the low-resource languages in our study (like Igbo), since these languages leave little room for improvement to begin with (e.g., Phi-4-reasoning already achieves 90% accuracy on Global PIQA in Hindi without fine-tuning).”
  • “There are many performance deteriorations by MT task SFT reported in Table 4 (e.g., Global MMLU of Phi-4-reasoning on Yoruba is 0.31[.27,.35]->0.26[.22,.30], Global PIQA native of Qwen3-8B on Yoruba is 0.48[.38,.58]->0.32[.24,.42]).”
    • The confidence intervals cited by the reviewer here overlap, so we cannot directly conclude that these differences are significant.
    • Our updated analysis in Figure 5 confirms that fine-tuning for open-domain MT has significant, positive Average Causal Mediation Effects on predicted accuracy gain through code-switching behaviors as the mediator.
    • The updated Figure 5 additionally shows a significant gain in reasoning task accuracy caused by fine-tuning for open-domain MT in Swahili.
Reviewer_hDtGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on code-switching in LLM reasoning and mainly addresses three questions: (1) how LLMs code-switch during reasoning, (2) how code-switching behaviors relate to reasoning performance, and (3) how fine-tuning can influence code-switching behaviors. The paper introduces a dataset of code-switched reasoning traces and investigates the relationship between different code-switching behaviors and reasoning accuracy. It further explores how fine-tuning on different tasks can influence code-switching behaviors associated with better reasoning performance.

Critical strengths

he paper provides an empirical investigation of code-switching behaviors in LLM reasoning, offering potentially useful insights for multilingual LLM research.

Critical weaknesses
  • The empirical findings, specifically to RQ1, are not particularly novel, and it is unclear what new insights they provide beyond existing studies of multilingual reasoning and code-switching.
  • The data generation and validation pipeline is unclear. It's hard to understand how you generate good data based on Gemini's brainstormed criteria. And the fluency judge is purely LLM-based without human evaluation.
  • While it is interesting that translation-based fine-tuning can influence code-switching behaviors, the paper mainly reports correlations between these behaviors and reasoning accuracy. It's less clear how much performance the proposed finetuning would improve.
Questions for authors

N/A

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer hDtG

We thank Reviewer hDtG for their thoughtful review and comment that our work “offer[s] potentially useful insights for multilingual LLM research.” Below, we provide further details addressing their fixable concerns about the paper.

  • “The empirical findings, specifically to RQ1, are not particularly novel, and it is unclear what new insights they provide beyond existing studies of multilingual reasoning and code-switching.”: To recap, Section 3.2 of our paper addresses RQ1 (“How do LLMs code-switch during reasoning?”). We agree that the discussion of our results in Section 3.2 can be sharpened with clearer insights and analysis. Accordingly, we have implemented these clarifying edits in the newly uploaded paper draft. Additionally, as substantiated by section 2 (Related Work), “prior work on code-switched reasoning lacks our coverage of diverse models, languages, domains, tasks, and reasoning behaviors.” Another reviewer (yJXf) also comments that “the authors have conducted experiments with diverse models, datasets, and languages for both the observational experiments (Section 3) and the SFT experiments (Section 4).”
  • “The data generation and validation pipeline is unclear. It's hard to understand how you generate good data based on Gemini's brainstormed criteria. And the fluency judge is purely LLM-based without human evaluation.” Could Reviewer hDtG confirm that this comment is directed to Section 3 of our paper?
  • How we generate good data informing the design of our code-switched reasoning behavior taxonomy and subsequent experiments, based on Gemini’s brainstormed criteria: Our procedure is summarized in Section 3.1, with exact prompts and human evaluation guidelines for reproducibility included in Appendices A5 and A6 (as stated in the paper text). We welcome questions on what is unclear about the description of our generation in these three sections. In brief, we prompt a frontier multilingual LLM to systematically extract descriptions of code-switching in examples of reasoning from our CoRE corpus, then use topic modeling and human curation to group these descriptions into a taxonomy. Our approach is validated by Lee et al. (2026 ICLR Paper “CoT Encyclopedia”): “This framework produces more interpretable and comprehensive analyses than existing methods.”
  • [Updated 11/12/2026] Human evaluation of LLM judge for measuring fluency of code-switching: We agree with Reviewer hDtG on the importance of validating our LLM judge’s annotations, motivating our original choice to use judging prompts calibrated with known human scores from Kuwanto et al. (see Section 3.1). While we would ideally like to conduct a larger evaluation on all our languages, recruiting bilingual English-speaking evaluators for the lower-resource languages in our paper is difficult. For example, searching for “Fluent bilingual English and Igbo speakers” on Prolific yields no results. As a next-best proxy for human evaluation, we apply our fluency judge to the human-evaluated SwitchLingua dataset of naturalistic, human-evaluated code-switched utterances. Crucially, SwitchLingua covers many of the languages in our paper (Arabic, Mandarin, French, German, Hindi, Italian, Japanese, Korean, Russian, and Spanish). [Updated 11/12/26] As described in section 3.1 of the updated uploaded paper, we obtain a moderately strong Spearman’s correlation of r = 0.45 (p < 0.001) between our scores and the SwitchLingua fluency scores, confirming that our judge captures significant signals of fluency (see appendix A.23 for details).
  • “While it is interesting that translation-based fine-tuning can influence code-switching behaviors, the paper mainly reports correlations between these behaviors and reasoning accuracy. It's less clear how much performance the proposed finetuning would improve.”: We agree with Reviewer hDtG about the importance of distinguishing between observational correlational and causal experimental results. We do report fine-tuning results in section 4, which another reviewer (ouHm) notes demonstrates “that SFT can encourage code-switching behaviors associated with higher accuracy.” Reporting effects on performance through code-switching metrics and regression coefficients provides greater insight than reporting raw accuracies alone. The regressions in Figures 2 and 3 allow us to separate the effects of factors such as the model and the prompt language from the effects of code-switching. For further clarity, we have updated our Figure 1 overview to indicate some of the reasoning task accuracy gains from fine-tuning, with more detailed results in the updated Section 4.2 as well. In brief, we achieve reasoning task accuracy improvements of up to 8%.
AuthorsOpenReview ↗

Fluency judge validation experiment results

We have updated our response to the reviewer to include the following results: We conducted further validation experiments to address reviewer concerns about the reliability of our judge for measuring code-switching fluency. As described in section 3.1 of the updated uploaded paper, we obtain a moderately strong Spearman’s correlation of r = 0.45 (p < 0.001) between our scores and fluency scores from the SwitchLingua dataset, confirming that our judge captures significant signals of fluency (see appendix A.23 for details).

Many thanks to the authors for their response. Regarding the use of LLMs for generation, could the authors further clarify in A6, what the topic representations are in "we use Gemini 2.5 Flash to generate more interpretable topic representations"? Could you provide some examples and how does that inform the final set of the taxonomy?

Regarding the updated section 3.2, I think some of the findings have also mentioned in prior works like [1].

[1] Winata et al. Can Large Language Models Understand, Reason About, and Generate Code-Switched Text?

Comments on the paper

4 comments
AuthorsOpenReview ↗

Summary of reviews and author responses

We thank the reviewers for their thoughtful reviews. We have uploaded an updated draft of the paper incorporating your feedback. We also highlight the specific sections of the revised paper version addressing reviewer concerns in blue in the newly uploaded file. Below, we provide a summary of key points and our individual responses to each reviewer:

  • Strengths of our work:
    • Reviewers hDtG and ouHm: We offer useful insights for multilingual LLM reasoning research, e.g., “translation-based training can encourage code-switching behaviors associated with correct answers, even without explicit code-switched reasoning demonstrations.”
    • Reviewer yJXf: The paper is organized naturally, with an observational study motivating a second experimental study.
    • Reviewers yJXf and ouHm: We conduct experiments with diverse models, datasets, tasks, and languages.
    • Reviewer ouHm: Our taxonomy of code-switched reasoning behaviors is helpful for understanding the association between code-switching and reasoning task performance.
  • Reviewer questions and concerns addressed:
    • Reviewers hDtG and yJXf: We address concerns about the novelty and clarity of our insights from the observational study by sharpening the discussion of results in section 3.2 and how these results motivate our SFT experiments in section 4.1.
    • Reviewers hDtG and ouHm:
      • We expand our methods appendices with additional information about the reliability of our language identification tools for analyzing code-switching.
      • [Updated 11/12/26] We conducted further validation experiments to address reviewer concerns about the reliability of our judge for measuring code-switching fluency. As described in section 3.1 of the updated uploaded paper, we obtain a moderately strong Spearman’s correlation of r = 0.45 (p < 0.001) between our scores and fluency scores from the SwitchLingua dataset, confirming that our judge captures significant signals of fluency (see appendix A.23 for details).
    • [Updated 11/16/26] Reviewers hDtG, yJXf, and ouHm: We update our Figure 1 overview and discussion of results in section 4.2 to clarify how much our fine-tuning interventions improve code-switching behaviors and reasoning task performance in specific experimental conditions. We also added mediation analysis.
    • Reviewer yJXf: We explain in our response to the reviewer how the designs of our experiments in section 4 and mixed-effects models of reasoning task performance allow us to make causal claims about the effects of our SFT interventions on code-switching behaviors and reasoning performance.
    • Reviewers yJXf and ouHm: We explain why the baseline comparisons requested by the reviewer represent fundamentally orthogonal directions to our goals for this work, introduce potential confounds, and are not directly comparable. We also clarify how our paper as written includes baseline comparisons that cover the most common approaches for multilingual reasoning SFT.
  • [Updated 11/15/2026] Resolved requests for clarification/questions for reviewers: We requested and received clarification from Reviewer hDtG on what was unclear to them about the description of our data synthesis process in section 3.1, since this concern was raised in their review. We followed up with the reviewer to explain our methods in more detail and updated the appendices A.24 - A.26 accordingly.
AuthorsOpenReview ↗

Fluency judge validation experiment results

We have updated our top-level summary above and the individual responses to reviewers below to include the following results: We conducted further validation experiments to address reviewer concerns about the reliability of our judge for measuring code-switching fluency. As described in section 3.1 of the updated uploaded paper, we obtain a moderately strong Spearman’s correlation of r = 0.45 (p < 0.001) between our scores and fluency scores from the SwitchLingua dataset, confirming that our judge captures significant signals of fluency (see appendix A.23 for details).

Dear Reviewers,

The authors have now submitted their responses. Please review them and indicate whether your concerns have been addressed. Please also update your scores if the responses have changed your assessment.

Thank you.

AC

We thank the AC and reviewers for their time. We have responded to all reviewer questions and concerns to the best of our ability and believe these concerns have been addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.