acceptodds
Under review as a conference paper at ICLR 2027

How Fine-Tuning Data Influences Reliance on Irrelevant Context in Math Reasoning

Abstract

Large language models have been shown to rely on and get distracted by irrelevant context (IC) when solving mathematical reasoning tasks. While there exist many works that show this failure mode, it is not well understood what factors contribute to such reliance on irrelevant context. In this work, we take a first step towards understanding the origins of this failure mode. First, we perform ablations on the irrelevant context and show that models often learn shortcut correlations between artifacts and operations without understanding the requirement of the question, which then causes them to rely on irrelevant context. Next, we discover and show that long chain-of-thought supervision exacerbates reliance on irrelevant context. We also show that introducing irrelevant context during fine-tuning can make these models more robust against irrelevant context. Interestingly, this is true even if the added IC is fully out-of-domain with respect to the original fine-tuning set. We conduct comprehensive generalization tests and also evaluate on more realistic and challenging ICs. Finally, we show that inclusion of such IC within the fine-tuning set does not negatively impact performance on clean test sets, while improving robustness by overcoming reliance on ICs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Irrelevant context reliance, Fine-tuning data, Math reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wUJvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how fine-tuning data affects mathematical reasoning under irrelevant context. It examines distractor components, CoT-length-based data selection, and forced reasoning continuation, and reports robustness gains from training with distractors, including out-of-domain text.

Critical strengths
  • The paper shows that retaining longer-CoT training examples can yield higher clean accuracy but worse performance under irrelevant context, revealing a useful trade-off in training-data selection.
  • The experiments settings cover two main model families, multiple pruning ratios, and several distractor types. Extending GSM-IC to 1,319 problems, auditing distractor relevance, and reporting three-seed variability strengthen the empirical study.
  • Out-of-domain distractor augmentation provides a simple, potentially useful training intervention. The reported gains with limited augmented samples and nearly unchanged clean accuracy in the Qwen3.5-4B setting are encouraging.
Critical weaknesses
  • The shortcut claim in Section 3.1 goes beyond the evidence. Removing an irrelevant number makes the model more accurate, but this does not show that CoT signals during SFT stage taught the model to use that shortcut. The model may already have had this tendency before fine-tuning; the paper does not compare the behavior before and after SFT or directly change the proposed training correlation.
  • The long-CoT and short-CoT training groups contain different questions. Their results could therefore differ because of what they learned, rather than how long the solutions were; the separate difficulty-pruning experiment does not resolve this.",
  • The uptake metric counts a distractor number whenever it appears in the output, even in a sentence such as '10 is irrelevant.' It therefore does not reliably measure whether the model used irrelevant information incorrectly. The 'wait' experiment also lacks a clean-question comparison, so the accuracy drop could partly reflect a general problem with forced continuation.
  • The claim of being the first to show that longer reasoning increases distraction is too broad. Inverse Scaling in Test-Time Compute (Gema et al., 2025) already reports this effect, although on different models and tasks.
Questions for authors
  • Could you define the “shortcut correlations” in Contribution 1(line 48 - 50) more clearly, and demonstrate their causal effect on the model’s reliance on irrelevant information by training on datasets with and without these correlations?
  • Could you train two models on the same questions, using short and long correct solutions that perform the same calculations? Does the model trained on longer solutions still suffer more from distractors?
  • Does adding 'wait' also reduce accuracy on clean questions? Can you check whether the extra distractor numbers in the output actually enter an incorrect calculation, rather than merely being mentioned?
  • What does your longer-generation experiment establish beyond Gema et al. (2025) [1]? Please state that difference explicitly and narrow the 'first work' claim.
  • In page 6, line 312–313, the paper claim that: "We observe similar patterns, where long-CoT pruning increases uptake of irrelevant context, as shown in Fig. 5.". But in Figure 5, it indicates that "Long CoT Supervision(Shortest Pruned)increases uptake". Do you want to say "long-CoT supervision" rather than "long-CoT pruning"?

[1] Gema, Aryo Pradipta, et al. "Inverse scaling in test-time compute." arXiv preprint arXiv:2507.14417 (2025).

2 comments

We thank the reviewer for their comments. We address their concerns below:


Q1 and W1

We thank the reviewer for this question - we believe the experiment they are requesting for already exists within the paper - but we value their input. Traditionally in GSM8k, almost every number is correlated with an operation. This is true for most math reasoning benchmarks. This lets the model learn a shortcut that numbers must be correlated with operations. Our fine tuning with IC experimental setup does not hold the shortcut that “every number must be correlated with an operation”. Fine-tuning with such ICs helps overcome reliance on irrelevant context (numbers) - breaking that shortcut. We will also ensure to soften that narrative that this behavior is learned from SFT alone, thank you for pointing this out.


Q2 and W2

To assess this, we use GPT-6-Astra to modify the original chain of thought in the GSM8k train set by making them more verbose. In the verbose setting, we add additional explanations for why the arithmetics are being performed. Such explanations were added for every step. We train 2 models - one on the original and one on the verbose. We test on the clean and the Extended GSM-IC benchmarks. And observe the following performances:

MetricOriginal CoTVerbose CoT
Clean accuracy80.52%64.06%
IC accuracy71.49%49.36%

We observe that for Verbose CoT, the drop in IC performance is significantly greater than for Original.

We note that synthetically making the train time chain of thought so much more verbose hurts the model’s overall performance. Thus, we repeated the experiment where we simply added 1 additional explanation sentence to the verbose setting. Here, the results are clearer:

MetricOriginal CoTVerbose CoT
Clean accuracy80.52%79.08%
IC accuracy71.49%67.85%

Again, we notice that for Verbose CoT, the drop in IC performance is greater than for Original.


Q3 and W3

We show that for the exact same setup, the accuracy does not reduce on the clean dataset but reduces significantly on the setup with IC (as already shown in the paper):

Requested Waits012
IC accuracy (%)69.45%58.73%60.55%
Clean accuracy (%)76.55%75.31%75.26%

With IC accuracy, we see a drop of up to 10-11%. We do not observe the same on clean accuracy.

In our paper, we present Unique Uptake values, which measures whether exact numeric tokens in the distractor appear within the generated response in those samples where the distractor does not contain the same numeric values as the question. We take it a step further and compute Unique Computational Uptake, which also ensures these numbers are used in arithmetic calculations. In GSM8k, all arithmetic computations occur within angular brackets <<>> - the model is instructed to do so. We parse for these, and observe if the number in the distractor appears within the left hand side of the operation. Such a search guarantees that the distractors are used within the arithmetic operations and not simply mentioned.

Unique Computational Uptake (%):

Unique Computational Uptake (%)Prune 30%Prune 50%Prune 70%
Shortest7.669.614.08
Longest4.514.644.45

With an unpruned baseline of 5.05. We observe that long CoT supervision (Prune Shortest), still exacerbates IC reliance, as seen by an increase in Unique Computational Uptake.


Q4 and W4

Our work is focused primarily on fine-tuning data and how it impacts reliance on irrelevant context. This will be of benefit to data selection/pruning and data generation techniques. We note that the cited work does not study fine-tuning/training data at all and instead only studies varying inference time compute. We will articulate this more clearly in the manuscript.


Q5

Apologies for this typo. You are correct. We meant long-CoT supervision increases uptake. We will fix this in the revised version of the manuscript.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer wUJv,

As we approach the end of the discussion period, we kindly as you to go through our rebuttal and let us know if we have addressed your concerns. Thank you for your time and effort!

Reviewer_5mWAOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines the effect of adding irrelevant information into Math Word Problems (MWPs). It demonstrates that models, when given specious numerical data, end up incorporating them erroneously into the solution reasoning process, without identifying that they have no bearing on the core problem. This vulnerability to including irrelevant data is shown to grow as chain-of-thought grows longer - either due to the complexity of the problem, or by forcing "wait" tokens to artificially inflate thinking time. This paper also proposes creating datasets with irrelevant data in the training samples to make models robust to distractors.

Critical strengths
  1. The paper investigates benchmark failure and probes for specific reasons and patterns influencing the extent of model distraction
  2. The characterization of IC changes and training on the various types - numeric, non-numeric, out-of-domain - offers useful comparisons
  3. The exploration and generalization to other ICs offers a useful study beyomd the effect of ICs.
Critical weaknesses
  1. Fine-tuning on IC examples lacks the setting where ICs are both out-of-domain, and numeric. This approach has been explored as a successful distractor in this work: "Cutting Through the Noise: Boosting LLM Performance on Math Word Problems", Anantheswaran et al. (2025). Incorporating this missing setting and contrasting with the fine-tuning approach enumerated here would help distinguish the improvements in this paper as well as novelty compared to existing work in the domain.
  2. Long and short CoT examples may differ in complexity, order of operations, and mathematical structure. While the setting established in the paper allows the conclusion that subsets chosen by CoT length are differently robust to distractors, the conclusion that CoT length or verbosity itself causes the distraction effect needs more support.
  3. The uptake metric checks whether distractor numbers appear in the generated response. But mentioning a distractor number is not equivalent to using it incorrectly as a model may explicitly identify and reject it. Longer responses also have more opportunity to mention numbers. Excluding examples where question and distractor numbers coincide helps, but does not distinguish copying, rejection, and arithmetic incorporation. This weakens uptake as evidence for the proposed CoT mechanism.
  4. The work could benefit from a more systematic design of distractors, beyond in/out-domain and train/test-sourced. Characterizing the distractors further along the lines of entity relevance, variable/unit relevance, operation cues, distractor position, and numerical plausibility, would further strengthen the investigation.
Questions for authors
  1. Does CoT verbosity still influence likelihood or extent of distraction when the same problem receives a concise versus a verbose solution? And how does a 'verbose' response variant for a problem compare to the 'concise' variant with forced 'wait' tokens?
  2. How much additional value does this approach of adding ICs offer over existing IC augmentation/distraction-testing methods for MWPs? Cutting Through the Noise: Boosting LLM Performance on Math Word Problems, Anantheswaran et al. (2025) MathAttack: Attacking Large Language Models towards Math Solving Ability, Zhou et al. (2024). Adversarial Math Word Problem Generation, Xie et al. (2024). Adversarial Examples for Evaluating Reading Comprehension Systems, Jia & Liang (2017). Evaluating Models’ Local Decision Boundaries via Contrast Sets, Gardner et al. (2020). Universal and Transferable Adversarial Attacks on Aligned Language Models, Zou et al. (2023). An LLM Can Fool Itself: A Prompt-Based Adversarial Attack, Xu et al. (2023)
  3. Do longer responses actually incorporate distractors into wrong calculations, or merely mention them more often?
2 comments

We thank the reviewer for their comments. We address their concerns below:


Q1 and W2

To assess this, we use GPT-6-Astra to modify the original chain of thought in the GSM8k train set by making them more verbose. In the verbose setting, we add additional explanations for why the arithmetics are being performed. Such explanations were added for every step. We train 2 models - one on the original and one on the verbose. We test on the clean and the Extended GSM-IC benchmarks. And observe the following performances:

MetricOriginal CoTVerbose CoT
Clean accuracy80.52%64.06%
IC accuracy71.49%49.36%

We observe that for Verbose CoT, the drop in IC performance is significantly greater than for Original.

We note that synthetically making the train time chain of thought so much more verbose hurts the model’s overall performance. Thus, we repeated the experiment where we simply added 1 additional explanation sentence to the verbose setting. Here, the results are clearer:

MetricOriginal CoTVerbose CoT
Clean accuracy80.52%79.08%
IC accuracy71.49%67.85%

Again, we notice that for Verbose CoT, the drop in IC performance is greater than for Original.

As for the differences between the verbose and concise CoTs in the forced ‘wait’ tokens experiment - we observe that forced-wait often produces repeated solutions and shifts towards previously ignored distractors. It is not equivalent to outputting the same solution with more detailed explanations.


Q2 and W1

We thank the reviewer for listing these works. We will ensure to cite all of these in our related works. We have gone through them and observed that the work with the closest overlap is Anantheswaran et al. (2025) - both discuss IC addition to fine-tuning data.

First, we note that adding IC to the fine-tuning data section is only a part of the contributions of our paper (Section 4). Even within just section 4, our work offers the following distinctive, novel insights:

  1. First to show that non-numeric distractors transfer well to numeric distractors.
  2. First to show that out-of-domain distractors transfer well to numeric, in-domain distractors. We note that, in our setting, out-of-domain sentences have nothing to do with arithmetics - they are sourced from an entirely different domain and do not contain numbers. The ICs used in Anantheswaran et. al. are arithmetic based (and contain numbers).
  3. Perform a strong transfer and generalization test across distractor types.

Additionally, Anantheswaran et. al. describes generating adversarial training explanations that explicitly identify irrelevant variables. Our approach preserves the original CoT without adding any explanations - only including the distractor in the question, making it an easier recipe to improve robustness.

We believe the above listed paper is important to cite in our background and we will ensure to do so. But we will respectfully push back on the claim that the cited paper contains out-of-domain distractors. We also note that they do not assess with in-domain non-numeric distractors.


Q3 and W3

In our paper, we present Unique Uptake values, which measures whether exact numeric tokens in the distractor appear within the generated response in those samples where the distractor does not contain the same numeric values as the question. We take it a step further and compute Unique Computational Uptake, which also ensures these numbers are used in arithmetic calculations. In GSM8k, all arithmetic computations occur within angular brackets <<>> - the model is instructed to do so. We parse for these, and observe if the number in the distractor appears within the left hand side of the operation. Such a search guarantees that the distractors are used within the arithmetic operations.

Unique Computational Uptake (%):

Unique Computational Uptake (%)Prune 30%Prune 50%Prune 70%
Shortest7.669.614.08
Longest4.514.644.45

With an unpruned baseline of 5.05. We observe that long CoT supervision (Prune Shortest), still exacerbates IC reliance, as seen by an increase in Unique Computational Uptake.

W4

We thank the reviewer for their comment. First we note that in the evaluation set with multiple distractors, distractors are scattered throughout the main questions, across different positions. However, we provide more results. In the original Extended GSM-IC, we maintained the IC right before the final question sentence, as explained in the appendix. We make a copy of the original Extended GSM-IC but move the distractor before the main, relevant question begins. We rerun the evals and compare performance against the original extended GSM-IC:

ModelOriginalFront IC
Qwen3.5-4B73.3973.01
Gemma3-4B35.71%44.28%

We note that for Qwen, there is no difference but for Gemma, performance improves as the distractor is moved to the front.

We provide additional categorical statistics for entity relevance - on the evaluations that are already present in the paper (Performance on Extended GSM-IC (Fig. 2)). As stated in the appendix, almost no samples have items that are shared between the question and the distractors. We provide accuracies on samples where the names are shared (and not shared) between the questions and the distractors:

ModelShared NameNo Shared Name
Qwen3.5-4B75.82%70.68%
Gemma3-4B37.36%37.03%

Finally, we note that distractors with operation cues are already evaluated in the paper (Fig. 8 (a) Right). We present the comparison with the original Extended GSM-IC performance below:

ModelOriginalOperation Cues Distractor
Qwen3.5-4B72.10%26.79%

We will include all additional results in the paper.

Reviewer_RNWzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study the effect of irrelevant context (IC) on LLMs' mathematical reasoning. The authors give two explanations for this problem: (1) models often learn shortcut correlations between artifacts and operations ; (2) long CoT supervision makes models more reliant on irrelevant context. They propose fine-tuning on examples with irrelevant context to improve robustness.

Critical strengths
  • The paper examines possible reasons why irrelevant context disrupts mathematical reasoning. This analysis is useful for developing better methods to address the problem.
  • The use of two LLMs and multiple random seeds strengthens the experimental design.
  • The writing of this paper is easy to follow.
Critical weaknesses
  • Some of the findings seem to overlap with prior works [1,2]. The paper needs a clearer statement of its main contributions.

  • Pruning examples by CoT length changes the distribution of training problems. The datasets differ in their problem types, arithmetic difficulty and total supervised tokens. It is unclear whether longer CoTs or these other factors cause the effect.

  • The forced-wait experiment shown in Figure 6 lacks a clean-input control. It is unclear if forced-wait hurts accuracy on clean inputs (w/o IC).

[1] Shi et al. Large Language Models Can Be Easily Distracted by Irrelevant Context. ICML, 2023.

[2] Yang et al. How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark. EMNLP, 2025.

Questions for authors
  • What are the main contributions, and how do they differ from previous work?

  • Does the effect of training CoT length persist with identical problems and controlled supervision budgets?

  • Does distractor augmentation help on other datasets as well?

6 comments

We thank the reviewer for their comments. We address their concerns below:


Q1 and W1

Thank you for your comment. We will articulate the main contributions. We kindly ask the reviewer to point us to the prior work they referred to and want us to compare against. In the meantime, we restate our contributions and novelty, and emphasize how they differ below:

We would like to note that this is the first work that studies what characteristics of the original fine-tuning data cause reliance on irrelevant context. The following are the novel insights:

When the fine-tuning samples contain no Irrelevant Context (IC) (Has not been explored before in literature):

  1. We show that samples learn shortcuts between input entities and operations. We also show that models are not mapping full training phases and sentences to operations.

  2. We show that samples containing long CoT supervision exacerbate reliance on irrelevant context. However, we also show that such samples are also responsible for good clean performance. This is important for data selection/pruning and generation strategies. When performing data selection or generation, a practitioner may generally choose or generate samples with longer CoTs due to good performance on clean benchmarks. But this will hurt the robustness of the models.

  3. We show that sample difficulty does not impact IC reliance.

When the fine-tuning samples contain Irrelevant Context (Has been explored before in literature but the following are our novel contributions):

  1. We are the first to show that non-numeric and fully out-of-domain ICs contained in the fine-tuning set improve robustness to ICs. We show that they transfer well to in-domain numerical ICs, making for a very easy fine-tuning modification for robustness.

  2. We discuss sample efficiency (show that only a few samples need to be augmented with distractors), training with a mixture of distractors and more - neither of which have been explored before and are of importance to the community.


Q2 and W2

To assess this, we use GPT-6-Astra to modify the original chain of thought in the GSM8k train set by making them more verbose. In the verbose setting, we add additional explanations for why the arithmetics are being performed. Such explanations were added for every step. We train 2 models - one on the original and one on the verbose. We test on the clean and the Extended GSM-IC benchmarks. And observe the following performances:

MetricOriginal CoTVerbose CoT
Clean accuracy80.52%64.06%
IC accuracy71.49%49.36%

We observe that for Verbose CoT, the drop in IC performance is significantly greater than for Original.

We note that synthetically making the train time chain of thought so much more verbose hurts the model’s overall performance. Thus, we repeated the experiment where we simply added 1 additional explanation sentence to the verbose setting. Here, the results are clearer:

MetricOriginal CoTVerbose CoT
Clean accuracy80.52%79.08%
IC accuracy71.49%67.85%

Again, we notice that for Verbose CoT, the drop in IC performance is greater than for Original.


W3

We show that for the exact same setup, the accuracy does not reduce on the clean dataset but reduces significantly on the setup with IC, as already shown in the paper:

Requested Waits012
IC accuracy (%)69.45%58.73%60.55%
Clean accuracy (%)76.55%75.31%75.26%

With IC accuracy, we see a clear drop of up to 10-11%. Such a drop is not observed on clean accuracy.


Q4

We thank the reviewer for this comment. To show that such distractor augmentation generalizes to other, non-math reasoning tasks, we perform the following experiment: We fine-tune two Qwen3.5-4B models on a subset of 2048 SQuAD [1] v1.1 training samples with identical answers and training settings. The baseline model was trained on the original samples while the augmented setup added irrelevant distractor sentences from AddSent [2]. We then evaluated both models on 787 held-out AddSent distractors with questions paired with the clean SQuAD passages. We report the exact match (outputted answer) and Token F1 (partial credit scores) below:

MetricClean BaselineDistractor-Augmented Fine-Tuning
Clean EM (%)89.71 %88.82%
IC EM (%)83.23%86.02%
Clean F1 (%)94.5%94.61%
IC F1 (%)89.16%92.3%

We notice that the Distractor-Augmented Fine-Tuned model reports higher scores on the IC benchmark, implying that it is more robust against distractors.

[1] SQuAD: Rajpurkar et al. 2016 EMNLP.

[2] AddSent: Jia and Liang 2017 EMNLP.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer RNWz,

As we approach the end of the discussion period, we kindly as you to go through our rebuttal and let us know if we have addressed your concerns. Thank you for your time and effort!

Dear Reviewer RNWz,

As there is less than one day remaining for the discussion phase, we kindly request that you let us know if we have addressed your concerns, and if so, we would appreciate it if you would consider raising the score. Thank you for your time.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Comment

We thank all the reviewers for their comments and concerns. We would like to note that this is the first work that studies what characteristics of the original fine-tuning data cause reliance on irrelevant context. The following are the novel insights:

When the fine-tuning samples contain no Irrelevant Context (IC) (Has not been explored before in literature):

  1. We show that samples learn shortcuts between input entities and operations. We also show that models are not mapping full training phases and sentences to operations.

  2. We show that samples containing long CoT supervision exacerbate reliance on irrelevant context. However, we also show that such samples are also responsible for good clean performance. This is important for data selection/pruning and generation strategies. When performing data selection or generation, a practitioner may generally choose or generate samples with longer CoTs due to good performance on clean benchmarks. But this will hurt the robustness of the models.

  3. We show that sample difficulty does not impact IC reliance.

When the fine-tuning samples contain Irrelevant Context (Has been explored before in literature but the following are our novel contributions):

  1. We are the first to show that non-numeric and fully out-of-domain ICs contained in the fine-tuning set improve robustness to ICs. We show that they transfer well to in-domain numerical ICs, making for a very easy fine-tuning modification for robustness.

  2. We discuss sample efficiency (show that only a few samples need to be augmented with distractors), training with a mixture of distractors and more - neither of which have been explored before and are of importance to the community.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewers,

We would really appreciate it if you could kindly go through our rebuttal and let us know if you need any additional clarifications! Thank you for your time and effort.

Dear Reviewer 5mWA,

Thank you for engaging with our rebuttal and for improving the quality of our work! We are glad that the concerns regarding novelty have been addressed. Thank you for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.