acceptodds
Under review as a conference paper at ICLR 2027

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

Abstract

Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical failure mode in this regime: repetitive copying, where models extensively copy text from the input into their reasoning traces rather than productively solving the problem. We show that this behavior is pervasive across frontier long-context LLMs and intensifies with context length. By separating each prompt into task-relevant key evidence and irrelevant distractor context, we further show that the root cause is insufficient grounding: models copy from the prompt indiscriminately, and those that fail to focus on key evidence are far more likely to answer incorrectly. Motivated by this diagnosis, we propose GEAR (Grounding Evidence-Aware Reward), a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context. To enable GEAR on natural-language data, we develop an automated pipeline that constructs evidence-annotated training data from arbitrary documents. We validate GEAR across multiple model scales and benchmarks, showing consistent improvements of up to +4.6 average points over standard RL with accuracy-based rewards, with larger gains at longer contexts, while also reducing repetitive copying and thinking length. Our findings suggest that, even as long-context evaluation shifts from simple retrieval toward complex reasoning, accurate grounding in relevant evidence remains an indispensable capability with substantial room for improvement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Long-context

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qjjkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies the problem of repetitive copying in large language models, where in long-context settings they copy text from the prompt in their reasoning traces. The paper argues that copying information from the context is only useful when that information overlaps with key evidence necessary for the question, and proposes a reward formulation that encourages copying relevant information and discourages copying irrelevant information. Experiments across models, context lengths, and tasks show improvements over accuracy-only rewards.

Critical strengths

The problem setting of excessive copying from the context is relevant and has implications for both performance and efficiency. The paper shows this behaviour across models and context length, and highlights how erroneously copying irrelevant information correlates with poor performance. The proposed reward shaping fix, GEAR, appears to improve performance and reduce reasoning length across tasks.

Critical weaknesses

The accuracy gains from GEAR reward shaping are small (1.5-4.6%), without any uncertainty estimates or variance. Due to the high variance in RL training, this small difference might not hold with repeated training, especially when considering the large impact of hyperparameters on GEAR performance (most beta/alpha changes drop accuracy to that of standard GSPO). In a related but smaller point: in multiple places there are claims of correlations and significance without corresponding statistical tests.

No other reward shaping methods are tested, making it unclear if other methods like length-based reward shaping or other evidence-based rewards would provide similar improvements. Methods like LongRLVR are cited in the related work but not compared against.

Questions for authors
  1. Can you provide statistical tests or confidence intervals for the correlation and significance claims? For example: the relationship between overlap rate and accuracy (Figure 2, lines 206-215), and between grounding ratio and correctness (Figure 3, lines 251-257). This would help substantiate the claim of "significantly more selective copying" on lines 51-53.

  2. Did you test GEAR on existing tasks without the evidence-annotations? In section 4.2 you propose a way to create new QA pairs with the evidence by construction, but I’m curious if the alternative mentioned on line 307 (post-hoc labeling which parts of a document are relevant) would allow you to apply GEAR to more existing datasets.

  3. Are the RL results from one training run each, or averaged across multiple seeds?

4 comments
AuthorsOpenReview ↗

Response to Reviewer qjjk (1/2)

Response to Reviewer qjjk

Thank you for recognizing the relevance of repetitive copying to long-context reasoning and the potential of GEAR to improve task performance while reducing reasoning length, and for your constructive suggestions on reward baselines, training variability, and statistical validation. We address your concerns point by point below.


Q1. Gain size, variability, and hyperparameters (Weakness 1)

The accuracy gains from GEAR reward shaping are small (1.5-4.6%), without any uncertainty estimates or variance. Due to the high variance in RL training, this small difference might not hold with repeated training, especially when considering the large impact of hyperparameters on GEAR performance (most beta/alpha changes drop accuracy to that of standard GSPO). In a related but smaller point: in multiple places there are claims of correlations and significance without corresponding statistical tests.

A1. To address this concern, we have added three-seed experiments for GSPO and GEAR on Qwen3.5-35B-A3B and report the mean and sample standard deviation below (Appendix B.1, Table 6 of the revised manuscript).

ModelMethodRulerLB-v2Bcomp-LCGraphwalksAA-LCRAvg
32k Context
Qwen3.5-35B-A3BGSPO92.2 ± 2.960.1 ± 0.575.4 ± 0.891.5 ± 0.3—79.8 ± 1.0
GSPO + GEAR95.9 ± 0.861.0 ± 1.475.7 ± 0.495.9 ± 0.1—82.1 ± 0.7
128k Context
Qwen3.5-35B-A3BGSPO85.7 ± 2.755.8 ± 0.968.3 ± 1.689.7 ± 0.868.0 ± 1.773.5 ± 1.4
GSPO + GEAR93.4 ± 1.457.4 ± 1.468.7 ± 0.791.2 ± 0.269.7 ± 0.676.1 ± 0.8

GEAR improves the aggregate benchmark score over GSPO by 2.4 and 2.6 percentage points at ≤32k and full ≤128k, respectively. Taken together, the cross-seed means and sample standard deviations of both methods provide further empirical support for the reliability of GEAR's performance gains across the three evaluated seeds.

Regarding hyperparameter sensitivity, we acknowledge that poorly chosen α and β can induce reward hacking, for example by incentivizing indiscriminate copying to collect overlap rewards. However, our chosen hyperparameter setting yields higher aggregate scores than accuracy-only GSPO on all three model scales—Qwen3.5-9B, Qwen3.5-27B, and Qwen3.5-35B-A3B—at both ≤32k and full ≤128k. Together with the consistent gains in the new multi-seed experiments, we believe these results provide sufficient empirical evidence for GEAR's advantage.

We address the questions about correlations in A3.


Q2. Other reward-shaping baselines (Weakness 2)

No other reward shaping methods are tested, making it unclear if other methods like length-based reward shaping or other evidence-based rewards would provide similar improvements. Methods like LongRLVR are cited in the related work but not compared against.

A2. We have added LongRLVR and length-penalty reward baselines on Qwen3.5-35B-A3B, using the same training data, backbone, and RL-update budget as GEAR. For LongRLVR, we directly train the initial model using the method's format (chunked input and evidence-ID output), and evaluate it with both the adapted protocol and the ordinary benchmark protocol; the length penalties use λ=0.1 and λ=0.01, following the reward definition and token-counting convention in Appendix B.2 (Eq. 4). The comparison results are shown below (Table 7).

MethodRULERLB-v2Bcomp-LCGraphWalksAA-LCRAverage
32k Context
LongRLVR, adapted protocol91.358.674.285.5—77.4
LongRLVR, ordinary protocol88.059.575.388.5—77.8
Length penalty, λ=0.194.858.669.388.7—77.8
Length penalty, λ=0.0191.853.271.157.7—68.4
GSPO93.160.475.691.5—80.2
GSPO + GEAR96.261.375.695.9—82.3
128k Context
LongRLVR, adapted protocol88.753.068.878.157.069.1
LongRLVR, ordinary protocol84.256.071.485.767.072.8
Length penalty, λ=0.193.750.867.684.947.068.8
Length penalty, λ=0.0191.749.065.355.231.058.5
GSPO86.455.468.889.667.073.4
GSPO + GEAR93.957.668.391.370.076.2
AuthorsOpenReview ↗

Response to Reviewer qjjk (2/2)

The results show that GEAR achieves the highest aggregate score among the tested methods at both context ranges. Under the same training data, backbone, and RL-update budget, the LongRLVR adaptation has lower aggregate scores than vanilla GSPO under both evaluation protocols. GEAR computes its reward directly from free-form reasoning without requiring an additional evidence-ID output protocol. In addition, we compare the methods' mean complete-response lengths on RULER ≤32k, as shown below (Appendix B.2, Table 8).

MethodAvg. response length (tokens)
LongRLVR, adapted protocol2,438
LongRLVR, ordinary protocol2,744
Length penalty, λ=0.189
Length penalty, λ=0.0124
GSPO2,853
GSPO + GEAR2,185

On the one hand, training with length penalties directly causes the model's reasoning ability to collapse. On the other hand, GEAR still has the shortest mean response length among GEAR, GSPO, and LongRLVR, further indicating that GEAR achieves a better balance between output length and task performance.


Q3. Statistical tests and confidence intervals (Original question 1)

Can you provide statistical tests or confidence intervals for the correlation and significance claims? For example: the relationship between overlap rate and accuracy (Figure 2, lines 206-215), and between grounding ratio and correctness (Figure 3, lines 251-257). This would help substantiate the claim of "significantly more selective copying" on lines 51-53.

A3. We have added Spearman correlation tests for the figures, using the revised results for Figure 2a. The table below reports the correlation coefficient ρ and p value (Appendix B.8, Table 16).

Two variablesCorrelation coefficient ρp value
Input overlap and answer correctness (Figure 2a)−0.424.85×10−474.85\times10^{-47}
Grounding ratio and answer correctness (Figure 3a)+0.291.22×10−211.22\times10^{-21}
Input overlap and reasoning length (Figure 2b)+0.275.68×10−195.68\times10^{-19}

The results show statistically significant correlations between the pairs of variables discussed above in the paper.


Q4. Existing QA data and post-hoc labels (Original question 2)

Did you test GEAR on existing tasks without the evidence-annotations? In section 4.2 you propose a way to create new QA pairs with the evidence by construction, but I’m curious if the alternative mentioned on line 307 (post-hoc labeling which parts of a document are relevant) would allow you to apply GEAR to more existing datasets.

A4. We have not conducted training experiments that retain existing QA pairs and add evidence annotations post hoc. Methodologically, GEAR's reward computation does not require questions to be generated through our data construction pipeline: once corresponding evidence annotations are available for existing QA pairs, they can be used to compute the GEAR reward. Post-hoc annotation could therefore extend GEAR to more existing datasets in principle.

We would like to emphasize that the main contributions of this paper are to identify and analyze repetitive copying in long-context reasoning and to propose an evidence-based reward design to improve reasoning behavior. To validate this design, we automatically construct training data with evidence annotations and demonstrate its effectiveness on long-context reasoning benchmarks.

Developing a reliable post-hoc evidence annotation pipeline for existing QA pairs is a valuable extension, whose annotation quality and training effectiveness require dedicated evaluation. This paper focuses on proposing and validating the GEAR reward design; a systematic comparison of such data extension pipelines is left to future work.


Q5. Number of independent RL runs (Original question 3)

Are the RL results from one training run each, or averaged across multiple seeds?

A5. The results in the original paper were obtained from a single training run per setting. We have added results for Qwen3.5-35B-A3B using three training seeds each for GSPO and GEAR; please see A1 and Appendix B.1 (Table 6) of the revised manuscript for details.


Thanks again for your constructive suggestions. Hope our explanations can ease your concerns. Please let us know if there is more to clarify.

I appreciate the additional experiments: in particular the three-seed RL experiments alleviate my concerns about training variance, and the statistical tests provide stronger support for the reported correlations. The additional reward-shaping baselines are also appreciated.

I have small concerns about the length-penalty baselines as they appear to produce extremely short responses (24-89 tokens on RULER), and the weaker penalty results in shorter responses than the stronger one. This suggests some training instability and that better performance may be possible with better hyper-parameters, although the setup may still be representative of a standard approach to length-penalty reward shaping.

Overall, the additional experiments have strengthened the empirical evidence for GEAR, and I am happy to increase my score from 2 to 3.

Thank you very much for raising the score! We agree with your observation about instability in length-penalized training. One possible reason may be that a simple length penalty provides a broad incentive to shorten responses, which may also penalize necessary, high-quality reasoning. However, GEAR was developed to address repetitive copying, a common issue in long-context models identified through our analysis. It therefore offers a more targeted objective for improving long-context capabilities, which can help make training more effective. We will investigate the causes of this instability and provide a more detailed analysis in future versions.

Thanks once again for your encouraging feedback. Have a great day!

Reviewer_ifN6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper examines copying from the input into reasoning traces on GSM-Infinite, using seven models and contexts from 8k to 64k tokens. Incorrect responses tend to contain more copying and, for most models, less selective use of relevant evidence. The proposed GEAR reward adds a bonus for n-gram overlap with key evidence and a penalty for overlap with distractors to the answer-correctness reward. The authors build 3,200 annotated examples from synthetic tasks and natural documents, then train three Qwen3.5 models with GSPO on 16k to 32k contexts. GEAR improves average performance over accuracy-only GSPO across five held-out benchmarks, including evaluation sets extending to 128k. On two benchmarks, it also reduces input overlap and reasoning length.

Critical strengths
  1. GEAR improves the average score for all three model sizes in both evaluation settings. Table 1 compares reward variants under the same stated data and GSPO setup. For the 9B model, the 32k average rises from 78.5 to 81.3, and the full 128k average rises from 70.2 to 74.8. The 35B-A3B and 27B models also improve. Achieving these gains across held-out benchmarks with 3,200 training examples makes the method practically interesting. The gains on inputs longer than those used in training also support its ability to transfer.

  2. The analysis gives a useful account of how copying can go wrong. The GSM-Infinite study separates evidence overlap from distractor overlap, and Appendix A.4 checks the copying-accuracy relationship within operation-count groups. Rewarding grounding alone lowers all six averages in Table 1; using the distractor penalty alone also underperforms accuracy-only GSPO in Table 3. With both terms, accuracy improves and copying falls. For the 35B-A3B model on RULER, Table 2 reports an overlap reduction from 35.7% to 27.0% and a reasoning-length reduction from 2,808 to 2,066 tokens. These ablations show why the behavioral effects of evidence-overlap rewards deserve attention.

Critical weaknesses
  1. I cannot tell how much the proposed reward adds over the closest existing grounding method. The paper discusses LongRLVR, whose evidence-selection F-score already balances relevant-evidence recall with selection precision. Table 1 compares GEAR with accuracy-only GSPO and its own reward components, so the grounding-only ablation does not answer this comparison. Scoring free-form reasoning with n-grams differs from requiring chunk identifiers, but the experiments do not test the benefit of that choice. A comparison with LongRLVR on the same annotated data, backbone, and training budget would help establish the contribution of this reward design. LongRLVR, Section 2.3.2

  2. The claim that GEAR works by improving selective grounding needs more direct evidence. The initial correlations, including the operation-count analysis, do not establish insufficient grounding as the root cause of copying. After training, Table 2 reports total input overlap and trace length, without separating evidence use from distractor copying. Eq. 1 measures the proportion of output windows that match the evidence; it does not measure how much of the necessary evidence is covered. Appendix A.3 includes both a non-truncated example and a token-exhaustion case, but these examples do not show how much of the overall gain comes from better grounding, shorter traces, or fewer truncated responses. An evidence/distractor breakdown after training and a generic repetition or length control would help resolve this issue.

Questions for authors
  1. How does GEAR compare with LongRLVR's evidence-selection reward on the same training mixture, backbone, and optimization budget? Please account for the different output interface. A matched evidence-coverage/precision reward on the existing traces would also help test the specific n-gram design.

  2. On held-out examples with evidence annotations, how do key-evidence overlap, distinct evidence coverage, and distractor overlap change after training? How much of the accuracy gain remains after accounting for token-budget exhaustion? A comparison with a generic length or repetition penalty at a similar reasoning-token budget would help separate improved grounding from reduced generation failure.

  3. Figure 2a labels 356 correct and 644 wrong samples. The counts in Table 5 sum to 1,050, and the rounded accuracies imply approximately 589 correct answers: 56.1%, compared with 35.6% in Figure 2a. Both describe DeepSeek-V3.2 on GSM-Infinite at 64k. Were different samples or filtering rules used, or does a figure or table need correction? This affects my interpretation of the diagnostic results.

  4. Please report the evaluation sample counts, decoding settings, uncertainty of the main differences, and the validation data used to choose alpha, beta, and n. Section 5.1 compares a 32k subset with the full set extending to 128k, with AA-LCR added only to the latter average. The larger gain therefore comes from an evaluation with a different task mixture as well as longer inputs. A paired comparison of the same problems with different distractor lengths would help assess whether the benefit increases with context length.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ifN6 (1/4)

Response to Reviewer ifN6

Thank you for your positive assessment of GEAR's performance across model scales and context lengths and our analysis of copying behavior, and for your constructive suggestions on baseline comparisons, mechanism analysis, and evaluation methodology. We address your concerns point by point below.


Q1. Contribution beyond the closest method (Weakness 1)

I cannot tell how much the proposed reward adds over the closest existing grounding method. The paper discusses LongRLVR, whose evidence-selection F-score already balances relevant-evidence recall with selection precision. Table 1 compares GEAR with accuracy-only GSPO and its own reward components, so the grounding-only ablation does not answer this comparison. Scoring free-form reasoning with n-grams differs from requiring chunk identifiers, but the experiments do not test the benefit of that choice. A comparison with LongRLVR on the same annotated data, backbone, and training budget would help establish the contribution of this reward design[1].

A1. Please refer to A3.


Q2. Evidence for the proposed mechanism (Weakness 2)

The claim that GEAR works by improving selective grounding needs more direct evidence. The initial correlations, including the operation-count analysis, do not establish insufficient grounding as the root cause of copying. After training, Table 2 reports total input overlap and trace length, without separating evidence use from distractor copying. Eq. 1 measures the proportion of output windows that match the evidence; it does not measure how much of the necessary evidence is covered. Appendix A.3 includes both a non-truncated example and a token-exhaustion case, but these examples do not show how much of the overall gain comes from better grounding, shorter traces, or fewer truncated responses. An evidence/distractor breakdown after training and a generic repetition or length control would help resolve this issue.

A2. Please refer to A4.


Q3. LongRLVR under the matched RL setting (Original question 1)

How does GEAR compare with LongRLVR's evidence-selection reward on the same training mixture, backbone, and optimization budget? Please account for the different output interface. A matched evidence-coverage/precision reward on the existing traces would also help test the specific n-gram design.

A3. We have added a LongRLVR comparison on Qwen3.5-35B-A3B, using the same training data, backbone, and RL-update budget as GEAR. We convert the evidence annotations in GEAR's data into the chunk IDs required by LongRLVR, retaining the original evidence labels. During training, the model is required to output evidence-ID and answer sections (the training reward requires this format), and we use the joint reward with the coefficients recommended in the original paper. At evaluation, we test the same LongRLVR checkpoint with both the adapted protocol and the ordinary benchmark protocol (without the added template). The results are shown below (Appendix B.2, Table 7).

MethodRULERLB-v2Bcomp-LCGraphWalksAA-LCRAverage
32k Context
LongRLVR, adapted protocol91.358.674.285.5—77.4
LongRLVR, ordinary protocol88.059.575.388.5—77.8
GSPO93.160.475.691.5—80.2
GSPO + GEAR96.261.375.695.9—82.3
128k Context
LongRLVR, adapted protocol88.753.068.878.157.069.1
LongRLVR, ordinary protocol84.256.071.485.767.072.8
GSPO86.455.468.889.667.073.4
GSPO + GEAR93.957.668.391.370.076.2

Under matched training settings, GEAR achieves higher benchmark averages than GSPO and both LongRLVR evaluation protocols at both context ranges. These results show that GEAR can apply its reward directly to the model's original free-form output and achieve stronger overall task performance without additional document chunking or an evidence-ID output protocol.


AuthorsOpenReview ↗

Response to Reviewer ifN6 (2/4)

Q4. Evidence use, truncation, and generic controls (Original question 2)

On held-out examples with evidence annotations, how do key-evidence overlap, distinct evidence coverage, and distractor overlap change after training? How much of the accuracy gain remains after accounting for token-budget exhaustion? A comparison with a generic length or repetition penalty at a similar reasoning-token budget would help separate improved grounding from reduced generation failure.

A4

On held-out examples with evidence annotations, how do key-evidence overlap, distinct evidence coverage, and distractor overlap change after training?

To answer your first question, we added a comparison of key-evidence overlap, distinct evidence coverage, and distractor overlap (Appendix B.4, Table 10). Specifically, using Qwen3.5-35B-A3B models trained with GSPO and GEAR, we computed key-evidence overlap (reasoning 3-gram windows matching key evidence as a fraction of all reasoning 3-gram windows), distinct evidence coverage (distinct key-evidence 3-grams matched in the reasoning as a fraction of all distinct key-evidence 3-grams), and distractor overlap (reasoning 3-gram windows matching distractor content as a fraction of all reasoning 3-gram windows) on the GSM-Infinite 64k evaluation set used in Figure 3.

Measure (%)GSPOGEAR
Key-evidence overlap26.010.5
Distinct evidence coverage73.885.8
Distractor overlap24.28.9

The results show that GEAR increases distinct evidence coverage by 12.0 percentage points while reducing distractor overlap from 24.2% to 8.9%, indicating broader coverage of key-evidence text together with less lexical overlap with distractors. Although key-evidence overlap decreases, the reasoning covers a larger fraction of key-evidence 3-grams, showing that the model covers a broader range of evidence with fewer evidence-matching windows, leaving more space for subsequent reasoning. Together, these metrics indicate that GEAR's reasoning output covers a broader set of evidence while reducing overlap with distractor content.

How much of the accuracy gain remains after accounting for token-budget exhaustion?

We added an output-budget comparison (Appendix B.5, Table 12): at 32k context length, we increased the maximum response budget for the Qwen3.5-35B-A3B GSPO and GEAR models from 16k to 64k and compared the changes in scores across four benchmarks. The results are shown below:

MethodRULERLongBench-v2BrowseComp-LCGraphWalksAvg
GSPO93.1→94.860.4→60.475.6→75.691.5→94.880.2→81.4
GSPO + GEAR96.2→96.861.3→61.375.6→75.695.9→97.882.3→82.9

Although increasing the response budget from 16k to 64k narrows GEAR's average accuracy advantage across the four benchmarks from approximately 2.1 to 1.5 percentage points, an additional analysis of mean response length on RULER (Appendix B.5, Table 13) shows a larger length advantage for GEAR:

Method16k budget: mean generated tokens64k budget: mean generated tokens
GSPO2,8544,396
GEAR2,1862,658
Relative reduction with GEAR23.4%39.5%

These results demonstrate GEAR's ability to produce more concise responses. Further analysis of responses truncated at the original 16k budget (Appendix B.5) shows that, on the same 1,040 RULER questions, 26 GSPO responses exhibited mechanical periodic repetition and exhausted the budget, compared with 9 for GEAR, approximately 3 times as many. After allowing up to 48k additional tokens from the original prefixes, 26/26 GSPO cases and 8/9 GEAR cases remained trapped in their original cycles and exhausted the budget again. This indicates that increasing the budget generally fails to rescue responses already trapped in a loop, while GEAR reduces such repetitive failures and wasted generation. Together with the preceding evidence-coverage and distractor-overlap analysis, these findings support GEAR's effectiveness in improving evidence use and reducing wasted generation.

A comparison with a generic length or repetition penalty at a similar reasoning-token budget would help separate improved grounding from reduced generation failure.

We added a decoding-time repetition-penalty control (Appendix B.6, Table 14, top): we set repetition penalty to 1.1 for the Qwen3.5-35B-A3B GSPO checkpoint and evaluated it in the 32k setting.

MethodRULERLongBench-v2BrowseComp-LCGraphWalksAvg
GSPO93.160.475.691.580.2
GSPO + repetition penalty93.963.075.292.081.0
GEAR96.261.375.695.982.3
AuthorsOpenReview ↗

Response to Reviewer ifN6 (3/4)

We also measured reasoning length, truncation, and internal repetition across all evaluation records (Appendix B.6, Table 14, bottom):

MethodMean reasoning tokensTruncation rate (%)Internal token 10-gram repetition (%)
GSPO (1.0)3,9954.218.7
GSPO + repetition penalty (1.1)3,6763.317.2
GEAR (1.0)3,1892.014.7

The results show that repetition penalty 1.1 helps improve GSPO's accuracy while reducing reasoning length and internal repetition. GEAR nevertheless performs better on these measures: its average score remains 1.3 percentage points higher, with shorter reasoning and lower truncation and internal repetition rates. This indicates that GEAR retains an advantage over a simple repetition penalty in reducing repetition.

Q5. Figure 2a and Table 5 inconsistency (Original question 3)

Figure 2a labels 356 correct and 644 wrong samples. The counts in Table 5 sum to 1,050, and the rounded accuracies imply approximately 589 correct answers: 56.1%, compared with 35.6% in Figure 2a. Both describe DeepSeek-V3.2 on GSM-Infinite at 64k. Were different samples or filtering rules used, or does a figure or table need correction? This affects my interpretation of the diagnostic results.

A5. Thank you for pointing out this error. We mistakenly included an intermediate development version as Figure 2a, which used a mixed dataset spanning 8k–128k rather than the stated 64k dataset. We have replaced Figure 2a with the plot corresponding to the Table 5 data, as shown in the updated PDF. After this correction, answer correctness and input overlap remain negatively correlated (Spearman ρ=−0.42, p=4.85×10−474.85\times10^{-47}), supporting the original analysis and insight. Please see the corrected Figure 2a (p. 4) and Appendix B.8 (Table 16) of the revised manuscript.


Q6. Evaluation, uncertainty, selection, and length (Original question 4)

Please report the evaluation sample counts, decoding settings, uncertainty of the main differences, and the validation data used to choose alpha, beta, and n. Section 5.1 compares a 32k subset with the full set extending to 128k, with AA-LCR added only to the latter average. The larger gain therefore comes from an evaluation with a different task mixture as well as longer inputs. A paired comparison of the same problems with different distractor lengths would help assess whether the benefit increases with context length.

A6.

Evaluation settings

The supplementary evaluation has the following entry counts:

BenchmarkSample count
32k Context
RULER1,040
LongBench-v2111
BrowseComp-LC287
GraphWalks450
AA-LCR—
128k Context
RULER1,498
LongBench-v2502
BrowseComp-LC580
GraphWalks650
AA-LCR100

Decoding uses temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.0, presence penalty 0, and a maximum output of 16,384 tokens.

Uncertainty of the main differences

To address this concern, we have added three-seed experiments for GSPO and GEAR on Qwen3.5-35B-A3B and report the mean and sample standard deviation below (Appendix B.1, Table 6 of the revised manuscript).

ModelMethodRulerLB-v2Bcomp-LCGraphwalksAA-LCRAvg
32k Context
Qwen3.5-35B-A3BGSPO92.2 ± 2.960.1 ± 0.575.4 ± 0.891.5 ± 0.3—79.8 ± 1.0
GSPO + GEAR95.9 ± 0.861.0 ± 1.475.7 ± 0.495.9 ± 0.1—82.1 ± 0.7
128k Context
Qwen3.5-35B-A3BGSPO85.7 ± 2.755.8 ± 0.968.3 ± 1.689.7 ± 0.868.0 ± 1.773.5 ± 1.4
GSPO + GEAR93.4 ± 1.457.4 ± 1.468.7 ± 0.791.2 ± 0.269.7 ± 0.676.1 ± 0.8

GEAR improves the aggregate benchmark score over GSPO by 2.4 and 2.6 percentage points at ≤32k and full ≤128k, respectively. Taken together, the cross-seed means and sample standard deviations of both methods provide further empirical support for the reliability of GEAR's performance gains across the three evaluated seeds.

AuthorsOpenReview ↗

Response to Reviewer ifN6 (4/4)

Validation data for hyperparameter selection

Regarding hyperparameter selection, we used RULER 32k as the validation benchmark to select α, β, and n on Qwen3.5-9B, choosing α=0.1, β=0.3, and n=3, and used the same settings across the other benchmarks and model scales. As shown in Tables 3 and 4, among the configurations tested in each ablation, the setting with the highest RULER 32k score also achieves the highest average across the other benchmarks, which were not used for hyperparameter selection. These results support the applicability of the selected hyperparameters across tasks.

Context length and task composition

Following your suggestion, we added a paired experiment on the same questions (Appendix B.7, Table 15) to examine the relationship between context length and GEAR's gains. We selected 220 synthetic questions from the RULER 32k bucket (input lengths of approximately 27k–32k tokens, forming a subset of the ≤32k evaluation set in Table 1). For each question, we kept the question, answer, and original evidence unchanged and extended the input to approximately 128k tokens by adding distractors. We evaluated models trained from Qwen3.5-35B-A3B on these paired inputs. The results are as follows:

MethodScoreScore drop from 32k
32k Context
GSPO91.3—
GSPO + GEAR99.6—
128k Context
GSPO86.74.6
GSPO + GEAR96.82.7

GEAR's score drops by 2.7 points, compared with 4.6 for GSPO. This shows that GEAR better handles context expansion caused by additional distractors.


Thanks for your careful reading and detailed review. Please let us know if there is more to clarify. We are happy to take your further questions.

[1] Shen et al. QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management, https://arxiv.org/abs/2512.12967

Reviewer_FEM2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies input copying in long-context reasoning. An analysis of seven models on GSM-Infinite shows that n-gram overlap between the input and the reasoning trace increases with context length. Correct responses also tend to have a higher ratio of overlap with key evidence to overlap with distractors. Based on these observations, the authors propose GEAR, which supplements the answer-correctness reward by rewarding lexical overlap with key evidence and penalizing overlap with distractors. The authors train three Qwen3.5 models with GSPO using 3,200 synthetic and natural-language examples. GEAR achieves the highest average score across five benchmarks in all six model-and-context-length settings, improving over accuracy-only GSPO by up to approximately 4.6 points. The paper also reports lower input-copying rates and shorter reasoning traces in some settings.

Critical strengths
  1. The problem is practically relevant, and distinguishing what gets copied is useful.

The paper distinguishes key evidence from distracting text instead of treating all input copying as harmful. This better reflects the needs of long-context tasks than simply shortening reasoning traces or reducing overall overlap.

  1. The reward is simple, transparent, and easy to integrate.

Given evidence annotations, the reward can be computed directly from n-gram statistics without an additional online verifier. Excluding n-grams shared by the evidence and distractor regions from the penalty is also a sensible design choice.

  1. The evaluation has good coverage and considers some alternative explanations.

The analysis covers seven models, four context lengths, and two levels of overlap granularity: 3-grams and 10-grams.

Critical weaknesses
  1. The claim that “insufficient grounding is the root cause” goes beyond the evidence.

Overlap differences between correct and incorrect responses, along with correlations after stratifying by the number of operations, do not establish the direction of causality. Retrieval failures, model uncertainty, or other reasoning difficulties could jointly lead to weaker grounding, longer outputs, and incorrect answers. GEAR’s effectiveness shows that the reward intervention is useful, but does not by itself establish the full causal mechanism proposed by the authors. Controlled interventions on the same questions, along with separate analyses of responses that were not truncated and responses of similar length, would strengthen the claim.

  1. Lexical overlap is a limited measure of semantic grounding.

Eq. (1) measures verbatim copying, but cannot tell whether copied evidence is used correctly or recognize evidence used through paraphrasing. It also does not directly measure repetition within the output. Figure 7 illustrates this limitation: a response has 0% input overlap but keeps repeating candidate words until it exhausts the token budget. A lower copying rate therefore does not, by itself, demonstrate less repetitive output or better semantic grounding.

  1. The assumption that all remaining text is distracting in natural-language data is not sufficiently validated.

The pipeline verifies that a question can be answered using only the selected 1–3 chunks. This establishes that those chunks are sufficient, but does not show that everything else is irrelevant. Other regions may contain alternative evidence, paraphrases, or qualifications that affect the answer. Excluding shared n-grams only addresses lexical overlap; it does not prevent semantically relevant content from being incorrectly penalized.

  1. The experiments lack comparisons with the closest evidence-aware rewards and simpler controls.

The related work discusses LongRLVR’s chunk-level F-score reward and LongR’s contextual density reward, but the main experiments primarily compare accuracy-only GSPO with GEAR and its component variants. The results therefore do not establish whether the gains come from adding evidence supervision in general or from GEAR’s specific n-gram reward design. Comparisons with other evidence-based rewards using the same data, backbone models, and training budget, alongside generic length and copying penalties, would make the contribution much clearer.

Questions for authors

please refer to the weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer FEM2 (1/4)

Response to Reviewer FEM2

Thank you for recognizing the practical relevance of the problem, the value of distinguishing key evidence from distractors, and the simplicity of our reward design, and for your constructive suggestions on causal evidence, lexical metrics, annotation validity, and baseline comparisons. We address your concerns point by point below.


Q1. Causal root-cause claim (Weakness 1)

The claim that “insufficient grounding is the root cause” goes beyond the evidence.

Overlap differences between correct and incorrect responses, along with correlations after stratifying by the number of operations, do not establish the direction of causality. Retrieval failures, model uncertainty, or other reasoning difficulties could jointly lead to weaker grounding, longer outputs, and incorrect answers. GEAR’s effectiveness shows that the reward intervention is useful, but does not by itself establish the full causal mechanism proposed by the authors. Controlled interventions on the same questions, along with separate analyses of responses that were not truncated and responses of similar length, would strengthen the claim.

A1. To further examine whether evidence localization helps mitigate repetitive generation, we conducted a same-question intervention (Appendix B.3, Table 9) on the GSM-Infinite 64k questions used in Section 3, using the original Qwen3.5-35B-A3B model (we initially planned to use DeepSeek-V3.2, as in Section 3, but it is unaccessible now through API service). We retained the full context and compared three input conditions: A uses the original input; B marks the annotated key evidence with <focus>...</focus>; C marks unannotated facts matched in number, position, and total marked length. B and C receive the same instruction to focus on the marked passages. We set a generous total output budget of 128k. The results are below.

MetricA: original inputB: key-evidence markersC: control markers
Accuracy60.0%79.2%70.0%
Evidence coverage65.9%71.7%65.4%
Internal repetition64.1%54.1%58.5%
Mean reasoning tokens67,66445,23356,257
Truncation rate37.5%20.0%28.8%

Evidence coverage is the fraction of distinct token 10-grams in the annotated evidence that appear in the model's reasoning; internal repetition is the fraction of whitespace-tokenized word 10-gram windows in the reasoning that repeat an earlier window.

Compared with A, B covers more evidence while reducing repetition and reasoning length and improving accuracy. Specifically, evidence coverage increases by 5.8 percentage points, internal repetition decreases by 10.0 points, mean reasoning length decreases by 33.2%, and accuracy increases by 19.2 points. With a generous output budget of 128k, B reduces the truncation rate by 17.5 percentage points and increases accuracy by 19.2 points, indicating that evidence localization helps more responses finish with a correct answer within the budget. C also shows higher accuracy and lower internal repetition than A. One possible explanation is that an explicit reading cue provides general grounding guidance, helping the model organize its reasoning around the context. B's additional improvement over C indicates the value of accurately locating key evidence beyond this general guidance. These same-question interventions show that helping the model locate key evidence can reduce repetitive generation, providing further support for insufficient grounding as a contributing factor in repetitive copying.

Q2. Lexical overlap and semantic grounding (Weakness 2)

Lexical overlap is a limited measure of semantic grounding.

Eq. (1) measures verbatim copying, but cannot tell whether copied evidence is used correctly or recognize evidence used through paraphrasing. It also does not directly measure repetition within the output. Figure 7 illustrates this limitation: a response has 0% input overlap but keeps repeating candidate words until it exhausts the token budget. A lower copying rate therefore does not, by itself, demonstrate less repetitive output or better semantic grounding.

AuthorsOpenReview ↗

Response to Reviewer FEM2 (2/4)

A2. We address three concerns raised in this comment:

  1. Whether evidence is used correctly: Copying key evidence does not imply that the model understands it or uses it correctly in reasoning.
  2. Whether paraphrased evidence is missed: The model may use evidence through paraphrasing, which lexical overlap cannot recognize.
  3. Whether the output still repeats itself: Input-output overlap does not directly measure repetition within the output.

1. Whether evidence is used correctly.

Eq. (1) measures verbatim copying, but cannot tell whether copied evidence is used correctly

We agree with the reviewer that long-context reasoning requires both locating relevant evidence in the context and combining that evidence to reason correctly. This is also a shared understanding in long-context reasoning research [1, 2]. Our central insight is that, even for models with strong reasoning capabilities, evidence localization can remain a bottleneck in long-context tasks: models spend substantial reasoning budgets repeatedly copying contextual content without effectively focusing on key evidence. We therefore propose GEAR to reward attention to key evidence and discourage copying from distractors.

We agree that lexical overlap alone cannot guarantee correct evidence use. This signal reflects whether the model's copying focuses on relevant evidence; its training value must be evaluated through actual task performance. The same-question intervention in A1 (Appendix B.3, Table 9) shows that guiding the model to locate key evidence increases accuracy from 60.0% to 79.2% while reducing internal repetition from 64.1% to 54.1%. The paper's main experiments further show that GEAR improves overall long-context task performance across model scales while reducing inference costs.

Combining located evidence to reason correctly is another capability that needs improvement. Work such as QwenLong-L1.5 [3] explores this direction by constructing multi-hop reasoning training tasks, complementing our focus. Our results show that strengthening evidence localization can already yield substantial gains in the long-context capabilities of existing reasoning models.

2. Whether paraphrased evidence is missed.

or recognize evidence used through paraphrasing.

We agree that lexical matching cannot fully recognize evidence used through paraphrasing, so the GEAR reward cannot cover all valid expressions of evidence. Nevertheless, A1 shows that guiding the model to attend to key evidence can reduce repetitive generation and improve task performance; the paper's main experiments further show that the GEAR reward, computed through simple n-gram matching, already improves long-context task performance and reduces inference costs across model scales. These results show that, even without recognizing all semantically equivalent expressions, evidence signals obtained through simple lexical matching can already provide effective training guidance.

3. Whether the output still repeats itself.

It also does not directly measure repetition within the output. Figure 7 illustrates this limitation: a response has 0% input overlap but keeps repeating candidate words until it exhausts the token budget.

We measured the proportion of repeated token 10-gram windows within the reasoning outputs of the Qwen3.5-35B-A3B GSPO/GEAR models on RULER and LongBench-v2, with the following results (Appendix B.4, Table 11):

BenchmarkGSPOGSPO + GEAR
RULER18.5%13.7%
LongBench-v213.4%8.7%

These results show that GEAR also reduces repetition within model outputs.


Q3. Validity of natural-document evidence labels (Weakness 3)

The assumption that all remaining text is distracting in natural-language data is not sufficiently validated.

The pipeline verifies that a question can be answered using only the selected 1–3 chunks. This establishes that those chunks are sufficient, but does not show that everything else is irrelevant. Other regions may contain alternative evidence, paraphrases, or qualifications that affect the answer. Excluding shared n-grams only addresses lexical overlap; it does not prevent semantically relevant content from being incorrectly penalized.

AuthorsOpenReview ↗

Response to Reviewer FEM2 (3/4)

A3. We agree with the reviewer's concern: the selected passages being sufficient to support the answer does not imply that the rest of the document is irrelevant. In natural documents, paraphrases of the evidence may occur in unannotated regions, so the automatically constructed negative reward signal can contain errors.

While acknowledging this limitation, we would like to emphasize that the main contributions of this paper are to identify and analyze repetitive copying in long-context reasoning and to introduce GEAR, which uses evidence-related rewards to guide improvements in reasoning behavior. To support this training objective, we designed automatic evidence-annotation schemes for synthetic tasks and natural documents, respectively. We use these two schemes to automatically construct training data with evidence annotations, then compute GEAR rewards from those annotations to train the models. Even with automatically constructed data whose annotations may contain errors, GEAR improves aggregate performance on long-context reasoning benchmarks, demonstrating that the proposed reward design is already effective under the current data conditions. Further improving evidence-annotation quality through semantic analysis or human annotation and reducing inappropriate penalties on relevant content may enable this reward mechanism to deliver greater gains. This paper focuses on proposing and validating the reward design; developing and systematically evaluating higher-quality annotation procedures is beyond its scope and is left for future work.

Q4. Evidence-aware baselines and simpler controls (Weakness 4; questions refer to weaknesses)

The experiments lack comparisons with the closest evidence-aware rewards and simpler controls.

The related work discusses LongRLVR’s chunk-level F-score reward and LongR’s contextual density reward, but the main experiments primarily compare accuracy-only GSPO with GEAR and its component variants. The results therefore do not establish whether the gains come from adding evidence supervision in general or from GEAR’s specific n-gram reward design. Comparisons with other evidence-based rewards using the same data, backbone models, and training budget, alongside generic length and copying penalties, would make the contribution much clearer.

please refer to the weaknesses

A4. To address the evidence-aware rewards and simpler controls in your question, we added separate comparisons of training-time rewards and a test-time repetition penalty.

Training-time reward comparisons. We added LongRLVR and length-penalty rewards on Qwen3.5-35B-A3B, using the same training data, backbone, and RL-update budget as GEAR. LongRLVR is trained with chunked inputs and evidence-ID outputs; the same checkpoint is evaluated under both the adapted protocol and the ordinary benchmark protocol. The length penalties use λ=0.1 and λ=0.01, following the reward definition and token-counting convention in Appendix B.2 (Eq. 4). The comparison results are shown below (Table 7).

MethodRULERLB-v2Bcomp-LCGraphWalksAA-LCRAverage
32k Context
LongRLVR, adapted protocol91.358.674.285.5—77.4
LongRLVR, ordinary protocol88.059.575.388.5—77.8
Length penalty, λ=0.194.858.669.388.7—77.8
Length penalty, λ=0.0191.853.271.157.7—68.4
GSPO93.160.475.691.5—80.2
GSPO + GEAR96.261.375.695.9—82.3
128k Context
LongRLVR, adapted protocol88.753.068.878.157.069.1
LongRLVR, ordinary protocol84.256.071.485.767.072.8
Length penalty, λ=0.193.750.867.684.947.068.8
Length penalty, λ=0.0191.749.065.355.231.058.5
GSPO86.455.468.889.667.073.4
GSPO + GEAR93.957.668.391.370.076.2

GEAR achieves benchmark averages of 82.3 and 76.2 at the 32k and 128k context ranges, respectively, exceeding both LongRLVR evaluation protocols and both length-reward controls. We additionally compare mean complete-response lengths on RULER ≤32k (Appendix B.2, Table 8):

MethodAvg. response length (tokens)
LongRLVR, adapted protocol2,438
LongRLVR, ordinary protocol2,744
Length penalty, λ=0.189
Length penalty, λ=0.0124
GSPO2,853
GSPO + GEAR2,185

The two length rewards compress mean responses to 89 and 24 tokens without matching GEAR's overall score. GEAR averages 2,185 tokens, below GSPO's 2,853 tokens and both LongRLVR protocols. These training-time controls show that GEAR combines shorter responses with higher task scores in this comparison, supporting the value of its reward design beyond the component ablations.

AuthorsOpenReview ↗

Response to Reviewer FEM2 (4/4)

Test-time repetition-penalty control. We set repetition penalty to 1.1 on the trained Qwen3.5-35B-A3B GSPO checkpoint and evaluate it in the 32k setting (Appendix B.6, Table 14, top). This control changes only decoding, without updating model parameters.

MethodRULERLongBench-v2BrowseComp-LCGraphWalksAvg
GSPO93.160.475.691.580.2
GSPO + repetition penalty93.963.075.292.081.0
GEAR96.261.375.695.982.3

We also measure reasoning-segment length, truncation, and internal repetition across all evaluation records from the four main benchmarks in the 32k setting, with the results shown below (Appendix B.6, Table 14, bottom).

MethodMean reasoning tokensTruncation rate (%)Internal token 10-gram repetition (%)
GSPO (1.0)3,9954.218.7
GSPO + repetition penalty (1.1)3,6763.317.2
GEAR (1.0)3,1892.014.7

Repetition penalty 1.1 raises GSPO's average score from 80.2 to 81.0 and reduces mean reasoning length, truncation, and internal repetition. GEAR's average score remains 1.3 percentage points higher, with shorter reasoning and lower truncation and internal repetition rates. This shows that simple test-time repetition suppression can improve GSPO, while GEAR retains an advantage on the reported task and behavioral measures. These results provide further support for the advantage of GEAR's reward design over other evidence-based rewards and generic length and repetition controls.


Thanks again for your detailed comments. Please let us know if there is more to clarify.

References

[1] Kuratov et al. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack, https://arxiv.org/abs/2406.10149

[2] Zhou et al. GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity? https://arxiv.org/abs/2502.05252

[3] Shen et al. QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management, https://arxiv.org/abs/2512.12967

Comments on the paper

1 comment

We thank the reviewers for their constructive feedback. In the revised manuscript, we have replaced Figure 2a with the plot corresponding to the Table 5 data (thanks to the careful review by reviewer ifN6) and added Appendix B to report the supplementary experiments and analyses. These additions include:

  • three-seed GSPO/GEAR results (Appendix B.1, Table 6);
  • LongRLVR and length-penalty baselines, including the length-penalty reward definition and response-length comparisons (Appendix B.2, Eq. 4, Tables 7–8);
  • a controlled evidence-localization intervention (Appendix B.3, Table 9);
  • analyses of evidence use and internal repetition after training (Appendix B.4, Tables 10–11);
  • output-budget and truncation-pattern analyses (Appendix B.5, Tables 12–13);
  • a decoding-time repetition-penalty control (Appendix B.6, Table 14);
  • paired context-extension results (Appendix B.7, Table 15);
  • and Spearman correlation tests for the diagnostic figures (Appendix B.8, Table 16).

The new appendix text and tables are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.