acceptodds
Under review as a conference paper at ICLR 2027

Tracing Repetitive Degeneration to the Post-Training Pipeline

Abstract

Large Language Models (LLMs) have achieved strong performance across diverse tasks. However, LLMs occasionally suffer from output repetition, which degrades output quality and wastes computational resources. This issue is widely observed, but difficult to reliably reproduce, leaving its underlying mechanisms underexplored. In this work, we present a controlled empirical study of how post-training choices are associated with this phenomenon. Firstly, to enable controlled analysis, we present the Self-Reinforced Loop Approach (SRLA), which amplifies output repetition and makes this otherwise sporadic phenomenon reproducible. Using SRLA, we find that models trained with recent reward-based post-training (e.g., GRPO) are more susceptible to repetition than conventional ones. This observation motivates our intuition that output repetition is closely associated with post-training choices, which we validate through cross-model comparisons. A weaker but consistent increase remains observable in models trained on GRPO-distilled data. Analyzing token-level distributions, we identify three crucial factors, including selected-token lock-in, insufficient candidate token diversity, and weak EOS activation. Motivated by these findings, we propose two mitigations that significantly reduce failure rates by an average of 86%. These findings bridge post-training and repetitive generation behaviors, and provide practical guidance for developing more robust models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
post-training, repetition, attribution

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UJmeOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work studies the repetition behaviour of LLMs at the sampling stage. Author first build a dataset by collecting the repetition samples from a post-trained open source model and then use this dataset to further study the reason/dynamic behind. Authors also found this problem is due to the GRPO style post training. Finally authors identified the root cause of this behaviour and then purposed two mitigations that significantly reduce failure rates.

Critical strengths
  1. Authors offers a very in-depth study on formulating the problem, understanding the root cause and then propose some simple and clean solutions. The progress of critical thinking and reasoning are also valuable.
  2. This repetition is indeed widely observed and have a major impact on user experience.
Critical weaknesses
  1. Does this behaviour fix have any impact on the model quality effect? I saw some results from Table 17 but these evals are all quite outdated and toy. Could you please try at least a few cutting edge more complicated evals such as the agent coding on relative larger models? (I understand these smaller models may be a bit struggle on the agentic tasks)
  2. IIUC, authors mainly focus on the repetition on vanilla decoding tokens. A straightforward question is, how is this interact with the tool use tokens and would this induce any positive or negative behavior for these long decoding tasks? (authors don't have to resolve this question in the paper or rebuttal but would be nice to write some discussion in somewhere of the paper)
Questions for authors

Minor: Are there any specific order to run these fixes verse the training stage. Is it always fine to just have a post-hoc fix or it is better to do it as early as possible?

2 comments

Thank you very much for your positive evaluation and your generous recognition of our problem formulation, root-cause analysis, and critical reasoning. We are especially encouraged that you found our study valuable and our proposed training solutions clean and effective. Your insightful question regarding harder agentic tasks prompted us to evaluate more challenging mathematics and coding benchmarks, which we address below.

W1. Impact on model quality

Table 18 (Appendix G.2) reports standard task accuracy and pass rates on MMLU, GSM8K, IFEval, and HumanEval. To assess whether the repetition mitigations preserve model quality on more challenging reasoning and coding tasks, we additionally evaluated answer accuracy on MATH-500 and code execution pass@1 on BigCodeBench-Instruct.

ModelMATH-500 accuracy (%)BigCodeBench-Instruct pass@1 (%)
Qwen DPO80.0023.68
Qwen DPO→GRPO77.6023.86
Multi-task80.8025.35
Concise80.2023.60
Meta-Llama-3-8B-Instruct31.0032.11

Note: Meta-Llama-3-8B-Instruct is included as an additional reference model.

Both mitigations score above DPO baseline on MATH-500 ( +0.80 and +0.20 percentage points), and on BigCodeBench-Instruct, Multi-task achieves the highest pass rate among the four DPO-derived models (+1.67 points) while Concise remains within 0.08 points of DPO baseline. Reducing repetitive degeneration therefore comes with minimal differences on these harder reasoning and coding benchmarks. These coding results cover single-response code generation at 7B; performance on multi-turn coding tasks at larger scales remains open.

W2. Interaction with tool-use tokens

Thank you for raising this direction. Our BigCodeBench-Instruct evaluation covers single-response code generation, which leaves the effect of our mitigations on multi-turn tool use open. Chen et al. (2026) found that compressing the interaction history made agents more likely to repeat a tool call and receive the same result. We will acknowledge this limitation and consider studying how call arguments, tool results, and eventual task success can distinguish a productive retry from a repetitive loop and whether our mitigations affect tool-call retries in future work.

Q. Order of the fixes relative to training

Both mitigations are applied independently during the GRPO stage—one modifying the reward and the other rebalancing the data mixture (§5.3). We train the two variants separately: Concise rewards shorter correct responses, and Multi-task changes the mixture of training tasks (§5.3).

Earlier intervention is also possible. For example, the Qwen3 team filtered repetitive reasoning traces from its cold-start data before GRPO (Qwen3 Technical Report, §4.1). In our Qwen pipeline, RDR was 14.76% after DPO and rose sharply to 100.00% after GRPO (Appendix C.2, Table 14), so we targeted the stage where we observed that increase. We have not compared earlier intervention with a post-hoc fix on an already trained model, and we would welcome further discussion of that question.

Once again, we sincerely thank you for your encouraging and inspiring feedback. We will highlight your valuable perspectives on multi-turn tool calling and training-stage comparisons as promising directions for future work.

Chen, T., Singh, P., Todi, K., Paria, B., He, Q., Wang, W., ... & Akula, A. (2026). DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents. arXiv preprint arXiv:2610.19376.

Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., ... & Qiu, Z. (2025). Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Thank you for the reply. Looks good to me and am happy to keep my scores positive

Reviewer_W9PcOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the relationship between post-training choices and repetitive degeneration in language models. It introduces the Self-Reinforced Loop Approach (SRLA), which appends model-generated repetitive seeds to task prompts to enable controlled stress testing. The authors compare models before and after an additional GRPO stage and examine publicly released reasoning-distilled models. They characterize generation using selected-token lock-in, effective candidate-set size, and EOS activation. Two training interventions, a conciseness reward and multi-task diversification, reduce the reported repetitive degeneration rate in the evaluated setting.

Critical strengths
  1. The paper addresses a practically relevant reliability problem: repetitive generation that exhausts the token budget. Studying this behavior is useful for understanding failures that degrade response quality and waste inference resources.

  2. SRLA provides a shared elicitation setting for comparing model susceptibility under fixed decoding conditions. The before-and-after comparisons using the same starting checkpoint are more informative about the added training stage than comparisons between unrelated public models alone.

  3. The paper evaluates two concrete training interventions rather than only reporting failure cases. Their different changes in EOS activation and candidate diversity provide useful evidence for investigating distinct mitigation paths.

Critical weaknesses
  1. The Related Work section has only one subsection heading, which seems to be an oversight.

  2. Why combine “reaching the maximum token length” and “ending with a repetitive loop” into a single metric? Could you report the proportions corresponding to each condition separately to further justify the design of RDR?

  3. All SRLA repetitive seeds are generated by Qwen and then used to evaluate other models. I believe this is a potential weakness, as different models may have different sensitivities to different repeated tokens. In the absence of additional experiments, each model under evaluation—such as Llama, Qwen, and DeepSeek—should generate its own set of repetitive seeds.

  4. Why does the standard capability check not report task accuracy?

  5. Has the use of EOS masking to induce repetition been validated by prior work? If not, why not use a repetition-induction method that has already been validated in other papers? How can you show that directly masking EOS produces genuine repetition rather than simply disrupting model performance? This issue also directly affects the EOS-based mitigation.

If you can address these concerns, I'll raise the score.

Questions for authors

You only test models around 7B parameters. Does this mean that larger models do not suffer from repetition problems?

4 comments

Thank you for your constructive review and helpful suggestions on how we elicit, measure, and mitigate repetitive degeneration. We address each point below.

W1. Related Work

Thank you for pointing this out. Section 2 now opens with the heading “Repetitive Degeneration in LLMs” to make the progression to “Mechanism Analyses and Post-Training Effects” clearer.

W2. Why combine a repetitive ending with reaching the token limit?

A model can repeat a passage several times and still emit EOS on its own. The failure we aim to count is runaway repetition in which a model keeps repeating until the token limit cuts it off. We therefore require both reaching the 4,096-token limit and finding the same six-token sequence at least four times in the final quarter of the answer (Appendix B.1). Counting length alone would also include a model that was still explaining its answer at the limit. Generating redundant solutions after reaching the correct answer can be a form of overthinking(Chen et al., 2025), the loop condition plays an actual role and better aligns with our claim of repetitive degeneration (§3.2); this is why we adopt the Repetitive Degeneration Rate (RDR) as the metric.

For the unmitigated Qwen DPO→GRPO model in the mitigation comparison (§5.4, Figure 4) evaluated on Deval\mathcal{D}_{\text{eval}} under the decoding settings of §3.2, 21.64% of all responses ended with a qualifying repetitive pattern: 5.13% repeated but stopped on their own, while the remaining 16.51% were still repeating at the token limit. Only the latter 16.51% count toward RDR, and the share of responses that reached the token limit without a qualifying loop was 0.00%.

On Deval\mathcal{D}_{\text{eval}}, we break down the proportions of responses with a repetitive ending and responses reaching the token limit across the unmitigated, mitigated, and reference models:

ModelTotal repetitive endingRepeated but stopped earlyRepeated at token limit (RDR)Reached limit without loop
Qwen DPO→GRPO21.64%5.13%16.51%0.00%
Concise4.20%1.08%3.12%0.00%
Multi-task2.21%0.80%1.41%0.00%
Meta-Llama-3-8B-Instruct0.60%0.60%0.00%0.00%

Note: Meta-Llama-3-8B-Instruct is included as an additional reference model.

Alongside the mitigation comparison on Deval\mathcal{D}_{\text{eval}} (§5.4, Figure 4), we also report two published metrics evaluated across all five models (Appendix G.1, Table 17). CREP reports the percentage of responses in which a repeated pattern covers at least 30% of the tokens (Lee et al., 2026). MAUVE measures distributional similarity between model responses and short reference answers (Pillutla et al., 2021).

ModelRDR (%)MAUVE ↑CREP (%) ↓
Qwen DPO→GRPO16.510.0072517.27
Concise3.120.0045414.55
Multi-task1.410.004277.47
Qwen2.5-7B-Instruct0.000.005340.00
Meta-Llama-3-8B-Instruct0.000.005810.00

Note: Meta-Llama-3-8B-Instruct is included as an additional reference model.

Both mitigations reduce the share of responses dominated by repetition (CREP), consistent with the reduction in RDR.

W3. Sensitivity to the seed model

We agree that evaluating a model on its own repetitive text is a valuable control. Starting from the same pool of multiple-choice questions, we applied the construction in §3.1 with three different seed models. Within each column, all five target models received the same retained prompts. The table reports RDR.

ModelQwen2.5-7B seedsLlama-3-8B-Instruct seedsDeepSeek-LLM-7B-Chat seeds
Qwen SFT2.22%3.33%3.89%
Qwen SFT→GRPO7.78%12.78%17.22%
Llama-3-8B-Instruct0.00%0.00%0.00%
Llama-GRPO2.22%1.67%2.78%
R1-Distill-Llama-8B1.11%0.56%1.11%

The RDR increase after GRPO holds across Qwen, Llama, and DeepSeek seeds for both model families, and Llama-3-8B-Instruct stays at 0.00% RDR, consistent with the training-stage comparison in Table 2.

W4. Task accuracy

Table 18 (Appendix G.2) evaluates MMLU, GSM8K, IFEval, and HumanEval. Following your suggestion, we added MATH-500 for competition mathematics and BigCodeBench-Instruct for code generation from natural-language instructions. All scores are percentages.

ModelsMMLUGSM8KIFEval (strict prompt accuracy)HumanEval pass@1MATH-500BigCodeBench-Instruct pass@1
Qwen DPO58.0889.9921.2648.7880.0023.68
Qwen DPO→GRPO59.4290.4524.2153.0577.6023.86
Multi-task58.1290.3021.2650.0080.8025.35
Concise58.1089.6920.8945.1280.2023.60
Meta-Llama-3-8B-Instruct66.7275.9768.5859.7631.0032.11

Note: Meta-Llama-3-8B-Instruct is included as an additional reference model.

Both mitigations reduced repetition while preserving performance on the two harder tasks we tested. Relative to DPO baseline, Multi-task scored +0.80 percentage points higher on MATH-500 and +1.67 points higher on BigCodeBench-Instruct, and Concise scored +0.20 points higher on MATH-500 and scored only 0.08 points lower on BigCodeBench-Instruct.

W5. EOS masking during seed construction

EOS masking is used only during seed extraction (§3.1) and never during target-model evaluation: all target models generate with their EOS token fully enabled throughout (§3.2). In §3.1, masking EOS serves a single purpose: it allows the seed model to continue past a point where it would otherwise stop, giving a repeated pattern time to appear while the model still chooses which non-EOS tokens to generate. Only continuations that actually contain a repeated pattern supply a seed. Prior work shows that repeated context encourages further repetition (Yan et al., 2024; Xu et al., 2022); we use EOS masking to collect such non-adversarial contexts (§3.1).

The target models then see the resulting prompt as ordinary text and generate with EOS enabled throughout (§3.2). One might worry that the EOS masking used only during seed extraction could artificially induce the repetitive patterns later observed in evaluation. Three results in the paper show that EOS masking is only for seed extraction and does not manufacture the evaluation findings:

  • Table 9: Qwen2.5-Math-7B-Instruct already loops on ordinary questions without any SRLA suffix (25.74% RDR on Base_mcq), and the rate rises to 91.87% on SRLA prompts, whereas Llama-3-8B-Instruct maintains 0.00% RDR on both sets under the same decoding settings.
  • Appendix A.2 (Table 6): The resulting SRLA prompts exhibit low perplexity under the evaluated models’ own native distributions.
  • Appendix B.1: Repetition in target-model outputs is identified by a fixed criterion (a six-token sequence occurring at least four times in the final quarter) verified against human judgment via manual spot-checks.

The conciseness reward likewise acts only during training. With EOS available throughout evaluation, the Concise model’s mean final-quarter EOS probability rises from 0.0132 to 0.0340 (Table 5).

Q. Can larger models also loop?

Yes—larger models can also loop. Pipis et al. (2026, Wait, Wait, Wait… Why Do Reasoning Models Loop?) observed looping in 32B reasoning models (e.g., 37% of responses for Qwen-32B at low temperature (§2)). Our controlled training-stage comparisons cover 7B–8B models, and we will test the effect of adding GRPO at larger scales in future work.

Chen, X., Xu, J., Liang, T., He, Z., Pang, J., Yu, D., ... & Yu, D. (2025, July). Do NOT think that much for 2+ 3=? on the overthinking of long reasoning models. In Forty-second International Conference on Machine Learning.

Xu, J., Liu, X., Yan, J., Cai, D., Li, H., & Li, J. (2022). Learning to break the loop: Analyzing and mitigating repetitions for neural text generation. Advances in Neural Information Processing Systems, 35, 3082-3095.

Pipis, C., Garg, S., Kontonis, V., Shrivastava, V., Krishnamurthy, A., & Papailiopoulos, D. Wait, wait, wait... why do reasoning models loop?. ICML 2026.

Lee, J., Park, S., Jang, S., Ryu, S., Kim, H., Lee, H., ... & Kim, Y. (2026). FOCUS & RePAIR: Mitigating Text Degeneration via Token-Level Guidance for Pruned Large Language Models. arXiv preprint arXiv:2608.26676.

Pillutla, K., Swayamdipta, S., Zellers, R., Thickstun, J., Welleck, S., Choi, Y., & Harchaoui, Z. (2021). Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances in Neural Information Processing Systems, 34, 4816-4828.

Yan, J. E., Xu, J., Song, C., Wu, C., Li, Y., & Zhang, Y. (2024, May). Understanding in-context learning from repetitions. In International Conference on Learning Representations (Vol. 2024, pp. 49968-49989).


Dear Reviewer, Thank you again for your valuable comments. As the discussion period is nearing its end, we would like to kindly ask whether our responses have adequately addressed your concerns. Please let us know if any further clarification is needed. If our responses have resolved your concerns, we would sincerely appreciate your consideration in updating your evaluations accordingly.

Thank you for your response, you have addressed my concerns. I have raised my score.

Reviewer_eVbAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines degradation issues in LLMs, in particular, for models trained or refined with GRPO for post-training. To this end, they proposed Self-Reinforced Loop Approach (SRLA), an evaluation method that extracts repetition patterns and injects them during the prompt construction to induce repetition, allowing quantifiable analysis for the degeneration phenomenon. With SRLA, they report that repetition issues can be exacerbated for the GRPO-trained models. Their analysis further identified three properties that can cause such degradation, (i) lock-in (ii) reduced candidate diversity, and (ii) weak EOS activation for selected tokens. They accordingly propose solutions to address these issues, including reward simplification and multi-task diversification.

Critical strengths
  • SRLA allows systematical evaluation/analysis for the repetition-related degeneration issues in a controlled manner.
  • By analyzing repetition rates based on SRLA, this paper provides insights about generative dynamics causing the degeneration, which otherwise would be hard to identify with simpler alternatives, e.g., output-level observations.
Critical weaknesses
  1. Their proposed SRLA method largely relies on the evaluation data samples built from their own procedure; it is not sufficiently clarified how they can obtain the data samples, e.g., which prompts are exactly used to induce the repetition and how they are designed/selected. Also, it's not clear that the SRLA-based method could outperform simpler alternatives. e.g., what if putting an arbitrary prompt repeatedly.
  2. They evaluated their methods on in-house custom models that they trained by applying the GRPO-based post training stage. Since the detailed settings that they used can also affect the metrics that they propose, e.g., RDR, it makes difficult differentiate if the reported issues came from the GRPO itself or their own setup. They didn't also evaluate whether the custom models were well trained as they didn't report common metrics used for evaluating GRPO-trained models. It should be clarified that the reported issues persist across the GRPO-trained models even when using other setup variants. Also, it would be helpful if they could share the official checkpoints.
Questions for authors
  1. In the evaluation, Qwen is used for the seed model. What if different models are used to build a dataset? For example, do the relative model rankings and RDR change?
  2. The proposed dataset construction procedure looks sophisticated; can the authors compare their proposed procedure with simpler alternatives?
  3. Table 11 suggests that the RDR changes significantly depending on the GRPO training hyperparameters. Can the authors also vary other training-related settings, e.g., using other corpus than the math-heavy ones? It will be helpful to understand if the reported degeneration issues indeed came from the GRPO or other potential factors related.
  4. It looks like the proposed mitigation methods suppress the degeneration issues. However, since the evaluation was mainly performed using their own metric, RDR, it would be helpful to report other publicly available metrics, e.g., MAUVE [1] and CREP [2], which measure the similar quantity. [1] Pillutla, Krishna, et al. "Mauve: Measuring the gap between neural text and human text using divergence frontiers." Advances in Neural Information Processing Systems 34, 2021 [2] Lee, Junyoung, et al. “FOCUS & RePAIR: Mitigating Text Degeneration via Token-Level Guidance for Pruned Large Language Models.” Proceedings of the Forty-Third International Conference on Machine Learning, 2026
3 comments

Thank you for your thoughtful review and constructive suggestions on the SRLA controls, seed sources, training settings, and evaluation metrics. We address each point in detail below.

W1 & Q2. SRLA construction and the simple loop comparison

We start from a multiple-choice question and its answer choices. Qwen2.5-7B generates a continuation with EOS excluded during seed construction (temperature 0.5, up to 512 tokens). Under this setting, not every continuation loops, and only continuations in which a repeated pattern actually appears supply a seed (§3.1). The extraction code takes the continuous span from the pattern’s first occurrence through the occurrence that triggers detection, including any intervening text. Each constructed SRLA prompt keeps the question and choices and appends a passage containing the loop. In some prompts, the suffix includes a brief explanation before the repeated answer. All target models then generate normally from the same SRLA inputs with EOS enabled, and we compute RDR on the resulting responses (§3.2).

As you suggest, we have also tested a simple loop that takes only the repeating segment at the end of each SRLA example. Changes are reported relative to the simple loop.

ModelSimple loop RDRFull SRLA RDRRelative change
Qwen DPO19.72%14.76%−25.15%
Qwen DPO→GRPO60.56%100.00%+65.13%
DeepSeek-R1-Distill-Llama-8B3.52%7.11%+101.99%
Llama-3-8B-Instruct0.00%0.00%—
Llama-3-8B-DPO0.00%0.70%—
DeepSeek-R1-Distill-Qwen-7B5.63%4.22%−25.04%

While a simple loop can elicit repetition (Xu et al., 2022), adding an SRLA seed to the original question makes degeneration more frequent in susceptible models under the same controlled decoding settings (Appendix B.2, Table 9; Appendix B.3, Table 10), giving us more instances of the failure to study.

Q1. Does the choice of seed model affect the result?

We agree that a model may respond differently to repetitive content produced by another model. Starting from the same pool of multiple-choice questions, we used Qwen2.5-7B, Meta-Llama-3-8B-Instruct, and DeepSeek-LLM-7B-Chat as seed sources following the SRLA construction in §3.1. Within each seed source, all five target models were evaluated on the same retained prompts. The table(Appendix A.3, Table 7) reports RDR.

ModelQwen2.5-7B seedsLlama-3-8B-Instruct seedsDeepSeek-LLM-7B-Chat seeds
Qwen SFT2.22%3.33%3.89%
Qwen SFT→GRPO7.78%12.78%17.22%
Llama-3-8B-Instruct0.00%0.00%0.00%
Llama-GRPO2.22%1.67%2.78%
R1-Distill-Llama-8B1.11%0.56%1.11%

With every seed source, both Qwen and Llama show higher RDR after GRPO, consistent with the training-stage comparison in Table 2. Qwen SFT→GRPO had the highest RDR for all three seed sources: 7.78% with Qwen seeds, 12.78% with Llama seeds, and 17.22% with DeepSeek seeds.

W2 & Q3. Training settings, data and reproducibility

We agree that the training recipe affects the magnitude of the increase. Section 4.1 compares the same backbone before and after GRPO across two model families under shared GRPO settings, and Table 14(Appendix C.2) separately varies the KL coefficient or mini-batch size on the Qwen DPO baseline.

Regarding training on broader corpora, our multi-task GRPO run combines English math (35%), Chinese math (20%), code (15%), and instruction following (30%) (§5.3 and Appendix F, Table 16). Broadening the training mixture while retaining math reduces RDR from 16.51% to 1.41% on the mitigation evaluation set Deval\mathcal{D}_{\text{eval}} (Figure 4).

The baselines, training datasets, and training details are provided in the paper. Specifically, Table 12(Appendix C.1) identifies the baselines, post-training stages, and training datasets; Appendix C provides the dataset links; Table 13(Appendix C.1) lists the public model checkpoints; §4.1 and Appendix C.2 specify the GRPO training settings; and Appendix G (Table 18) reports the standard capability evaluation. To further support community reproducibility, we commit to publicly releasing the models alongside the code, configuration files, and dataset splits upon camera-ready publication.

Q4. Additional metrics

Your suggestion to measure how much of an answer repeats is very insightful. RDR counts responses that reach the token limit in a loop. CREP complements it by measuring repetition coverage: following Lee et al. (2026), we report the percentage of responses in which a repeated pattern covers at least 30% (τ=0.3\tau = 0.3) of the tokens. MAUVE measures how closely the distribution of generated texts resembles that of a reference corpus (Pillutla et al., 2021). We evaluate both metrics on Deval\mathcal{D}_{\text{eval}} (Appendix G.1, Table 17).

ModelRDR (%)MAUVE ↑CREP (%) ↓
Qwen DPO→GRPO16.510.0072517.27
Concise3.120.0045414.55
Multi-task1.410.004277.47
Qwen2.5-7B-Instruct0.000.005340.00
Meta-Llama-3-8B-Instruct0.000.005810.00

Note: Meta-Llama-3-8B-Instruct is included as an additional reference model.

CREP falls substantially with both mitigations, whereas a low MAUVE score alone cannot establish repetitive degeneration in this task. On the same dataset, Qwen2.5-7B-Instruct and Meta-Llama-3-8B-Instruct both scored 0.00% on RDR and CREP, yet their MAUVE scores (0.00534 and 0.00581) were close to the unmitigated model’s (Qwen DPO→GRPO) 0.00725.

For example, one question asks for a synonym of happy. The short reference identifies joyful. A model can explain why joyful fits, discuss the other three options, and stop normally. That response contributes zero to RDR and CREP even though its length and style differ from the short reference. Pillutla et al. (2021) evaluated MAUVE on open-ended continuations of web articles (WebText), news (RealNews), and stories (WritingPrompts). Repetition can shift the text distribution that MAUVE measures, but so can ordinary step-by-step explanations. Consequently, a low MAUVE score by itself cannot establish repetitive degeneration in multiple-choice QA.

Xu, J., Liu, X., Yan, J., Cai, D., Li, H., & Li, J. (2022). Learning to break the loop: Analyzing and mitigating repetitions for neural text generation. Advances in Neural Information Processing Systems, 35, 3082–3095.

Lee, J., Park, S., Jang, S., Ryu, S., Kim, H., Lee, H., ... & Kim, Y. (2026). FOCUS & RePAIR: Mitigating Text Degeneration via Token-Level Guidance for Pruned Large Language Models. arXiv preprint arXiv:2608.26676.

Pillutla, K., Swayamdipta, S., Zellers, R., Thickstun, J., Welleck, S., Choi, Y., & Harchaoui, Z. (2021). Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances in Neural Information Processing Systems, 34, 4816-4828.


Dear Reviewer, Thank you again for your valuable comments. As the discussion period is nearing its end, we would like to kindly ask whether our responses have adequately addressed your concerns. Please let us know if any further clarification is needed. If our responses have resolved your concerns, we would sincerely appreciate your consideration in updating your evaluations accordingly.

Comments on the paper

2 comments

Dear Reviewers, please review the authors' rebuttal and participate in the discussion to address any remaining questions before the period closes on Nov 18 AoE. Thank you!

Dear reviewers, please note that the author–reviewer discussion period ends on November 18 (AoE).

eVbA, you asked how SRLA prompts are constructed and selected, how they compare with simpler repetition prompts, and how the seed model affects RDR and model rankings. You also asked whether degeneration stems from GRPO or the particular training recipe, and requested standard capability results, checkpoints, and mitigation metrics beyond RDR.

W9Pc, you asked why RDR combines token-limit hits with repetitive endings and what the separate rates are. You also raised concerns about using Qwen-generated seeds across models, missing task-accuracy results, and whether EOS masking elicits genuine repetition rather than disrupting performance, with implications for the EOS-based mitigation. You also asked whether larger models exhibit repetition.

Do the authors’ responses adequately address these concerns? If any remain, please let us know before the discussion deadline.

UJme, thank you for following up. Please feel free to share any additional comments before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.