- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work studies the repetition behaviour of LLMs at the sampling stage. Author first build a dataset by collecting the repetition samples from a post-trained open source model and then use this dataset to further study the reason/dynamic behind. Authors also found this problem is due to the GRPO style post training. Finally authors identified the root cause of this behaviour and then purposed two mitigations that significantly reduce failure rates.
Critical strengths
- Authors offers a very in-depth study on formulating the problem, understanding the root cause and then propose some simple and clean solutions. The progress of critical thinking and reasoning are also valuable.
- This repetition is indeed widely observed and have a major impact on user experience.
Critical weaknesses
- Does this behaviour fix have any impact on the model quality effect? I saw some results from Table 17 but these evals are all quite outdated and toy. Could you please try at least a few cutting edge more complicated evals such as the agent coding on relative larger models? (I understand these smaller models may be a bit struggle on the agentic tasks)
- IIUC, authors mainly focus on the repetition on vanilla decoding tokens. A straightforward question is, how is this interact with the tool use tokens and would this induce any positive or negative behavior for these long decoding tasks? (authors don't have to resolve this question in the paper or rebuttal but would be nice to write some discussion in somewhere of the paper)
Questions for authors
Minor: Are there any specific order to run these fixes verse the training stage. Is it always fine to just have a post-hoc fix or it is better to do it as early as possible?
2 comments
Thank you for the reply. Looks good to me and am happy to keep my scores positive
Thank you very much for your positive evaluation and your generous recognition of our problem formulation, root-cause analysis, and critical reasoning. We are especially encouraged that you found our study valuable and our proposed training solutions clean and effective. Your insightful question regarding harder agentic tasks prompted us to evaluate more challenging mathematics and coding benchmarks, which we address below.
W1. Impact on model quality
Table 18 (Appendix G.2) reports standard task accuracy and pass rates on MMLU, GSM8K, IFEval, and HumanEval. To assess whether the repetition mitigations preserve model quality on more challenging reasoning and coding tasks, we additionally evaluated answer accuracy on MATH-500 and code execution pass@1 on BigCodeBench-Instruct.
Note: Meta-Llama-3-8B-Instruct is included as an additional reference model.
Both mitigations score above DPO baseline on MATH-500 ( +0.80 and +0.20 percentage points), and on BigCodeBench-Instruct, Multi-task achieves the highest pass rate among the four DPO-derived models (+1.67 points) while Concise remains within 0.08 points of DPO baseline. Reducing repetitive degeneration therefore comes with minimal differences on these harder reasoning and coding benchmarks. These coding results cover single-response code generation at 7B; performance on multi-turn coding tasks at larger scales remains open.
W2. Interaction with tool-use tokens
Thank you for raising this direction. Our BigCodeBench-Instruct evaluation covers single-response code generation, which leaves the effect of our mitigations on multi-turn tool use open. Chen et al. (2026) found that compressing the interaction history made agents more likely to repeat a tool call and receive the same result. We will acknowledge this limitation and consider studying how call arguments, tool results, and eventual task success can distinguish a productive retry from a repetitive loop and whether our mitigations affect tool-call retries in future work.
Q. Order of the fixes relative to training
Both mitigations are applied independently during the GRPO stage—one modifying the reward and the other rebalancing the data mixture (§5.3). We train the two variants separately: Concise rewards shorter correct responses, and Multi-task changes the mixture of training tasks (§5.3).
Earlier intervention is also possible. For example, the Qwen3 team filtered repetitive reasoning traces from its cold-start data before GRPO (Qwen3 Technical Report, §4.1). In our Qwen pipeline, RDR was 14.76% after DPO and rose sharply to 100.00% after GRPO (Appendix C.2, Table 14), so we targeted the stage where we observed that increase. We have not compared earlier intervention with a post-hoc fix on an already trained model, and we would welcome further discussion of that question.
Once again, we sincerely thank you for your encouraging and inspiring feedback. We will highlight your valuable perspectives on multi-turn tool calling and training-stage comparisons as promising directions for future work.
Chen, T., Singh, P., Todi, K., Paria, B., He, Q., Wang, W., ... & Akula, A. (2026). DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents. arXiv preprint arXiv:2610.19376.
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., ... & Qiu, Z. (2025). Qwen3 technical report. arXiv preprint arXiv:2505.09388.