acceptodds
Under review as a conference paper at ICLR 2027

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Abstract

Large Language Models generate text by sampling tokens autoregressively, but open-ended generation is prone to verbatim looping, where the model repeats spans already present in its context. Standard defenses, such as repetition, presence, and frequency penalties, and n-gram blocking, act on token recurrence rather than on the sequential structure of a loop, and suppress looping only at strengths that also degrade formatting and fluency. We propose Don’t Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context, with sequence breakers that protect chat templates and formatting tokens. Our experiments across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study show that DRY reduces the suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no such reduction, identifying suffix-matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces the loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8k, where standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks, including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its impact on practical text generation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
repetition, penalty, LLM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_o6bxOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DRY, a novel decoding algorithm for LLMs that avoid repetitions in a principled way. DRY reduces repetition by penalizing a token if the token reproduces a suffix that has already appeared in the context and if the suffix is of at least certain length L, which is a hyperparameter. While simple, the algorithm shows good performance in practice with reducing repetitions in long generation, while preserving performance in tasks including reasoning tasks in Math, coding, reasoning etc. The paper provides experiments for Qwen2.5, Llama-3-70B and Gpt-oss-120b, for several benchmarks like MT-Bench, MMLU, GSM8k etc. The algorithm is being used in popular openweight local model repositories like llama.cpp.

Critical strengths
  • The paper provides a simple yet impactful LLM text decoding algorithm to avoid repetitions by using a soft penalty for recurring phrases. The algorithm does not modify the decoding if there are no repetitions of length >= L, with L being a hyperparameter. Hence, it preserves the natural decoding performance if there are no repetitions.
  • The paper provides several experiments for repetitions and performance on Qwen2.5, Llama-3-70B and Gpt-oss-120b, for several benchmarks like MT-Bench, MMLU, GSM8k etc.
  • The paper is also added as a decoding algorithm option in the popular llama.cpp github repository.
Critical weaknesses
  • There needs to be more study on the required additional compute for this decoding algorithm. How does the compute require scale with the hyperparam L, do we need to store all seen tokens of length L?
  • Although Section I.1 provides ablation on the need of a sequence breaker, it is not completely clear why removing the sequence breaker leads to high false positives of penalized tokens. Further analysis with examples with different choices of sequence breakers would help explain the false positives better.
Questions for authors

Please answer the questions asked in the weaknesses.

Reviewer_QoJkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents the DRY method, which is Don’t Repeat Yourself, which is a sampling-time logit adjustment that reframes repetition control from token recurrence to suffix continuation. It tries to solve the problem of verbatim looping, where the model begins repeating spans that already appeared in the context. A really simple and straightforward method that basically checks previous tokens generated and penalises if the token is the same. Shows improvement on several different LLMs with basically no computational cost.

Critical strengths
  • A simple, easy-to-implement technique to address the limitation of the LLM’s problem.
  • Shows great performance across several LLMs with little computational overhead.
Critical weaknesses
  • The “Pareto frontier” claim. In Table 1 and Figure 3, n-gram blocking beats DRY on both loop rate and diversity, so DRY isn’t strictly on the frontier by those two metrics. Its advantage only shows up on structure and capability.
  • GPT-OSS-120B is described as a “dense transformer.” As far as I know, it’s a mixture-of-experts model.
  • The discussion overstates the human study. It says frequency penalty and n-gram blocking scored lower with human raters, but the study only compared DRY against the baseline and the repetition penalty.
Questions for authors

In addition to my concerns raised above in the weaknesses section, I would like to ask for further clarification on the following:

  • Upon checking the reference, according to Ref. #2, it seems that DRY was released in 2024 and is already in llama.cpp and other frameworks. Would you please clarify what exactly the paper’s contributions are beyond evaluating an existing method?
  • It looks like there are some inconsistencies in the results. For example, Figure 2 shows DRY at 0.032, 0.097, and 0.086 per model, but Table 2 says 0.026, 0.086, and 0.084. Figure 12 and Table 15 also differ slightly. Figure 12 has a “full stack” bar with no row in the table. Would you please clarify these discrepancies? Not sure if it is because of different runs?
  • In Table 1, No-repeat n-gram beats DRY on both SER@4 and distinct-4. Would you please clarify in what sense is DRY on the Pareto frontier?
Reviewer_3V6vOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper evaluates DRY, a sampling-time penalty used in text generation tools. It discourages repetition by exponentially lowering the probability of tokens that would repeat a previously seen text snippet, resetting only at specific "sequence breaker" tokens like newlines or colons.

Critical strengths

Critical Strengths

The one and only strength in the paper is that the paper proposed a method a soft, variable-length n-gram blocking with reset tokens. Which, in my opinion, is a very thin contribution to the community as most of the recently published decoding method papers also proposed their own repetition stoppage method.

Critical weaknesses

Core Weaknesses

The paper does not have any novelty from my point of view, for anyone who has read enough decoding method papers, this contribution is too thin to be even called a contribution, the authors claimed that "reframes repetition control from token recurrence to suffix continuation", yet if you read your own baseline paper carefully, it already reframes repetition control from token recurrence to suffix continuation, the ablation experiments design was also confusing, three crucial(yes, crucial) experiments missing are: (1): hard versus soft penalty. (2): fixed n versus the exponential growth in match length. (3): breakers, yet none of them are in the paper.

Questions for authors
  1. Why the nearest-neighbor ablations are missing? To show that the delta really matters, you need at least show the experiment results for soft fixed-n penalty and a n-gram blocking with breakers. Otherwise you are not beating the baseline, but rather compare with the worst baseline settings. I believe the gains could be wiped out once those two ablation experiments are done.

  2. If I understood correctly, the 8 point MMLU drop and 11 point GSM8K are actually wrong, it came simply from the implementation of the no_repeat_ngram_size, it just blocked n-grams appreaing anywhere in the input_ids, including the prompt. This means that the hard-blocking result without excluding the prompts reported in the paper are wrong, everywhere. I think this is a severe flaw beyond salvation and this single issue needs a major revision.

  3. The CD baseline is broken, if you check the results of the ablation experiments,it's even worse than no intervention, this is ridiculous, write an undertuned basedline in the paper is highly suspicious to me that other baselines were also undertuned.

  4. There were tons of inconsistencies in the paper, the max tokens​ were reported as 256 , then in figure 9 it was reported up to 500. Table 15 no intervention was reported as 0.106​, then in Table 2 it was 0.112​. The dry alone number kept changing from 0.056​ to 0.059​ to 0.058​ from table 2 to table 15 to figure 12. There is no anonymous codes for me to reproduce those results, I hope the authors could explain what happened here.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.