- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies catastrophic repetition in long-context retrieval-augmented generation. It introduces DE-RP, a stress benchmark containing 731 failure-enriched long-context RAG prompts, together with a detector that identifies periodic word-level repetition and retrospectively returns a character-offset boundary. Based on this detector, the authors observe that prefixes preceding detected repetition substantially increase the probability of subsequent repetition. They further report that prefixes mined from one model can induce failures in several other model families and that repetition-associated hidden-state signals become prominent near the detected prefix boundary. The proposed PrefixRLVR method combines two components. Prefix-Guided Rollout (PGR) conditions RLVR rollouts on mined high-risk prefixes, increasing the probability that a rollout group contains both positive and negative verifier outcomes. Length-Aware Advantage Estimation (LAAE) rescales group-relative advantages according to continuation length in order to discourage a shortcut in which the policy avoids repetition by terminating early.
Across Qwen3-0.6B, Qwen3-4B, Qwen3-8B, and Llama-3.1-8B, PrefixRLVR substantially reduces detector-defined repetition on DE-RP. Compared with prefix-conditioned GRPO alone, LAAE also reduces the observed shortening of non-repetitive responses. The paper additionally reports results on standard reasoning and instruction-following benchmarks, RULER, LaRA, LongWeave, and YaRN-extrapolated 64K/128K DE-RP variants.
Critical strengths
-
The method produces large and consistent anti-repetition gains on the evaluated stress benchmark. PrefixRLVR reduces DE-RP repetition from 9.90% to 0.03% on Qwen3-0.6B, from 3.44% to 0.07% on Qwen3-4B, from 4.10% to 0.03% on Qwen3-8B, and from 14.16% to 0.09% on Llama-3.1-8B. The models are evaluated using the original prompts rather than trigger-prefixed inputs, so the gains are not restricted to the exact conditioning format used during training. The continuation experiments and mining comparison also provide meaningful behavioral evidence that small-model failure prefixes can be useful for other models.
-
The paper identifies an important optimization shortcut and presents evidence that LAAE moderates it. Prefix-conditioned GRPO without LAAE reduces repetition but shortens non-repetitive outputs by approximately 35–40% on several backbones. On Qwen3-4B, adding LAAE changes the reported length shift from −40.3% to +0.7% while retaining a very low repetition rate. The binary-gate, ungated-scaling, and positive-only ablations exhibit substantially larger positive or negative length drift. This is consequential because an anti-repetition method is of limited value if it mainly learns to terminate before completing the answer.
Critical weaknesses
-
The DE-RP quality study evaluates only 100 Qwen3-4B queries for which both Base and PrefixRLVR already produce non-repetitive answers. This clean-clean selection is a useful conditional sanity check, but it excludes cases in which the method changes the repetition outcome and therefore does not measure end-to-end utility over the test population. Matching the mean length of non-repetitive answers also does not establish completeness, relevance, faithfulness, or correct citation use. Table 4 presents a more serious comparison issue. Several LaRA and LongWeave Base scores marked with † or ‡ come from the original benchmark papers, whereas PrefixRLVR scores are obtained from the authors’ local pipeline. These results may differ in checkpoint revision, prompt formatting, decoding, data processing, or LLM-judge version. In the LongWeave rows, the published Overall score and the locally measured repetition statistics also appear to come from different generations, so their relationship cannot be interpreted directly. Several reported LaRA improvements are extremely small, such as 46.45 to 46.50, and no uncertainty is provided. Consequently, Table 4 supports the conclusion that repetition robustness transfers to some long-context settings, but it does not yet establish a general improvement in long-context capability.
-
The LAAE component ablations are useful, but detailed comparisons are concentrated on Qwen3-4B and do not fully compare the proposed transformation with a simple, well-specified length or EOS correction under the same training and quality-evaluation budget. The reward-side correction shown in Figure 4 is not reported with the same final repetition, length, and answer-quality metrics as the full method. The LZ comparison uses one fixed configuration rather than characterizing the quality–length–repetition trade-off. The related-work framing also needs correction. Tsai et al. (2008), cited in support of entropy-based repetition evaluation, is a medical image-quality paper and is unrelated to text repetition. The statement that prior repetition-mitigation work does not assess capability preservation is too broad: LZ Penalty explicitly evaluates capability preservation, Xu et al. (2022) evaluates perplexity and generation quality after repetition-focused training, and Dong et al. (2025) evaluates downstream code-generation benchmarks. Similarly, the claim that prior work typically intervenes only after repetition becomes observable overlooks training-based approaches such as DITTO. These issues do not invalidate PrefixRLVR, but they overstate the novelty and motivation in its current form.
Questions for authors
- Since the paper states that local Base evaluations were already conducted, why are published Base scores used for several LaRA and LongWeave comparisons in Table 4? Please provide local Base scores using exactly the same checkpoint revision, prompting, decoding, data processing, and judge as PrefixRLVR. Are LongWeave Overall, Rep., and n-gram values computed from the same generated outputs for both variants?
- Can the authors report quality measurements on an unconditioned or randomly sampled DE-RP test population, including cases in which only one model produces a repetitive response? Per-dimension results, paired uncertainty, and evaluation of completeness and faithfulness would clarify whether repetition reduction preserves useful answers.
- How many independent training runs were performed for each method? Please report raw repetition-failure counts, prompt-clustered confidence intervals, and training-run variability for the principal results. For general benchmarks, are the observed changes larger than normal evaluation or checkpoint variance?
- What exactly is the reward-side length correction labeled GRPO+RP in Figure 4, how was it tuned, and what are its final repetition, length, and answer-quality results? A compute-matched comparison with a simple EOS/minimum-length or length-distribution correction would materially affect my assessment of LAAE.
- How were the 200 human-evaluation samples selected, did they include outputs from post-training models, and what was the annotator agreement before discussion? How sensitive are the main results to the exclusion of periods 2–5 and to alternative detector thresholds?
- Please replace the unrelated Tsai et al. image-quality citation, qualify the claims about prior capability evaluation and post-onset intervention, and update incomplete bibliographic information. How does PrefixRLVR differ empirically from training-based anti-repetition methods such as DITTO under a comparable evaluation?
3 comments
Thanks for the detailed comments and for the recognition of our work. We have also run several additional experiments. Our point-by-point responses are below. All new results will be added to the revision, some overstated claims will be corrected, and the revised PDF will be uploaded before the end of the discussion period.
Q1. We originally used the published scores as a conservative choice. These scores were produced by the benchmark authors with the official pipelines, and our local Base scores are lower than the published ones in 4 of 5 cases. In the original paper, the Base Overall score was taken from the original benchmark paper, while Rep. and n-gram were computed from our local generations. This mixed two evaluation pipelines, which was our oversight. Therefore, in the revision, all Base scores in Table 4 are replaced by local results, using the same checkpoint, prompt, decoding, data processing, and judge as PrefixRLVR.
Numbers (published Base in the original paper / local Base / PrefixRLVR):
- Llama-3.1-8B, LaRA-LC 32K / 128K: 66.58 / 55.10 ; 65.48 / 53.82 ; 67.94 / 55.57
- Llama-3.1-8B, LongWeave: 20.25 / 16.23 / 25.69
- Qwen3-8B, LongWeave: 27.17 / 25.82 / 31.98
- Qwen3-4B, LongWeave: 19.18 / 21.78 / 27.83
Q2. On Qwen3-4B, we used GPT-4o, the same judge as in the original paper, to score answers on five dimensions from 1 to 5 (the prompt will be included in the revision). Intervals are 95% CIs from a paired bootstrap over questions.
Random 100 questions, not filtered by repetition status (Base / PrefixRLVR / difference [95% CI]):
- Relevance: 3.92 / 4.17 / +0.25 [0.10, 0.41]
- Faithfulness: 2.82 / 3.11 / +0.29 [0.13, 0.46]
- Completeness: 2.71 / 2.85 / +0.14 [0.02, 0.26]
- Coherence: 3.67 / 4.28 / +0.61 [0.43, 0.80]
- Citation quality: 2.63 / 3.04 / +0.41 [0.25, 0.58]
- Overall: 3.15 / 3.49 / +0.34 [0.24, 0.44]
All five dimensions improve significantly, including completeness and faithfulness. There is no evidence that reducing repetition comes at the cost of informativeness.
Q3
Due to GPU compute constraints, each method was trained only once on each backbone, and the revision will state this limitation explicitly. The same recipe was trained independently on 4 backbones, with consistent results. Raw failure counts (731 prompts x 8 = 5,848 generations) with prompt-clustered 95% CIs (Base vs. PrefixRLVR):
- Qwen3-0.6B: 579 (9.90% [8.93, 10.89]) vs. 2 (0.03% [0.00, 0.09])
- Qwen3-4B: 201 (3.44% [2.72, 4.19]) vs. 4 (0.07% [0.02, 0.14])
- Qwen3-8B: 240 (4.10% [3.21, 5.04]) vs. 2 (0.03% [0.00, 0.09])
- Llama-3.1-8B: 828 (14.16% [12.48, 15.87]) vs. 5 (0.09% [0.02, 0.19])
Q4
- P+GRPO+RP (reward-side correction): within each rollout group, we compute the mean and the population standard deviation of the lengths of all non-repetitive responses. A non-repetitive response whose length is below the mean minus k standard deviations has its reward changed from 1 to 0, and GRPO advantages are then computed from the modified rewards (no penalty is applied if a group has fewer than 2 non-repetitive responses). This baseline directly rewrites the verifier reward labels, whereas LAAE leaves the reward unchanged and only adjusts the magnitude of the advantages. The curve in Figure 4 of the original paper corresponds to k = 1.5: during training, the length kept drifting and all metrics were unsatisfactory, so no checkpoint was saved. For the rebuttal, we ran full training and evaluation with k = 2, with results shown below.
- P+Dr.GRPO (aggregation correction): removes the per-response length normalization and the standard-deviation normalization in GRPO (Liu et al., 2025).
Both baselines use exactly the same prefix data, number of training steps, and other hyperparameters as PrefixRLVR. Results on Qwen3-4B (failures / repetition rate / length change / overall quality):
- Base: 201 / 3.44% / - / 3.150
- P+GRPO+RP (k = 2): 7 / 0.12% / -18.8% / 3.318
- P+Dr.GRPO: 39 / 0.67% / -9.8% / 3.282
- PrefixRLVR: 4 / 0.07% / +0.7% / 3.490
The reward-side correction is very sensitive to the threshold: with k = 1.5 the length grows out of control, and with k = 2 responses are still 18.8% shorter, with both repetition and quality worse than PrefixRLVR. Dr.GRPO removes most of the shortening (from -40.3% to -9.8%), but its repetition rate is about 10 times that of PrefixRLVR, and its quality is lower by 0.21 [0.11, 0.31]. Only LAAE achieves low repetition, unchanged length, and higher quality at the same time.
Q5
- Samples: randomly drawn from Qwen3-0.6B Base outputs on DE-RP and labeled by humans until there were 100 repetitive and 100 non-repetitive samples, then compared with the detector results. Both sampling and labeling were independent of the detector.
- Agreement: before discussion, only 2 of the 200 samples were labeled inconsistently (99.0% agreement), and both were finally judged repetitive.
- Post-training models: the human-evaluation samples do not include outputs from post-trained models. To rule out the possibility that the policy exploits blind spots of the detector, we ran a sensitivity analysis on the 5,848 Qwen3-4B generations. After relaxing the rule to 3 occurrences, Base failures increase from 201 to 225, while PrefixRLVR stays at 4. When detecting only the excluded periods 2 to 5, Base has 24 failures and PrefixRLVR has 0.
Q6
Our revision will remove the unrelated references, complete the bibliographic information, and qualify some overstated claims.
DITTO: we reproduced DITTO on Qwen3-4B (official "nl" loss, mixed 50/50 with MLE). A repeated-sentence probe shows that the implementation is correct: the probability of a repeated sentence decays with each repetition (0.689, 0.335, 0.168, 0.084), consistent with the trend in the original paper. On DE-RP, however, its repetition rate is 24.23% (1417/5848), higher than Base, and its overall quality is 2.294, also lower than Base. We believe the reason is that DITTO learns from offline synthetic sentence-level repetitions, whereas loops in long RAG outputs mostly occur at the level of lists, citations, and sub-sentence spans, and come entirely from real samples rather than constructed ones. PrefixRLVR instead optimizes directly on high-risk prefixes generated by the model itself, and therefore generalizes better.
As the author have not submitted the rebuttal, I will keep my initial score.