acceptodds
Under review as a conference paper at ICLR 2027

Long-Zero: Self-Evolving Long-Context Reasoning via Information Degradation

Abstract

Long-context language models can ingest increasingly long documents, yet post-training them for evidence-sensitive reasoning remains difficult because open-ended tasks rarely provide automatic verifiers or scalable answer annotations. We introduce Long-Zero, a self-evolving reinforcement learning framework that converts this supervision bottleneck into response ranking induced by controlled information degradation. For each document–question pair, Long-Zero progressively removes query-relevant evidence, uses a frozen generator to produce answer candidates from the resulting context views, and trains a policy to recover their latent quality order under the full context. A machine-checkable Kendall- reward enables group-relative policy optimization, while a feedback controller adapts the information gap between views to the policy's ranking competence. Across LongBench, Long-Zero improves over the corresponding base model in all four scored model–decoding settings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Mqw3OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes Long-Zero, a post-training framework for long-context language models that constructs answer-ranking pseudo-labels by progressively removing query-relevant evidence from a document. It further introduces a feedback-driven curriculum controller that adjusts the relevance-mass gap between successive context views based on the policy's ranking rewards. Experiments on LongBench v2 and HELMET show improvements over the corresponding base models, with particularly promising gains in passage reranking.

Critical strengths
  1. Interesting supervision strategy. Using controlled evidence degradation to construct ranking targets without reference answers is a potentially useful direction for long-context post-training.
  2. Clear presentation and empirical motivation. The overall framework is relatively easy to follow, and the experiments provide initial evidence that the proposed procedure can improve downstream long-context performance.
Critical weaknesses

CW1. The related-work review systematically misses important research directions.

The related-work section does not adequately cover several important subareas of recent long-context training research: (i) RL with verifiable context-level or process-level rewards, such as LongRLVR (Chen et al., 2026) and LongPAS (Peng et al., 2026); (ii) long-context preference optimization and context-chunk-based preference construction, such as LongMab (Duan et al., 2026); and (iii) long-context reward modeling, as studied by LongRM (Tang et al., 2025). More broadly, it lacks systematic coverage of reference-free pseudo-labeling and self-feedback methods, such as Self-Rewarding Language Models (Yuan et al., 2024), as well as reward-driven adaptive curriculum learning, exemplified by AdaRFT (Shi et al., 2025). The issue is therefore not simply a few missing citations, but an incomplete coverage of the research landscape directly relevant to the paper's two central contributions.

CW2. The novelty of both contributions is insufficiently established.

The missing related work directly undermines the novelty claims. LongMab already uses feedback-guided context-chunk selection to generate responses and construct preference pairs for long-context post-training; Self-Rewarding Language Models explores automatically generated preference signals; and AdaRFT adjusts training difficulty according to recent reward feedback. These methods are not identical to Long-Zero, but they establish important precedents for its central ideas. The paper needs to explain more precisely what is fundamentally new about using evidence degradation to define answer rankings and about adapting the relevance-mass gap, rather than presenting these broad ideas as novel without sufficiently distinguishing them from existing approaches.

CW3. The experimental evaluation lacks essential baselines at multiple levels.

The current comparisons against Base and SFT do not adequately establish the value of the proposed mechanisms. I encourage the authors to consider three levels of baselines: (i) long-context alignment methods, including LongMab, LongReward, and, where the task and supervision permit a fair comparison, LongRLVR; (ii) reference-free pseudo-labeling or self-feedback methods, such as Self-Rewarding Language Models and related approaches; and (iii) mechanism-level controls, including random ranking targets, random evidence deletion, fixed-gap training without the adaptive controller, and training with the ranking reward in Equation 6 instead of Equation 7. Not every method necessarily admits a fair direct comparison, but the paper needs to select suitable alternatives and justify the comparison protocol. Without such evidence, it is difficult to determine whether the reported gains result from information-degradation supervision, adaptive curriculum learning, or simply performing additional post-training.

CW4. Missing ablations leave the mechanism and downstream results insufficiently validated.

The paper provides no component ablations that isolate the contributions of evidence degradation, the ranking objective, and adaptive curriculum control. The LongBench v2 results also do not consistently outperform SFT in Overall scores, while the stronger HELMET results are concentrated in a task structurally close to the training objective. The relationship between learning to rank candidate answers and improving direct answer generation is not sufficiently investigated. Moreover, the training-dynamics analysis relies on a single 7B trajectory, and the paper acknowledges that repeated seeds and matched-compute comparisons remain necessary. Together, these limitations prevent the experiments from establishing which proposed components are responsible for the observed improvements and how robust those improvements are.

CW5. The core assumptions underlying the pseudo-labels are not empirically validated.

Long-Zero relies on two assumptions: more available relevant evidence tends to produce better answers, and the relevance scores used to remove document chunks adequately reflect the amount of useful evidence retained. Neither relationship is independently established. Semantic or query relevance is not equivalent to evidential sufficiency, and removing evidence need not monotonically reduce answer quality. Moreover, when all chunks have already been removed, subsequent views can contain identical contexts while still receiving strictly ordered labels. Acknowledging these issues in the limitations section does not establish that they have a negligible effect on training. The paper should quantify the reliability of the induced rankings, examine cases where the proxy ordering disagrees with actual answer quality, and investigate how relevance scores relate to the amount of usable evidence.

CW6. The reward actually used for training does not appear to implement the stated ranking objective.

Equation 6 defines standard candidate-pair Kendall agreement using candidate identities and their positions, whereas Equation 7—the reward used in the experiments—compares the numerical values in the predicted and target candidate-ID sequences. These formulations are not equivalent: Appendix A reports that they disagree on 296 out of 576 pairs of permutations for four candidates, and even gives an example in which their rewards have opposite signs. In particular, Equation 7 appears to make the reward dependent on the arbitrary numerical assignment of candidate IDs, rather than solely on the agreement between the predicted and target rankings. This raises a fundamental concern about whether the policy is optimized toward the intended objective. The authors should justify why Equation 7 is used as the training reward despite its disagreement with the standard candidate-pair ranking objective in Equation 6. More importantly, they should establish that Equation 7 measures the intended ranking agreement and verify whether the main results hold when training with a candidate-identity-based ranking reward.

References

  1. Chen, G., Shieh, M. Q., & Bing, L. (2026). Longrlvr: Long-context reinforcement learning requires verifiable context rewards. arXiv preprint arXiv:2603.02146.
  2. Peng, M., Shen, W., Chen, N., Li, C., Yan, M., & Li, J. (2026). Incentivizing in-depth reasoning over long contexts with process advantage shaping. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 26201-26228). First published on 18/1/2026: https://arxiv.org/abs/2601.12465
  3. Duan, S., Huang, P., Li, X., Liu, Z., Yi, X., Yan, Y., ... & Sun, M. (2026). Chunks as arms: Multi-armed bandit-guided sampling for long-context llm preference optimization. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 3559-3575). First published on 19/8/2025: https://arxiv.org/abs/2508.13993
  4. Tang, Z., Ji, B., Qiu, Q., Wang, H., Liang, X., Li, J., & Zhang, M. (2025). Longrm: Revealing and unlocking the context boundary of reward modeling. arXiv preprint arXiv:2510.06915.
  5. Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., & Weston, J. (2024). Self-rewarding language models. arXiv preprint arXiv:2401.10020.
  6. Shi, T., Wu, Y., Song, L., Zhou, T., & Zhao, J. (2025). Efficient reinforcement finetuning via adaptive curriculum learning. arXiv preprint arXiv:2504.05520.
Questions for authors

What exactly does δt\delta_t represent operationally? Please clarify how the target relevance-mass gap determines the evidence removed between successive views, including the effects of chunk-level overshoot and the special case of diffuse relevance scores.

Reviewer_6zvEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Long-Zero masks query-relevant chunks to create nested views of each document. A generator held fixed within a cycle answers each view. Their source order supplies proxy ranking labels, and GRPO trains a policy to rank the shuffled answers given the full document. A controller adjusts the masking gap from ranking rewards. On two Qwen checkpoints, the strongest gains are on HELMET MS MARCO reranking; LongBench v2 QA beats SFT in only one of four model/decoding settings, by 0.2 points.

Critical strengths
  1. Controlled removal of document evidence produces proxy ranking labels without reference answers. Because the policy compares answers against the original document, this is a useful setup for studying evidence use when long-answer annotation is costly (Sections 2.1-2.2).
  2. HELMET MS MARCO macro NDCG@10 rises over SFT by 5.45 points (7B) and 2.05 (4B), with gains over Base at both scales (Table 2). Four late checkpoints from one 4B run also exceed those baselines. This is a substantive positive reranking signal.
Critical weaknesses
  1. The cause for reranking gain is not clear. LOGO [1] already scores chunks by relevance to synthesize preference data; LongPO [2] also contrasts answers from different context conditions. Long-Zero adds nested views, listwise ranking, and an adaptive gap, but does not isolate them or compare with preference training on the same generated answers. Its clearest win is MS MARCO reranking, which resembles its training task. On LongBench v2 QA, it beats SFT in only one of four settings, by 0.2 points, with no uncertainty estimates. The paper tests no other HELMET tasks or short-context knowledge, reasoning, or instruction-following benchmarks (e.g., MMLU, GSM8K, IFEval), leaving skill retention untested. It also omits the training corpus size and exact scorer and generator checkpoints. I would want a fixed-gap run, a same-data preference baseline, and results on other abilities.
  2. The paper does not check whether source-view order tracks answer quality. Generator variation and redundant evidence can make adjacent views yield tied or reversed answers; if masks exhaust the relevant evidence, different view ranks may even encode identical evidence. This proxy supplies every ranking target, so a small answer-quality audit reporting ties and reversals would help. Since training uses Eq. 7 instead of the candidate-pair Eq. 6, a matched reward ablation would show whether that choice matters. Reference: [1] Tang et al. LOGO: Long cOntext aliGnment via efficient preference Optimization. arXiv:2410.18533 (2024). [2] Chen et al. LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization. arXiv:2502.13922 (2025).
Questions for authors

What is the performance of the proposed method when tackling skill retention?

Reviewer_y1GeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Long-Zero, which generates candidate answers from document views with different degrees of redaction and then has the model rank them using the complete document. The method constructs a ranking reward from the source-view order without reference answers and adjusts the relevance gap based on training feedback. Experiments on long-context QA and passage reranking show some improvements.

Critical strengths
  1. Constructing a ranking reward without reference answers by controlling evidence availability is a useful idea.
  2. The paper evaluates the method on long-context QA and passage reranking, with some improvements in reranking.
Critical weaknesses
  1. The implemented reward does not directly measure the ability to rank candidate pairs. Equations (7) and (6) assign different scores to some rankings, and the score under Equation (7) depends on how candidate IDs are labeled. A higher reward does not directly indicate a corresponding improvement in ranking candidate pairs.
  2. The reliability of the source-view order as a proxy for answer quality has not been validated. The source-view order can be obtained automatically, but the paper does not verify whether the actual quality of the candidate answers follows this predefined order.
Questions for authors
  1. Redundant evidence and generation variability may disrupt the answer quality order. What is the agreement rate between the source-view labels and independent assessments of answer quality?
  2. The same prediction and target in Appendix A receive a score of +2/3 under Equation (7), but −2/3 under Equation (6). A common relabeling of candidate IDs may also change the score under Equation (7). Why is Equation (7) still used?
  3. Key ablations and diagnostics are missing, making it hard to tell whether the gains come from the proposed core mechanisms. The paper lacks training comparisons using a fixed gap, random removal, and different reward definitions. It also does not compare the ranker's performance with and without the document. Could the authors provide these experiments to separate the effects of relevance-based redaction, adaptive control, and ranking training?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.