- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an improvement to LLMs' performance on long context (i.e. >> training length), namely a log-ratio version of RoPE that in hybrid (local, vs. global attention) that outperforms the RoPE-NoPE hybrid on long-context tasks. The paper also proves a theorem that shows an inverse relation between length generalization and positional resolution, such that the number of keys the model keeps in the trained range is inversely related to how well it generalizes beyond that range.
Critical strengths
-
Provides a new hybrid model (log-ratio RoPE hybrid) that outperforms other hybrid or non-hybrid RoPE/NoPE-based models on extending to long context on the RULER task.
-
Proves a theorem about the relation between positional encoding resolution and length generation that can inform future work on improving long context generalization.
Critical weaknesses
-
The title seems a bit hyperbolic compared to the actual contribution of the paper. One could argue that Theorem 1 is a contribution to a theory of the "Science of Long Context", but apart from that, the paper seems to be an instance of a genre that has seen a lot of work recently, namely variations on position encoding methods, or lack thereof, in order to try to improve LLMs' performance on long context.
-
As far as I can tell, the method is only evaluated on one set of long-context tasks, namely RULER. In past work even within this task, various subtasks (e.g. NIAH, QA), can be expected to show a wide range of performance.
-
The method only compares against PE approaches that are static. What about methods that learn to reorder positional indices to optimize for specific tasks, such as RePO (Li et al, 2026, ICML)?
Questions for authors
-
Which subsets of RULER are you reporting? Is this an aggregate score?
-
It seems odd to have the related work section be all the way back in Section 7.
-
Not a question, but it looks as if the AI use statement could use some editing.
2 comments
Response
Thank you for your updates, and for the additional experiments. I am happy to raise my score.
We thank the reviewer for the feedback and for the pointer to RePo, which we have now trained in our setup and compared against.
W1. Is the title too broad for the paper's contribution, which looks like one more positional-encoding variant?
W2. Is RULER alone enough, given that its subtasks can vary widely in performance?
W3. How does Log-Ratio RoPE compare with learned positional methods such as RePo?
Questions
Q1. Is the reported RULER score an aggregate over tasks?
Q2. Why is the related work in Sec. 7?
Q3. Not a question, but it looks as if the AI use statement could use some editing.
Summary of paper changes:
Thank you for taking the time to review our paper and helping us further improve it.
[1] https://arxiv.org/abs/2512.17351