- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes T-RoPE, a time-aware rotary position embedding for sequential recommendation. Instead of rotating by position index, it encodes timestamps directly into rotary angles with learnable coefficients, multiscale frequencies, shifted query alignment, and non-stationary keys, capturing elapsed time, periodic cycles, and calendar phase. The paper theoretically proves it breaks time-translation invariance, and validates it on public benchmarks, an interaction industrial dataset, and an online A/B test, showing consistent improvements over existing temporal RoPE variants.
Critical strengths
(1) It formally proves the time-translation invariance of standard RoPE and TO-RoPE, and proposes non-stationary key encoding to break the symmetry, with relatively solid theoretical contribution. (2) It maintains full interface compatibility with standard RoPE, serving as a drop-in replacement with very low migration cost. (3) Evaluations span public benchmarks, industrial-scale data, and online A/B tests, with clear ablation analysis quantifying each component’s contribution. Improvements are consistent and significant.
Critical weaknesses
(1) This paper belongs to component-level innovation. The work improves position encoding but does not change the overall Transformer architecture, which indicates its granularity of innovation is relatively fine. (2) Boundary with prior work (TO-RoPE) is not sharply defined. The core difference in temporal distance modeling is not sufficiently distinguished from related works. (3) The A/B test only reports conversion rate and order count, lacking finer-grained recommendation metrics like click-through rate or user retention, which may due to the restriction of paper pages. (4) Only tested in recommendation tasks and therefore generalizability to other temporal tasks (e.g. time series forecasting) is not verified. (5) The multiscale frequency bank uses fixed geometric spacing as a hyperparameter. You have to re-tune it for every new dataset, i.e., it can’t adapt automatically to different periodic patterns, which adds real tuning cost for deployment.
Questions for authors
(1) Non-stationary key encoding breaks the relative position symmetry of standard RoPE. Does this damage the modeling of relative distances in long sequences? Have you tested performance on very long sequence lengths? (2) The method is only validated in recommendation scenarios. Can it generalize to other temporal tasks such as time series forecasting or speech processing?
2 comments
Adjust rating to 3
Overall, the responses address most of my concerns, and the shared frequency bank and long-sequence tests are helpful. However, the authors confirm that the non-stationary key is equivalent to standard rotation with independent coefficients, so the theoretical novelty is weaker than initially presented. I adjust my rating to 3 (weak accept), while recognizing the solid empirical contribution across benchmarks and the online test.
Thank you for your time and thoughtful review. We really appreciate that!
Q2 Generalization beyond recommendation
Thank you for raising the question of generalization beyond recommendation. This paper focuses on generative recommendation. However, we have also extended T-RoPE to other internal temporal tasks and observed performance improvements. We did not include these extensions here in hopes to make the paper more focused and easier to read.
Q1 Relative distances and long sequences
Thank you for raising this concern about relative distance modeling in long sequences. In additional internal recommendation experiments, we tested sequence lengths up to 2,048 without observing any degradation. We have conducted experiments internally, on other tasks involving user events, where we trained on user sequences ranging in lengths between 8K and 16K, and we did not observe any performance degradation.
We believe that retaining Time RAB helps preserve relative time modeling alongside T-RoPE's sensitivity to absolute time, because RAB continues to provide explicit relative time information. As discussed in Section 5.2 (page 9, lines 439 to 443) and Table 3 (page 8), removing Time RAB hurts performance, consistent with its complementary role in modeling relative recency alongside T-RoPE.
Frequency settings across datasets
Thank you for raising this concern about the tuning cost of the frequency bank. We use the same geometric frequency bank, with bases ranging from 102 to 108, across all public benchmarks, internal experiments, and production deployments. We chose this range solely based on the finest and coarsest temporal scales relevant to our user event data. Learnable temporal coefficients adapt the effective frequencies, and we did not retune the bank for individual datasets or tasks. The positive results across these settings demonstrate the robustness of this shared configuration.
Nonetheless, all the tasks we tested consist of user event data. We cannot yet determine whether substantially different data types would require different frequency settings, and establishing transfer without tuning across arbitrary data types is beyond the problem addressed in this paper.
Additional online metrics
Thank you for suggesting that we report additional online metrics. We also measured high-quality click-through rate (HQCTR) in the online evaluation, which is a constrained form of click-through rate which focuses on clicks that lead to high intent purchase signal: favoriting, cart addition, or purchase. HQCTR increased by 0.52% relative to the baseline, with p=0.14. This increase was not statistically significant. The evaluation lasted only seven days, which may have limited our ability to detect significant effects. We previously omitted this result to save space and have now included it in Table 4 (page 9, lines 432 to 436) and Section 5.3 (page 9, lines 454 to 455).
Thank you again for your time and effort to kindly review this paper! The comments and feedback really encourage us, really appreciate that!