acceptodds
Under review as a conference paper at ICLR 2027

T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation

24 views

Abstract

Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices made for text, including Rotary Position Embedding (RoPE). In language models, RoPE encodes token indices for relative position reasoning, but in recommendation, an interaction index records only event order, saying nothing about elapsed time, behavioral cycles across scales, or calendar phase. We revisit this choice and propose T-RoPE, a time-aware RoPE for sequential generative recommendation that replaces index-only rotation with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. We prove that standard RoPE, even on timestamps, remains time-translation invariant and cannot distinguish seasonal contexts, and that T-RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks, T-RoPE achieves the best result on every metric on every dataset, improving over the strongest baseline by 78–130% in HR@10 on the sparse PixelRec data and 8–12% across metrics on Amazon Books. On an industrial-scale e-commerce dataset with more than 6B interactions, it improves every metric over the HSTU + Time RAB backbone by 13–82%, with ablations attributing the largest gains to multiscale frequencies ( NDCG@50) and non-stationary keys (). An online A/B test in the Shop app yields positive lifts in conversion rate () and order count (). We also provide forward and backward algorithms whose added cost is linear in sequence length and head dimension, keeping time-aware RoPE practical for large generative recommenders.

Primary area:
learning on time series and dynamical systems
Keywords:
generative recommendation, positional encoding, temporal modeling, sequential recommendation, Rotary Position Embedding (RoPE)

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_uS94OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes T-RoPE, a time-aware rotary position embedding for sequential recommendation. Instead of rotating by position index, it encodes timestamps directly into rotary angles with learnable coefficients, multiscale frequencies, shifted query alignment, and non-stationary keys, capturing elapsed time, periodic cycles, and calendar phase. The paper theoretically proves it breaks time-translation invariance, and validates it on public benchmarks, an interaction industrial dataset, and an online A/B test, showing consistent improvements over existing temporal RoPE variants.

Critical strengths

(1) It formally proves the time-translation invariance of standard RoPE and TO-RoPE, and proposes non-stationary key encoding to break the symmetry, with relatively solid theoretical contribution. (2) It maintains full interface compatibility with standard RoPE, serving as a drop-in replacement with very low migration cost. (3) Evaluations span public benchmarks, industrial-scale data, and online A/B tests, with clear ablation analysis quantifying each component’s contribution. Improvements are consistent and significant.

Critical weaknesses

(1) This paper belongs to component-level innovation. The work improves position encoding but does not change the overall Transformer architecture, which indicates its granularity of innovation is relatively fine. (2) Boundary with prior work (TO-RoPE) is not sharply defined. The core difference in temporal distance modeling is not sufficiently distinguished from related works. (3) The A/B test only reports conversion rate and order count, lacking finer-grained recommendation metrics like click-through rate or user retention, which may due to the restriction of paper pages. (4) Only tested in recommendation tasks and therefore generalizability to other temporal tasks (e.g. time series forecasting) is not verified. (5) The multiscale frequency bank uses fixed geometric spacing as a hyperparameter. You have to re-tune it for every new dataset, i.e., it can’t adapt automatically to different periodic patterns, which adds real tuning cost for deployment.

Questions for authors

(1) Non-stationary key encoding breaks the relative position symmetry of standard RoPE. Does this damage the modeling of relative distances in long sequences? Have you tested performance on very long sequence lengths? (2) The method is only validated in recommendation scenarios. Can it generalize to other temporal tasks such as time series forecasting or speech processing?

2 comments

Thank you for your time and thoughtful review. We really appreciate that!

Q2 Generalization beyond recommendation

The method is only validated in recommendation scenarios. Can it generalize to other temporal tasks such as time series forecasting or speech processing?

Thank you for raising the question of generalization beyond recommendation. This paper focuses on generative recommendation. However, we have also extended T-RoPE to other internal temporal tasks and observed performance improvements. We did not include these extensions here in hopes to make the paper more focused and easier to read.

Q1 Relative distances and long sequences

Non-stationary key encoding breaks the relative position symmetry of standard RoPE. Does this damage the modeling of relative distances in long sequences? Have you tested performance on very long sequence lengths?

Thank you for raising this concern about relative distance modeling in long sequences. In additional internal recommendation experiments, we tested sequence lengths up to 2,048 without observing any degradation. We have conducted experiments internally, on other tasks involving user events, where we trained on user sequences ranging in lengths between 8K and 16K, and we did not observe any performance degradation.

We believe that retaining Time RAB helps preserve relative time modeling alongside T-RoPE's sensitivity to absolute time, because RAB continues to provide explicit relative time information. As discussed in Section 5.2 (page 9, lines 439 to 443) and Table 3 (page 8), removing Time RAB hurts performance, consistent with its complementary role in modeling relative recency alongside T-RoPE.

Frequency settings across datasets

The multiscale frequency bank uses fixed geometric spacing as a hyperparameter. You have to re-tune it for every new dataset, i.e., it can’t adapt automatically to different periodic patterns, which adds real tuning cost for deployment.

Thank you for raising this concern about the tuning cost of the frequency bank. We use the same geometric frequency bank, with bases ranging from 10210^2 to 10810^8, across all public benchmarks, internal experiments, and production deployments. We chose this range solely based on the finest and coarsest temporal scales relevant to our user event data. Learnable temporal coefficients adapt the effective frequencies, and we did not retune the bank for individual datasets or tasks. The positive results across these settings demonstrate the robustness of this shared configuration.

Nonetheless, all the tasks we tested consist of user event data. We cannot yet determine whether substantially different data types would require different frequency settings, and establishing transfer without tuning across arbitrary data types is beyond the problem addressed in this paper.

Additional online metrics

The A/B test only reports conversion rate and order count, lacking finer-grained recommendation metrics like click-through rate or user retention, which may due to the restriction of paper pages.

Thank you for suggesting that we report additional online metrics. We also measured high-quality click-through rate (HQCTR) in the online evaluation, which is a constrained form of click-through rate which focuses on clicks that lead to high intent purchase signal: favoriting, cart addition, or purchase. HQCTR increased by 0.52% relative to the baseline, with p=0.14p = 0.14. This increase was not statistically significant. The evaluation lasted only seven days, which may have limited our ability to detect significant effects. We previously omitted this result to save space and have now included it in Table 4 (page 9, lines 432 to 436) and Section 5.3 (page 9, lines 454 to 455).

Thank you again for your time and effort to kindly review this paper! The comments and feedback really encourage us, really appreciate that!

Reviewer_uS94OpenReview ↗

Adjust rating to 3

Overall, the responses address most of my concerns, and the shared frequency bank and long-sequence tests are helpful. However, the authors confirm that the non-stationary key is equivalent to standard rotation with independent coefficients, so the theoretical novelty is weaker than initially presented. I adjust my rating to 3 (weak accept), while recognizing the solid empirical contribution across benchmarks and the online test.

Reviewer_whrWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes T-RoPE, a temporal rotary positional encoding for sequential recommendation. It replaces position indices with Unix timestamps and combines learnable query/key coefficients, multiscale frequencies, shifted query alignment, and reverse key rotation to capture elapsed time and absolute-time effects. The theoretical analysis characterizes when timestamp-based rotary attention is time-translation invariant. Experiments report the best results across five public benchmarks, improvements on an industrial dataset with over 6B interactions, and a relative conversion-rate lift of 0.33% in an online A/B test.

Critical strengths
  1. Practical Design with Industrial-scale Validation T-RoPE preserves the RoPE interface, adds only 256 parameters in the public-benchmark configuration, and shows throughput comparable to TO-RoPE in Table 5. On the Shop dataset, it improves the HSTU + Time RAB backbone’s NDCG@50 from 13.81 to 22.94. The statistically significant conversion-rate improvement in the online A/B test provides further evidence of practical value beyond offline benchmarks.
  2. Clear Problem Formulation and Useful Empirical Findings The paper identifies temporal information omitted by index-based RoPE and uses time-translation invariance to distinguish relative-time encoding from absolute-time sensitivity. Table 3 also shows substantial gains from combining timestamp-based rotation with multiscale frequencies. These findings are useful for designing temporal encodings for sequential recommenders.
Critical weaknesses
  1. Unclear Contributions of Reverse Key Rotation and Coefficient Learning With unrestricted, independent query/key coefficients, reverse key rotation is equivalent to standard rotation after negating the key coefficient. Comparison against the shared-coefficient formulation in Theorem 1 therefore does not establish additional expressivity. If row 2 of Table 3 retains independent coefficients, its reported 4.0% performance drop needs explanation; if it shares coefficients, the ablation confounds rotation direction with coefficient sharing. The exact configuration and a comparison with sign-matched initialization are needed. Moreover, temporal coefficients adjust frequencies rather than directly weighting temporal bands. Learned effective periods and a frozen-coefficient ablation would help substantiate the interpretation and benefit of coefficient learning.
  2. Baseline Comparisons and Temporal Evaluation Need Further Validation The abstract’s 78–130% PixelRec HR@10 gains compare against TO-RoPE; gains over the strongest baseline, HSTU + Time RAB, are 2.8%, 7.1%, and 25.5%. TO-RoPE includes Time RAB yet substantially underperforms that baseline, requiring clarification of implementation and tuning. Single-seed results also leave the approximately 1% ML-20M gains uncertain. Furthermore, user-wise leave-one-out splitting can include other users’ observations after a test timestamp in training. A global temporal split is needed to assess future-time generalization, although this concern alone does not establish that leakage explains the gains. The final offline query timestamp and its availability at recommendation-request time should also be specified.
Questions for authors

Q1. Reverse Key Rotation Does row 2 of Table 3 retain independent query/key coefficients? Can the authors explain the reported gap and provide a comparison using sign-matched initialization across multiple seeds? Q2. TO-RoPE Baseline Which TO-RoPE variant, timestamp scale, frequency configuration, and tuning procedure were used? Why does it underperform HSTU + Time RAB on PixelRec? Q3. Temporal Evaluation Which timestamp is assigned to the final offline query, and is it available at recommendation-request time? Do the improvements persist under a global temporal split of the public benchmarks? Q4. Temporal Coefficient Learning Does learning the coefficients outperform freezing them at initialization? How do their distributions and effective periods change during training, and how does this support the temporal-band interpretation?

4 comments

Thank you for your time and detailed technical review. Really appreciate that!

Q3 Temporal evaluation

Which timestamp is assigned to the final offline query, and is it available at recommendation-request time? Do the improvements persist under a global temporal split of the public benchmarks?

Thank you for raising this. We hope the table below clarifies how query timestamps are determined under different settings. In particular:

  • Public Offline evaluations (Table 2, page 7): we use ground-truth timestamps of the held-out interactions
  • Internal Offline evaluations (Table 3, page 8): uses midnight UTC with histories frozen throughout the held out day
  • Online experiment (Table 4, page 9) and production inference load: we use a timestamp available at recommendation time
SettingQuery timestampHistory and evaluation protocol
TrainingNext interaction timestamp tm+1t_{m+1}The query is aligned with the event being predicted.
Public benchmarks, Table 2 (page 7)Ground truth timestamp of the held out interactionLeave one out for each user, following the target time convention used by HSTU Time RAB. Specifically: HSTU + Time RAB, HSTU + TO-RoPE, and HSTU + T-RoPE all receive the same held out interaction timestamp through Time RAB. T-RoPE also uses that timestamp in the query rotation
Internal Shop evaluation, Table 3 (page 8)UTC (00:00:00)A global temporal holdout covering one day. Training interactions and histories precede the cutoff. The day's interactions serve as evaluation labels.
Production, Section 5.3 (page 9)Actual user login timeRecommendations use events already available in the database at that moment, including earlier events from the same day when available.

We also agree that leave-one-out splitting for each user does not enforce a global temporal cutoff, since another user's training interaction may occur after a particular user's test event. For timestamp shifting setting, T-RoPE follows the same approach as HSTU+TimeRAB, which also uses shifted timestamps for public datasets in the original HSTU paper. In Table 2 (page 7), we retained this established public protocol for comparability. However, to address this concern, we in fact used a global temporal holdout covering one day when we produced the results in Table 3 (page 8) (internal Shop evaluation). Training interactions and user histories precede midnight UTC at the start of the held out day, when we generate one recommendation list per user with the query timestamp set to that cutoff.

We have clarified the query timestamp sources in Section 5.1 (page 7, lines 371 to 374), Section 5.2 (page 8, lines 411 to 414), and Section 5.3 (page 9, lines 447 to 451).

Q2 TO-RoPE configuration

Which TO-RoPE variant, timestamp scale, frequency configuration, and tuning procedure were used? Why does it underperform HSTU + Time RAB on PixelRec?

Thank you for asking us to clarify the baseline configuration and tuning procedure. We use the split by head variant with a split ratio of 0.3, following Table 5 of the original TO-RoPE paper (page 9). Our common training configuration follows the hyperparameters reported for ML-20M in its Section 5.1.4 (page 7 of the TO-RoPE paper). For temporal frequencies, we adopt the base form construction with baset=104\mathrm{base}_t = 10^4, within the range [103,105][10^3,10^5] specified in its Section 3.2 (page 3). We retain the timestamps in seconds as provided by the public datasets, with no additional rescaling, corresponding to a normalization factor of s=1.0s=1.0.

All compared methods, including T-RoPE, use the same HSTU backbone, training configuration, and evaluation protocol without additional tuning for individual datasets or methods. T-RoPE also retains the same frequency bank configuration across all five public benchmarks and all internal benchmarks and productions. Its consistent improvements under these settings provide evidence of robustness across the evaluated datasets without additional tuning for each dataset. TO-RoPE’s lower PixelRec scores are the observed results under this common setup.

We have update the paper and clarified the common training settings and encoding specific configurations in Appendix D (page 17, lines 916 to 917, and page 18, lines 930 to 934) and Table 7 (page 18). If you consider it helpful, we would be happy to tune TO-RoPE’s timestamp normalization factor ss and temporal frequency base baset\mathrm{base}_t on Pixel200K and report the results.

Q1 and Q4 Rotation and temporal coefficient learning

Does row 2 of Table 3 (page 8) retain independent query/key coefficients? Can the authors explain the reported gap and provide a comparison using sign-matched initialization across multiple seeds?

Temporal Coefficient Learning Does learning the coefficients outperform freezing them at initialization? How do their distributions and effective periods change during training, and how does this support the temporal-band interpretation?

Thank you for identifying the parameter equivalence and suggesting these controlled comparisons. We agree that standard and reverse rotation represent the same function class when query and key coefficients are independent and unconstrained. Theorem 1 assumes shared coefficients (page 6, lines 291 to 295), and Remark 1 now explicitly states the equivalence (page 6, lines 300 to 305). Row 2 of Table 3 (page 8) retains independent coefficients, shifted queries, and the multiscale frequency bank.

Following your suggestions, we have added a controlled ablation on Amazon Books in Appendix E (page 18, lines 938 to 959), with results in Table 8 (page 20, lines 1027 to 1047). Each industrial experiment in Table 3 (page 8) requires approximately 10,000 B300 GPU hours, so Amazon Books provides an accessible setting for comparing initialization, sharing, and freezing. All configurations use the same backbone and training settings with seed 42, and frozen coefficients remain at their initial values. Full T-RoPE achieves the best or tied best scores across all five metrics. Independently learned coefficients also outperform their corresponding frozen controls under both rotations and both initialization choices.

Figure 5 (page 19) helps interpret these results. At shorter base scales, reverse rotation predominantly learns matching query and key signs, consistent with Figure 3(a) (page 4), while standard rotation predominantly learns opposite signs. Both patterns support additive timestamp contributions and absolute time sensitivity. Together with Table 8 (page 20), these observations suggest that T-RoPE’s initialization may provide a favorable starting point under the evaluated training setup. We interpret the approximately 4% NDCG@50 gap in Table 3 (page 8) as an empirical training result. The Figure 5 (page 19) caption defines the effective period as Pd=2πβd/∣αd∣P_d = 2\pi\beta_d/|\alpha_d| (page 19, lines 1017 to 1019), and we have revised the wording around Figure 3 (page 4) to describe frequency and period adaptation (page 4, lines 194 to 197 and 212 to 215).

We also recognize a training limitation shared by all variants. As described in Appendix B.1 (page 16, lines 837 to 844), backpropagation through ϕ=αt/β\phi = \alpha t/\beta introduces a t/βt/\beta factor into coefficient gradients. With absolute Unix timestamps, these gradients are large, and we observe global norm clipping in nearly every update. We would be happy to discuss this limitation explicitly. We are actively working to address it and hope to share our findings in future work.

Abstract comparison

The abstract’s 78–130% PixelRec HR@10 gains compare against TO-RoPE; gains over the strongest baseline, HSTU + Time RAB, are 2.8%, 7.1%, and 25.5%.

Thank you for identifying the error in the abstract. The 78% to 130% figures refer to TO-RoPE, as stated in Section 5.1 (page 8, lines 399 to 401). The corresponding HR@10 gains over HSTU + Time RAB are 2.8%, 7.1%, and 25.5% on Pixel200K, Pixel1M, and Pixel8M. We have corrected the abstract (page 1, lines 23 to 24) to report the 25.5% HR@10 gain over the strongest baseline on Pixel8M.

Thank you again for your time and effort to kindly review this paper! Hope the update paper and comments helps to answer your questions and concerns!

Thank you for the detailed response and the additional experiments, which clarified the query-timestamp sources and the TO-RoPE configuration. I also appreciate that the revision now states the equivalence between standard and reverse rotation explicitly, and I have raised my Presentation score to 3 accordingly. However, I will keep my overall rating. Given this equivalence, I still find the contribution of the non-stationary key limited, since it offers the same expressivity as standard rotation with independent coefficients. A comparison against TO-RoPE with normalized timestamps would go a long way toward addressing my remaining concerns.

Thank you for the helpful follow up and for raising the Presentation score. Following your suggestion, we have added an ablation of TO-RoPE’s timestamp normalization and frequency base in Appendix F (page 20, lines 1052 to 1068), with results in Figure 6 (page 21).

Updated TO-RoPE results in Table 2 results.

Tuning substantially improves TO-RoPE’s performance. One point we would like to highlight is that T-RoPE still performs better using its production configuration without additional tuning on Pixel200K, providing further evidence of its robustness. We appreciate this suggestion, which has made the comparison more informative.

#####Figure 6 link to check#####

DatasetMethodHR@10HR@50NDCG@10NDCG@50MRR
ML-20MHSTU + TO-RoPE32.1358.1218.5524.3015.86
ML-20MHSTU + T-RoPE32.3758.5518.6224.4215.91
Amazon BooksHSTU + TO-RoPE7.0414.534.015.653.53
Amazon BooksHSTU + T-RoPE7.6515.454.406.113.87
Pixel200KHSTU + TO-RoPE7.0516.633.966.033.60
Pixel200KHSTU + T-RoPE7.4117.374.156.303.76
Pixel1MHSTU + TO-RoPE12.6026.307.3210.306.51
Pixel1MHSTU + T-RoPE13.3527.447.8110.886.95
Pixel8MHSTU + TO-RoPE9.3919.375.517.674.91
Pixel8MHSTU + T-RoPE11.4723.026.859.356.11

Clarify Non-stationary in Practice

Regarding rotation equivalence, we agree that reverse rotation does not enlarge the function class when coefficients are independent and unconstrained. Our original Remark 1 already acknowledged that standard rotation with independent coefficients can break time translation invariance, and the revision states the equivalence explicitly (page 6, lines 300 to 305). Our empirical comparison in Table 8 focuses on initialization and training. Figure 5 (page 19) extends Figure 3(a), showing that both rotation conventions ultimately learn patterns consistent with nonstationary attention and absolute time sensitivity. These results support the usefulness of absolute time modeling, but practical performance still differs under the tested initializations and training settings despite the theoretical equivalence.

Table 8. Temporal coefficient ablation on Amazon Books

Non-stationary and Stationary denote reverse and standard key rotation, respectively. Scores are percentages. Parentheses show relative changes from full T-RoPE in the first row.

RotationCoefficient configurationHR@10HR@50NDCG@10NDCG@50MRR
Non-stationaryIndependent, same sign init. (T-RoPE)7.6515.454.406.113.87
Non-stationaryIndependent, opposite sign init.7.46 (−2.51%)15.24 (−1.35%)4.23 (−3.82%)5.97 (−2.33%)3.72 (−3.89%)
Non-stationaryShared7.41 (−3.11%)15.21 (−1.57%)4.23 (−3.82%)5.94 (−2.74%)3.73 (−3.67%)
Non-stationaryFrozen, same sign init.7.31 (−4.43%)15.09 (−2.36%)4.16 (−5.34%)5.87 (−3.98%)3.67 (−5.19%)
Non-stationaryFrozen, opposite sign init.7.19 (−5.98%)14.75 (−4.55%)4.11 (−6.48%)5.77 (−5.63%)3.62 (−6.49%)
StationaryIndependent, same sign init.7.51 (−1.79%)15.37 (−0.51%)4.28 (−2.67%)6.11 (0.00%)3.77 (−2.59%)
StationaryIndependent, opposite sign init.7.42 (−2.99%)15.23 (−1.40%)4.22 (−4.01%)5.93 (−2.88%)3.71 (−4.11%)
StationaryShared7.28 (−4.79%)14.90 (−3.54%)4.17 (−5.15%)5.84 (−4.39%)3.67 (−5.19%)
StationaryFrozen, same sign init.7.24 (−5.39%)14.71 (−4.77%)4.16 (−5.34%)5.80 (−5.08%)3.67 (−5.19%)
StationaryFrozen, opposite sign init.7.35 (−3.95%)15.10 (−2.25%)4.19 (−4.77%)5.88 (−3.70%)3.69 (−4.76%)

We hope this normalized comparison addresses your remaining concern. If our response addresses your concerns, we’d appreciate it if you would consider updating your assessment. We’re happy to clarify anything that remains open.

Reviewer_haznOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to leverage the actual timestamps of user actions in sequential recommendation to address the limitations of sequence-based Rotary Position Embeddings, which cannot directly represent actual time intervals or calendar time. The authors propose T-RoPE, which integrates Unix timestamps, learnable query/key time coefficients, a multiscale frequency bank, Shifted Query Alignment, and non-stationary key rotation into the rotary position encoding framework; the authors also analyze its time-translation invariance. T-RoPE was evaluated across five public benchmarks, a proprietary Shop dataset containing over 6 billion interactions, and an online A/B test. The results show that T-RoPE achieves state-of-the-art performance across all metrics on the public benchmarks, while also delivering improvements on industrial data and online conversion rates.

Critical strengths

The consistent performance of the proposed method across multiple public benchmarks lends high credibility to the experimental results. As shown in Table 2, T-RoPE achieves the best scores across all five metrics on all five benchmarks. Compared to the best-performing baselines for each metric, T-RoPE yields relative improvements of approximately 8.3%–11.1% on the Amazon Books dataset and 22.0%–27.8% on the Pixel8M dataset. These findings demonstrate that the performance gains are not limited to a single dataset or metric, further reinforcing the reliability of the results.

Critical weaknesses

Section 3.4 mentions that the timestamp of the next event t_m+1 is used during training, while the current session time t_now is used during deployment. However, Section 5 does not explicitly clarify which timestamp is used for the final offline evaluations on the public benchmarks and the Shop dataset. If the task involves predicting a future, unknown user action based on a historical sequence, the actual timestamp of the next event would not necessarily be available. The authors should clarify the source of the query timestamp used in the offline evaluations, whether this timestamp is actually available at the time of recommendation generation, and whether all time-aware baselines were provided with the same temporal information.

Questions for authors

Could the authors clarify the specific source of the query timestamp used in the offline evaluations, and whether this timestamp is known at the time when recommendations are actually generated? Furthermore, did all time-aware baselines utilize the same temporal information? Additionally, please specify the exact point in time at which the recommendation list is generated, and clarify whether the user's history is updated to include new actions that occur on that same day.

3 comments

Thank you for your time and careful review. We really appreciate that!

Could the authors clarify the specific source of the query timestamp used in the offline evaluations, and whether this timestamp is known at the time when recommendations are actually generated?

We hope the table below clarifies how query timestamps are determined under different settings. In particular:

  • Public Offline evaluations (Table 2, page 7): we use ground-truth timestamps of the held-out interactions
  • Internal Offline evaluations (Table 3, page 8): uses midnight UTC with histories frozen throughout the held out day
  • Online experiment (Table 4, page 9) and production inference load: we use a timestamp available at recommendation time
SettingQuery timestampHistory and evaluation protocol
TrainingNext interaction timestamp tm+1t_{m+1}The query is aligned with the event being predicted.
Public benchmarks, Table 2 (page 7)Ground truth timestamp of the held out interactionLeave one out for each user, following the target time convention used by HSTU Time RAB. Specifically: HSTU + Time RAB, HSTU + TO-RoPE, and HSTU + T-RoPE all receive the same held out interaction timestamp through Time RAB. T-RoPE also uses that timestamp in the query rotation
Internal Shop evaluation, Table 3 (page 8)UTC (00:00:00)A global temporal holdout covering one day. Training interactions and histories precede the cutoff. The day's interactions serve as evaluation labels.
Production, Section 5.3 (page 9)Actual user login timeRecommendations use events already available in the database at that moment, including earlier events from the same day when available.

We have clarified the query timestamp sources in Section 5.1 (page 7, lines 371 to 374), Section 5.2 (page 8, lines 411 to 414), and Section 5.3 (page 9, lines 447 to 451).

Furthermore, did all time-aware baselines utilize the same temporal information?*

Thank you for asking about the temporal information available to the baselines. Yes. The public benchmark (Table 2, page 7), HSTU + Time RAB, HSTU + TO-RoPE, and HSTU + T-RoPE all receive the same held out interaction timestamp through Time RAB. T-RoPE also uses that timestamp in the query rotation, so the principal HSTU comparisons share the same information about the target event time. As for TiSASRec, it only uses historical timestamps.

Additionally, please specify the exact point in time at which the recommendation list is generated, and clarify whether the user's history is updated to include new actions that occur on that same day.*

Thank you for asking us to clarify when recommendations are generated and histories are updated to make sure there is no message leaky. We assume that you are referring to inference in production. In this case, the user's actual login time is used to generate the recommendations. When available, earlier actions from the same day are appended to the user’s history.

Thank you again for your time and effort to kindly review this paper!

Thank you for the detailed clarification and for specifying where these details have been added to the manuscript. Your explanation addresses my questions about the timestamp sources and history updates in the internal Shop evaluation and production setting. It is also helpful to know that the three HSTU variants receive the same target timestamp, which alleviates my concern about unequal access to temporal information within these comparisons.

My remaining concern is about the interpretation of the public benchmark results. Since these evaluations use the ground-truth timestamp of the held-out interaction, they evaluate recommendation conditioned on a known target time. This does not directly establish performance when predicting a future interaction whose time is unknown. I would appreciate an explicit statement of this distinction and its implications for the scope of the results.

Additionally, since TiSASRec uses only historical timestamps, it would be helpful to make this difference in available temporal information clear when discussing comparisons with that baseline.

Overall, the response substantially addresses my questions about the evaluation protocol. My remaining concern is primarily about clearly communicating the assumptions and scope of the public benchmark comparisons. Taking these clarifications into account, I will maintain my original rating for now.

Thank you for this helpful follow up. We agree that the public evaluation protocol and its scope should be explicit. Our evaluation follows the target time convention used by HSTU with Time RAB and across our timestamp conditioned generative recommendation models. We have clarified in Section 5.1 (page 7, lines 372 to 377) that the compared HSTU variants receive the same ground truth timestamp of the held out interaction. We now also explicitly state in Section 5.1 (page 7, lines 375 to 376) that Table 2 evaluates recommendation conditioned on a known target time and does not directly establish performance when the future interaction time is unknown.

One point we would like to highlight is that our online A/B test uses the actual user login time, which is available when recommendations are generated. The positive online results provide direct evidence that T-RoPE remains effective in actual serving conditions without access to future interaction timestamps. We have specified this timestamp source in Section 5.3 (page 9, lines 450 to 452), making the distinction between the public benchmark protocol and actual serving conditions explicit.

We also agree that the difference in temporal inputs for TiSASRec should be explicit when interpreting the comparison. As summarized in Table 1 (page 2, lines 54 to 61) and discussed in the Introduction (page 2, lines 66 to 69), TiSASRec models historical time intervals. We have now stated directly in Section 5.1 (page 7, lines 376 to 377) that TiSASRec uses only historical timestamps.

Thank you for helping us communicate these assumptions and the scope of the results more clearly.

Reviewer_qUGwOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper provides a theoretical analysis of time-translation invariance and demonstrates that T-RoPE can represent absolute-time-dependent attention patterns. Experiments on five public benchmarks, a large-scale industrial dataset, and an online A/B test demonstrate consistent improvements over existing recommendation baselines, with relatively low computational overhead.

Critical strengths
  1. The paper identifies a meaningful limitation of conventional positional embeddings in sequential recommendation, where interaction indices fail to capture irregular time intervals and periodic behavioral patterns. T-RoPE addresses this limitation within the existing rotary attention framework, making it potentially applicable to large-scale recommendation systems without substantial architectural modifications.
  2. The experiments cover five public benchmarks and an industrial dataset containing over six billion interactions. The consistent improvements across datasets, together with component-level ablations and an online A/B test, provide encouraging evidence of the method's practical effectiveness. The computational analysis further supports its feasibility for large-scale deployment.
Critical weaknesses
  1. The theoretical analysis establishes that T-RoPE breaks time-translation invariance. However, Remark 1 explicitly acknowledges that standard timestamp-based RoPE with independent query and key coefficients can also achieve this property. More importantly, since the proposed coefficients are unconstrained, reversing the key rotation is mathematically equivalent to negating the learnable key coefficient in standard RoPE. Consequently, the proposed non-stationary mechanism does not appear to introduce additional representational capacity beyond standard timestamp RoPE with independently learnable coefficients. Without a direct comparison against this equivalent parameterization, the necessity of the proposed mechanism and its claimed theoretical contribution remain insufficiently established.
  2. The shifted query mechanism uses the next interaction's timestamp during training but the current session time during inference. The paper does not clearly explain how query timestamps are obtained during offline evaluation, raising concerns about potential future-information leakage or training–inference discrepancies. Furthermore, the abstract highlights improvements ranging from 78% to 130% over the strongest baseline on PixelRec, whereas Table 2 shows substantially smaller improvements over the actual strongest competing method. These issues weaken confidence in the interpretation of the reported performance gains.
Questions for authors
  1. Given that the learnable query and key coefficients are unconstrained, how does the proposed non-stationary key rotation differ mathematically from standard timestamp-based RoPE with independently learnable coefficients? Can the authors explain what additional benefit the proposed parameterization provides beyond this equivalent formulation?
  2. How are query timestamps determined during offline evaluation, particularly under the leave-one-out protocol? Does the model use the ground-truth timestamp of the held-out interaction or a timestamp available at recommendation time? How is consistency between training, offline evaluation, and online inference ensured?
  3. Could the authors explain the exact configuration of the "- Non-Stationary" ablation in Table 3? In particular, are the query and key coefficients still learned independently, and how should the observed performance difference be interpreted given the potential equivalence between the two rotation parameterizations?
2 comments

Thank you for your time and careful review. We really appreciate that!

Q2 Query timestamps

How are query timestamps determined during offline evaluation, particularly under the leave-one-out protocol? Does the model use the ground-truth timestamp of the held-out interaction or a timestamp available at recommendation time? How is consistency between training, offline evaluation, and online inference ensured?

Thank you for raising this. We hope the table below clarifies how query timestamps are determined under different settings. In particular:

  • Public Offline evaluations (Table 2, page 7): we use ground-truth timestamps of the held-out interactions
  • Internal Offline evaluations (Table 3, page 8): uses midnight UTC with histories frozen throughout the held out day
  • Online experiment (Table 4, page 9) and production inference load: we use a timestamp available at recommendation time
SettingQuery timestampHistory and evaluation protocol
TrainingNext interaction timestamp tm+1t_{m+1}The query is aligned with the event being predicted.
Public benchmarks, Table 2 (page 7)Ground truth timestamp of the held out interactionLeave one out for each user, following the target time convention used by HSTU Time RAB. Specifically: HSTU + Time RAB, HSTU + TO-RoPE, and HSTU + T-RoPE all receive the same held out interaction timestamp through Time RAB. T-RoPE also uses that timestamp in the query rotation
Internal Shop evaluation, Table 3 (page 8)UTC (00:00:00)A global temporal holdout covering one day. Training interactions and histories precede the cutoff. The day's interactions serve as evaluation labels.
Production, Section 5.3 (page 9)Actual user login timeRecommendations use events already available in the database at that moment, including earlier events from the same day when available.

For timestamp shifting, T-RoPE follows the same approach as HSTU+TimeRAB, which also uses shifted timestamps for public datasets in the original HSTU paper. Regarding consistency between offline and online inference, in all our offline evaluations we apply a temporal split to construct a hold-out evaluation set comprising events occurring at timestamps strictly larger than those occurring in the users’ sequences. In this fashion, we are essentially mimicking the online serving setting, where recommendations are generated a priori, and are evaluated based on future user interactions. As for training, we rely on the correlation between the next-token prediction loss and retrieval metrics (e.g., recall) computed on the hold-out set to ensure consistency.

We have updated our paper and clarified the query timestamp sources in Section 5.1 (page 7, lines 371 to 374), Section 5.2 (page 8, lines 411 to 414), and Section 5.3 (page 9, lines 447 to 451).

Abstract comparison

Furthermore, the abstract highlights improvements ranging from 78% to 130% over the strongest baseline on PixelRec, whereas Table 2 (page 7) shows substantially smaller improvements over the actual strongest competing method.

Thank you for catching the abstract error. The 78% to 130% figures refer to TO-RoPE, as stated in Section 5.1 (page 8, lines 399 to 401). The corresponding HR@10 gains over HSTU + Time RAB are 2.8%, 7.1%, and 25.5% on Pixel200K, Pixel1M, and Pixel8M. We have corrected the abstract (page 1, lines 23 to 24) to report the 25.5% HR@10 gain over the strongest baseline on Pixel8M.

Q1 and Q3 Rotation and ablation

Given that the learnable query and key coefficients are unconstrained, how does the proposed non-stationary key rotation differ mathematically from standard timestamp-based RoPE with independently learnable coefficients? Can the authors explain what additional benefit the proposed parameterization provides beyond this equivalent formulation?

Could the authors explain the exact configuration of the "- Non-Stationary" ablation in Table 3 (page 8)? In particular, are the query and key coefficients still learned independently, and how should the observed performance difference be interpreted given the potential equivalence between the two rotation parameterizations?

Thank you for identifying the parameter equivalence and suggesting these controlled comparisons. We agree that standard and reverse rotation represent the same function class when query and key coefficients are independent and unconstrained. Theorem 1 assumes shared coefficients (page 6, lines 291 to 295), and Remark 1 now explicitly states the equivalence (page 6, lines 300 to 305). Row 2 of Table 3 (page 8) retains independent coefficients, shifted queries, and the multiscale frequency bank.

Following your suggestions, we have added a controlled ablation on Amazon Books in Appendix E (page 18, lines 938 to 959), with results in Table 8 (page 20, lines 1027 to 1047). Each industrial experiment in Table 3 (page 8) requires approximately 10,000 B300 GPU hours, so Amazon Books provides an accessible setting for comparing initialization, sharing, and freezing. All configurations use the same backbone and training settings with seed 42, and frozen coefficients remain at their initial values. Full T-RoPE achieves the best or tied best scores across all five metrics. Independently learned coefficients also outperform their corresponding frozen controls under both rotations and both initialization choices.

Figure 5 (page 19) helps interpret these results. At shorter base scales, reverse rotation predominantly learns matching query and key signs, consistent with Figure 3(a) (page 4), while standard rotation predominantly learns opposite signs. Both patterns support additive timestamp contributions and absolute time sensitivity. Together with Table 8 (page 20), these observations suggest that T-RoPE’s initialization may provide a favorable starting point under the evaluated training setup. We interpret the approximately 4% NDCG@50 gap in Table 3 (page 8) as an empirical training result. The Figure 5 (page 19) caption defines the effective period as Pd=2πβd/∣αd∣P_d = 2\pi\beta_d/|\alpha_d| (page 19, lines 1017 to 1019), and we have revised the wording around Figure 3 (page 4) to describe frequency and period adaptation (page 4, lines 194 to 197 and 212 to 215).

We also recognize a training limitation shared by all variants. As described in Appendix B.1 (page 16, lines 837 to 844), backpropagation through ϕ=αt/β\phi = \alpha t/\beta introduces a t/βt/\beta factor into coefficient gradients. With absolute Unix timestamps, these gradients are large, and we observe global norm clipping in nearly every update. We would be happy to discuss this limitation explicitly. We are actively working to address it and hope to share our findings in future work.

Thank you again for your time and effort to kindly review this paper!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.