acceptodds
Under review as a conference paper at ICLR 2027

Longer History or the Right History? Candidate-Focal History Compression for Long-Sequence Recommendation

Abstract

Long-sequence recommendation seeks to improve ranking by leveraging longer user histories, under the premise that more historical interactions provide richer evidence of user preference. However, more history is not always better: historical evidence is often redundant, and, crucially, what is useful depends on the candidate being ranked. This observation motivates us to reframe long-sequence recommendation from blindly processing longer histories to *candidate-focal history compression*: preserving the right historical evidence for each candidate. This reframing raises two coupled challenges: a shared compressed history may discard candidate-specific evidence (**C1**), while independently constructing a history for every candidate multiplies redundant computation (**C2**). We develop a **quality–write–state tradeoff framework** to study this quality–efficiency tradeoff, which characterizes the first-order contribution of each recurrent write to the ranking objective, and then quantifies when candidates can share sparse histories with bounded error. Building on this theoretical frame, we introduce **DeltaRec**, which distills history contributions into a lightweight selector to guide both candidate-focal history-event writing and utility-based state sharing. Across three datasets and four backbones, we show that DeltaRec (1) improves ranking metrics by up to ( on average) across all datasets over the corresponding backbones; (2) achieves a P50 prefill speedup over per-candidate state construction; and (3) extends the empirical quality–efficiency Pareto frontier compared to other baselines, achieving higher ranking quality at lower prefill cost than full-history execution. Together, these results support selecting and sharing the right history rather than blindly extending it. We release our code base at https://anonymous.4open.science/r/delta-rec-DB35.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Recommendation System, History Compression, Recurrent Linear Attention

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_C1uxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeltaRec, a candidate-focal history compression framework for long-sequence recommendation. Motivated by the redundancy in long user histories and candidate-dependent historical relevance, DeltaRec uses Counterfactual Write Influence (CWI) to identify important interactions and groups candidates with similar historical utility patterns to share recurrent states. Experiments demonstrate improved recommendation accuracy and computational efficiency across multiple datasets and backbones.

Critical strengths
  1. The paper addresses a meaningful problem in long-sequence recommendation, with convincing empirical observations highlighting historical redundancy and candidate-dependent relevance.
  2. The proposed Quality–Write–State framework provides theoretical insights into candidate-specific history selection and utility-preserving state sharing.
  3. DeltaRec integrates CWI-guided history selection with utility-based candidate grouping, offering a coherent approach to balancing recommendation quality and computational efficiency.
Critical weaknesses
  1. The core idea of selecting candidate-relevant historical interactions has been extensively explored in prior studies, such as SIM [1] and TWIN [2]. Although the proposed CWI-based selection and state-sharing mechanisms offer additional contributions, the paper could better clarify its methodological novelty over existing target-aware history selection approaches.
  2. The theoretical analysis relies on first-order approximations around the full-history trajectory, while the proposed method performs substantial history compression. Moreover, the error bound in Theorem 2 may become loose under aggressive compression. This raises concerns about how effectively the theoretical results explain the empirical performance of DeltaRec.
  3. Although DeltaRec achieves substantial speedups over per-candidate state construction, the efficiency gains over the original backbones are less convincing. As shown in Table 2, DeltaRec introduces higher overall latency than the original FuXi-Linear backbone, while the improvement over HSTU is relatively modest. This somewhat weakens the practical efficiency claims.

References:

[1] Pi, Qi, et al. "Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction." Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020.

[2] Chang, Jianxin, et al. "TWIN: TWo-stage interest network for lifelong user behavior modeling in CTR prediction at kuaishou." Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023.

Questions for authors

In Table 1, DeltaRec achieves substantial improvements over the original backbones. However, Section 5 indicates that DeltaRec replaces the original attention mechanisms with GDR, making it difficult to attribute these gains solely to the proposed history compression strategy. Although Table 3 includes a full-length GDR variant, it only reports limited settings and metrics. Could the authors provide more comprehensive controlled comparisons across all backbones to isolate the contributions of GDR, CWI-based history selection, and state sharing?

1 comment
AuthorsOpenReview ↗

Rebuttal to Reviewer #C1ux

We sincerely thank the reviewer for recognizing our 'meaningful problem', 'convincing observations', 'theoretical insights', and 'coherent approach'. We will answer the questions in the following.

Q1: ... Could the authors provide more comprehensive controlled comparisons across all backbones to isolate the contributions of GDR, CWI-based history selection, and state sharing?

Re: We thank you for the valuable suggestions. To address your concern, we separate GDR and the candidate-focused CWI compression strategy. In rTable 8, we keep all settings the same, including loaded checkpoints, and only change the history selection strategy. rTable 9 uses the various selection strategies on dense self-attention. The two results show the effectiveness of the proposed selection strategy.

rTable 8. Varies selection strategy on freezed GDR.

Selection StrategyGAUC
Full history0.53501
Recency0.53921
Candidate similarity0.54075
CWI-distilled selector (ours)0.54197

rTable 9. Varies selection strategy on native dense self-attention HSTU.

Native HSTU inputMacro-GAUC
Full history0.5347
Recency0.5375
Candidate similarity0.5400
CWI-distilled selection (ours)0.5418

We also show the cumulative components ablations, which demonstrate the effectiveness of both DeltaRec with GDR as a unified framework and DeltaRec selection strategy.

rTable 10. Cumulative components ablations

ComponentsGAUC
HSTU baseline0.5295
HSTU + GDR0.5287
HSTU + GDR + CWI selection0.5364
HSTU + GDR + CWI selection + grouping/shared state(DeltaRec, G=4)0.5418

W1: ...the paper could better clarify its methodological novelty over existing target-aware history selection approaches.

Re: Thanks for the valuable suggestion and acknowledgements of our contributions. We clarify that candidate-aware methods were mainly explored via two-stage traditional recommendation, e.g., history modeling and candidate cross-attention, and the user model itself is indeed not target-aware. However, our perspective pushes the target-aware principle forward to unified scalable generative models where target-aware is usually cost-unacceptable, via the state-write tradeoff mechanism. We highlight the quality-efficiency frontier in a unified model instead of target-aware alone.

We will add a more detailed discussion to the Related Work to clarify this distinction. "Existing target-aware methods, such as SIM and TWIN, demonstrate the importance of candidate awareness, but they model the candidate information outside the model itself. Our work addresses the quality–computation trade-off in unified generative recommendation, where candidate-specific histories can require repeated construction. We use the same candidate-focal write utility to jointly determine which historical updates to retain and which candidates can share the resulting recurrent states."

W2: ... raises concerns about how effectively the theoretical results explain the empirical performance of DeltaRec.

Re: We clarify that neither theorem claims that compression necessarily improves performance. Theorem 1 explains candidate-dependent write importance, while Theorem 2 characterizes the quality–efficiency trade-off of state sharing at a fixed retention budget. We agree that aggressive compression can indeed loosen the finite-deletion bound; while tightly predicting compression error is not their objective. Moreover, we empirically examine the effect of retention on ranking quality in Figure 4(c ), including degradation when too little history is retained.

W3: ...the efficiency gains over the original backbones are less convincing.... higher overall latency...

Re: Thanks for the detailed review. We clarify that the overall latency shown in Table 2 simply averages prefill and serving latency, while the 100% cache hit rate implies prefill only needs to run once in cache construction time, and serving latency dominates the real serving cost, which shows significant gains over the backbone: looking into the serving latency in rTable 11, DeltaRec achieves significant gains with 37% and 89% less serving latency. We will revise the related claims in the final version.

rTable 11. Latency and cache hits across all 11,010 eligible KuaiRand-1K histories.

MethodPrefillServing latencyState-cache hit rate
Native HSTU8.738 ± 0.0346.663 ± 0.039100%
DeltaRec (HSTU)9.540 ± 0.0494.175 ± 0.014100%
Native FuXi-Linear15.516 ± 0.0032.157 ± 0.008100%
DeltaRec (FuXi-Linear)36.223 ± 0.0840.234 ± 0.007100%
Reviewer_YSL7OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper reformulates long-sequence recommendation as a problem of candidate-aware history compression. The authors argue that different candidate items may rely on different parts of a user's historical behavior; consequently, using the entire history or a uniformly compressed version of it, often retains a significant amount of irrelevant information. Conversely, selecting a unique history and constructing a recurrent state for each individual candidate would lead to excessive redundant computation. To resolve this trade-off between quality and efficiency, the paper introduces DELTAREC. First, the authors define "Counterfactual Write Influence" (CWI), which uses a first-order approximation to measure the impact of a specific historical "write" (an interaction event) on the ranking objective for a given candidate; the method then selects the most influential historical events subject to a fixed budget. Second, a lightweight selector is trained to emulate CWI, thereby avoiding the need for online gradient computation. Furthermore, candidate items are grouped based on their historical utility patterns, with candidates in the same group sharing a compressed history and a corresponding recurrent state (GDR state). This reduces the cost of state construction from the candidate level to the group level. The paper also provides a theoretical analysis regarding the selection of candidate-aware historical events and the error introduced by state sharing. Experiments were conducted on the ML-20M, Amazon Books, and KuaiRand-1K datasets, utilizing four different backbone models: HSTU, LinRec, FuXi-Linear, and BlossomRec. The authors report results covering recommendation quality, prefill latency, cached serving latency, the proportion of history retained, and the number of candidate groups.

Critical strengths
  1. The paper offers a meaningful reframing of long-sequence recommendation as candidate-aware history compression. Instead of assuming that a longer history is universally beneficial, it recognizes that different candidates may depend on different historical interactions. This perspective directly addresses the trade-off between preserving candidate-specific evidence and avoiding redundant computation.
  2. The proposed method is technically comprehensive. Counterfactual Write Influence (CWI) estimates the importance of each recurrent state write for candidate-specific ranking, while the theoretical analysis characterizes both history selection and state sharing. The distilled selector, candidate grouping, sparse fine-tuning, and cached execution together form a reasonably complete system. The evaluation also covers multiple datasets and backbones and examines recommendation quality, prefill latency, serving latency, and the quality-efficiency trade-off.
Critical weaknesses
  1. The main-table gains cannot be clearly attributed to candidate-aware history selection. DELTAREC replaces the native attention module with GDR and additionally introduces full-history warm-up, CWI distillation, and sparse-history fine-tuning. As acknowledged in the appendix, comparisons with native backbones evaluate the complete adapted system rather than the selector alone. On ML-20M, full-history GDR achieves an HR@10 of 0.2664, exceeding both DELTAREC (0.2364) and native HSTU (0.1496), suggesting that much of the reported improvement may come from architectural and training changes. A similar concern arises on Amazon Books: its mean and median history lengths are 12.47 and 5, while the experiment retains at least 32 events. Most samples are therefore not compressed, yet substantial gains are still reported.
  2. The efficiency evidence does not establish a general speedup over the native backbones. The reported 17.48× and 27.11× prefill improvements are measured against the expensive K-state implementation rather than the original models. Compared with native HSTU, G-state prefill increases from 8.672 ms to 10.434 ms; for FuXi-Linear, it increases from 15.429 ms to 36.479 ms, with overall latency rising from 18.568 ms to 36.847 ms. Moreover, the measurements exclude utility prediction, candidate grouping, Top-B selection, and cache preparation, while cache hit rates, memory overhead, update frequency, and invalidation costs are not reported. The main quality results also lack multi-seed variance and significance testing, which is important because some gains are small and DELTAREC with BlossomRec decreases KuaiRand performance by 0.4%.
Questions for authors
  1. Could the authors conduct a controlled comparison using the same GDR architecture, initialization, training data, optimization budget, and final checkpoint, while varying only the history-selection strategy, such as Random, Recency, candidate similarity, and CWI? This would help isolate the contribution of the proposed selector.
  2. How much of the improvement over the native backbones comes from replacing attention with GDR, sparse-history fine-tuning, and the CWI selector, respectively? Please provide a step-by-step cumulative ablation that separates these factors.
  3. On Amazon Books, what proportion of evaluation samples are actually compressed under the minimum retention rule of 32 events? Please also report the realized retention ratio and performance separately for users whose histories exceed 32 events.
  4. Could the authors report end-to-end latency including utility prediction, candidate grouping, Top-B selection, cache lookup and construction, and cache misses? Please compare this complete latency directly with the native backbones and report cache hit rates, memory overhead, and results across multiple random seeds.
4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer #YSL7. Part 1/2

We sincerely thank the reviewer for acknowledging the "meaningful problem framing", the "comprehensive" technical contribution, and the evaluations. We address the questions below.

Q1&W1: Could the authors conduct a controlled comparison using the same GDR architecture, initialization, training data, optimization budget, and final checkpoint, while varying only the history-selection strategy, such as Random, Recency, candidate similarity, and CWI?

Re: Thanks for the valuable suggestion. We first clarify that the ablation study in Table 3 uses the same GDR architecture, initialization, training data, and optimization budget but retrains the model. To address your concern, we compare alternative selection strategies under fully controlled conditions using the same checkpoint on KuaiRand-1k. The results are shown below.

rTable 4. Varying selection strategy on frozen GDR.

Selection StrategyGAUC
Full history0.535010
Random0.537242
Recency0.539215
Candidate similarity0.540751
CWI-distilled selector0.541472

These results further confirm the findings of our ablation study (Table 3), showing that the CWI-distilled selector performs better.

Q2&W1: How much of the improvement over the native backbones comes from replacing attention with GDR, sparse-history fine-tuning, and the CWI selector, respectively? Please provide a step-by-step cumulative ablation that separates these factors.

Re: We appreciate the valuable suggestion. To address your concern, we ran a step-by-step cumulative ablation on KuaiRand-1k by retraining the model under fully controlled conditions. The results are shown below:

rTable 5. Cumulative ablation.

ComponentsGAUC
HSTU baseline0.5295
HSTU + GDR0.5287
HSTU + GDR + CWI selection0.5364
HSTU + GDR + CWI selection + grouping/shared state(DeltaRec, G=4)0.5418

The results show that when GDR combines with the proposed CWI and state grouping, the results can be greatly improved, which demonstrates the effectiveness of both the unified framework DeltaRec and the candidate-focal CWI selection strategy.

Q3: On Amazon Books, what proportion of evaluation samples are actually compressed under the minimum retention rule of 32 events? Please also report the realized retention ratio and performance separately for users whose histories exceed 32 events.

Re: We appreciate the insightful comments. After analyzing the Amazon Books dataset, we find that approximately 93% of samples have history lengths <32. We first evaluate the subsets separately and then analyze different minimum numbers of recent events that must be retained:

rTable 6. Separate results.

SubsetMandatory Recent RReal retention ratio over datasetNative HSTU full length HR@10 / NDCG@10DeltaRec HR@10 / NDCG@10
n≤32321.000000.03338 / 0.017400.04514 / 0.02447
n≤32160.97729same0.04515 / 0.02447
n≤3280.89512same0.04522 / 0.02456
n≤3240.72830same0.04576 / 0.02491
n≤3200.33814same0.04081 / 0.02229
n>32320.566940.01562 / 0.007420.03923 / 0.02097

The results show that all DeltaRec configurations outperform the HSTU baseline. For n≤32, DeltaRec's performance improves as R decreases from 32 to 4, showing the effectiveness of the proposed strategy. However, performance deteriorates when R=0, showing the importance of short-term interests.

AuthorsOpenReview ↗

Rebuttal to Reviewer #YSL7. Part 2/2

Q4 & W2: Could the authors report end-to-end latency including utility prediction, candidate grouping, Top-B selection, cache lookup and construction, and cache misses? Please compare this complete latency directly with the native backbones and report cache hit rates, memory overhead, and results across multiple random seeds.

Re: Thanks for the valuable comments. We clarify that Table 2 reports prefill and serving latency separately. Serving latency excludes disk loading and CPU-to-GPU cache transfer for both the baselines and DeltaRec, while the candidate grouping, utility prediction, Top-B selection, and cache construction are performed offline.

Prefill constructs reusable history states, while serving performs readout and scoring. Online requests first check the state cache, falling back to cached indices or online selection when necessary. The following shows the request process after considering cache misses:

Assign candidates to fixed groups → state-cache lookup
State hit: serving (readout → score)
State miss: index-cache lookup
  Index hit: prefill → serving
  Index miss: utility prediction → Top-B selection → prefill → serving

Newly computed indices and states are written back to their respective caches. A state hit therefore bypasses both selection and prefill; utility prediction and Top-B selection are required online only on a double miss.

Following your suggestion, we report cache coverage and measured cache storage alongside latency on three random seeds. Although the 100% coverage is measured after offline construction for both baselines and ours, we agree these methods need further dedication in real industrial systems.

rTable 7a. Prefill, cached serving, and measured cache storage.

ModelPrefill (ms/batch)Serving (ms/batch)Prewarmed cache coverageTotal cache (MiB)
Native HSTU8.738 ± 0.0346.663 ± 0.039100%220845.201
DeltaRec HSTU9.540 ± 0.0494.175 ± 0.014100%55222.115
Native FuXi15.516 ± 0.0032.157 ± 0.008100%1078.723
DeltaRec FuXi36.223 ± 0.0840.234 ± 0.007100%193.619

Additionally, we report individual operation costs in rTable 7b and directly measured cache-path latency in rTable 7c. Values are mean ± std across three request seeds. Candidate assignment takes approximately 4 μs/batch, while each cache lookup takes less than 0.5 μs/batch. Finally, when a state hits, the process takes 22.52 ms; when both state and index miss, it takes 54.45 ms on average.

rTable 7b. Individual operation latency per batch. Lookup values indicate hit / miss.

OperationUnitDeltaRec HSTUDeltaRec FuXi-Linear
Candidate assignmentμs4.474 ± 0.0394.179 ± 0.019
State-cache lookupμs0.4138 ± 0.0005 / 0.3762 ± 0.00050.4077 ± 0.0033 / 0.3827 ± 0.0025
Index-cache lookupμs0.3876 ± 0.0017 / 0.3712 ± 0.00220.3826 ± 0.0019 / 0.3748 ± 0.0025
Utility predictionms5.456 ± 0.0122.722 ± 0.022
Top-B selectionms0.906 ± 0.0040.843 ± 0.011

rTable 7c. Routed latency with CPU-resident caches (ms/batch).

ModelState hitState miss, index hitDouble miss
DeltaRec HSTU22.524 ± 0.61448.014 ± 0.30954.450 ± 0.286
DeltaRec FuXi-Linear4.116 ± 0.06843.181 ± 0.33346.885 ± 0.399

We acknowledge the reviewer for indicating that cache miss latency may occur in industrial serving. We will highlight this in the revisions.

Thanks for your response. The authors have addressed most of my concerns, such as the more rigorous experiment and more clear experiment description. However, the current score still reflects the quality and novelty of this paper.

Dear Reviewer #YSL7,

Thanks for your valuable time, and we sincerely appreciate your constructive suggestions. We will include these revisions in our next version.

However, we would greatly appreciate further details regarding Novelty and Quality, which were not explicitly raised as concerns in your original review. In your latest reply, you acknowledge that our responses have addressed most of your original concerns, which primarily focused on experimental rigor and efficiency. Nevertheless, the current score is maintained based on Novelty and Quality, without specifying what remains insufficient. We believe this may not align with the Review Instructions "Be specific: Do not make vague statements in your review, as they are unfairly difficult for authors to address.", and "Be fair: Do not let personal feelings affect your review."

Regarding Novelty, we would like to clarify that all three reviewers recognized the "problem framing" in their strengths. Specifically, reviewer #vX4M described our idea as "intuitive" and "impactful"; both your original review and reviewer #C1ux highlighted the "meaningful problem" and technical contribution. We believe these comments in the strengths sections reflect the reviewers' recognition of the novelty of our work. We further clarify that our novelty lies not only in the theoretical insights and coherent system design, but also in reframing the problem from scaling history length to finding the right histories, while jointly optimizing history selection and recurrent-state sharing.

Regarding Quality, we would like to clarify that the presentation, technical contributions, and experiments have been carefully polished in this paper. Moreover, we have provided additional controlled experiments, cumulative ablations, and detailed efficiency measurements to address your original concerns. Given that your latest reply explicitly acknowledges the improved experimental rigor and clarity, we would appreciate an objective explanation of which aspects of the paper's quality remain insufficient.

Finally, we sincerely respect the reviewer for your constructive comments and valuable time. However, we believe that maintaining the original score based on general statements about Novelty and Quality, without any specific explanations, is "unfair" and may contradict the Reviewer Principles of ICLR . Given your acknowledgment that most of the original concerns have been addressed, we would sincerely appreciate it if you could reconsider the score in light of the additional evidence provided. If any concerns regarding novelty or quality remain, we would also be grateful for your further clarification to help us strengthen the paper.

Best wishes

Reviewer_vX4MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the inefficiency and redundancy of processing ultra-long user histories in sequential recommendation systems. Instead of blindly scaling up history length, the authors reframe the problem as candidate-focal history compression, identifying and preserving only the right historical evidence for each specific candidate item. To tackle the tension between candidate-specific ranking quality and computational efficiency, the authors propose DELTAREC, a system that distills CWI into a lightweight selector for sparse history selection and groups candidates based on utility profiles to share state construction.

Critical strengths
  • Shifting the focus from "longer history" to "the right history" is a highly intuitive and impactful perspective. It directly addresses the core bottleneck of long-sequence recommendation: redundancy and candidate-dependent relevance.
  • The experiments are thorough, covering three datasets with vastly different history length distributions (ML-20M, Amazon Books, KuaiRand-1K) and four state-of-the-art backbones. The ablation studies and parameter sensitivity analyses further validate the robustness of the proposed components.
Critical weaknesses
  • While the online serving efficiency is heavily emphasized, the paper lacks a detailed quantification of the offline costs. Specifically, the computational overhead of clustering training-data CWI and the storage/memory footprint of maintaining the index-cache and state-cache for dynamic, large-scale candidate pools are not thoroughly discussed.
  • The proposed "write" and "state" abstraction is inherently tied to linear recurrent attention mechanisms. It is unclear how easily this framework can be adapted to traditional dense self-attention models or other non-recurrent architectures.
  • The CWI-distilled selector relies on event and candidate features to predict utility. The paper does not explicitly discuss how DELTAREC can strike balance between focusing on selecting the right histories from long sequences and learning from short-term interests, and how it deals with cold start settings (items with sparse or no historical interaction features), where the selector's utility predictions might be highly unreliable.
Questions for authors

NA

1 comment
AuthorsOpenReview ↗

Rebuttal to Reviewer #vX4M

We sincerely thank the reviewer for recognizing our 'intuitive', 'impactful' idea, 'thorough' experiments, and 'robust' analysis, and for raising concerns about offline costs, transferability, short-term interests, and cold start. We address them below.

W1 (offline costs): ... the paper lacks a detailed quantification of the offline costs. ...

RE: We appreciate this suggestion. DeltaRec primarily targets online serving efficiency. We further report offline costs, including CWI generation, selector distillation, sparse-history training, grouping, and cache preparation.

rTable 1a. Offline training cost on KuaiRand-1K.

Offline componentTime (h)Percentage
Training on sparse histories12.079183.72%
CWI generation0.74585.17%
Selector distillation1.14907.96%
Grouping<0.00010.00%
Cache preparation0.45423.15%
Total offline14.4281100.00%

Training dominates offline costs (83.72%), followed by selector distillation (7.96%), while other overhead is relatively low.

For cache storage, rTable 1b reports FP32 payloads per user-group combination. And rTable 1c show the real cache memory and compares with KV cache memory used in Native HSTU and state cache used in FuXiLinear.

rTable 1b. Cache storage payload.

DatasetIndex (KiB/group)State (MiB/group)
ML-20M1.22850.250000
Amazon Books1.22850.015625
KuaiRand-1K1.07911.250000

rTable 1c. Real cache storage on KuaiRand-1k.

ModelState cache (MiB)Index cache (MiB)Total (MiB)
Native HSTU220845.201—220845.201
DeltaRec HSTU55050.084172.03155222.115
Native FuXi1078.723—1078.723
DeltaRec FuXi21.588172.031193.619

W2 (transferability): ... how easily this framework can be adapted to traditional dense self-attention models or other non-recurrent architectures.

RE: DeltaRec builds on linear attention, with efficiency gains from write and state utilization, where state-sharing error is bounded by Theorem 2.

The selection strategy also applies to dense self-attention. Beyond the cross-backbone results in Figure 8, we evaluate CWI-distilled selection on native HSTU.

rTable 2. Native HSTU with dense self-attention under different selection strategies.

Native HSTU inputMacro-GAUCMean / P95 (ms/batch)
Full history0.534715.505 / 15.684
Recency0.537522.864 / 23.565
Candidate similarity0.540035.602 / 36.230
CWI-distilled selection (ours)0.541887.855 / 88.450

CWI-distilled selection achieves the highest GAUC, demonstrating transferability beyond linear attention, while DeltaRec's efficiency gains rely on state sharing.

W3: .. how .. balance between ... the right histories... short-term interests, and how .. cold start...

RE: Thanks for your valuable comments. We answer the short-term interests and cold-start questions below.

Short-term interests. As we stated in Eq. (14), DeltaRec always preserves the 32 most recent events. To test how it balances the short-term interests, we vary this count under a fixed history budget (25% retention in all). From rTable 3a, removing recent-history preservation reduces GAUC, while too many recency events also reduce performance, confirming the importance of balancing them.

rTable 3a. Different minimal recent-history retention numbers.

Retained recent count RMacro-GAUC
1280.5393
640.5413
320.5415
160.5419
00.5349

Cold-start settings. As we mentioned in Line 326, we clarify that the selector uses both interaction histories and candidate embeddings, providing additional features for sparse histories. To show the performance, we stratify candidates by training interaction frequency on KuaiRand to test the cold-start performance.

rTable 3b. Performance of candidate-interaction frequency stratification on KuaiRand.

Training interactionsData sizeRecency (GAUC)Similarity (GAUC)Ours CWI (GAUC)
07251160.5184270.5232570.526487
1–102827800.5193930.5153310.518038
>101443280.5402870.5416280.542572

CWI performs best for unseen (0) and frequent (>10) candidates. Recency slightly outperforms CWI for sparse candidates (1–10), motivating the recent-history safeguard in Eq. (14).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal for AC

Dear Area Chair,

We sincerely appreciate your efforts throughout the review process. To assist your evaluation, we summarize the reviewers' assessments and our responses below.

Main strengths of our work

  • Significance and motivation: All three reviewers acknowledged the meaningful problem framing of candidate-focal history compression. Reviewer #vX4M described the idea as "intuitive" and "impactful", directly addressing redundancy and candidate-dependent relevance in long-sequence recommendation. Reviewer #YSL7 also recognized the "meaningful reframing" of long-sequence recommendation.
  • Technical contributions and novelty: Our work develops a Quality–Write–State framework that connects candidate-focal history selection with recurrent-state sharing. Reviewer #YSL7 acknowledged the "technically comprehensive" contribution, while Reviewer #C1ux recognized the "theoretical insights" and "coherent approach" to balancing recommendation quality and computational efficiency.
  • Experimental evaluation: Our experiments cover three datasets and four backbones, with extensive ablations and efficiency analyses. Reviewer #vX4M explicitly described the experiments as "thorough" and acknowledged the "robustness" demonstrated by the ablation and sensitivity studies.

Main concerns and how we addressed them

  • Reviewer #vX4M raised concerns about offline costs, generalizability, short-term interests, and cold-start settings. We provided additional results in rTables 1a–1c (offline costs and storage), rTable 2 (generalizability), and rTables 3a–3b (short-term interests and cold start), directly addressing the original concerns.

  • Reviewer #YSL7 questioned the experimental comparisons, attribution of improvements, and practical efficiency. We addressed these concerns through rTables 4–5 (controlled comparisons and ablations), rTable 6 (dataset-specific analysis), and rTables 7a–7c (detailed efficiency measurements). Importantly, in the follow-up discussion, the reviewer explicitly acknowledged that most concerns had been addressed, including improved experimental rigor and clarity. However, the score remained unchanged based on general concerns about novelty and quality, for which we requested further clarification.

  • Reviewer #C1ux raised concerns regarding related works, theoretical analysis, and efficiency, and requested more controlled experiments. We clarified the difference with related works and theoretical contributions, and provided rTables 8–10 (additional controlled comparisons and ablations) and rTable 11 (efficiency comparisons) to further support our claims.

Summary of discussion process

  • Overall, all three reviewers acknowledged the significance, motivation, theoretical insights, and technical contributions of DeltaRec. Their main concerns focused on cache implementation details, ablations, and overall efficiency. Our rebuttal provided additional evidence in rTables 1–11 and clarified the remaining methodological and theoretical questions. Notably, Reviewer #YSL7 explicitly confirmed that most original concerns had been addressed.

We sincerely hope the additional evidence and clarifications will be fully considered in the final assessment. We would greatly appreciate your consideration of the reviewers' positive evaluations and the substantial progress made during the discussion.

Best regards,

Authors of Submission #50408

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.