acceptodds
Under review as a conference paper at ICLR 2027

Fast-Weight Memory for Adaptive Recommendation

Abstract

Modern sequential recommenders repeatedly process interaction histories to capture evolving user preferences, coupling preference adaptation with sequential history modeling. We propose FWPRec, a recommendation model that separates user representation into a stable preference anchor and a fast associative memory for recent behavioral changes. The anchor captures persistent preference information, while fast-weight matrices incorporate new positive and negative feedback through learned gradient-free updates at inference time without modifying global model parameters. This separation allows item representations and long-term user states to be reused across requests, while preference changes are directly incorporated into a fixed-size user-specific state. We evaluate FWPRec on six real-world interaction datasets against static, lightweight adaptation, recurrent, attention-based, and state-space recommenders under controlled ranking settings. FWPRec achieves competitive ranking performance while providing effective adaptation to preference changes, with particularly strong gains when informative positive and negative feedback is available. Further analysis shows that learned memory construction and the separation of stable preferences from fast behavioral corrections are important for effective adaptation. The results demonstrate that fast-weight memory provides an alternative and complementary mechanism to sequential history modeling for capturing dynamic user preferences.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Recommender Systems, Fast Weight Programmers, Adaptive preference modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4cc4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FWPRec, a recommender that splits the user state into a fixed “anchor” u₀ (e.g., a mean-content profile, query, or cached long-term representation) and two d×d fast-weight matrices, M⁺ and M⁻, for positive and negative feedback. Each interaction (item, signed reward) is passed through small MLP “writers” that produce a key, a correction vector, and a write gate. The authors claim a competitive ranking quality against static, lightweight-update, and sequential baselines on six datasets. Ablation studies support learned writes, separate positive and negative memories, and the anchor.

Critical strengths

A practical design with a clear deployment story. User-side-only adaptation over frozen item vectors leaves the item index and ANN stage untouched. This addresses a real industrial constraint.

Critical weaknesses
  1. Comparison fairness and strength of the empirical claims. FWPRec and the lightweight baselines receive signed feedback. The sequential baselines receive clicked-only histories. The headline wins on MIND and EB-NeRD may therefore come from extra negative-feedback information rather than the architecture. The paper itself says gains are strongest when negative feedback is informative. A signed-input HSTU/SASRec is the necessary control.
  2. Many comparisons (e.g., MIND, where Mamba4Rec, HSTU, LRURec, and GRU4Rec are within ~0.001–0.005 nDCG@10) are within plausible noise.
  3. Mixed outcomes compared to the “competitive” baselines. FWPRec is worse than HSTU, LRURec, and GRU4Rec on Amazon (MRR/nDCG). On ML-1M it trails ItemKNN and even Popularity. It is second on KuaiRand and loses on KuaiRec MRR. Absolute MIND AUC (~0.63) is far below tuned news models, and NRMS/LSTUR are described but not reported.
  4. Efficiency claims are thin. No latency or throughput is reported. The datasets use histories of only 50–100, so the “O(1) vs growing” advantage is not demonstrated in the tested regime. Recurrent baselines are also O(1).
Questions for authors
  1. How exactly is u₀ built in each dataset? If it is a mean of history items, are those the same interactions then written into memory? If so, is history processed twice, and how does this fit the “decoupled from history” claim?
  2. How were training instances and the negatives N_t built (impression non-clicks vs. random)? What is the training cost of backpropagating through 50–100 memory updates?
  3. KuaiRand “uses learned ID embeddings in each model’s native configuration”. How does this reconcile with the frozen-embedding protocol?
  4. Report scaling with much longer histories and wall-clock/QPS.
2 comments

Response to Reviewer 3

We thank the reviewer for the constructive feedback. We have added experiments addressing comparison fairness, statistical reliability, recommendation performance, and computational efficiency. We clarify the experimental protocol below.

Responses to Critical Weaknesses

1. Comparison fairness regarding signed feedback.

To investigate whether FWPRec's gains come from access to negative feedback, we added Appendix E.6 (Signed-Input Sequential Baselines), extending HSTU to accept signed interactions through learned feedback embeddings. Please see the results below.

Table 1: Comparison with positive-only and signed-input HSTU (test nDCG@10).

DatasetFWPRecHSTU (Positive)HSTU (Signed)
MIND0.40430.40050.3977
EB-NeRD0.48210.47170.4768
Amazon0.24960.26740.2723
MovieLens-1M0.24210.33780.3533
KuaiRand0.48580.41520.6785
KuaiRec0.08140.08010.0822

FWPRec remains stronger on MIND and EB-NeRD despite HSTU receiving signed feedback and using up to 100 historical interactions, compared with FWPRec's 20. The news recommendation gains therefore cannot be explained solely by access to negative feedback.

2. Statistical reliability.

We added Appendix E.2 (Experimental Results with Statistics), reporting results across three training seeds and paired hierarchical bootstrap analysis (Appendix A.5).

On MIND, FWPRec achieves nDCG@10 of 0.4048 ± 0.0015, compared with 0.4013 ± 0.0018 for Mamba4Rec and 0.4007 ± 0.0028 for HSTU. Both differences are distinguishable from zero under paired bootstrap and seed-level confidence intervals. These results support a small but statistically reliable improvement rather than an effect of random variation.

3. Mixed ranking performance and comparison with news recommenders.

As discussed in Section 4.2, FWPRec targets lightweight preference adaptation rather than replacing sequential recommendation architectures. To further investigate its performance, Appendix C.3 (Effect of Enhanced Writer Inputs) introduces feedback-type and temporal features, improving nDCG@10 on four datasets, including Amazon (0.2496 → 0.2825), MovieLens-1M (0.2421 → 0.2800), and KuaiRand (0.4858 → 0.5613).

We also added Appendix E.4 (Comparison with News Recommendation Models), evaluating NRMS (Wu et al., 2019) and LSTUR (An et al., 2019) on the same test impressions. Please see the results below.

Table 2: Comparison with established news recommendation models (test nDCG@10).

MethodMINDEB-NeRD
NRMS0.40740.4757
LSTUR0.39590.4699
FWPRec0.40430.4821

FWPRec achieves comparable performance to NRMS on MIND and outperforms both models on EB-NeRD, despite using frozen item representations rather than jointly trained news encoders. These results support the effectiveness of FWPRec within the representation-reuse setting studied in our paper.

4. Computational efficiency.

We added Appendix E.5 (Update Latency and Throughput), measuring incremental serving latency for histories of 20–10,000 interactions. FWPRec maintains constant latency of 0.31 ms per user and 0.67 ms per batch of 1,024 users, achieving approximately 1.5 million user updates per second. In contrast, SASRec's batch latency increases from 0.46 ms to 38.2 ms as history length grows from 20 to 10,000.

As discussed in Section 4.4, recurrent models also support constant-size states. FWPRec's advantage is combining history-independent updates with learned associative corrections while retaining existing item representations and retrieval infrastructure.

Responses to Questions for Authors

Q1. How is u0u_0 constructed, and is history processed twice?

As clarified in Appendix D (Detailed Analysis of User Anchor), u0u_0 is the normalized mean of earlier positive interaction embeddings, using frozen content representations or learned ID embeddings on KuaiRand. These interactions precede the adaptation window and are not replayed when writing memory.

Once constructed, u0u_0 remains fixed, and only the latest 20 interactions contribute to memory updates (Eq. 9). Appendix D, Table 7 further evaluates alternative anchor constructions, including user ID embeddings and profile features. Thus, the history-independence claim concerns online adaptation, not the initial construction of u0u_0.

Q2. How are training instances and negatives constructed, and what is the training cost?

As described in Section 3.3 and Appendix A.1–A.2, each training instance contains a causal history, a positive target, and a fixed candidate set. The target interaction is excluded from its prediction history. Please see the negative-sampling protocol below.

Table 3: Construction of training negatives.

DatasetNegative candidates NtN_t
MIND / EB-NeRDUnclicked items from the same logged impression
Amazon / MovieLens-1MRandom catalog items, excluding the target and previously positive items
KuaiRand / KuaiRecPreviously observed non-positive interactions, with catalog fallback

Candidate sets are constructed using fixed random seeds and are not resampled across training epochs. Sampled candidate negatives are not automatically treated as negative feedback in the user history.

We also added Appendix A.8 (Training Cost and the Memory Write Window). FWPRec backpropagates through only 20 memory updates, rather than the full 50–100-event history. Please see the measured training costs below.

Table 4: Training time per epoch, including validation (seconds, RTX 4080 SUPER).

MethodAmazonMovieLens-1MKuaiRandKuaiRec
GRU4Rec8192059
SASRec9242279
HSTU14213481
Mamba4Rec374980222
FWPRec174324125

FWPRec has training costs comparable to sequential baselines and is consistently faster than Mamba4Rec. Its primary computational advantage remains gradient-free online adaptation rather than training speed.

Q3. How does KuaiRand's learned ID embedding setting relate to the frozen-embedding protocol?

As stated in Table 2 and Appendix A.2, KuaiRand has no item text in our setup. We therefore use learned ID embeddings in each model's native configuration. KuaiRand serves as an additional ID-based evaluation, while the other datasets provide representation-controlled comparisons using shared frozen content embeddings.

Q4. Can the authors report scaling to longer histories and wall-clock throughput?

We added Appendix E.5 (Update Latency and Throughput), extending the efficiency evaluation to 10,000 historical interactions on an RTX 4080 SUPER GPU. Each serving step incorporates one interaction and scores 100 candidates. Please see the results below.

Table 5: Median incremental serving latency (ms).

MethodB=1, L=20B=1, L=10,000B=1,024, L=10,000
GRU4Rec0.080.090.24
LRURec0.240.280.34
Mamba4Rec0.250.250.34
SASRec0.252.0438.20
HSTU0.420.43OOM
FWPRec (dense)0.310.310.67

B denotes batch size and L denotes history length.

FWPRec maintains constant latency as history length increases, achieving approximately 1.5 million updates per second with dense memory at batch size 1,024. These results directly demonstrate history-independent serving costs, complementing the persistent-memory analysis in Section 4.4.

Reviewer_QNVBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FWPRec, a recommendation framework that separates stable user preferences from recent behavioral changes using fast-weight associative memory. The model maintains a fixed user representation together with separate positive and negative memories, which are updated through learned functions as new feedback arrives. The goal is to support online preference adaptation without repeatedly processing the full interaction history or updating the global model parameters.

I find the problem interesting and practically relevant. The overall framework is clearly explained, and the idea of combining a stable preference representation with lightweight memory updates is appealing. The experiments cover six recommendation datasets and include comparisons with both traditional preference-update methods and more advanced sequential recommenders. The results on MIND and EB-NeRD are encouraging, and the ablation studies provide some evidence for the usefulness of the proposed components.

However, I have concerns about the methodological novelty and whether the experiments fully support the central claims. Fast-weight associative memory and delta-rule updates are established techniques, and the proposed method appears to adapt these mechanisms to recommendation through learned update functions and separate positive and negative memories. While this is a reasonable design, I am not fully convinced that the paper demonstrates a sufficiently distinct contribution beyond existing memory-based and adaptive recommendation approaches.

My second concern is the experimental evidence for preference adaptation and efficiency. FWPRec performs well on some datasets but is less competitive on others, particularly MovieLens-1M and Amazon Video Games. More importantly, the paper mainly reports overall ranking metrics rather than directly demonstrating how quickly and reliably the model responds to changes in user preferences. The efficiency analysis also focuses on memory consumption, without a clear comparison of actual online update latency or throughput. These results make it difficult to assess the practical advantage of the proposed approach.

Overall, I appreciate the motivation and the effort put into the experimental evaluation. The approach is technically reasonable and shows promising results in some settings. However, I would like to see stronger evidence that the fast-weight mechanism provides a benefit that cannot be achieved with simpler learned updates, as well as more direct validation of the claimed adaptation and efficiency advantages. I therefore lean toward rejection in its current form.

Critical strengths

1. Practical motivation and a lightweight adaptation mechanism. The paper addresses an important problem in recommendation systems: incorporating recent user feedback without repeatedly processing the full interaction history. Separating stable preferences from fast-changing behavioral information is a reasonable design choice, and the use of user-specific fast-weight memory provides a concrete way to implement it. I find this appealing because it could potentially fit into existing recommendation pipelines without requiring changes to the item representation and retrieval infrastructure.

2. Broad experimental evaluation with useful ablations. The authors evaluate the method on six datasets covering different recommendation scenarios and compare it with both lightweight adaptation methods and sequential recommendation models. The results on MIND and EB-NeRD are encouraging, and the ablation experiments help illustrate the roles of the learned memory updates, stable preference anchor, and separate positive and negative memories. These results provide some support for the proposed design, even though the improvements are not consistent across all datasets.

Critical weaknesses

1. The methodological novelty is not sufficiently established. The proposed method builds on existing fast-weight memory and delta-rule update mechanisms. Although the combination of a stable user anchor, learned memory writers, and separate positive and negative memories is reasonable, it is unclear which aspects constitute a substantial methodological advance. The current experiments do not sufficiently isolate the benefit of associative matrix memory from that of using learned feedback-dependent update functions. A comparison with a similarly expressive learned vector-update mechanism would help clarify whether the added complexity of fast-weight memory is necessary.

2. Limited direct evidence for the central adaptation and efficiency claims. Most of the experimental results focus on overall ranking performance, which does not directly establish how well the model handles changes in user preferences. Appendix A.5 describes adaptation-specific evaluation metrics and statistical analysis, but the corresponding confirmatory results are not clearly reported. In addition, the efficiency experiments mainly compare persistent state size and peak inference memory rather than actual update latency or end-to-end serving performance. Since recurrent and state-space models can also maintain fixed-size states, the practical efficiency advantage of FWPRec remains unclear. These gaps are important because fast adaptation and computational efficiency are central motivations of the paper.

Questions for authors

1. Could the authors clarify what benefits are specifically attributable to the fast-weight associative memory? The model uses learned writer networks to determine how interactions modify the user representation. How much of the improvement comes from the matrix-based memory itself, rather than from having a learned feedback-dependent update mechanism? A comparison with a parameter-matched learned vector-update baseline would help me better understand the methodological contribution.

2. Could the authors provide the adaptation-specific results described in Appendix A.5? The appendix mentions AdaptNDCG@10, negative-rich evaluation, and hierarchical bootstrap confidence intervals, but I could not find the corresponding confirmatory results in the paper. Since responding to changing user preferences is a central motivation, these results would be important for assessing the effectiveness of the proposed approach. I would also appreciate clarification of the comparison protocol on MIND, where methods using signed feedback receive information from non-clicked impressions while positive-only sequential baselines use clicked histories.

3. Could the authors clarify the practical computational advantage of FWPRec over efficient recurrent and state-space recommenders? Figure 3 demonstrates fixed-size user memory, but recurrent models can also maintain compact states. Are there measurements of per-interaction update latency, inference latency, or throughput under comparable hardware and serving conditions? Such results would help determine whether the proposed mechanism offers an advantage beyond its fixed memory footprint.

2 comments

Response to Reviewer 2

We thank the reviewer for the constructive feedback and for recognizing the practical motivation, lightweight adaptation mechanism, and broad experimental evaluation of FWPRec. We have revised the paper with additional experiments and analysis to address the concerns.

Responses to Critical Weaknesses

1. Methodological novelty.

FWPRec is inspired by Fast Weight Programmers (Schlag et al., 2021), but applying this mechanism to recommendation requires a different formulation. As described in Sections 3.1–3.2 (Fast-Weight Preference Adaptation), we introduce a stable user anchor, learned feedback-dependent memory writers, separate positive and negative memories, and anchor-conditioned associative readout. These components allow recent interactions to modify user preferences without updating the global model or changing the existing item representations and retrieval infrastructure.

To the best of our knowledge, fast-weight programming has not previously been investigated for this recommendation adaptation setting. We believe the contribution lies in formulating and validating fast-weight associative memory for online user preference adaptation, rather than proposing a new general-purpose memory update rule.

Beyond the main formulation, Section 4.3 (Ablation Study) demonstrates the importance of the proposed components, while Appendix E.1 (Matrix Memory versus Learned Vector Updates) further isolates the contribution of associative matrix memory from learned update functions. We provide the detailed comparison in our response to Question 1.

2. Direct evidence for adaptation and efficiency.

We have added two experiments directly addressing these concerns.

First, Appendix E.7 (Adaptation after Preference Shifts) evaluates ranking performance immediately after detected changes in user preferences using AdaptNDCG@10. FWPRec achieves the highest performance on MIND and EB-NeRD and outperforms all sequential baselines on KuaiRand.

Second, Appendix E.5 (Update Latency and Throughput) measures incremental serving latency as interaction histories grow from 20 to 10,000. FWPRec maintains constant update latency and supports approximately 1.5 million user updates per second under batched inference.

These experiments complement the ranking comparisons in Sections 4.1–4.2 and the memory analysis in Section 4.4. Together, they provide direct evidence for the adaptation effectiveness and computational properties motivating FWPRec. Detailed results are provided in our responses to Questions 2 and 3.

Responses to Questions for Authors

Q1. What benefits are specifically attributable to fast-weight associative memory?

To isolate the effect of associative matrix memory, we added Appendix E.1 (Matrix Memory versus Learned Vector Updates). We compare FWPRec against a parameter-matched vector-update model using the same learned writer networks, user anchor, interaction window, and training objective. Both models contain 83,330 trainable parameters. Please see the results below.

Table 1: Comparison between associative matrix memory and learned vector updates (test nDCG@10).

DatasetFWPRecVector + Writers
MIND0.40430.4008
EB-NeRD0.48210.4804
Amazon0.24960.2512
MovieLens-1M0.24210.2420
KuaiRand0.48580.4823
KuaiRec0.08140.0757

The learned updates account for much of the ranking improvement, while associative matrix memory provides additional gains on five datasets. More importantly, matrix memory stores key-addressed corrections rather than compressing all interactions into a single evolving vector, allowing interaction-level contributions to be recovered directly from the memory.

As demonstrated in Section 4.5 (Understanding Preference Updates) and Appendix C.1 (Interaction Attribution), these contributions can be exactly decomposed for each recommendation. Figure 4 further shows that removing highly attributed interactions produces 1.57× and 1.41× larger score changes than random removal for the top three and five interactions, respectively.

These results establish both the ranking contribution and the additional structural benefits of associative matrix memory.

Q2. Can the authors provide adaptation-specific results and clarify the signed-feedback comparison?

We added Appendix E.7 (Adaptation after Preference Shifts) to directly evaluate recommendation performance following preference shifts. We detect shifts from changes in item-category distributions using Jensen–Shannon divergence and measure AdaptNDCG@10 over the first three impressions following each shift. All methods are evaluated using their existing test predictions without retraining. Please see the results below.

Table 2: AdaptNDCG@10 over the first three impressions after detected preference shifts.

MethodMINDEB-NeRDAmazonKuaiRandKuaiRec
BPRMF0.37240.48400.07550.53780.0637
GRU4Rec0.38580.49850.27510.44290.0938
SASRec0.35260.47790.25730.44790.1043
LRURec0.38910.50120.29940.43270.1059
Mamba4Rec0.39070.49940.28650.44200.0935
HSTU0.38920.49850.31820.44880.0922
FWPRec0.39360.50350.28780.52780.0944

Selected baselines from Appendix E.7; results averaged over three training seeds.

FWPRec achieves the highest AdaptNDCG@10 on MIND and EB-NeRD, with statistically significant improvements over all baselines on MIND. On KuaiRand, it outperforms all sequential baselines and approaches BPRMF. We additionally evaluate negative-rich recent histories, where FWPRec outperforms all nine baselines on MIND and EB-NeRD and six of nine on Amazon. These results demonstrate the effectiveness of FWPRec in responding to recent preference changes.

Regarding the MIND comparison protocol, Appendix A.1 (Datasets and Preprocessing) specifies that non-clicked impressions are assigned weak negative feedback (−0.25) for signed-input methods, while sequential baselines use positive-only histories following their standard formulations.

To directly examine this difference, we added Appendix E.6 (Signed-Input Sequential Baselines), extending HSTU to incorporate both positive and negative feedback. FWPRec achieves higher nDCG@10 than signed-input HSTU on MIND (0.4043 vs. 0.3977) and EB-NeRD (0.4821 vs. 0.4768), despite using only 20 recent interactions compared with HSTU's 100 historical interactions. This provides evidence that FWPRec's news recommendation performance is not solely due to access to negative feedback.

Q3. What is the practical computational advantage over recurrent and state-space recommenders?

We added Appendix E.5 (Update Latency and Throughput) to directly measure incremental update latency and throughput under the same serving conditions. Each step processes one new interaction and scores 100 candidates on an NVIDIA RTX 4080 SUPER GPU. Please see the results below.

Table 3: Median incremental serving latency (ms) across history lengths and batch sizes.

MethodB=1, L=20B=1, L=10,000B=1,024, L=10,000
GRU4Rec0.080.090.24
LRURec0.240.280.34
Mamba4Rec0.250.250.34
SASRec0.252.0438.20
HSTU0.420.43OOM
FWPRec (dense)0.310.310.67

FWPRec maintains constant update latency as history length increases, achieving approximately 1.5 million user updates per second with batch size 1,024. Compared with attention-based methods such as SASRec, this avoids computation that grows with interaction history.

We recognize that efficient recurrent and state-space models also maintain compact states and can achieve lower latency. The practical advantage of FWPRec is therefore not universally faster inference, but learned associative preference adaptation with history-independent updates while retaining existing cached user/item representations, dot-product scoring, and approximate nearest-neighbor retrieval. As discussed in Section 2.2 (Lightweight Adaptation and Representation Reuse), these are common design choices in industrial recommendation systems (Yi et al., 2019; Pancha et al., 2022; Satuluri et al., 2020).

Reviewer_sZDhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FWPRec, a framework for adaptive recommendation that separates stable user preferences from recent behavioral changes. It maintains a fixed user representation alongside positive and negative fast-weight memories, which are updated through learned, gradient-free rules as new interactions arrive. The resulting user representation is used for recommendation without updating global model parameters at inference time. Experiments on six real-world datasets compare FWPRec with lightweight adaptation methods and sequential recommenders, along with ablation studies and memory-efficiency analyses.

Critical strengths
  1. The paper addresses a practically relevant problem in recommendation: adapting to recent user interactions while preserving long-term preferences. Separating stable preferences from recent behavioral changes is an intuitive direction.

  2. The proposed framework is relatively simple and lightweight, requiring only a small set of trainable writer networks. It enables per-user adaptation without gradient-based optimization at inference time or changes to the pretrained item representations.

  3. The paper provides experiments across six real-world datasets, comparing FWPRec with lightweight adaptation methods and various sequential recommendation models. The ablation studies further examine the contributions of individual components, including positive and negative memories, learned memory updates, and low-rank variants.

Critical weaknesses
  1. The motivation and empirical justification for introducing separate user-specific memory are not fully convincing. Incremental fine-tuning offers a straightforward (but often efficient and effective) alternative for adapting existing recommenders to new interactions, while prior work has also explored preserving long-term preferences during continual adaptation [1, 2]. Although FWPRec enables per-user, gradient-free adaptation, the paper does not sufficiently discuss these alternatives or empirically establish the advantages of its design over them. In particular, Section 4.4 focuses on memory usage without directly evaluating adaptation latency or computational cost.

  2. The methodological novelty appears somewhat limited. The core fast-weight memory mechanism, including learned key-value representations and delta-rule updates, largely follows prior work [3]. However, it is unclear what new technical contributions are introduced beyond adapting this mechanism to recommendation. Related key-value memory approaches have also been explored for modeling evolving user preferences in recommendation [2, 4]. More broadly, prior work such as ReLoop2 [5] has investigated memory-based online adaptation without repeatedly updating the global model, although through a different mechanism.

  3. The ranking performance is inconsistent across datasets. While FWPRec achieves strong results on MIND and EB-NeRD, it underperforms several baselines on other datasets in Table 2. The gap is particularly large on MovieLens-1M, where FWPRec achieves an nDCG@10 of 0.2421 compared with 0.3378 for HSTU.

References

[1] Mi et al., "ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation," RecSys 2020.

[2] Lee et al., "Capturing User Interests from Data Streams for Continual Sequential Recommendation," WSDM 2026.

[3] Schlag et al., "Linear Transformers Are Secretly Fast Weight Programmers," ICML 2021.

[4] Lin et al., "Self-Evolving Memory for Generative Recommendation," CIKM 2026.

[5] Zhu et al., "ReLoop2: Building Self-Adaptive Recommendation Models via Responsive Error Compensation Loop," KDD 2023.

Questions for authors

NA

2 comments

We thank the reviewer for the constructive feedback and for recognizing the practical relevance of preference adaptation and the lightweight, gradient-free nature of FWPRec. We have revised the paper with additional experiments and analysis to address the concerns.

1. Motivation and empirical justification for user-specific memory

As discussed in the Introduction and Section 2.2 (Lightweight Adaptation and Representation Reuse), FWPRec targets recommendation systems where user and item representations are already computed and cached, as commonly seen in industrial applications (Yi et al., 2019; Pancha et al., 2022; Satuluri et al., 2020). Our objective is to incorporate new interactions without modifying existing encoders, replaying interaction histories, or performing gradient-based optimization. This differs from incremental fine-tuning, which updates model parameters, and from deploying a new sequential recommendation architecture.

We have added direct latency and throughput experiments in Appendix E.5 (Update Latency and Throughput) to evaluate the computational advantages beyond memory usage. The following table summarizes representative results from Table 12. Each serving step incorporates one interaction and scores 100 candidates.

MethodBatch=1, L=20Batch=1, L=10,000Batch=1,024, L=10,000
GRU4Rec0.080.090.24
SASRec0.252.0438.20
HSTU0.420.43OOM
FWPRec (dense)0.310.310.67

Median serving latency in milliseconds on an NVIDIA RTX 4080 SUPER. L denotes history length.

FWPRec maintains constant update latency as the interaction history grows, reaching approximately 1.5 million user updates per second at batch size 1,024. While efficient recurrent models can also maintain constant-size states and achieve lower latency, FWPRec avoids repeated history processing without changing the existing item representations or retrieval infrastructure.

Additionally, Section 4.4 (Computational Efficiency) shows that FWPRec maintains approximately 32 KB of persistent user memory.

2. Methodological novelty

We agree that FWPRec builds upon the Fast Weight Programmer framework [3], and we do not claim the delta-rule update itself as a new contribution. Our contribution is a preference adaptation formulation that combines a stable user anchor, separate positive and negative associative memories, and learned feedback-dependent corrections, while preserving conventional dot-product scoring and item retrieval.

Sections 3.1–3.2 describe this formulation, while Section 4.3 (Ablation Study) evaluates the importance of its components. In particular, removing memory or replacing the learned writers with fixed updates substantially reduces performance on several datasets.

To directly address the comparison with ReLoop2, we added Appendix E.3 (Comparison with an Error-Compensation Memory). We implement a ReLoop2-style error-compensation baseline under the same user anchor, interaction window, and evaluation setting. Please see results below.

DatasetFWPRecReLoop2-styleDifference (95% CI)
MIND0.40430.3814+0.023 [+0.021, +0.025]
EB-NeRD0.48210.4698+0.012 [+0.011, +0.014]
Amazon0.24960.2459+0.004 [−0.003, +0.010]
MovieLens-1M0.24210.1206+0.121 [+0.116, +0.127]
KuaiRand0.48580.5126−0.027 [−0.031, −0.023]
KuaiRec0.08140.0582+0.023 [+0.021, +0.025]

FWPRec outperforms this baseline on four of six datasets, while ReLoop2-style compensation performs better on KuaiRand. The results support the benefit of learned associative corrections over non-parametric error compensation in several recommendation settings.

Furthermore, Appendix E.1 (Matrix Memory versus Learned Vector Updates) introduces a parameter-matched vector-state control. Both models contain 83,330 trainable parameters, and their nDCG@10 differs by at most 0.006 across the six datasets. This indicates that much of the ranking improvement comes from learned feedback-dependent updates rather than matrix storage alone. The matrix formulation additionally supports key-addressed corrections and exact interaction-level attribution, as demonstrated in Section 4.5. I worth to highlight that, the vector updates still need the learned mapping function fcf_c as we described in our methodology section.

3. Performance compared with sequential models

We acknowledge that FWPRec does not universally outperform sequential models, particularly on MovieLens-1M. However, as emphasized in the Introduction and Section 4.2, our objective is not to replace sequential recommendation, but to provide a lightweight adaptation mechanism compatible with existing recommendation infrastructure.

We understand that reviewer wants to know if the signed feedback can also influence other baselines. But based on our study, we note that SASRec does not support signed input as the algorithm was designed for the implicit feedback. For HSTU, the algorithm does support signed input with additional training and additional token usage.

Appendix E.6 (Signed-Input Sequential Baselines) extends HSTU to incorporate positive and negative feedback. This comparison examines whether FWPRec's performance advantage comes primarily from its access to signed feedback. In this experiment, HSTU uses up to 100 historical interactions, while FWPRec uses only the latest 20 interactions for adaptation. The following is the test nDCG@10 that reflects our revision in the paper.

DatasetFWPRecHSTU (Positive)HSTU (Signed)
MIND0.40430.40050.3977
EB-NeRD0.48210.47170.4768
Amazon0.24960.26740.2723
MovieLens-1M0.24210.33780.3533
KuaiRand0.48580.41520.6785
KuaiRec0.08140.08010.0822

FWPRec maintains higher nDCG@10 on MIND and EB-NeRD despite using a shorter adaptation window, suggesting that its performance on news recommendation cannot be explained solely by access to negative feedback. However, signed-input HSTU performs substantially better on KuaiRand and MovieLens-1M, demonstrating the importance of longer interaction histories and stronger sequential modeling in these settings. BUT, as mentioned in the earlier experiments. HSTU cannot scale well (OOM) and it requires additional infra to support its deployment. It is probably good for being later stage in the full recommender system where FWPRec is candidate generation system that needs less resource to scale up.

We want to highlight that FWPRec is proposed to address practical concern of reusing established recommendation infra and offers online adaptation while reducing deployment cost, rather than universally outperforming sequential recommendation models. While our paper shows the comparison of it with SOTA sequential models, we only treat it as an opportunity gap for better completion of this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.