- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes explainability-guided history distillation for efficient sequential recommendation. A trained recommender is frozen, and post-hoc attribution scores are computed for historical interactions with respect to the model's top-predicted item. The method retains the top-K positions, restores their chronological order, and feeds the shortened history into the frozen model. Attribution is performed offline, while online prediction uses the distilled history without retraining. Experiments evaluate Integrated Gradients, GradientSHAP, and, where applicable, attention-based attribution on SASRec, BERT4Rec, Mamba, and LONGER. The evaluation covers three datasets with four configurations, including two history lengths for KuaiRand-1K. Table 1 shows that the benefits are dataset dependent: attribution-based selection improves normalized NDCG@10 AUC on KuaiRand, while Recency remains competitive or stronger on MovieLens. Additional experiments examine peak performance versus GFLOPs, synthetic history corruption, and model capacity.
Critical strengths
1. A practical approach to compressing histories without retraining. The procedure in Section 3.2 is straightforward and can be applied to an existing frozen model. Evaluating gradient-based selection across four backbones gives evidence that its usefulness extends across several architectures. This is consequential for systems where retraining a recommender to accommodate a separate compression mechanism would be costly.
2. Evaluation across compression budgets exposes useful differences between datasets. Table 1 and Figure 6 summarize behavior across retained history lengths, and Figures 4 and 5 examine corruption and capacity effects. In particular, the contrast between MovieLens and KuaiRand provides useful evidence about when attribution-based selection may offer an advantage over recency. This empirical observation is more informative than a result at a single compression ratio.
Critical weaknesses
1. The peak-based analysis does not establish the claimed performance–efficiency advantage. Section 3.4 selects each method's accuracy-maximizing K and plots only that operating point. Comparing such points does not establish dominance of the methods' full accuracy–compute frontiers: a method with a weaker accuracy peak can still be preferable at a smaller serving budget. This distinction matters because Section 4.2 uses the peak comparison to support deployment-efficiency conclusions. The stated extension across architectures is also too strong. The end of Section 4.2 says that the BERT4Rec, Mamba, and LONGER columns show identical efficiency trends, but Figure 3(d), LONGER on ML-1M, places the Recency mean point above and to the left of the IG and GradientSHAP points. Table 1 likewise gives LONGER/ML-1M AUC values of 0.85 for Recency, 0.79 for IG, and 0.77 for GradientSHAP. These results do not support a uniform advantage across the evaluated architectures. Please evaluate quality at shared compute budgets and distinguish full-frontier evidence from comparisons of individual peaks. In addition, Section 3.4 mentions validation-based K selection only as a deployment option, while Section 3.5 places evaluation on the test set. The data actually used to select Figure 3's K values remain unspecified. Validation curves, selected K values, and test scores at those fixed choices are needed before interpreting the peaks as deployable operating points.
2. The main positive benchmark is insufficiently specified to verify what task supports the gains. Section 3.5 specifies full-vocabulary next-item ranking, while Section 3.1 also allows feature-based inputs and binary or multiclass outputs. Table 2 lists 1,101,222 KuaiRand-1K items and Table 3 reports only 341K trainable SASRec parameters; the LONG setting has 1,068,905 items and 405K parameters. An ordinary trainable catalog-wide item lookup cannot fit these totals even at one scalar per item. Feature-based or fixed representations could explain the counts, so this is not proof that the numbers are impossible. The problem is that the manuscript does not identify the implemented KuaiRand input/output mapping, effective prediction vocabulary, candidate construction, or excluded parameters. Those choices also determine the full-catalog projection cost claimed in Section 3.4. Since the clearest gains in Table 1 come from KuaiRand, the omission affects the central evidence. Section 3.6 further establishes improvements only against Random and Recency selection. The four backbones do not supply stronger competing selectors. The current results therefore support a narrower conclusion about attribution versus these heuristics under an incompletely specified task.
Questions for authors
-
Operating-point selection and frontier evidence: Which data select the K values in Figure 3, and is selection performed separately per seed or after seed averaging? Please provide validation results over K and test performance at the selected K. Can the claimed efficiency advantage be shown at common compute budgets, including the LONGER/MovieLens cases where Recency is competitive or superior?
-
KuaiRand task reconciliation: Please specify the actual prediction labels, output dimension, post-preprocessing catalog and candidate set, feature encoding, and parameter-counting exclusions. How do these choices reconcile Tables 2–3 with Section 3.5, and how is full-catalog scoring included in the GFLOPs calculation?
-
Comparative scope: Can a stronger feasible model-aware selector be compared using the same frozen backbones and budgets? Recent learned compression such as SITA provides relevant context, but its target-aware training setting differs. Please explain that boundary and prioritize a fair comparator rather than assuming that a different task can be inserted unchanged.
-
Uncertainty and practical costs: Please provide paired differences or confidence intervals for the main comparisons and measurements of online latency and offline attribution cost. Three-seed means with the same direction do not establish robustness to variance, as suggested in Section 4.1. The two KuaiRand lengths also share a data source, so their agreement supplies limited independent evidence about distributional robustness.
-
Reporting and reproducibility: Figure 4's caption says ML-1M for panels d–f, while Section 4.3.1 says ML-10M. Which dataset was used? The Reproducibility Statement mentions supplementary code and public release upon acceptance. Please identify any anonymous artifact available to reviewers; otherwise, I recommend making preprocessing, K selection, parameter counting, and evaluation code available anonymously during review. I have not verified a separate supplement from the supplied PDF.
4 comments
Official Comment by Authors: Part 2
(Q2) KuaiRand task reconciliation. Response to Reviewer: Clarifications on Prediction Task, Parameter Counting, and GFLOPs Accounting
We thank the reviewer for raising these questions. We clarify our evaluation protocol, parameter scaling, and compute calculations below:
-
Full-Catalog Evaluation vs. Sampled Training: All reported test metrics evaluate full-catalog ranking against the complete item catalog without negative sampling ( candidates on KuaiRand). The sampled negatives referenced in Appendix A were utilized strictly to accelerate training and validation sweeps; the underlying backbone architecture and inner-product scoring function () remain identical.
-
Decoupled Parameter Scaling on KuaiRand: Items on KuaiRand are content-encoded from static metadata (-dimensional feature vectors of video types and tags) mapped via a learned projection , rather than instantiated via a discrete ID embedding table. Consequently, model capacity scales strictly with feature dimensionality, hidden width , and context length ---not with catalog size . The parameter delta between KuaiRand-1K () and KuaiRand-1K-Long () is exclusively attributable to the additional learned position embeddings ().
-
End-to-End GFLOPs Accounting: Full-catalog evaluation is explicitly integrated into our inference compute as the per-query dot product over all items, adding FLOPs. The projected item representations are precomputed and cached offline (amortized across requests), and the final top- ranking relies on comparisons rather than floating-point multiply--adds, conforming to standard efficiency benchmarking practices.
We have updated the Appendix to incorporate these exact operational details to ensure complete transparency.
Official Comment by Authors: Part 3
(Q3) Comparative scope.
We thank the reviewer for highlighting this distinction regarding task boundaries and fair comparators. We have updated Sections 2, 3.6, and 4.2 to clarify our scope and expand our comparisons:
-
Task Boundary (Frozen-Backbone Selection vs. Target-Aware Learned Compression): Learned compression methods like SITA require target-aware end-to-end training or architectural modifications to learn selection modules. In contrast, our framework operates in a strictly retraining-free, model-agnostic setting: given a frozen pretrained backbone, we distill raw input sequences offline before deployment without altering weights or conditioning on target items. Within this frozen-backbone boundary, we compare four model-aware attribution selectors (Integrated Gradients, GradientSHAP, Grad-SAM, and Attention Tracing) under identical weights and compute budgets.
-
Benchmarking Against Purpose-Built Long-Sequence Models (GLASS & LONGER): To benchmark against recent long-sequence recommendation paradigms at full context (), we evaluate GLASS (Cao et al, RecSys 2026), a recently published coarse-to-fine generative recommender utilizing semantic ID search, and LONGER, an industrial linear-complexity long-sequence architecture. As shown in Figure 3 (Section 4.2), while full-length GLASS and LONGER () incur low inference GFLOPs, neither achieves the peak ranking accuracy of attribution-distilled backbones.
-
Orthogonality and Synergy with Efficient Architectures: Because our approach distills the raw input sequence prior to encoding, it is orthogonal to efficient model architectures and can be layered directly on top of them. Applying Integrated Gradients and GradientSHAP directly onto LONGER improves ranking quality over both undistilled LONGER () and Recency truncation on KuaiRand-1K and KuaiRand-1K-Long across Table 1, Figure 7, Figure 8 and the fixed-budget Friedman analysis in Appendix B.1. On MovieLens (ML-1M and ML-10M), attribution-guided distillation on LONGER likewise matches or outperforms Recency along the Pareto frontier (Figure 3).
(Q4) Uncertainty and practical costs.
We thank the reviewer for these constructive suggestions. We have updated the manuscript as follows:
-
Statistical Significance (Friedman & Nemenyi Tests): In Section 4.2 and Appendix B.1, we now evaluate all methods under matched compute budgets () across all configurations. A Friedman test decisively rejects equivalent performance (), with Integrated Gradients (IG) securing the best overall rank ( on and on vs. and for Recency), exceeding the Nemenyi critical difference ().
-
Offline Attribution vs. Online Serving Costs: We report end-to-end online serving GFLOPs (including the full-catalog scoring layer, Figure 3) alongside offline attribution GFLOPs (Appendix Table 4). While gradient-based scoring incurs higher offline compute ( to GFLOPs for IG; to GFLOPs for GradientSHAP), this cost is strictly isolated to offline preprocessing and adds zero latency to online serving. We explicitly discuss this trade-off in our new Limitations section (Section 5), noting that this work establishes gradient attribution as a strong relevance reference to motivate future lightweight, single-pass attribution estimators.
-
Role of KuaiRand-1K () vs. KuaiRand-1K-Long (): Rather than treating the two KuaiRand variants as independent domains, evaluating both lengths isolates how distillation scales as context windows double (). Cross-domain distributional generalization is instead evaluated between dense movie ratings (ML-1M, ML-10M) and sparse, million-scale short-video logs (KuaiRand).
(Q5) Reporting and reproducibility.
We thank the reviewer for catching this discrepancy and for checking our reproducibility artifacts. We address both points below:
-
Correction of Dataset Label in Section 4.3.1 (ML-1M): The noise robustness experiments in panels (d)--(f) of Figure 4 (as well as Appendix Figure 14) were conducted on ML-1M, matching the figure captions. The reference to "ML-10M" in Section 4.3.1 was a typographical error, which we have now corrected to ML-1M in the revised manuscript.
-
Supplementary Code. Due to internal open-sourcing procedures, the code will be made publicly available upon paper acceptance. We understand this is not ideal for verifying reproducibility during review, and we are happy to answer any questions regarding implementation details.
The two weaknesses mentioned by the reviewer has been addressed by answering Questions 1-5.
Dear Reviewer, as the discussion period nears its end, we wanted to kindly check in to confirm whether our updates, including leak-free validation-selected operating points, fixed-budget Friedman/Nemenyi frontier comparisons, KuaiRand task/parameter reconciliations, and GLASS/LONGER benchmarks, have resolved your questions.
Official Comment by Authors: Part 1
We sincerely thank the reviewer for the thoughtful feedback and for recognizing the practical value of our retraining-free sequence compression across frozen architectures, as well as our multi-budget evaluation revealing domain differences between MovieLens and KuaiRand. Below, we address each of your questions and detail the corresponding statistical analyses, baseline comparisons, and methodological clarifications added to the revised manuscript. The updated part of the revised paper is highlighted in blue in the paper.
(Q1) Operating-point selection and frontier evidence. Response to Reviewer: Operating-Point Selection and Fixed-Budget Frontier Evidence
We thank the reviewer for highlighting the importance of decoupling operating-point selection from test evaluation and for suggesting fixed-budget comparisons. We have revised our evaluation protocol and expanded our statistical analysis accordingly:
Leak-Free Operating-Point Selection (K⋆) via Validation Sweep: In our initial draft, K⋆ was selected directly from the test trajectories. Following the reviewer's constructive feedback, we updated our evaluation protocol (Section 3.4 and Section 3.7) so that K⋆ is determined strictly a priori by sweeping over the validation set: K⋆=argmaxK<LmDvalid(K). Specifically, K⋆ is selected independently per run/seed on Dvalid (reflecting a fixed-length operational deployment budget rather than per-sample adaptive truncation), and the resulting operating points are then evaluated and plotted on the held-out test set Dtest in Figure 3 (and Appendix Figure 79. In addition, full validation compression trajectories across all evaluated sequence budgets K are summarized via AUCNDCG@10 and AUCHR@10 on Dvalid in Table 1 and Table 7.
Performance Comparisons Under Strictly Matched (Common) Compute Budgets: To isolate distillation selection quality from operating-point bias, we added a fixed-budget evaluation in Section 4.2 and Appendix B.1 comparing all methods across identical sequence lengths K∈{100,50,25,12,6}. Across all seed×K×architecture configurations (including LONGER and MovieLens), a Friedman test decisively rejects equivalent performance across both NDCG@10 and HR@10 (p<10−60):
We have updated Sections 3.4, 3.7, and 4.2, Figure 3, and Appendix B.1 in the revised manuscript to reflect these changes.