acceptodds
Under review as a conference paper at ICLR 2027

Explainability-Guided History Distillation for Efficient Sequential Recommendation

Abstract

Sequential recommendation models, spanning self-attention based Transformer architectures to state-space models, rely on processing complete interaction histories, incurring inference costs that scale with sequence length. While efficient attention variants and heuristic truncation mitigate this computational footprint, they overlook internal model attribution signals and neglect the efficiency gains achievable by distilling the raw interaction sequence itself. To bridge this gap, we propose explainability-driven sequence distillation, a retraining-free framework that leverages post-hoc attribution methods to score each interaction's predictive contribution. By retaining only the highest-scoring positions, our framework compresses user histories offline, enabling shorter sequences to accelerate online inference without altering underlying model parameters. Across four dataset configurations and four architectural paradigms, including autoregressive, bidirectional, state-space, and long-sequence backbones, Integrated Gradient-based sequence distillation preserves or improves ranking quality while substantially reducing inference GFLOPs, outperforming recency-based truncation by an average of 11.5% across all evaluated settings. Crucially, even on benchmarks with extreme temporal recency bias, our approach matches or exceeds recency truncation in the performance-efficiency trade-off, an advantage that widens under noisy input sequences. These findings demonstrate that post-hoc explainability, traditionally restricted to model interpretation, serves as a general, retraining-free principle for distilling raw input contexts and reducing online serving costs in sequential recommendation.

Primary area:
interpretability and explainable AI
Keywords:
Sequential Recommendation Model, Input distillation, Efficient online inference, explainability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jhbSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes explainability-guided history distillation for efficient sequential recommendation. A trained recommender is frozen, and post-hoc attribution scores are computed for historical interactions with respect to the model's top-predicted item. The method retains the top-K positions, restores their chronological order, and feeds the shortened history into the frozen model. Attribution is performed offline, while online prediction uses the distilled history without retraining. Experiments evaluate Integrated Gradients, GradientSHAP, and, where applicable, attention-based attribution on SASRec, BERT4Rec, Mamba, and LONGER. The evaluation covers three datasets with four configurations, including two history lengths for KuaiRand-1K. Table 1 shows that the benefits are dataset dependent: attribution-based selection improves normalized NDCG@10 AUC on KuaiRand, while Recency remains competitive or stronger on MovieLens. Additional experiments examine peak performance versus GFLOPs, synthetic history corruption, and model capacity.

Critical strengths

1. A practical approach to compressing histories without retraining. The procedure in Section 3.2 is straightforward and can be applied to an existing frozen model. Evaluating gradient-based selection across four backbones gives evidence that its usefulness extends across several architectures. This is consequential for systems where retraining a recommender to accommodate a separate compression mechanism would be costly.

2. Evaluation across compression budgets exposes useful differences between datasets. Table 1 and Figure 6 summarize behavior across retained history lengths, and Figures 4 and 5 examine corruption and capacity effects. In particular, the contrast between MovieLens and KuaiRand provides useful evidence about when attribution-based selection may offer an advantage over recency. This empirical observation is more informative than a result at a single compression ratio.

Critical weaknesses

1. The peak-based analysis does not establish the claimed performance–efficiency advantage. Section 3.4 selects each method's accuracy-maximizing K and plots only that operating point. Comparing such points does not establish dominance of the methods' full accuracy–compute frontiers: a method with a weaker accuracy peak can still be preferable at a smaller serving budget. This distinction matters because Section 4.2 uses the peak comparison to support deployment-efficiency conclusions. The stated extension across architectures is also too strong. The end of Section 4.2 says that the BERT4Rec, Mamba, and LONGER columns show identical efficiency trends, but Figure 3(d), LONGER on ML-1M, places the Recency mean point above and to the left of the IG and GradientSHAP points. Table 1 likewise gives LONGER/ML-1M AUC values of 0.85 for Recency, 0.79 for IG, and 0.77 for GradientSHAP. These results do not support a uniform advantage across the evaluated architectures. Please evaluate quality at shared compute budgets and distinguish full-frontier evidence from comparisons of individual peaks. In addition, Section 3.4 mentions validation-based K selection only as a deployment option, while Section 3.5 places evaluation on the test set. The data actually used to select Figure 3's K values remain unspecified. Validation curves, selected K values, and test scores at those fixed choices are needed before interpreting the peaks as deployable operating points.

2. The main positive benchmark is insufficiently specified to verify what task supports the gains. Section 3.5 specifies full-vocabulary next-item ranking, while Section 3.1 also allows feature-based inputs and binary or multiclass outputs. Table 2 lists 1,101,222 KuaiRand-1K items and Table 3 reports only 341K trainable SASRec parameters; the LONG setting has 1,068,905 items and 405K parameters. An ordinary trainable catalog-wide item lookup cannot fit these totals even at one scalar per item. Feature-based or fixed representations could explain the counts, so this is not proof that the numbers are impossible. The problem is that the manuscript does not identify the implemented KuaiRand input/output mapping, effective prediction vocabulary, candidate construction, or excluded parameters. Those choices also determine the full-catalog projection cost claimed in Section 3.4. Since the clearest gains in Table 1 come from KuaiRand, the omission affects the central evidence. Section 3.6 further establishes improvements only against Random and Recency selection. The four backbones do not supply stronger competing selectors. The current results therefore support a narrower conclusion about attribution versus these heuristics under an incompletely specified task.

Questions for authors
  1. Operating-point selection and frontier evidence: Which data select the K values in Figure 3, and is selection performed separately per seed or after seed averaging? Please provide validation results over K and test performance at the selected K. Can the claimed efficiency advantage be shown at common compute budgets, including the LONGER/MovieLens cases where Recency is competitive or superior?

  2. KuaiRand task reconciliation: Please specify the actual prediction labels, output dimension, post-preprocessing catalog and candidate set, feature encoding, and parameter-counting exclusions. How do these choices reconcile Tables 2–3 with Section 3.5, and how is full-catalog scoring included in the GFLOPs calculation?

  3. Comparative scope: Can a stronger feasible model-aware selector be compared using the same frozen backbones and budgets? Recent learned compression such as SITA provides relevant context, but its target-aware training setting differs. Please explain that boundary and prioritize a fair comparator rather than assuming that a different task can be inserted unchanged.

  4. Uncertainty and practical costs: Please provide paired differences or confidence intervals for the main comparisons and measurements of online latency and offline attribution cost. Three-seed means with the same direction do not establish robustness to variance, as suggested in Section 4.1. The two KuaiRand lengths also share a data source, so their agreement supplies limited independent evidence about distributional robustness.

  5. Reporting and reproducibility: Figure 4's caption says ML-1M for panels d–f, while Section 4.3.1 says ML-10M. Which dataset was used? The Reproducibility Statement mentions supplementary code and public release upon acceptance. Please identify any anonymous artifact available to reviewers; otherwise, I recommend making preprocessing, K selection, parameter counting, and evaluation code available anonymously during review. I have not verified a separate supplement from the supplied PDF.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors: Part 1

We sincerely thank the reviewer for the thoughtful feedback and for recognizing the practical value of our retraining-free sequence compression across frozen architectures, as well as our multi-budget evaluation revealing domain differences between MovieLens and KuaiRand. Below, we address each of your questions and detail the corresponding statistical analyses, baseline comparisons, and methodological clarifications added to the revised manuscript. The updated part of the revised paper is highlighted in blue in the paper.

(Q1) Operating-point selection and frontier evidence. Response to Reviewer: Operating-Point Selection and Fixed-Budget Frontier Evidence

We thank the reviewer for highlighting the importance of decoupling operating-point selection from test evaluation and for suggesting fixed-budget comparisons. We have revised our evaluation protocol and expanded our statistical analysis accordingly:

  1. Leak-Free Operating-Point Selection (K⋆K^\star) via Validation Sweep: In our initial draft, K⋆K^\star was selected directly from the test trajectories. Following the reviewer's constructive feedback, we updated our evaluation protocol (Section 3.4 and Section 3.7) so that K⋆K^\star is determined strictly a priori by sweeping over the validation set: K⋆=arg⁡max⁡K<LmDvalid(K)K^\star = \arg\max_{K < L} m_{D_{\text{valid}}}(K). Specifically, K⋆K^\star is selected independently per run/seed on DvalidD_{\text{valid}} (reflecting a fixed-length operational deployment budget rather than per-sample adaptive truncation), and the resulting operating points are then evaluated and plotted on the held-out test set DtestD_{\text{test}} in Figure 3 (and Appendix Figure 79. In addition, full validation compression trajectories across all evaluated sequence budgets KK are summarized via AUCNDCG@10AUC_{NDCG@10} and AUCHR@10{AUC_{HR@10}} on DvalidD_{\text{valid}} in Table 1 and Table 7.

  2. Performance Comparisons Under Strictly Matched (Common) Compute Budgets: To isolate distillation selection quality from operating-point bias, we added a fixed-budget evaluation in Section 4.2 and Appendix B.1 comparing all methods across identical sequence lengths K∈{100,50,25,12,6}K \in \{100, 50, 25, 12, 6\}. Across all seed×K×architecture\text{seed} \times K \times \text{architecture} configurations (including LONGER and MovieLens), a Friedman test decisively rejects equivalent performance across both NDCG@10\mathrm{NDCG@10} and HR@10\mathrm{HR@10} (p<10−60p < 10^{-60}):

    • Overall Superiority: Integrated Gradients (IG) achieves the strongest overall Friedman rank across benchmarks (2.002.00 on NDCG@10\mathrm{NDCG@10} and 2.072.07 on HR@10\mathrm{HR@10} vs. 3.113.11 and 3.003.00 for Recency), outperforming heuristic baselines well beyond the Nemenyi critical difference threshold (CD0.05=0.69\mathrm{CD}_{0.05} = 0.69).
    • Domain-Specific Trade-Offs (MovieLens vs. KuaiRand): As detailed in Appendix Tables 5 and 6 (and per-budget bar plots in Appendix Figures 10 and 11), we explicitly characterize where Recency remains competitive: on compact, strongly recency-biased sequences in ML-1M, Recency achieves the top rank (1.681.68 vs. 2.082.08 for IG on NDCG@10\mathrm{NDCG@10}), whereas on ML-10M (2.182.18 vs. 2.732.73) and across all long-horizon KuaiRand configurations (1.35–2.521.35\text{--}2.52 vs. 3.78–4.073.78\text{--}4.07), Integrated Gradient consistently dominates.

We have updated Sections 3.4, 3.7, and 4.2, Figure 3, and Appendix B.1 in the revised manuscript to reflect these changes.

AuthorsOpenReview ↗

Official Comment by Authors: Part 2

(Q2) KuaiRand task reconciliation. Response to Reviewer: Clarifications on Prediction Task, Parameter Counting, and GFLOPs Accounting

We thank the reviewer for raising these questions. We clarify our evaluation protocol, parameter scaling, and compute calculations below:

  • Full-Catalog Evaluation vs. Sampled Training: All reported test metrics evaluate full-catalog ranking against the complete item catalog ∣V∣|\mathcal{V}| without negative sampling (>106>10^6 candidates on KuaiRand). The 100100 sampled negatives referenced in Appendix A were utilized strictly to accelerate training and validation sweeps; the underlying backbone architecture and inner-product scoring function (h⊤ev\mathbf{h}^\top \mathbf{e}_v) remain identical.

  • Decoupled Parameter Scaling on KuaiRand: Items on KuaiRand are content-encoded from static metadata (6161-dimensional feature vectors of video types and tags) mapped via a learned projection Linear(61→E)\mathrm{Linear}(61 \to E), rather than instantiated via a discrete ID embedding table. Consequently, model capacity scales strictly with feature dimensionality, hidden width EE, and context length LL---not with catalog size ∣V∣|\mathcal{V}|. The +64K+64\text{K} parameter delta between KuaiRand-1K (341K341\text{K}) and KuaiRand-1K-Long (405K405\text{K}) is exclusively attributable to the additional learned position embeddings (+500×E+500 \times E).

  • End-to-End GFLOPs Accounting: Full-catalog evaluation is explicitly integrated into our inference compute as the per-query dot product over all items, adding 2E(∣V∣+1)2E(|\mathcal{V}|+1) FLOPs. The projected item representations are precomputed and cached offline (amortized across requests), and the final top-KK ranking relies on comparisons rather than floating-point multiply--adds, conforming to standard efficiency benchmarking practices.

We have updated the Appendix to incorporate these exact operational details to ensure complete transparency.

AuthorsOpenReview ↗

Official Comment by Authors: Part 3

(Q3) Comparative scope.

We thank the reviewer for highlighting this distinction regarding task boundaries and fair comparators. We have updated Sections 2, 3.6, and 4.2 to clarify our scope and expand our comparisons:

  1. Task Boundary (Frozen-Backbone Selection vs. Target-Aware Learned Compression): Learned compression methods like SITA require target-aware end-to-end training or architectural modifications to learn selection modules. In contrast, our framework operates in a strictly retraining-free, model-agnostic setting: given a frozen pretrained backbone, we distill raw input sequences offline before deployment without altering weights or conditioning on target items. Within this frozen-backbone boundary, we compare four model-aware attribution selectors (Integrated Gradients, GradientSHAP, Grad-SAM, and Attention Tracing) under identical weights and compute budgets.

  2. Benchmarking Against Purpose-Built Long-Sequence Models (GLASS & LONGER): To benchmark against recent long-sequence recommendation paradigms at full context (K=LK=L), we evaluate GLASS (Cao et al, RecSys 2026), a recently published coarse-to-fine generative recommender utilizing semantic ID search, and LONGER, an industrial linear-complexity long-sequence architecture. As shown in Figure 3 (Section 4.2), while full-length GLASS and LONGER (K=LK=L) incur low inference GFLOPs, neither achieves the peak ranking accuracy of attribution-distilled backbones.

  3. Orthogonality and Synergy with Efficient Architectures: Because our approach distills the raw input sequence prior to encoding, it is orthogonal to efficient model architectures and can be layered directly on top of them. Applying Integrated Gradients and GradientSHAP directly onto LONGER improves ranking quality over both undistilled LONGER (K=LK=L) and Recency truncation on KuaiRand-1K and KuaiRand-1K-Long across Table 1, Figure 7, Figure 8 and the fixed-budget Friedman analysis in Appendix B.1. On MovieLens (ML-1M and ML-10M), attribution-guided distillation on LONGER likewise matches or outperforms Recency along the Pareto frontier (Figure 3).

(Q4) Uncertainty and practical costs.

We thank the reviewer for these constructive suggestions. We have updated the manuscript as follows:

  1. Statistical Significance (Friedman & Nemenyi Tests): In Section 4.2 and Appendix B.1, we now evaluate all methods under matched compute budgets (K∈{100,50,25,12,6}K \in \{100, 50, 25, 12, 6\}) across all seed×K×architecture\text{seed} \times K \times \text{architecture} configurations. A Friedman test decisively rejects equivalent performance (p<10−60p < 10^{-60}), with Integrated Gradients (IG) securing the best overall rank (2.002.00 on NDCG@10\mathrm{NDCG@10} and 2.072.07 on HR@10\mathrm{HR@10} vs. 3.113.11 and 3.003.00 for Recency), exceeding the Nemenyi critical difference (CD0.05=0.69\mathrm{CD}_{0.05} = 0.69).

  2. Offline Attribution vs. Online Serving Costs: We report end-to-end online serving GFLOPs (including the O(∣I∣⋅d)\mathcal{O}(|\mathcal{I}|\cdot d) full-catalog scoring layer, Figure 3) alongside offline attribution GFLOPs (Appendix Table 4). While gradient-based scoring incurs higher offline compute (6.996.99 to 97.0697.06 GFLOPs for IG; 0.880.88 to 10.3910.39 GFLOPs for GradientSHAP), this cost is strictly isolated to offline preprocessing and adds zero latency to online serving. We explicitly discuss this trade-off in our new Limitations section (Section 5), noting that this work establishes gradient attribution as a strong relevance reference to motivate future lightweight, single-pass attribution estimators.

  3. Role of KuaiRand-1K (L=500L=500) vs. KuaiRand-1K-Long (L=1000L=1000): Rather than treating the two KuaiRand variants as independent domains, evaluating both lengths isolates how distillation scales as context windows double (L=500→1000L=500 \to 1000). Cross-domain distributional generalization is instead evaluated between dense movie ratings (ML-1M, ML-10M) and sparse, million-scale short-video logs (KuaiRand).

(Q5) Reporting and reproducibility.

We thank the reviewer for catching this discrepancy and for checking our reproducibility artifacts. We address both points below:

  1. Correction of Dataset Label in Section 4.3.1 (ML-1M): The noise robustness experiments in panels (d)--(f) of Figure 4 (as well as Appendix Figure 14) were conducted on ML-1M, matching the figure captions. The reference to "ML-10M" in Section 4.3.1 was a typographical error, which we have now corrected to ML-1M in the revised manuscript.

  2. Supplementary Code. Due to internal open-sourcing procedures, the code will be made publicly available upon paper acceptance. We understand this is not ideal for verifying reproducibility during review, and we are happy to answer any questions regarding implementation details.

The two weaknesses mentioned by the reviewer has been addressed by answering Questions 1-5.

Dear Reviewer, as the discussion period nears its end, we wanted to kindly check in to confirm whether our updates, including leak-free validation-selected K⋆K^\star operating points, fixed-budget Friedman/Nemenyi frontier comparisons, KuaiRand task/parameter reconciliations, and GLASS/LONGER benchmarks, have resolved your questions.

Reviewer_GRpNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for compressing historical sequences based on interpretability methods, aimed at reducing the online inference cost of sequential recommendation models. The core workflow is as follows: first, a recommendation model is trained using the full historical sequence; next, existing post-hoc attribution methods—such as Integrated Gradients, GradientSHAP, Attention Tracing, or Grad-SAM—are employed to calculate importance scores for each position in the historical sequence; then, the top-K positions with the highest scores are retained and reordered according to their original temporal sequence; finally, this compressed historical input is fed into the frozen, pre-trained model to generate predictions. Consequently, this approach eliminates the need for model retraining and shifts the attribution computation to an offline phase. The authors conducted experiments across four dataset configurations using four types of models—SASRec, BERT4Rec, Mamba, and LONGER. Performance at various compression ratios was evaluated using normalized AUC and Pareto curves (balancing performance against computational cost), with further analysis into the impacts of input noise and model capacity. The results indicate that certain attribution methods outperform random selection and truncation based on recency on the KuaiRand dataset, though this advantage is less pronounced or absent in the case of the MovieLens dataset.

Critical strengths
  1. The method is simple, model-agnostic, and does not require retraining the recommendation model. As long as an attribution method can produce position-level importance scores, this framework can, in principle, be applied to various sequential models, offering both versatility and deployment flexibility.
  2. The experiments encompass four model architectures, four dataset configurations, and multiple compression ratios, with results reported as the mean and standard deviation across three random runs. The paper also examines performance under conditions of input noise, suggesting that attribution-based selection may offer a robustness advantage when recent user behaviors are unreliable.
Critical weaknesses
  1. The technical contribution is limited, and the baselines are insufficient to demonstrate the advantages of the proposed method. In essence, the approach employs existing attribution algorithms to calculate the importance of historical items and retains the top-K items with the highest scores; the paper does not propose any novel attribution algorithms, selection mechanisms, training objectives, or theoretical analyses. Consequently, the contribution is more akin to a direct application of existing techniques combined with an empirical study. Furthermore, the experiments primarily compare the method against random selection and simple truncation (retaining only the most recent items). It lacks comparisons with stronger baselines—such as learned history selection, attention sparsification, token pruning, or behavior denoising—making the current results insufficient to establish a clear advantage for this framework over existing history compression methods.
  2. The efficiency analysis and experimental conclusions lack rigor. The paper reports only the inference GFLOPs of the compressed model, failing to account for the computational overhead—multiple forward and backward passes required by methods like Integrated Gradients and GradientSHAP—as well as the costs associated with updating attributions, storing them, and performing periodic recalculations. Moreover, the attribution-based methods do not consistently outperform the "recency" baseline on the ML-1M and ML-10M datasets. On the KuaiRand dataset, some results exhibit high variance; relying solely on aggregated averages or box plots may obscure this dataset-dependent behavior. Additionally, the paper should clarify whether the optimal compression length K is determined exclusively using a validation set; if the optimal value is selected based on performance curves from the test set, it could introduce test-set selection bias.
Questions for authors
  1. Is the optimal compression length K for the report selected a priori using a validation set, or is it determined from the full compression curve generated on the test set? Please provide the complete model selection procedure.
  2. Could you include learned history selection, behavioral denoising, token pruning, or other sequence compression methods as stronger baselines for comparison?
  3. What is the offline attribution cost for Integrated Gradients and GradientSHAP? Please report the actual runtime, GPU costs, storage requirements, and the frequency of recomputation—either per user or per million interactions.
  4. How are the history selection results updated as users generate new interactions? Furthermore, how are new user actions handled during the interval before the attribution is recomputed?
  5. Could you report absolute NDCG@10, HR@10, and actual end-to-end latency at a fixed compression ratio, rather than relying solely on normalized AUC, GFLOPs, or the "optimal" operating point specific to each individual method?
4 comments
AuthorsOpenReview ↗

Official Comment by Authors: Part 1

We sincerely thank the reviewer for recognizing our simple, retraining-free, model-agnostic framework and our comprehensive evaluation across architectures, datasets, compression ratios, and input-noise conditions. Below, we address your weaknesses (W) and questions (Q) point by point. The updated part of the revised paper is highlighted in blue in the paper.

(W1/Q2) Additional baselines:

We now explicitly benchmark against GLASS (Cao et al., RecSys'26), a coarse-to-fine generative recommender that compresses long histories via semantic ID search, as well as full-context (K=LK=L) LONGER, which incorporates internal sequence compression mechanisms. As shown in Figure 3 (Section 4.2), while GLASS and full-sequence LONGER (K=LK=L) operate at low online GFLOPs, neither reaches the peak ranking quality of attribution-distilled backbones. Moreover, because our framework distills raw input sequences prior to encoding, it is complementary to architectural compression: applying Integrated Gradients and GradientSHAP directly onto LONGER improves its ranking quality over both undistilled LONGER (K=LK=L) and Recency truncation across KuaiRand and MovieLens (Table 1, Figure 3, and Appendix B.1).

(W2/Q3) Response to Question: Offline Attribution Compute, Storage Overhead, and Recomputation Frequency

We thank the reviewer for asking about the offline resource footprint. We summarize the per-sequence compute cost, storage reduction, and recomputation cadence below, with complete GFLOP measurements reported in Appendix Table 4 and discussed in Section 5:

  1. Offline Compute Cost (Per-Sequence GFLOPs): As detailed in Appendix Table 4, per-sequence offline attribution compute for Integrated Gradients (nsteps=20n_{\text{steps}} = 20) scales from 6.996.99 GFLOPs at L=200L = 200 (ML-1M) to 97.0697.06 GFLOPs at L=1,000L = 1{,}000 (KuaiRand-1K-Long), while GradientSHAP (nsamples=2n_{\text{samples}} = 2) requires 0.880.88 GFLOPs (L=200L = 200) to 10.3910.39 GFLOPs (L=1,000L = 1{,}000). Because this attribution step executes asynchronously in offline batches, it incurs zero latency or GPU cost during live online serving. As noted in Section 5 (Limitations), developing lightweight single-pass estimators to reduce this multi-pass offline cost is a key direction for future work.

  2. Storage Footprint (Zero Attribution Storage and Smaller Distilled Sequences): Raw attribution scores ϕ\boldsymbol{\phi} do not need to be stored persistently. Instead, attribution is computed on the fly during offline preprocessing solely to select and store the compact distilled sequence of length K⋆≪LK^\star \ll L for each user. As shown across Figures 3, 7, 8, and 9, retaining only the distilled K⋆K^\star interactions substantially reduces sequence length, lowering persistent feature-store footprint per user compared to storing full histories.

  3. Recomputation Frequency: Because historical interaction relevance decays gradually and newly arriving interactions are appended directly to the distilled sequence at serving time (Section 3.4), full attribution recomputation is triggered periodically in asynchronous batch jobs (e.g., daily or after NN new interactions per user) rather than on every user action.

AuthorsOpenReview ↗

Official Comment by Authors: Part 2

(Q1) Validation-Based Selection of Optimal Compression Length (K⋆K^\star) and Model Selection Procedure

The optimal compression length K⋆K^\star is selected strictly a priori on the validation set (Dvalid\mathcal{D}_{\mathrm{valid}}) and is never tuned on the test set (Dtest\mathcal{D}_{\mathrm{test}}). We have clarified this complete model selection protocol in Sections 3.2, 3.4, and 3.7 of the revised manuscript:

  1. Backbone Training (Frozen Prior to Distillation; Section 3.7): For each dataset and random seed, the recommendation backbone is first trained to convergence on the training split DtrainD_{train} using full-length sequences (LL) and early stopping on DvalidD_{valid}. Once trained, all backbone parameters are completely frozen.

  2. Offline Attribution Scoring (Section 3.2): Using the frozen backbone, attribution scores ϕ\boldsymbol{\phi} are computed offline over the full input histories in DvalidD_{valid} and DtestD_{test} with respect to each sequence's top-1 predicted item.

  3. Validation Sweep for K⋆K^\star Per Dataset-Seed Pair (Sections 3.4 & 3.7): We sweep over a predefined candidate grid of retained sequence lengths K<LK < L (with ∣K∣=11|K| = 11 for ML-1M, ∣K∣=12|K| = 12 for ML-10M and KuaiRand-1K, and ∣K∣=17|K| = 17 for KuaiRand-1K-Long; Section 3.7). For each attribution method and seed, we evaluate ranking quality mDvalid(K)m_{D_{valid}}(K) (NDCG@10\mathrm{NDCG@10} and HR@10\mathrm{HR@10}) across all candidate KK on DvalidD_{valid} and select the single global operating point K⋆=arg⁡max⁡K<LmDvalid(K)K^\star = \arg\max_{K < L} m_{D_{valid}}(K) (Section 3.4).

  4. Held-Out Test Evaluation (Sections 3.4 & 3.7): Finally, we fix K=K⋆K = K^\star and evaluate the frozen backbone once on the held-out test set DtestD_{test} to report the operating-point results in Figure 3 and Appendix Figure 7 (while full test trajectories across all KK are shown in Figures 7 and 8 solely for diagnostic analysis).

(Q4) How are the new user interactions handled? During live serving, newly generated user interactions are immediately appended as the most recent tokens to the precomputed distilled sequence. Because recent interactions capture immediate user intent and short-term temporal dynamics, appending them ensures that real-time behavioral shifts are reflected without requiring online attribution computation. However, with the development of more light-weight attribution methods for RecSys application can lead to the online distillation as mentioned in Limitation section in the paper.

(Q5) Response to Question: Absolute Metrics and Fixed-Budget Evaluation Beyond Normalized AUC and Method-Specific Peaks

We thank the reviewer for this suggestion. We have added a dedicated fixed-budget evaluation in Section 4.2 and Appendix B.1, alongside complete unnormalized compression trajectories across all retained sequence lengths in Figures 7 and 8:

Appendix B.1 evaluates all selectors at identical, shared sequence budgets (K∈{100,50,25,12,6}K \in \{100, 50, 25, 12, 6\}) across every seed×K×architecture\text{seed} \times K \times \text{architecture} configuration. Under these fixed budgets, a Friedman test confirms statistically significant differences across methods (p<10−60p < 10^{-60}), with Integrated Gradients achieving the top overall rank (2.002.00 on NDCG@10\mathrm{NDCG@10} and 2.072.07 on HR@10\mathrm{HR@10} vs. 3.113.11 and 3.003.00 for Recency; Nemenyi CD0.05=0.69\mathrm{CD}_{0.05} = 0.69).

Thanks for your response. The authors have addressed most of my concerns, such as the more clear experiment description including how to address the new user interactions, the seletion of the optimalcompression length, and so on. However, the current score still reflects the quality of this paper. I will maintain this score.

Thank you for confirming that our revisions clarified the experimental protocol, online handling of new interactions, and validation-based selection of K⋆K^\star. Since you noted that most of your concerns were addressed, we briefly summarize how our evidence resolves the remaining points from your review:

  • Distinct Contribution: We establish post-hoc attribution as a retraining-free, model-agnostic mechanism for raw input distillation on frozen backbone, a paradigm orthogonal to architectural modifications that consistently boosts even already-efficient backbones like Mamba and LONGER.
  • Stronger Baselines & Statistical Significance: Beyond benchmarking against GLASS and full-sequence LONGER (K=LK=L) (Figure 3), our Friedman/Nemenyi analysis across fixed compute budgets (K∈{100,50,25,12,6}K \in \{100, 50, 25, 12, 6\}; Appendix B.1) confirms statistically significant gains (p<10−60p < 10^{-60}, exceeding CD0.05=0.69\mathrm{CD}_{0.05}=0.69) and cleanly delineates domain boundaries (Recency on ML-1M vs. Integrated Gradients dominating ML-10M and KuaiRand).
Reviewer_f7rtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores post-hoc feature attribution methods to compress user histories after training a sequential recommender. That way, it keeps the most relevant interactions according to the trained model and maintains most of its original performance while improving inference latency without the need to retrain the model. Experiments across multiple datasets and backbone models show a considerable inference GFLOPs reduction while preserving ranking utility.

Critical strengths
  • The paper tackles a really relevant topic for industrial applications with long user sequences. The paper is well-structured and motivated from a experimental point of view.
  • The framework is simple, applicable across different backbones and it does not require retraining or online extra compute.
  • Improvements on average performance on KuaiRand suggest potential of the method for history distillation on real domains.
Critical weaknesses
  • The paper, while providing a simple framework, still shows limited novelty. The methodology simply applies existing feature attribution methods to prune sequence interactions on the long sequence modeling problem. The main contribution is on the experimental results but no significant improvement is provided in the results and confidence intervals are mostly ignored.
  • Baselines are limited to random and recency. The paper would benefit from comparison against models like LONGER, SIM, GLASS, etc. that perform long sequence compression during inference to check against models that do not precompute sequence reduction. In the paper, LONGER is only used as backbone and not as competitor.
  • Certain clarifications are needed in the paper: reported optimal K values are validation-selected or test-set oracle peaks? Are these optimal K values selected per user sequence or per dataset-seed pair? XAI implementation details should be part of the main paper, along with their complexity for the offline dataset distillation, and how signed-scores are handled.
  • Check AUC sensitivity to the irregular, sparse K-grid (it may introduce bias inadvertently). Also, the paper should improve its statistical reporting with paired significance tests, consistent decimal positions for mean and standard deviations. Qualify Pareto-dominance claims using complete frontiers and uncertainty.
  • The paper claims supplementary code availability but that's not the case so it's not possible to verify reproducibility and clarify certain implementation details.

Note: Potential error in Section 3.6: "For non-attention models (Mamba and LONGER), Attention Tracing and Grad-SAM are excluded as they rely on explicit attention matrices;" but they are actually provided for LONGER.

Questions for authors

How does the model behave against long-sequence modeling backbones that do not pre-distill user sequences? Is there statistical significance in any of the reported results?

6 comments
AuthorsOpenReview ↗

Official Comment by Authors: Part 1

We sincerely thank the reviewer for recognizing the industrial relevance of long-sequence compression, our simple, retraining-free, backbone-agnostic framework with zero online overhead, and our strong empirical gains on KuaiRand. Below, we address your weaknesses (W) and questions (Q) point by point. The updated part of the revised paper is highlighted in blue in the paper.

(W1) Confidence Interval and Statistical Tests:

  1. Conceptual Contribution (Repurposing Post-Hoc Attribution for Retraining-Free Serving Efficiency): While our framework builds upon established post-hoc attribution estimators, our core contribution lies in repurposing explainability from a passive model-diagnostic tool into an active, retraining-free mechanism for compressing raw input sequences prior to online inference. Unlike architectural modifications (e.g., linear/sparse attention or SSMs) or trainable token-pruning modules that alter model weights and training pipelines, our formulation operates entirely offline on frozen pretrained backbones. Furthermore, we establish that raw input-level sequence distillation is orthogonal to architectural efficiency, demonstrating measurable gains even when layered directly onto modern efficient backbones such as Mamba and LONGER.

  2. Statistical Significance Across Matched Compute Budgets (Appendix B.1): To address the concern regarding variance and statistical significance, we added a formal non-parametric evaluation in Section 4.2 and Appendix B.1 comparing all methods under strictly matched compute budgets (K∈{100,50,25,12,6}K \in \{100, 50, 25, 12, 6\}) across all seed×K×architecture\text{seed} \times K \times \text{architecture} configurations. A Friedman test decisively rejects equivalent performance across both NDCG@10\mathrm{NDCG@10} and HR@10\mathrm{HR@10} (p<10−60p < 10^{-60}). Specifically, Integrated Gradients achieves the top overall rank (2.002.00 on NDCG@10\mathrm{NDCG@10} and 2.072.07 on HR@10\mathrm{HR@10}) compared to Recency (3.113.11 and 3.003.00), exceeding the Nemenyi critical difference threshold (CD0.05=0.69\mathrm{CD}_{0.05} = 0.69) and confirming statistically significant improvements.

(W2) Additional baselines:

We have now expanded our experimental baselines in Sections 3.6 and 4.2 to address the comment:

  1. Comparison with Full-Sequence LONGER (K=LK=L) and GLASS: We now explicitly benchmark against full-context (K=LK=L) executions of LONGER and GLASS (coarse-to-fine generative recommender utilizing semantic ID search). As shown in Figure 3 (Section 4.2), while full-length LONGER (K=LK=L) and GLASS incur low online inference GFLOPs, neither achieves the peak ranking quality of attribution-distilled backbones. Furthermore, because our framework distills raw input sequences prior to encoding, it is orthogonal to efficient architectures: applying Integrated Gradients and GradientSHAP directly onto LONGER boosts its ranking performance over both undistilled LONGER (K=LK=L) across KuaiRand (Table 1, Figure 3, and Appendix B.1).

  2. Incompatibility of Target-Conditioned CTR Models (SIM) with Full-Catalog Ranking: Unlike our next-item prediction setting, which scores and ranks the entire item catalog (∣I∣|\mathcal{I}|, exceeding 10610^6 items on KuaiRand) in a single forward pass, target-conditioned CTR architectures such as SIM search and filter user history conditioned on a specific candidate target item. Evaluating SIM under full-catalog ranking would require O(∣I∣)\mathcal{O}(|\mathcal{I}|) separate forward passes per user query, making it computationally infeasible for full-vocabulary retrieval benchmarks.

AuthorsOpenReview ↗

Official Comment by Authors: Part 2

W3. Clarifications on Operating-Point Selection (K⋆K^\star), Signed Attribution Handling, and Offline XAI Complexity

  1. Validation-Selected Operating Points (K⋆K^\star) Per Dataset-Seed Pair: As detailed in Sections 3.4 and 3.7, K⋆K^\star is not a test-set oracle peak. Instead, for each method, K⋆K^\star is selected strictly a priori by sweeping over the validation set: K⋆=arg⁡max⁡K<LmDvalid(K)K^\star = \arg\max_{K < L} m_{\mathcal{D}_{\mathrm{valid}}}(K). Specifically, K⋆K^\star is determined globally per dataset-seed pair rather than adaptively per user sequence, reflecting a fixed-length operational deployment budget. The held-out test metrics at these fixed operating points are then reported in Figure 3 and Figure 9.

  2. Handling of Signed Attribution Scores: As described in Section 3.1 and Equation 2 (Section 3.2), attribution scores ϕj∈R\phi_j \in \mathbb{R} are signed scalars computed with respect to the full-sequence top-1 predicted item c=arg⁡max⁡cy^c = \arg\max_{c} \hat{\mathbf{y}}. During distillation, we apply ArgTopK⁡(ϕ,K)\operatorname{ArgTopK}(\boldsymbol{\phi}, K) directly over the raw signed scores (without taking absolute values). Interactions with high negative values (ϕj<0\phi_j < 0) suppress the predicted score of item cc and are therefore dropped, ensuring that the retained KK positions preserve high-confidence prediction of the full-sequence top-1 item.

  3. XAI Implementation Details and Offline GFLOPs Overhead: We summarize the XAI hyperparameters in Appendix A.2 (nsteps=20n_{\text{steps}} = 20 for Integrated Gradients; nsamples=2n_{\text{samples}} = 2 with stdev=0.05\text{stdev} = 0.05 for GradientSHAP, both attributing the topk=1\text{top}_k = 1 prediction) and quantify their exact per-example offline GFLOPs in Appendix Table 4. As shown in Table 4, gradient-based methods require substantial offline compute (6.996.99 to 97.0697.06 GFLOPs for IG and 0.880.88 to 10.3910.39 GFLOPs for GradientSHAP across L∈[200,1000]L \in [200, 1000]). While this offline overhead is strictly isolated from live serving latency, we explicitly discuss in our new Limitations section (Section 5) that high precomputation cost can constrain large-scale production viability. Crucially, by establishing gradient attribution as an effective relevance signal, this work motivates future research into lightweight, single-pass attribution estimators that eliminate iterative forward and backward passes.

We have updated Sections 3.1, 3.2, 3.4, 3.7, and 5, as well as Appendix A.2 and Table 4, to clarify each point

W4. K-Grid Sensitivity, Statistical Significance, and Reporting Consistency

We thank the reviewer for these constructive suggestions to strengthen our empirical rigor and reporting standards. We have updated Sections 4.1 and 4.2, Table 1, Figure 6, Appendix Table 7, and Appendix B.1 accordingly:

  1. AUC Sensitivity to K-Grid Cardinality (∣K∣|K|): To ensure our irregular KK-grid does not introduce numerical integration bias into AUCm\mathrm{AUC}_m, we evaluated grid cardinality sensitivity across varying sizes of ∣K∣|K| (Section 4.1 and Appendix Figure 6). Our analysis confirms that AUCm\mathrm{AUC}_m stabilizes once ∣K∣>4|K| > 4. All benchmarks in our evaluation strictly exceed this threshold (∣K∣=11|K| = 11 for ML-1M, ∣K∣=12|K| = 12 for ML-10M and KuaiRand-1K, and ∣K∣=17|K| = 17 for KuaiRand-1K-Long; Section 3.7).

  2. Fixed-Budget Statistical Significance and Qualified Pareto Claims: In Section 4.2 and Appendix B.1, we performed a non-parametric Friedman test with post-hoc Nemenyi critical difference intervals under strictly matched compute budgets (K∈{100,50,25,12,6}K \in \{100, 50, 25, 12, 6\}) across all seed×K×architecture\text{seed} \times K \times \text{architecture} configurations. The Friedman test decisively rejects equivalent performance across both NDCG@10\mathrm{NDCG@10} and HR@10\mathrm{HR@10} (p<10−60p < 10^{-60}), with Integrated Gradients achieving the strongest overall rank (2.002.00 and 2.072.07 vs. 3.113.11 and 3.003.00 for Recency), well beyond the Nemenyi critical difference threshold (CD0.05=0.69\mathrm{CD}_{0.05} = 0.69). In Section 4.2, we also qualified our Pareto-frontier claims to note that gradient-based attribution matches or dominates Recency across 14 of the 16 dataset-architecture configurations, while explicitly acknowledging the two ML-1M cases where Recency retains an edge.

  3. Standardized Decimal Precision Across Tables: We have unified the decimal formatting across Table 1 and Appendix Table 7 so that all mean and standard deviation pairs report consistent significant digits.

(W5) Supplementary Code. Due to internal open-sourcing procedures, the code will be made publicly available upon paper acceptance. We understand this is not ideal for verifying reproducibility during review, and we are happy to answer any questions regarding implementation details.

AuthorsOpenReview ↗

Official Comment by Authors: Part 3

(Q1) Comparison against the long-sequence modeling backbones: We benchmark directly against full-sequence GLASS and LONGER (K=LK=L) as dedicated long-sequence baselines in Figure 3 (Section 4.2). Furthermore, the full compression trajectories in Figures 7 and 8 explicitly evaluate distilled sequence performance relative to the undistilled full-context model (K=LK=L, plotted at the rightmost endpoint of each curve). On the KuaiRand benchmarks, these curves demonstrate that operating at the distilled sequence length consistently improves ranking quality over the full-context K=LK=L baseline across all evaluated architectures.

Comments on the paper

1 comment
Area_Chair_Q8wqOpenReview ↗

Discussion encouraged during this period!

Hi all reviewers,

We’re about halfway through the discussion period. This is a friendly reminder to check the authors' replies. If the authors have answered questions or responded to critiques in the reviews, please leave a comment for the authors!

Thank you so much!

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.