acceptodds
Under review as a conference paper at ICLR 2027

Graph-to-Sequence Knowledge Distillation Method for Sequential Recommendation

Abstract

The graph structure of user-item interactions and users' sequential behaviors are two fundamental signals in sequential recommendation (SR). Graph-based models excel at capturing high-order connectivity in the interaction graph, while sequential models are effective in modeling dynamic user preferences. However, the two embedding models learn different representation spaces, and naive fusion strategies often suffer from representation conflicts. To bridge this gap, we design FROm Graph to Sequence Knowledge Distillation (FROGS), a theoretically grounded method that transfers structural knowledge from a graph-based teacher to a sequential student model. Under common assumptions on embeddings, noise, and sample size, we derive upper bounds on the prediction error of three schemes, sequential-only, direct fusion, and graph-to-sequence distillation. Theoretically, we establish a distillability criterion for when knowledge distillation improves the prediction. We also analyze the bias-variance trade-off in FROGS distillation. Empirically, experiments on publicly available datasets and real-world data from an aggregated payment platform demonstrate consistent improvements over the evaluated baselines.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Knowledge Distillation, Sequential Recommendation, Recommender Systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_w8GxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies when graph information can be effectively transferred to sequential models in sequential recommendation. The authors show that direct fusion suffers from an irreducible margin, and that this limitation can be mitigated through knowledge distillation based on output distributions. They theoretically characterize the prediction error of three settings (sequence-only, direct fusion, and graph-to-sequence distillation) by deriving high-probability error bounds for each. Building on this analysis, the authors propose a criterion for assessing the distillability. Empirically, the proposed approach achieves state-of-the-art performance on various real-world datasets.

Critical strengths

[S1] The paper provides both a theoretical analysis and empirical observations on representation learning, which aligns well with the scope of ICLR. To the best of my knowledge, this direction has not yet been explored in the context of graph-enhanced sequential recommendation.

[S2] The paper's attempt to approach the problem from a theoretical perspective is positive and valuable.

[S3] The experimental results are averaged over a sufficient number of random seeds, so the reported performance appears statistically reliable.

Critical weaknesses

[W1] The core contributions (the proposed criterion and its theoretical proofs) rely on the alignment residual measured by the orthogonal map defined in Eq. (1). However, I'm afraid I have to strongly disagree with the claim that a measurement based on an orthogonal map is an appropriate metric for capturing representation conflict. The counterexample is intuitive.

Let G∈RN×dG \in \mathbb{R}^{N \times d} and S∈RN×dS \in \mathbb{R}^{N \times d} denote the user embedding matrices obtained from the graph and sequence, respectively.
Now suppose that G=kSG = kS for some constant k∈(0,∞)k \in (0, \infty). Then the Eq (1) is reformulated in matrix form: Δalign=min⁡Q⊤Q=I1N∑i=1N∣Si−kSiQ∣2,\Delta_{\mathrm{align}} = \min_{Q^\top Q = I} \frac{1}{N} \sum_{i = 1}^{N} \\| S_{i} - k S_{i} Q \\|_{2}, where Si∈RdS_i \in \mathbb{R}^{d} denotes the ii-th row of SS.

Since QQ is orthogonal, we have
∣Si−kSiQ∣2≥∣1−k∣⋅∣Si∣2.\\| S_i - k S_i Q \\|_{2} \ge |1 - k| \cdot \\| S_i \\|_2.

This implies the optimal orthogonal matrix is an identity, Q∗=IQ^* = I. Therefore, Δalign=∣1−k∣N∑i=1N∣Si∣2.\Delta_{\mathrm{align}} = \frac{| 1- k |}{N} \sum_{i = 1}^{N} \\| S_i \\|_2.

This counterexample shows that when G=kSG = kS, the two embeddings have no practical difference in representational capacity, yet the orthogonality constraint can produce an arbitrarily large alignment residual driven purely by the difference in scale.

In other words, the residual can reflect a simple scale difference rather than a genuine conflict between the representations. This is a critical issue because the measure is used to motivate the proposed criterion and support the theoretical analysis. If the measure does not capture the claimed conflict, the conclusions based on it become less convincing. The authors therefore need to show that the measure captures representation conflict beyond scale differences. Without this evidence, the paper’s main contribution is substantially weakened.

[W2] The experiments are conducted on relatively small-scale datasets, and the proposed method is not compared against recent graph-based or sequential models (published in 2025 or later).

[W3] I am concerned about potential scalability issues arising from the matrix operations.

[W4] The theoretically developed criterion and the proposed method do not appear to be well connected; they seem to constitute separate contributions.

Questions for authors

[Q1] In line 54, the paper supports the claim that graph and sequence embeddings conflict with each other by citing a work on multimodal collapse. Although the two embeddings are intuitively different, don't sequence embeddings also learn temporal collaborative signals, at least implicitly?

[Q2] What are the GPU VRAM usage and elapsed time during training and inference?

2 comments
AuthorsOpenReview ↗

Refining Alignment Analysis and Clarifying Computational Cost and Distillability

Thank you for the helpful comments. We address each point below.

1. Alignment under global rescaling (W1, Q1)

We agree that orthogonal-only alignment could mistake a scale difference for a mismatch. Sections 3.1, 3.4, and 4 now use aQaQ, with a global scale a≥0a\ge0 shared by all users and Q⊤Q=IQ^\top Q=I. For your example G=kSG=kS, k>0k>0, choosing a=1/ka=1/k and Q=IQ=I gives zero residual. Theorem 1 and Appendix B now account for the scale factor, including its effect on graph-side noise in the fusion bound.

Under the revised assumptions, the fusion lower bound still contains a penalty proportional to the residual after scaled alignment, which can persist as estimation noise decreases. Distillation uses shared item-indexed output distributions and avoids embedding alignment. We also agree that sequential models can implicitly learn collaborative signals; the revised claim concerns representation geometry even when the encoders capture overlapping signals.

We repeated Appendix G's analysis using scaled orthogonal alignment, fitting on 70% of users and evaluating on the remaining 30%. Ten independent pairs of training runs established within-model references. On all nine datasets, the held-out graph–sequence residual exceeded the larger within-model reference on average, with mean paired gaps of 0.055–0.167 (Table 8). This supports mismatch beyond global scaling for the models and datasets studied.

2. Recent graph–sequence baseline (W2)

We added LS4SRec (He et al., 2025), a self-supervised graph neural sequential recommender. The comparison covers all nine datasets, including four public benchmarks and five transaction datasets (Section 5.2, Tables 1 and 9).

3. Training and inference cost (W3, Q2)

Section 5.7 and Table 3 report training costs on all nine datasets using one NVIDIA A10 GPU (24 GB) and sparse graph propagation. Mean epoch time was 2.88–29.02 seconds; total wall-clock time, including graph construction and validation, was about 2–32 minutes; peak PyTorch-allocated GPU memory was 0.17–4.52 GiB. All runs fit on one GPU. Inference uses only the sequential student; the teacher is needed during training. Separate inference latency and VRAM measurements remain to be reported.

4. How the criterion informs FROGS (W4)

The criterion screens whether graph-induced variance reduction can offset propagation bias and the student's matching residual before committing to full distillation. This trade-off motivates using the graph branch as a teacher and transferring item-level probabilities to the student.

εopt\varepsilon_{\mathrm{opt}} is the mean per-user L1L_1 distance between teacher and student item distributions. We would estimate it on a held-out subset after a small pilot distillation run, before full training. CC is an unspecified bound constant. We would choose a plausible range using prior experience or validation data, then combine a common CC with Section 3.4's data estimates in all three proxies in Eq. (3). We would check whether the KD proxy stays below both the sequential and fusion proxies across that range. This is approximate screening, not a calibrated guarantee of improvement.

Unspecified constants also occur in transfer-learning results: Chen et al. (2026, Theorem 4.5) discuss cross-validating a tuning constant; Fan et al. (2025, Theorem 6) suppress a multiplicative constant; Li et al. (2022, Theorem 3) use an unspecified constant in a sufficient condition. These examples distinguish theoretical characterization from empirical calibration.

For fixed data quantities, Eq. (9) defines a linear boundary in the (εopt,C)(\varepsilon_{\mathrm{opt}},C) plane. For an informative neighborhood, less per-user evidence lowers the CC required for the KD bound to improve on the sequential bound at a fixed residual. At a chosen CC, the boundary gives the largest admissible residual for each dataset. Larger neighborhoods reduce variance but may increase propagation bias (Corollary 3). Thus the criterion provides qualitative guidance and dataset-specific sensitivity analysis without requiring a single calibrated constant.

References

Chen, E., Chen, X., & Jing, W. (2026). Data-driven knowledge transfer in batch Q∗Q^* learning. JASA, 121(553), 276–288.

Fan, J., Gao, C., & Klusowski, J. M. (2025). Robust transfer learning with unreliable source data. Ann. Stat., 53(4), 1728–1752.

Li, S., Cai, T. T., & Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. JRSS B, 84(1), 149–173.

He, L., Yan, W., Yi, T., & Sun, H. (2025). Self-supervised graph neural sequential recommendation with disentangling long and short-term interest. ACM Trans. Recomm. Syst., 4(1), 1–25.

Thank you for your responses and for the additional experiments and analyses.

Considering the purpose of knowledge distillation, the absence of large-scale experiments is a critical weakness. In particular, the O(N2+M2)O(N^2 + M^2) memory complexity (Appendix E) is practically infeasible at industrial scale. Therefore, W2 and W3 remain unresolved.


In addition, the claim that direct fusion suffers from the alignment residual is mathematically supported only by the lower bound in Eq. (7). Eq. (6), Eq. (8), and the upper bound in Eq. (7) cannot establish this, since an upper bound containing wΔalignw\Delta_{\mathrm{align}} does not imply that fusion actually incurs the penalty. Below, we show, using only the facts presented in the paper, that this lower bound is practically zero. Please correct me if I have misread any constant.

Let P:=c,η,wτd,e−2Rmax⁡/τ,ΔalignP := \frac{c\\,\eta\\,w}{\tau d}\\, e^{-2R_{\max}/\tau}\\,\Delta_{\mathrm{align}} denote the penalty term in Eq. (7), so that the lower bound on EfuseE_{\mathrm{fuse}} is

P−CσSΛδ/τ≤P.P - C\sigma_S\Lambda_\delta/\tau \le P.

We proceed in three steps.

(i) c=1/(32cˉV)c = 1/(32\bar{c}_V) (line 1069).

(ii) By the covariance stability condition (Eq. 41) and the item-norm bound ∣v^jS∣2≤cˉV\\|\hat{v}^S_j\\|_2 \le \bar{c}_V, η2=d⋅η2d≤d,λmin⁡(Covp^i⋅S(v^S))≤tr,Covp^i⋅S(v^S)≤∑jp^ijS,∣v^jS∣22≤cˉV2,\frac{\eta}{2} = d\cdot\frac{\eta}{2d} \le d\\,\lambda_{\min}\big(\mathrm{Cov}_{\hat{p}^S_{i\cdot}}(\hat{v}^S)\big) \le \mathrm{tr}\\,\mathrm{Cov}_{\hat{p}^S_{i\cdot}}(\hat{v}^S) \le \sum_j \hat{p}^S_{ij}\\,\\|\hat{v}^S_j\\|_2^2 \le \bar{c}_V^2, so η≤2cˉV2\eta \le 2\bar{c}_V^2 and hence c,η≤cˉV/16c\\,\eta \le \bar{c}_V/16.

(iii) By the triangle inequality used in the proof of Lemma 8 (line 998), ∣ri∣2≤ru+max⁡k∣μkG∣2\\|r_i\\|_2 \le r_u + \max_k\\|\mu^G_k\\|_2 for every ii. Together with Eq. (42), this gives Δalign=1N∑i∣ri∣2≤ru+max⁡k∣μkG∣2≤Rmax⁡2wcˉV,\Delta_{\mathrm{align}} = \frac{1}{N}\sum_i \\|r_i\\|_2 \le r_u + \max_k \\|\mu^G_k\\|_2 \le \frac{R_{\max}}{2w\bar{c}_V}, which holds for any optimal scale a∗≥0a^* \ge 0.

Combining (i)-(iii) and writing x:=wcˉVΔalign/τ≥0x := w\bar{c}_V\Delta_{\mathrm{align}}/\tau \ge 0, P;≤;cˉV16⋅wτd,Δalign,e−4wcˉVΔalign/τ;=;x,e−4x16d;≤;116d⋅14e;=;164,e,d;≈;9.0×10−5(d=64).P \\;\le\\; \frac{\bar{c}_V}{16}\cdot\frac{w}{\tau d}\\,\Delta_{\mathrm{align}}\\, e^{-4w\bar{c}_V\Delta_{\mathrm{align}}/\tau} \\;=\\; \frac{x\\,e^{-4x}}{16d} \\;\le\\; \frac{1}{16d}\cdot\frac{1}{4e} \\;=\\; \frac{1}{64\\,e\\,d} \\;\approx\\; 9.0\times 10^{-5} \quad (d = 64).

This bound holds uniformly in ww, τ\tau, cˉV\bar{c}_V, a∗a^*, and Δalign\Delta_{\mathrm{align}}. Since Efuse∈[0,2]E_{\mathrm{fuse}} \in [0, 2], the lower bound in Eq. (7) is numerically vacuous. In particular, it cannot grow with the residual: since Rmax⁡≥2wcˉVΔalignR_{\max} \ge 2w\bar{c}_V\Delta_{\mathrm{align}} by (iii), the factor e−2Rmax⁡/τe^{-2R_{\max}/\tau} decays at least as fast as e−4wcˉVΔalign/τe^{-4w\bar{c}_V\Delta_{\mathrm{align}}/\tau}.

Therefore, the lower bound in Eq. (7) never exceeds 1/(64ed)1/(64ed) and does not scale with wΔalignw\Delta_{\mathrm{align}}. The theory certifies at most a≈10−4a \approx 10^{-4} penalty for direct fusion, and guarantees that KD outperforms fusion only when KD's own error bound in Eq. (8) is already below that level.

Reviewer_dpvFOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FROGS, a graph-to-sequence knowledge distillation method for sequential recommendation, in which a LightGCN-based teacher built on bipartite and second-order graphs transfers its item prediction distribution to a SASRec student via a reweighted KL loss. The authors argue that direct embedding fusion suffers from a non-vanishing alignment residual. They derive error bounds for sequential-only, fusion, and distillation predictors, along with a distillability criterion.

Critical strengths
  • The paper proposes a clearly motivated method that transfers graph knowledge to a sequential model at the prediction level, avoiding direct fusion of heterogeneous embedding spaces.
  • FROGS achieves consistent performance improvements across nine datasets, including real-world industrial transaction data.
Critical weaknesses
  • The framework in Figure 1 is largely an assembly of existing components (LightGCN, SASRec), and it shows little methodological innovation.
  • The theory is weakly connected to the actual method. The KD bound reduces to a triangle inequality with a free εoptε_{opt} term. The distillability criterion depends on unknown constants and is never actually computed on any dataset.
  • The ablation variant without distillation (Only S) already outperforms nearly all baselines (e.g., 0.5524 vs. SASRec’s 0.4488 on campus 13), so most of the reported gain does not come from distillation. In addition, closely related graph–sequence fusion baselines are missing.
  • The significance tests in Table 10 are unreliable: the t-statistics come from bootstrap resampling of a single trained model. They also contradict Table 2, since Amazon H@10 shows a negative t-statistic while Table 2 reports FROGS ahead.
Questions for authors

Please refer to Critical Weaknesses.

1 comment
AuthorsOpenReview ↗

Clarifying FROGS’s Contribution and Strengthening Statistical Evidence

Thank you for the helpful comments. We address each point below.

1. Contribution and Figure 1

We agree that Figure 1 emphasizes the established LightGCN and SASRec encoders. Our contribution is their connection: FROGS transfers the graph teacher's item-level prediction distribution through a reweighted loss separating observed and unobserved items. Shared item indices allow direct transfer in prediction space. Section 4 compares sequence-only prediction, weighted embedding fusion, and distillation, identifying when graph-induced variance reduction can offset propagation bias and the teacher-matching residual.

2. Connection to FROGS and calibration

The analysis separates teacher prediction error from the student's matching residual. The teacher bound captures the neighborhood bias–variance trade-off; Eq. (4) trains the student to match the teacher's item distributions.

εopt\varepsilon_{\mathrm{opt}} is the mean per-user L1L_1 distance between teacher and student item distributions. We would estimate it on a held-out subset after a small pilot distillation run, before full training. CC is an unspecified bound constant. We would choose a plausible range using prior experience or validation data, then combine a common CC with Section 3.4's data estimates in all three proxies in Eq. (3). We would check whether the KD proxy stays below both the sequential and fusion proxies across that range. This is approximate screening, not a calibrated guarantee of improvement.

Unspecified constants also occur in transfer-learning results: Chen et al. (2026, Theorem 4.5) discuss cross-validating a tuning constant; Fan et al. (2025, Theorem 6) suppress a multiplicative constant; Li et al. (2022, Theorem 3) use an unspecified constant in a sufficient condition. These examples distinguish theoretical characterization from empirical calibration.

For fixed data quantities, Eq. (9) defines a linear boundary in the (εopt,C)(\varepsilon_{\mathrm{opt}},C) plane. For an informative neighborhood, less per-user evidence lowers the CC required for the KD bound to improve on the sequential bound at a fixed residual. At a chosen CC, the boundary gives the largest admissible residual for each dataset. Larger neighborhoods reduce variance but may increase propagation bias (Corollary 3). Thus the criterion provides qualitative guidance and dataset-specific sensitivity analysis without requiring a single calibrated constant.

3. Incremental contribution and additional baseline

We agree that Only S is strong: it retains SASRec and the sequence-side contrastive objective. Comparing FROGS with Only S therefore isolates the incremental contribution of distillation, which we acknowledge is modest in several settings (Section 5.5). We reassessed its significance across matched training seeds, as described below.

We also added LS4SRec (He et al., 2025), a recent graph–sequence integration method using a global interaction sequence graph. The comparison covers all nine datasets (Section 5.2, Tables 1 and 9).

4. Revised significance test

We agree that resampling test users from one trained pair did not capture training variability. The negative Amazon H@10 statistic in the previous Table 10 also conflicted with Table 2.

We replaced that analysis with 20 matched random seeds on Campus 13, 38, and 107. For each seed, FROGS and Only S were trained from scratch on the same fixed, full training split; each checkpoint was selected by validation N@10 and evaluated on the full test split. We applied two-sided paired tt-tests to the 20 within-seed differences for H@KK and N@KK, K=5,10,20K=5,10,20. All 18 comparisons favor FROGS with individual p<0.05p<0.05, consistent in direction with Table 2 for shared datasets and metrics. These results are limited to the three campus datasets and do not establish significance on Amazon. See Section 5.5 and Appendix I.2 (Table 11).

References

Chen, E., Chen, X., & Jing, W. (2026). Data-driven knowledge transfer in batch Q∗Q^* learning. JASA, 121(553), 276–288.

Fan, J., Gao, C., & Klusowski, J. M. (2025). Robust transfer learning with unreliable source data. Ann. Stat., 53(4), 1728–1752.

Li, S., Cai, T. T., & Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. JRSS B, 84(1), 149–173.

He, L., Yan, W., Yi, T., & Sun, H. (2025). Self-supervised graph neural sequential recommendation with disentangling long and short-term interest. ACM Trans. Recomm. Syst., 4(1), 1–25.

Reviewer_QESyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FROGS, a graph-to-sequence knowledge distillation framework for sequential recommendation. The method combines a graph-based teacher that captures collaborative structure from the user–item interaction graph and second-order user and item graphs with a SASRec-based sequential student. The teacher transfers item-level prediction distributions to the student through a weighted distillation objective over observed and unobserved items. The paper further develops a theoretical analysis of sequential-only learning, embedding fusion, and graph-to-sequence distillation, deriving prediction-error bounds, a distillability condition, and a bias–variance characterization based on graph neighborhoods. The method is evaluated on four public benchmarks and five real-world transaction datasets against 14 baselines, together with ablation studies, hyperparameter analyses, statistical tests, and an empirical analysis of graph–sequence representation mismatch.

Critical strengths

1. The paper provides a clear and coherent motivation for graph-to-sequence knowledge distillation.

FROGS formalizes the mismatch between graph-based and sequential representation spaces and uses their shared item-indexed prediction space as the interface for knowledge transfer. The accompanying theoretical analysis gives this design a principled structure by relating sequential learning, embedding fusion, and distillation to representation mismatch, graph neighborhood size, and propagation bias. The empirical analysis of graph–sequence embedding mismatch further supports the motivation across multiple datasets and random runs. This connection between the modeling idea, theoretical formulation, and empirical evidence is one of the main strengths of the paper and weighs positively in my assessment.

2. The empirical evaluation is extensive and provides useful evidence for the effectiveness of the proposed approach.

FROGS is evaluated on four public benchmarks and five real-world transaction datasets against 14 baselines covering sequential, graph-based, knowledge-distillation, and contrastive-learning approaches. The paper also includes sequential-only, graph-only, and direct-fusion controls, together with hyperparameter studies, embedding-mismatch analyses, and additional statistical evaluations. FROGS achieves consistently strong results across the evaluated datasets, and the ablation results support the usefulness of transferring graph-derived information to the sequential student. The breadth of the empirical study increases my confidence that the reported improvements are robust across the evaluated settings and are not driven by a single benchmark.

Critical weaknesses

1. The proposed distillability criterion is not yet convincingly demonstrated as an actionable pre-distillation criterion.

A major theoretical contribution of the paper is the claim that FROGS can predict, before distillation, when graph-to-sequence KD should be preferable to sequential-only learning or direct fusion. However, the practical bounds still depend on quantities such as the student–teacher matching residual ϵopt\epsilon_{\mathrm{opt}}, and the paper itself notes that the criterion does not yield a single numerical threshold but rather upper-bound curves involving unknown quantities. The experiments establish that FROGS is effective and that graph–sequence representation mismatch exists, but they do not directly show how accurately the proposed criterion predicts the relative benefit of KD across datasets or user regimes. This is consequential because the predictive distillability criterion is one of the paper’s main theoretical contributions, and a direct empirical validation would substantially strengthen the connection between the theory and the proposed method.

2. The comparison with existing graph–sequence integration approaches is incomplete.

The theoretical analysis considers an orthogonally aligned weighted embedding-fusion formulation, while the empirical ablations compare FROGS with adaptive weighted fusion and an MLP-based fusion variant. The paper also cites more flexible graph–sequence or multi-view fusion approaches, including Zhou and Lee (2024), but these methods are not included in the main comparison. This limits the evidence that the empirical advantage of distillation over representation fusion extends beyond the relatively simple fusion baselines considered in the paper. A controlled comparison with directly relevant existing graph–sequence or multi-view integration methods cited by the paper would materially affect my assessment of the empirical novelty and advantage of the proposed approach.

Questions for authors

1. How is the proposed distillability criterion operationalized before distillation in practice?

Section 3.4 explains how Δ̂(align), κ̂, b̂, and ν̂ can be estimated, but the bound proxies still involve quantities such as ϵopt\epsilon_{\mathrm{opt}} and an unspecified constant CC. Could the authors clarify how B̂(S), B̂(fuse), and B̂(KD) are instantiated numerically before distillation, and whether their predicted ordering agrees with the observed performance ordering of FROGS, Only S, and direct fusion across datasets?

2. Can the authors provide a more direct empirical test of the regime predicted by the theory?

The theoretical analysis predicts larger benefits from distillation in the small-nn regime when the graph provides an informative effective neighborhood κ\kappa. Could the authors evaluate FROGS versus Only S across controlled history-length regimes and different effective-neighborhood regimes, using these as empirical proxies for the theoretical quantities? This would help establish whether the observed gains follow the dependence on nn and κ\kappa predicted by the theory.

3. Could the authors strengthen the comparison with the closest graph–sequence or multi-view integration methods?

The paper cites existing approaches that explicitly combine graph and sequential representations, including Zhou and Lee (2024), while the current experiments mainly compare against adaptive weighted fusion and MLP fusion. Could the authors provide a controlled comparison with the closest available integration approaches, or clarify why such comparisons are not feasible? This would materially help me assess the empirical advantage of the proposed graph-to-sequence distillation strategy.

4. Could the authors clarify the exact optimization protocol for the graph teacher and the sequential student?

Equation (5) combines the graph contrastive objective, the student objective, and the distillation loss in the overall training objective. Is the graph teacher pretrained and then fixed during distillation, are its predictions detached when computing LdistillL_{\mathrm{distill}}, or are the teacher and student optimized jointly throughout training? Please also clarify whether the graph teacher receives a supervised recommendation loss in addition to its contrastive objective. This detail affects how I interpret both the teacher–student formulation and its connection to the theoretical analysis.

5. Could the authors confirm which interactions are used to construct the graph structures during training and evaluation?

The graph teacher constructs the user–item matrix RR, the user graph RR⊤RR^\top, and the item graph R⊤RR^\top R, while the evaluation uses chronological leave-one-out splitting. Are all graph structures constructed exclusively from interactions available in the corresponding training history, with validation and test interactions excluded? Please also clarify how the observed-item set used in the distillation loss is defined under this protocol.

2 comments
AuthorsOpenReview ↗

Clarifying the Distillability Criterion, Experimental Validation, and Training Protocol

Thank you for the helpful comments. We address each point below.

1. Operationalizing the distillability criterion

εopt\varepsilon_{\mathrm{opt}} is the mean per-user L1L_1 distance between teacher and student item distributions. We would estimate it on a held-out subset after a small pilot distillation run, before full training. CC is an unspecified bound constant. We would choose a plausible range using prior experience or validation data, then combine a common CC with Section 3.4's data estimates in all three proxies in Eq. (3). We would check whether the KD proxy stays below both the sequential and fusion proxies across that range. This is approximate screening, not a calibrated guarantee of improvement.

Unspecified constants also occur in transfer-learning results: Chen et al. (2026, Theorem 4.5) discuss cross-validating a tuning constant; Fan et al. (2025, Theorem 6) suppress a multiplicative constant; Li et al. (2022, Theorem 3) use an unspecified constant in a sufficient condition. These examples distinguish theoretical characterization from empirical calibration.

For fixed data quantities, Eq. (9) defines a linear boundary in the (εopt,C)(\varepsilon_{\mathrm{opt}},C) plane. For an informative neighborhood, less per-user evidence lowers the CC required for the KD bound to improve on the sequential bound at a fixed residual. At a chosen CC, the boundary gives the largest admissible residual for each dataset. Larger neighborhoods reduce variance but may increase propagation bias (Corollary 3). Thus the criterion provides qualitative guidance and dataset-specific sensitivity analysis without requiring a single calibrated constant.

2. Controlled history-length and neighborhood experiment

We added a supplementary experiment on Campus 13, limited to this dataset by time and compute. We fixed the user cohort and validation/test targets, then retrained FROGS and Only S with the most recent 5 or 15 training interactions per user. At each history length, the bipartite and item graphs were fixed; we retained each user's 8 or 32 strongest neighbors before symmetrization as low/high effective-neighborhood proxies. The same Only S baseline was used for both graph settings.

FROGS improved HR@10 and NDCG@10 in all four settings. Gains were larger with 5 interactions than with 15 at either neighborhood level. For high κ\kappa, the NDCG@10 gains were 0.19 and 0.09 percentage points, respectively. Increasing κ\kappa improved both metrics at either history length. These trends are consistent with the regime predicted by Corollary 1 on Campus 13.

3. Graph–sequence baseline

We added LS4SRec (He et al., 2025), which enriches sequential recommendation using a global interaction sequence graph. FROGS achieves higher H@10 and N@10 on all nine datasets (Section 5.2, Tables 1 and 9).

4. Teacher and student optimization

Both branches are optimized in the same training loop. The graph teacher receives only its graph contrastive loss; next-item cross-entropy supervises the student. Teacher logits are detached in the distillation loss, so no distillation gradient reaches graph parameters. The teacher keeps learning while supplying soft targets. The independently trained encoders in Section 3.4 serve the screening procedure; Section 3.5 and Eqs. (4)–(5) describe full FROGS training.

5. Graph construction and observed items

Under the leave-one-out protocol (Section 5.1), for interactions (i1,…,il)(i_1,\ldots,i_l), training predicts iti_t from (i1,…,it−1)(i_1,\ldots,i_{t-1}) for 2≤t≤l−22\le t\le l-2. Validation predicts il−1i_{l-1} from the first l−2l-2 interactions; testing predicts ili_l from the first l−1l-1. Shared input histories contain already-observed events: the validation interaction is available by test time, while the test target stays held out.

We build RR from the longest training input prefix, (i1,…,il−3)(i_1,\ldots,i_{l-3}), and derive the bipartite graph, RR⊤RR^\top, and R⊤RR^\top R from it. Held-out events are excluded, and graphs stay fixed during evaluation. In Eq. (4), Vu+\mathcal V_u^+ contains items in the student's current input; Vu−\mathcal V_u^- contains the remaining candidates. A held-out event remains excluded even if its item ID appeared earlier.

References

Chen, E., Chen, X., & Jing, W. (2026). Data-driven knowledge transfer in batch Q∗Q^* learning. JASA, 121(553), 276–288.

Fan, J., Gao, C., & Klusowski, J. M. (2025). Robust transfer learning with unreliable source data. Ann. Stat., 53(4), 1728–1752.

Li, S., Cai, T. T., & Li, H. (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. JRSS B, 84(1), 149–173.

He, L., Yan, W., Yi, T., & Sun, H. (2025). Self-supervised graph neural sequential recommendation with disentangling long and short-term interest. ACM Trans. Recomm. Syst., 4(1), 1–25.

Reviewer_QESyOpenReview ↗

Response to authors

Thank you for the response and the additional experiments. The controlled experiments on Campus 13 and the inclusion of LS4Rec are helpful additions. The clarifications regarding the teacher–student optimization and graph construction protocols also help address my questions on these aspects. However, two concerns remain. The additional experiments provide useful evidence for the theoretical trends, but do not directly demonstrate how reliably the proposed distillability criterion can predict when KD will outperform the alternatives. Similarly, while LS4Rec is a useful addition, the comparison with more directly related graph–sequence or multi-view fusion approaches could be further strengthened. I therefore maintain my original assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.