acceptodds
Under review as a conference paper at ICLR 2027

DAME: Depth-Aware Hybrid Structural Mixture-of-Experts for Graph Recommendation

Abstract

Graph neural networks have achieved remarkable success in collaborative recommendation tasks by modeling high-order connections. However, existing methods often force diverse behaviors to follow homogenized aggregation paths, fail to disentangle complex user intents, and lack explicit modeling of the hierarchical semantics inherent in propagation depth. To address this, we propose DAME, a Depth-Aware Mixture-of-Experts framework for graph-based recommendation. DAME employs a structural expert mechanism to achieve node-adaptive graph propagation and incorporates Rotational Embedding Propagation, integrating propagation depth information into node representations through layer-wise orthogonal rotations. This design combines adaptive modeling of interaction structures with explicit depth encoding, facilitating the distinction between shallow and high-order collaborative signals. Experiments on three public datasets demonstrate that DAME achieves state-of-the-art performance. Ablation studies further validate the effectiveness of these two components, while routing analysis reveals the correlation between expert specialization and user interaction patterns.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
GNN, Recommendation, RoPe, MoE

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_iA5wOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DAME, a graph recommendation framework that combines structural mixture-of-experts with depth-aware rotary encoding. It addresses two challenges: a shared graph propagation structure may mix diverse user preferences and noisy interactions, and directly pooling layer representations may obscure the distinction between shallow and high-order collaborative signals. Experiments on Gowalla, Yelp, and Amazon compare DAME with 19 baselines. The authors report improvements across all evaluated metrics, with the largest gains on Amazon.

Critical strengths
  1. The motivation of this paper is clear and easy to understand.
  2. The improvement of this methods is pretty significant.
  3. Ablation study is implemented to show the effectiveness of the methods.
Critical weaknesses
  1. it seems that the improvement on amazon dataset is very significant but on other datasets, the results seems to be modest. Is it possible to explain the differences? Is it related to dataset character?
  2. This method is only tested on 3 datasets. The improvement is less convincing. Is it possible to test it on other datasets?
  3. τ\tau seems to be a very important hyperparameter but it is not tested in sensitivity analysis. Can you explain how τ\tau is selected?
  4. Using RoPE to encode propagation depth and improve cross-layer fusion is reasonable. However, I remain unconvinced that it preserves differences between nodes and alleviates oversmoothing. Could the authors clarify this mechanism and provide more direct evidence?
  5. Although the appendix provides a theoretical complexity analysis, the paper lacks empirical comparisons of training time, inference time, total parameter counts, and peak GPU memory usage. Could the authors evaluate the method on a substantially larger dataset and compare these costs with representative baselines? Reporting how runtime and memory consumption grow with the interaction graph size would further strengthen the scalability evaluation.
Questions for authors

See Weaknesses

5 comments

Response to Reviewer iA5w

We sincerely thank the reviewer for recognizing DAME's clear motivation, strong empirical improvements, and comprehensive ablation design. Below, we address the cross-dataset gain differences (W1), benchmark coverage (W2), binarization threshold τ\tau (W3), RoPE's geometric and anti-oversmoothing mechanism (W4), and computational efficiency and scalability (W5).


W1: Why Is DAME's Performance Gain Much Larger on Amazon?

Key Takeaway: Under the exact same embedding dimensionality as the corresponding baselines on each dataset, node-conditional structural routing yields its largest gains when two structural conditions hold simultaneously: (1) high structural heterogeneity across a large candidate item space, so different nodes genuinely require distinct propagation subgraphs; and (2) sufficiently rich local neighborhoods, providing both multi-hop paths to reweight per node and abundant edge-level supervision to train 32 structural experts per layer.

DatasetUsersItemsTraining interactionsInteractions / userInteractions / itemAvg. node degree
Gowalla25,55719,747294,98311.5414.9413.02
Yelp42,71226,822182,3574.276.805.25
Amazon76,46983,761966,68012.6411.5412.07

Comparing these graph statistics alongside our module ablations and routing diagnostics reveals a clear, self-consistent mechanism across the three regimes:

  1. Yelp (Sparse Local Neighborhood Regime): Bounded by limited propagatable paths rather than routing failure. Yelp has only 4.274.27 interactions/user and an average degree of 5.255.25 (40%–43%40\%\text{--}43\% of Gowalla and Amazon), leaving fewer neighbors and higher-order paths per node to reweight and sparser supervision per expert. Routing itself remains crisp (31.5×31.5\times within-/cross-expert behavioral similarity; 3.0%3.0\% Top-50 overlap), but thin local connectivity naturally caps downstream gain from structural filtering.
  2. Gowalla (Dense Local Neighborhood, Compact Catalog Regime): Dominated by multi-hop depth aliasing. Gowalla has dense neighborhoods (degree 13.0213.02), yet its item catalog (19,74719{,}747) is less than a quarter of Amazon's. Multi-hop propagation over a compact item graph causes hop neighborhoods to overlap rapidly, making depth disambiguation via RoPE more critical than separating diverse item subspaces.
  3. Amazon (High Heterogeneity + Ample Supervision Regime): Satisfies both conditions simultaneously. With 4.24×/3.12×4.24\times / 3.12\times more items and 3.28×/5.30×3.28\times / 5.30\times more interactions than Gowalla / Yelp alongside 12.6412.64 interactions/user, Amazon's multi-category e-commerce graph creates diverse, conflicting co-interaction subgraphs while providing nearly one million interactions to estimate sparse expert masks reliably.

Direct validation from ablations and mask diagnostics: In Table 3, removing Structural MoE on Amazon drops Recall by 0.59%0.59\%—exceeding the 0.39%0.39\% drop from removing RoPE—whereas on Gowalla and Yelp, removing RoPE causes larger drops (2.18%2.18\% and 0.59%0.59\%) than removing Structural MoE (1.36%1.36\% and 0.22%0.22\%). Moreover, all 128/128128/128 layer-experts remain active after training, with a 2.73×2.73\times higher hard mask keep rate on edges incident to nodes selecting an expert (Reviewer wAt9, W1). We will incorporate this structural regime analysis into the revised manuscript.


W2: Evaluation Across Three Benchmark Datasets

Key Takeaway: Although evaluated on three public benchmarks, Gowalla (location check-ins), Yelp (local business reviews), and Amazon (large-scale e-commerce) deliberately span three distinct application domains, a 5.3×5.3\times range in training interactions (182K–967K182\text{K}\text{--}967\text{K}), a 4.2×4.2\times range in catalog size (19.7K–83.8K19.7\text{K}\text{--}83.8\text{K}), and sparse-to-dense local connectivity (5.25–13.025.25\text{--}13.02 avg. degree)—further reinforced by 5-seed statistical validation and a controlled 4×4\times graph-scaling sweep.

  1. Consistent cross-regime superiority: DAME outperforms 19 competitive baselines on every reported metric across all three domains under identical embedding dimensionality, exhibiting regime-consistent module behavior (W1).
  2. Multi-seed statistical stability (Table 6): Across 5 independent random seeds on all three datasets, relative standard deviations stay tightly bounded within 0.56%–1.08%0.56\%\text{--}1.08\% for Recall and 0.63%–1.51%0.63\%\text{--}1.51\% for NDCG:
DatasetRecall relative std. (5 seeds)NDCG relative std. (5 seeds)
Gowalla0.56%0.63%
Yelp0.74%0.72%
Amazon1.08%1.51%
  1. Controlled 4×4\times edge-scale sweep: Evaluating nested 25%/50%/75%/100%25\% / 50\% / 75\% / 100\% interaction subgraphs on Amazon with fixed node sets (W5) confirms predictable sub-linear runtime and memory growth. In the revision, we will explicitly state these domain/scale characteristics and discuss how catalog heterogeneity and local density govern expected gains.

W3: Selection Rationale and Sensitivity of Mask Threshold τ\tau

Key Takeaway: In Eq. (6), τ\tau is the hard binarization cutoff in the straight-through estimator (not a Gumbel-Softmax temperature). Setting τ=0.5\tau=0.5 corresponds algebraically to the exact zero-logit sign boundary (α>0\alpha > 0), requires zero per-dataset tuning, and exhibits minimal variation (0.55%–0.60%0.55\%\text{--}0.60\%) across perturbations.

Specifically, Eq. (6) generates discrete edge masks via:

pe,k(l)=σ(αe,k(l)),ze,k(l)=I(pe,k(l)>τ)−sg(pe,k(l))+pe,k(l)p_{e,k}^{(l)} = σ(α_{e,k}^{(l)}), z_{e,k}^{(l)} = I(p_{e,k}^{(l)} > τ) - sg(p_{e,k}^{(l)}) + p_{e,k}^{(l)}

Because σ(0)=0.5\sigma(0)=0.5, the condition pe,kl>0.5p_{e,k}^{l}>0.5 is strictly equivalent to αe,kl>0\alpha_{e,k}^{l}>0 (positive logits retain an edge; negative logits prune it), providing a natural parameter-sign boundary shared across all datasets without tuning. Sweeping τ∈[0.475,0.525]\tau \in [0.475, 0.525] on Gowalla under held-out validation confirms strong stability:

Threshold τ\tauRelative Recall@20 vs. τ=0.5\tau=0.5Relative NDCG@20 vs. τ=0.5\tau=0.5
0.475-0.56%-0.55%
0.490-0.24%-0.18%
0.500 (default)0.00%0.00%
0.510+0.04%0.00%
0.525-0.04%-0.06%

Across the entire interval, Recall@20 and NDCG@20 vary within 0.60%0.60\% and 0.55%0.55\%, with τ=0.5\tau=0.5 tying for best NDCG@20 and within 0.04%0.04\% of peak Recall@20 while keeping all 128/128128/128 layer-experts active (Reviewer wAt9, W1). We will clarify τ\tau's exact role next to Eq. (6) and add this sensitivity table in the revision.


W4: Does RoPE Preserve Node Distinctions and Mitigate Over-Smoothing?

Key Takeaway: Yes. RoPE strictly preserves within-layer node geometry via orthogonal transformations while disambiguating shallow vs. high-order collaborative signals during cross-layer summation via relative-depth inner products—outperforming learned depth-encoding alternatives with zero extra parameters and cutting 2-to-8 layer Recall degradation by more than half.

  1. Exact within-layer geometry preservation (Isometry): At depth ll, multiplying all nodes by the block-diagonal orthogonal rotation RlR_l (Rl⊤Rl=IR_l^\top R_l = I) leaves pairwise cosines, Euclidean distances, and norms strictly invariant:

    \left\|R_l h_u^{(l)} - R_l h_v^{(l)}\right\|_2 = \left\|h_u^{(l)} - h_v^{(l)}\right\|_2.$$ Empirically, pre-/post-rotation numerical error is $<4.8\times 10^{-7}$ (FP32 precision), and pre-rotation layer-4 within-layer user/item mean cosines remain low at $0.032 / 0.025$ (far from collapse $\to 1$). Thus, RoPE never distorts within-hop node distinctions.
  2. Relative-depth cross-layer pooling (Rl⊤Rm=Rm−lR_l^\top R_m = R_{m-l}): Unlike uniform or scalar-weighted summation—which conflates shallow and high-order hops in one shared subspace—cross-layer inner products under RoPE satisfy ⟨Rlhu(l),Rmhv(m)⟩=(hu(l))⊤Rm−lhv(m)\langle R_l h_u^{(l)}, R_m h_v^{(m)}\rangle = (h_u^{(l)})^\top R_{m-l} h_v^{(m)}, injecting explicit relative-hop awareness (m−l)(m-l) with zero extra parameters and unchanged embedding dimensionality.

  3. Controlled depth-encoding ablation and 2-to-8 layer scaling (Gowalla): Under identical held-out validation controls (Reviewer pHsz, W2), parameter-free RoPE improves Recall@20 / NDCG@20 by +0.44%/+0.37%+0.44\% / +0.37\% over no depth encoding, outperforming both learnable additive depth embeddings (−0.56%/−0.68%-0.56\% / -0.68\%, which perturb vector norms/cosines) and learnable scalar layer weights (−6.40%/−7.31%-6.40\% / -7.31\%). Furthermore, across 2 to 8 layers:

Propagation layersDAME Recall@20 gain over same-depth pure GCN
2 layers+0.40%
4 layers+1.41%
6 layers+1.21%
8 layers+1.13%

From 2 to 8 layers, pure GCN degrades by 1.28%1.28\% in Recall@20 whereas DAME degrades by only 0.56%0.56\% (less than half), maintaining a clear advantage at every depth.


Reviewer_NJMvOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DAME, a novel graph-based recommendation approach. DAME addresses two main challenges in the current literature, namely, the uniform treatment of all interactions through an homogeneous aggregation procedure within the user-item graph, and the message passing propagation which is not aware of the contribution of different depths. To this end, DAME integrates a hybrid structural mixture-of-experts where one of them is shared and always active to preserve global connectivity, while the others learn different sparse representations of the adjacency matrix to allow node-specific propagation. For every user node in the system, a router applies the routed expert distribution and retains only the top-kk most relevant experts. Finally, a rotational embedding propagation ensures embeddings computed at each propagation depth are not equally summed for pooling, but it leaves a stamp specific to each considered depth. Results on three popular recommendation datasets and against 19 state-of-the-art (graph)-based recommender systems demonstrate the efficacy of the proposed approach. Further ablation studies and hyper-parameter sensitivity analyses, alongside a routing interpretability and expert specialization and a stability depth analysis, provide complementary evaluation dimensions to support the goodness of the architectural choices.

Critical strengths

I think the paper has the merits to address some critical limitations in the graph-based recommendation literature, namely, the uniformity of aggregation and the depth-agnostic propagation. Then, the proposed methodology is adequately sound and relatively simple (which I consider as a positive aspect). Finally, the empirical evaluation considers a large range of baselines spanning different families of (graph)-based recommendation approaches.

Critical weaknesses

I recognize two main weaknesses in the work, where one is slightly minor while the other is more severe.

As for the former, I think the presentation of the methodology would require a more clarity in terms of notation and terminology. For instance, I am not sure I understood if formulas in Section 3.2 only refer to the user-side part of the recommendation system, or if they also apply to the item side.

Regarding the latter, I think the experimental evaluation would require some more careful considerations. For instance, while it is admirable that the model can largely outperform the other baselines on 2 out of 3 datasets (Gowalla and Amazon-Book), I would like to hear from the authors some possible intuitions on why the same is not observable on Yelp, where the margin is much more limited. Additionally, I think it might be useful to consider additional experiments regarding the running time of the proposed DAME methodology with respect to some of the other baselines; this would complement the evaluation dimensions to understand whether the improved recommendation performance comes at the detriment of speedup during the training or inference time.

Questions for authors

My questions are mainly related to the observed weaknesses from above, especially the last one.

  1. Do the authors have an intuition on why the performance gap on Yelp is more limited with respect to the other datasets? Maybe taking a look at the dataset characteristics might help in this sense. Additionally, running further experiments on other recommendation datasets might help improving the generalization of results.

  2. Can the authors provide further evidence of the running times (training and/or test) for the proposed DAME model and some of the most competitive baselines?

5 comments

Response to Reviewer NJMv

We thank the reviewer for the positive assessment of DAME's motivation, simplicity, and empirical study. We address the bipartite equations, dataset-dependent gains and scope, and runtime against strong baselines below.

W1: Do the structural-MoE equations apply only to users?

No. They apply symmetrically to users and items. Every layer updates the complete bipartite node matrix

H(l)=[HU(l)HI(l)]∈R(∣U∣+∣I∣)×d.H^{(l)}=\begin{bmatrix}H_U^{(l)}\\H_I^{(l)}\end{bmatrix}\in\mathbb R^{(|U|+|I|)\times d}.

Each target node i∈U∪Ii\in U\cup I selects its Top-KK experts and aggregates opposite-side neighbors through the effective edge weights:

h_v^{(l)}=\sum_{u\in\mathcal N(v)}w_{vu}^{(l)}h_u^{(l-1)}.$$ Shared and routed masks cover both edge directions, and final recommendation scores are user-item inner products. Section 3.2 used a user only for exposition. We will define $i\in U\cup I$ explicitly and include these two symmetric update blocks. ## W2: Performance regimes, generalization, and efficiency This weakness raises two issues developed in Q1/Q2. First, Yelp's smaller gain is consistent with its thin local neighborhoods, not routing collapse. Three domains, five seeds, and a fixed-node scaling study support stability across the evaluated settings, while we will state the external-validity boundary and add another benchmark. Complementary evidence appears in our responses to **Reviewer iA5w (W1, W2, W5)** and **Reviewer pHsz (W1)**. Second, DAME's extra cost lies offline. Exact edge fusion accelerates naive multi-expert forward/training by $5.24$--$8.55\times$/$6.68$--$9.88\times$; cached online scoring uses only inner products. Q2 compares the core training paths of LightGCN, a single-mask proxy, SGL-ED, SimGCL, and DAME on common operators and hardware. ## Q1: Why is the gain smaller on Yelp, and how general are the results? **Yelp has the thinnest local interaction evidence, limiting the paths that structural experts can reweight.** Large routing gains require both diverse local/higher-order paths and enough supervision to estimate 32 expert masks. |Dataset|Users|Items|Training interactions|Interactions/user|Avg. node degree| |---|---:|---:|---:|---:|---:| |Gowalla|25,557|19,747|294,983|11.54|13.02| |Yelp|42,712|26,822|182,357|4.27|5.25| |Amazon|76,469|83,761|966,680|12.64|12.07| Yelp's degree is only $40\%$--$43\%$ of Gowalla/Amazon. Hence: **lower degree $\rightarrow$ fewer candidate paths $\rightarrow$ sparser supervision per expert $\rightarrow$ less headroom for structural filtering**. Two findings support this mechanism and rule out routing failure. First, removing MoE reduces Yelp Recall@40 by $0.22\%$, less than removing RoPE ($0.59\%$); on Amazon the ordering reverses ($0.59\%$ vs. $0.39\%$). Our response to **Reviewer iA5w (W1)** analyzes this complementary Amazon regime. Second, Yelp has the strongest within-/cross-expert behavioral-similarity ratio ($31.5\times$) and lowest cross-expert Top-50 overlap ($3.0\%$). In our responses to **Reviewer pHsz (Q3)** and **Reviewer wAt9 (W1)**, we further report load entropy $0.954$--$1.000$, $27.32/32$ effective groups, and no dead experts with load balancing. The smaller gain therefore reflects limited propagatable signal, not collapsed specialization. For scope, Gowalla, Yelp, and Amazon span location, local-business, and e-commerce recommendation, a $5.30\times$ interaction range, a $4.24\times$ item-count range, and degrees $5.25$--$13.02$. Five-seed relative standard deviations are $0.56\%$--$1.08\%$ (Recall) and $0.63\%$--$1.51\%$ (NDCG), showing low run-to-run variation. Our response to **Reviewer iA5w (W5)** adds a fixed-node $4\times$ edge-scale study. These results establish stability over the tested regimes, not every recommendation setting.

Q2: Training and testing runtime against strong baselines

Naive full-batch execution performs a full sparse convolution for every selected expert, costing O((1+Nexp)∣E∣d)O((1+N_{exp})|E|d). Since gi,eg_{i,e} depends only on target ii and mijem^e_{ij} only on edge (i,j)(i,j), selected masks can be fused first:

wij=Aij(mijs+∑e∈TopK⁡(i)gi,emije),H=WX,w_{ij}=A_{ij}\left(m^s_{ij}+\sum_{e\in\operatorname{TopK}(i)}g_{i,e}m^e_{ij}\right),\qquad H=WX,

reducing each layer to O(K∣E∣+∣E∣d)O(K|E|+|E|d) with one feature aggregation. Maximum fused-vs.-naive output error is 3.73×10−93.73\times10^{-9} and auxiliary-loss error is 0. Binary masks pack from 218.7/319.9/1054.2 MB to 6.8/10.0/32.9 MB.

We separately time a training iteration, offline embedding refresh, and full-test all-item ranking on the same GPU.

Entries below are LightGCN / SGL-ED / SimGCL / DAME-fused for training and memory, LightGCN / single-mask / DAME-fused for forward, and DAME-naive -> fused in the final column:

DatasetForward (ms)Train iteration (ms)Peak memory (MB)Full ranking range (ms)Fusion gain F/T/M
Yelp12.06 / 12.24 / 22.4631.34 / 82.08 / 100.33 / 60.523323 / 6468 / 8878 / 465863.62--63.917.62x / 9.88x / 49.1%
Gowalla9.14 / 9.24 / 16.8824.56 / 62.27 / 74.63 / 50.683070 / 4903 / 6446 / 494330.07--30.238.55x / 9.22x / 33.1%
Amazon18.50 / 18.63 / 34.0243.59 / 116.81 / 137.65 / 102.904605 / 8691 / 11344 / 11840214.47--215.795.24x / 6.68x / 27.3%

Fusion removes the main repeated expert computation. Fused DAME forward is 1.841.84--1.86×1.86\times LightGCN; its training iteration is 1.141.14--1.36×1.36\times faster than the matched SGL-ED path and 1.341.34--1.66×1.66\times faster than SimGCL. These are controlled core-path comparisons, not official end-to-end timings. Full-test ranking differs only within measurement noise; after embeddings are cached, online scoring uses the same inner-product and Top-KK operators without router or GNN work.

As detailed in our responses to Reviewer iA5w (W5), Reviewer wAt9 (W3), and Reviewer pHsz (Q2), increasing Amazon interactions 4×4\times with fixed nodes raises fused-DAME forward/training time only 1.42×/1.50×1.42\times/1.50\times, while peak memory remains below 12 GB. This supports smooth scaling over the measured range; mask storage itself remains linear in edge count.

Dear Authors,

Thanks for your rebuttal, which I carefully assessed.

  • W1. Thanks for the clarification regarding symmetric structural-MoE equations to users and items. As you said, I would suggest you integrate this explanation within the methodology section.

  • W2-Q1. Thanks for this further theoretical and empirical evidence on the performance gap observed between the other datasets and Yelp. I agree the lower node degree (both user- and item-wise) might be the reason for the underperformance on that benchmark. I also acknowledge the other answers provided to the other reviewers which somehow complement this one.

  • Q2. Thanks for the additional experiments regarding the efficiency of the proposed methodology against the other baselines. From the results, I guess DAME is not the most efficient approach overall, but it can reach a decent trade-off on different metrics and dimensions.

Reviewer_pHszOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DAME, a graph-based recommender built on top of LightGCN with two additions. First, a hybrid structural MoE learns several sparse, edge-masked views of the interaction graph at each layer, and a router picks the Top-K views for each node, together with an always-on shared expert. Second, a RoPE-style rotation is applied after each propagation layer, using the propagation depth as the “position”, so that embeddings from different layers remain distinguishable before layer pooling. Experiments on Gowalla, Yelp and Amazon report improvements over 19 baselines, with the largest gain on Amazon, along with ablations, sensitivity analysis on the number of experts and K, and some analysis of routing behavior.

Critical strengths
  1. The idea of treating graph propagation itself as the experts, i.e. routing each node over learned sparse views of the interaction graph instead of over FFN branches, is simple and easy to plug into LightGCN-style models. It is a reasonable way to let different users aggregate from different neighborhoods.
  2. The paper goes beyond the main table. It includes routing analysis across user activity groups, expert load distributions, multi-seed results and a depth study, and the appendix discusses complexity and limitations fairly openly.
Critical weaknesses
  1. The experimental comparison is hard to interpret. The baseline numbers seem to be taken directly from the LightGNN paper, while DAME’s own setting (embedding size, number of layers, validation protocol) is not reported. Also, in the ablation, each single-module variant is already far above LightGCN and LightGNN, while combining the two modules adds very little. So it is unclear where the gain actually comes from.
  2. I am somewhat confused about how RoPE alleviates over-smoothing. The rotation is the same for all nodes, so it does not seem to change node similarities within a layer, and the drop in inter-layer cosine similarity (Tables 7 and 12) may simply come from the rotation itself. Could the authors explain this more clearly?
  3. There is no related work section in either the main text or the appendix, so it is hard to judge what is new relative to prior work on graph MoE, disentangled CF and edge masking. Please add this discussion.
Questions for authors
  1. What embedding size and how many layers are used for DAME? Were the baselines re-run under the same setting, or taken from prior work? Are the hyperparameters selected on a validation set?
  2. The conclusion mentions controllable overhead. Could you report training and inference time and memory compared with LightGCN and LightGNN?
  3. For the routing analysis in Table 5, how does the same-expert item overlap compare with a simple baseline, such as clustering LightGCN embeddings into the same number of groups? This would show whether the specialization comes from the routing itself.
2 comments

Response to Reviewer pHsz

We thank the reviewer for the careful assessment and for identifying unclear experimental details and questions about the interpretation of RoPE.

W1 / Q1: Experimental settings, baseline provenance, and model selection

DAME uses 4 graph-propagation layers, 32 routed experts, and Top-4 routing, with the corresponding LightGNN embedding dimension on each dataset. For the added controls, hyperparameters are fixed before training and the checkpoint with the best validation Recall is selected before one final test evaluation.

The main table uses the datasets and baseline values reported by LightGNN; we will state this source explicitly.

For the ablation study, each single-module variant still retains the other proposed component. Its gap to the full model therefore measures the marginal contribution of adding that module on top of the other, rather than the full gain over a plain graph backbone. A controlled Gowalla depth study makes this interaction clearer:

LayersDAME Recall@20 relative to same-depth pure GCN
2+0.40%
4+1.41%
6+1.21%
8+1.13%

The shallow-layer parity and the larger gains at greater depth are consistent with DAME addressing depth redundancy and structural noise rather than merely adding capacity. To make the joint gain over a plain backbone explicit, we will add a vanilla GCN row to the ablation table. The 5-seed results in Table 6 show low run-to-run variation, with relative standard deviations of 0.56%0.56\%–1.08%1.08\% for Recall and 0.63%0.63\%–1.51%1.51\% for NDCG. In the additional controlled comparisons, DAME also outperforms LightGCN by +1.25%/+0.86%+1.25\% / +0.86\% on Gowalla, supporting the robustness of the design.

W2: How does RoPE preserve node differences and mitigate depth degradation?

Orthogonal layer rotations RlR_l preserve exact within-layer geometry (cos⁡(Rlhu(l),Rlhv(l))=cos⁡(hu(l),hv(l))\cos(R_l h_u^{(l)},R_l h_v^{(l)})=\cos(h_u^{(l)},h_v^{(l)}) and Euclidean distances), while RlTRm=Rm−lR_l^TR_m=R_{m-l} makes cross-layer interactions explicitly depend on relative depth before summation.

Depth treatmentRelative Recall@20 vs. no encodingRelative NDCG@20 vs. no encoding
No depth encoding0.00%0.00%
Learnable scalar layer weights-6.40%-7.31%
Additive depth embeddings-0.56%-0.68%
RoPE+0.44%+0.37%

RoPE outperforms learned alternatives with zero extra parameters. Numerically, pre-/post-rotation error is <4.8×10−7<4.8\times10^{-7}, and pre-rotation layer-4 user/item cosine similarity remains low (0.032/0.0250.032/0.025, far from node collapse). These results verify geometry preservation, while the task metrics support improved depth robustness: from 2 to 8 layers, DAME's Recall drops by only 0.56%0.56\% (vs. 1.28%1.28\% for pure GCN), maintaining a positive advantage at every depth. We do not claim that isometry alone proves the absence of cross-layer aliasing.

W3: Missing related work and novelty positioning

DAME's contribution is a structured coupling of target-node-conditioned, layer-specific graph operators with parameter-free relative-depth encoding. GraphDIVE/GMoE route over graph experts; DGCF/DCCF separate latent intent channels; graph-denoising methods and LightGNN suppress unreliable or redundant interactions; LightGCN/GPR-GNN aggregate or reweight propagation depths. DAME differs by routing each target node over layer-specific edge-mask operators while encoding propagation depth inside the representation with an isometric rotation. We will add this explicit comparison and the corresponding citations to Related Work.

Q2: Training, inference, and memory cost

DAME's extra cost is confined to offline training and full-graph embedding refresh; peak training memory stays below 12 GB across all datasets. Naive full-batch execution costs O((1+Nexp⁡)∣E∣d)O((1+N_{\exp})\vert{}E\vert{}d) because any selected expert runs a full graph convolution. By fusing Top-KK masks at the edge level prior to a single sparse propagation, we reduce per-layer complexity to O(K∣E∣+∣E∣d)O(K\vert{}E\vert{}+\vert{}E\vert{}d) with exact numerical equivalence (max element-wise output error <3.73×10−9<3.73\times 10^{-9}; auxiliary-loss error =0=0). Relative to naive expert-by-expert execution, edge-level fusion yields substantial speedups and memory savings:

Absolute entries follow LightGCN / single-mask proxy / DAME-fused; the last column reports DAME-naive -> DAME-fused forward/training speedup and peak-memory reduction:

DatasetForward (ms)Training iteration (ms)Peak memory (MB)Fusion vs. naive: forward / training / memory
Yelp12.06 / 12.24 / 22.4631.34 / 33.96 / 60.523323 / 3357 / 46587.62x / 9.88x / 49.1%
Gowalla9.14 / 9.24 / 16.8824.56 / 27.48 / 50.683070 / 3120 / 49438.55x / 9.22x / 33.1%
Amazon18.50 / 18.63 / 34.0243.59 / 49.23 / 102.904605 / 4765 / 118405.24x / 6.68x / 27.3%

Fused DAME runs at ∼1.85×\sim 1.85\times LightGCN forward time. As detailed in our response to Reviewer NJMv (Q2), it cuts per-iteration training time by 1.141.14–1.36×1.36\times versus SGL-ED and 1.341.34–1.66×1.66\times versus SimGCL. For deployment, bit-packing compresses FP32 masks to 6.86.8–32.932.9 MB; once final embeddings are cached, online inference uses the same inner-product scoring operator and adds no router or graph-propagation work. As reported in our response to Reviewer iA5w (W5), a 4×4\times increase in Amazon interactions raises forward and training time by only 1.42×1.42\times and 1.50×1.50\times over the tested range.

Q3: Is routing specialization stronger than simple clustering?

Not on every criterion. K-means explicitly optimizes post-hoc compactness, whereas DAME optimizes assignments jointly with the graph operators that construct each representation. We train LightGCN on the same graph, cluster its normalized user embeddings into 32 groups, and compare them with DAME's deepest-layer routing. Entropy/effective groups measure expert use (higher is better), CV/Gini measure size imbalance (lower), and NMI/JSD measure activity-conditioned structure (higher).

MetricDAME routingLightGCN K-meansPreferred direction
Normalized load entropy0.9540.937Higher
Effective groups (out of 32)27.3225.70Higher
Group-size CV0.5900.703Lower
Group-size Gini0.3140.365Lower
Activity-quartile NMI0.05270.0483Higher
Low/high-activity JSD0.1360.116Higher

DAME uses 6.3%6.3\% more effective groups, lowers CV/Gini by 16.1%/14.0%16.1\%/14.0\%, and raises NMI/JSD by 9.2%/17.6%9.2\%/17.6\%. K-means is more compact (within-/cross-group similarity 18.61×18.61\times vs. 9.56×9.56\times; Top-50 overlap 1.03%1.03\% vs. 5.44%5.44\%), so we do not claim superiority on that objective. DAME's advantage is propagation-aware specialization: in our response to Reviewer wAt9 (W1), we show that load balancing raises routing entropy from 0.7120.712 to 1.0001.000 and removes zero-assignment experts, while selected-expert edges receive 2.73×2.73\times higher hard-mask retention. Thus DAME provides behavior-conditioned structural routing, not merely a tighter static clustering.

Reviewer_wAt9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DAME, a graph-based recommendation framework that employs a Mixture-of-Experts (MoE) mechanism to capture diverse behavioral patterns in user-item interactions, while incorporating Rotary Position Encoding (RoPE) into the GNN propagation process to capture different semantic patterns and mitigate overfitting.

Critical strengths
  1. The paper identifies two important challenges in GNN-based recommendation: heterogeneous interaction patterns and the loss of propagation-depth information during multi-layer aggregation. Both problems are relevant to collaborative filtering, particularly in sparse interaction scenarios.
  2. DAME consistently achieves the best reported Recall and NDCG results across Gowalla, Yelp, and Amazon among the 19 listed baselines. The improvements on Amazon are particularly notable, suggesting potential benefits in sparse recommendation scenarios.
  3. The paper investigates expert routing, behavioral specialization, sensitivity, and representation similarity, providing clear evidence for the contribution of each module.
Critical weaknesses
  1. The paper claims to achieve structural decoupling. However, it only employs a load-balancing loss to prevent expert collapse and encourage a relatively balanced distribution of nodes across experts. Such a constraint does not guarantee that different experts learn distinct structural patterns, particularly given the lack of explicit constraints or regularization on the learnable edge masks to encourage inter-expert structural diversity.
  2. Given the fixed 1:1 weighting between the shared and routed experts in Eq. (10), the shared branch continuously propagates information across the entire original topology. Consequently, even if certain noisy edges are removed by the routed experts, their information can still leak into the final node representations via the shared expert. This raises the question of whether genuinely effective structural denoising is attained.
  3. Compared with conventional baseline models, introducing multiple routing experts may incur substantial additional computational and memory overhead, particularly for large-scale recommendation graphs with a large number of user-item interactions.
  4. The necessity of introducing RoPE for propagation-depth encoding remains unclear. Simpler alternatives, such as learnable layer-wise aggregation weights and learnable layer-specific positional embeddings, could potentially capture propagation-depth information without requiring explicit rotational transformations. However, the authors do not provide comparisons with these alternatives.
Questions for authors
  1. Regarding Weakness 1, are there any theoretical guarantees or explicit methodological constraints that encourage structural disentanglement and specialization among different experts?
  2. Regarding Weakness 2, could the authors introduce an adjustable weighting coefficient to control the relative contributions of the shared and routed experts? Additionally, could the authors provide a sensitivity analysis of this coefficient to investigate how different weighting configurations affect recommendation performance and structural denoising effectiveness?
  3. Regarding Weakness 3, could the authors provide a comparison of training efficiency and memory consumption between DAME and the baseline models?
  4. Regarding Weakness 4, could the authors provide additional experiments to verify the superiority of the proposed rotational depth encoding over simpler layer-wise perturbation mechanisms?
3 comments

We thank the reviewer for recognizing DAME's motivation, empirical performance, and analyses.

W1 / Q1: Can DAME guarantee that different experts learn distinct structures?

We distinguish an unconditional identifiability guarantee from the specialization mechanism that DAME actually provides. Like most MoE models, DAME does not claim that expert identities are unique up to permutation. It does, however, impose conditional structural optimization: every routed expert has an independent, layer-specific edge mask, and its mask receives recommendation gradients only from target nodes that select that expert. This mechanism explains how routing can break expert symmetry. At one layer, write the routed output of node ii as

hir=∑e∈Siqie∑jAijmijexj,h_i^r=\sum_{e\in S_i}q_{ie}\sum_j A_{ij}m^e_{ij}x_j,

where SiS_i is the Top-KK expert set and qieq_{ie} is the normalized routing weight. Under the straight-through estimator, the gradient of edge-mask logit sijes^e_{ij} is

=\mathbf 1[e\in S_i]q_{ie}\sigma'(s^e_{ij})A_{ij} \left\langle\nabla_{h_i^r}\mathcal L,x_j\right\rangle.$$ Thus, even if two experts currently have identical masks, any difference in their selection status or routing weight on a node-edge pair with a nonzero task gradient gives them different gradients, so mask equality need not persist after the next optimization step. Top-$K$ routing can therefore couple experts to different conditional node distributions, while the load-balancing term keeps those supports populated. To ensure that discrete experts remain sparse yet participate from the first optimization step, we initialize $s^e_{ij}=-0.3+0.1\epsilon^e_{ij}$. The expected initial hard keep rate is only $0.135\%$ (about 850 edges per expert on Gowalla), yet every expert is nonempty. After training, the model keeps $0.160\%$ of edges on average and all $128/128$ layer-experts remain structurally alive; relative to LightGCN trained through the same code path, Recall/NDCG improve by $1.25\%/0.86\%$. We report **hard binary** masks rather than infer sparsity from sigmoid means. Among active initializations, $-0.3$ is best: changing the mean to $-0.2/-0.1/0.0$ changes Recall by $-0.72/-3.62/-3.27\%$ and NDCG by $-0.67/-5.08/-4.10\%$. The load-balancing loss provides an explicit anti-collapse constraint on expert use. With it, normalized routing entropy is $1.000$ at every layer and mean load CV is only $0.008$. Removing it reduces mean entropy to $0.712$, raises CV to $1.476$, and leaves some experts with zero assignments at every layer. This ablation directly shows that the constraint is necessary to keep all conditional supports populated. Finally, the learned structures align with those supports. In layer 1, the hard keep rate is $0.428\%$ on edges whose target node selects the expert, versus $0.157\%$ on other edges, a $2.73\times$ enrichment that remains positive through all four layers. The deepest routing groups also have a $9.56\times$ within-/cross-group behavioral-similarity ratio, only $5.44\%$ Top-50 item overlap, and load entropy $0.954$. Two complementary controls further exclude accidental grouping: in our response to **Reviewer pHsz (Q3)**, we show that DAME has more effective groups than post-hoc LightGCN K-means ($27.32$ versus $25.70$) and more balanced group sizes; in our response to **Reviewer iA5w (W3)**, we show that Recall/NDCG vary by only $0.60\%/0.55\%$ over $\tau\in[0.475,0.525]$. We therefore do not claim expert identifiability up to permutation. The precise conclusion is that DAME maintains non-collapsed conditional supports and learns sparse structures that align significantly with routing decisions.

W2 / Q2: Can the complete shared graph leak noisy information?

The shared branch does retain baseline messages from the original graph, including potentially noisy ones, so DAME does not claim to set every noisy edge exactly to zero. This does not invalidate structural selection because the final message combines the shared path with node-conditional routed paths. The shared branch preserves connectivity for low-degree nodes and general collaborative signal, while routed experts increase the relative contribution of edges supported by the current node. DAME therefore controls noise through relative structural reweighting, rather than absolute edge deletion.

For target node ii, the effective edge weight is

wij=Aij(λsmijs+λr∑e∈TopK(i)gi,emije)w_{ij}=A_{ij}(\lambda_s m^s_{ij}+\lambda_r\sum_{e\in\mathrm{TopK}(i)}g_{i,e}m^e_{ij})

The model uses λs=λr=1\lambda_s=\lambda_r=1. The trained shared masks retain all edges on Gowalla and Amazon, preserving the original graph connectivity exactly. Let rij=∑egi,emije∈[0,1]r_{ij}=\sum_e g_{i,e}m^e_{ij}\in[0,1] denote the routed contribution. The effective multiplier is then 1+rij1+r_{ij}: an edge supported by selected experts receives up to twice the baseline contribution, while an unsupported edge retains the residual baseline. DAME thus amplifies node-supported evidence relative to unsupported evidence rather than absolutely removing the latter, avoiding disconnection of low-degree users or irreversible deletion of globally useful interactions.

We directly test this trade-off by sweeping λs∈{0,0.25,0.5,1,2}\lambda_s\in\{0,0.25,0.5,1,2\} with all other settings fixed:

Shared coefficient λs\lambda_sRelative Recall@20 vs. submittedRelative NDCG@20 vs. submitted
0-1.83%+0.12%
0.25-0.80%-1.65%
0.5-0.80%-1.35%
1.0 (default)0.00%0.00%
2.0-10.20%-10.84%

Removing the shared path reduces Recall by 1.83%1.83\%, confirming the value of the connectivity-preserving residual. Conversely, making the shared graph dominant (2.02.0) reduces Recall by 10.20%10.20\%, showing that routed structural evidence must remain influential. The slight NDCG increase at λs=0\lambda_s=0 (+0.12%+0.12\%) comes with a clear Recall loss. The full curve therefore supports the intended balance between shared robustness and routed selectivity.

W3 / Q3: Computation and memory overhead

DAME's main extra cost is offline multi-expert graph propagation. The exactly equivalent fused implementation reduces its forward time to 1.841.84--1.86×1.86\times that of LightGCN, keeps peak training memory below 12 GB on all three datasets, and gives cached online scoring the same inner-product operator as LightGCN. All forward and training-iteration values are warmed microbenchmarks on the same device; a training iteration includes forward, backward, and parameter update, rather than a full epoch. The appendix analyzes naive full-batch execution as O((1+Nexp⁡)∣E∣d)O((1+N_{\exp})|E|d). Because gi,eg_{i,e} depends only on target node ii and mijem^e_{ij} only on edge (i,j)(i,j), selected masks can be fused before propagation, reducing each layer to O(K∣E∣+∣E∣d)O(K|E|+|E|d) and executing the dd-dimensional sparse aggregation only once.

Fusion is exact rather than approximate: compared with expert-by-expert execution, the maximum element-wise output error is below 3.73×10−93.73\times10^{-9} and the load-balancing-loss error is 00. Absolute entries below follow LightGCN / single-mask / DAME-fused; the final column reports DAME-naive -> fused forward/training speedup and memory reduction:

DatasetForward (ms)Training iteration (ms)Peak memory (MB)FP32 mask (MB)Fusion gain F/T/M
Yelp12.06 / 12.24 / 22.4631.34 / 33.96 / 60.523323 / 3357 / 46580 / 6.6 / 218.77.62x / 9.88x / 49.1%
Gowalla9.14 / 9.24 / 16.8824.56 / 27.48 / 50.683070 / 3120 / 49430 / 9.7 / 319.98.55x / 9.22x / 33.1%
Amazon18.50 / 18.63 / 34.0243.59 / 49.23 / 102.904605 / 4765 / 118400 / 32.0 / 1054.25.24x / 6.68x / 27.3%

The fused forward is 1.841.84--1.86×1.86\times LightGCN rather than the 9.69.6--15.8×15.8\times overhead of the naive implementation. In our response to Reviewer NJMv (Q2), we further compare SGL-ED/SimGCL-style matched computation paths on a common backbone: fused DAME has 1.141.14--1.36×1.36\times and 1.341.34--1.66×1.66\times lower training-iteration time, respectively. Along the scaling axis, our response to Reviewer iA5w (W5) shows that when Amazon interactions increase by 4×4\times, forward and training-iteration time grow by only 1.42×1.42\times and 1.50×1.50\times.

W4 / Q4: Why use RoPE rather than simpler depth encodings?

RoPE jointly provides two properties that the simpler alternatives do not. First, it exactly preserves within-layer node geometry. Since RlR_l is orthogonal, for any nodes u,vu,v,

\qquad \cos(R_lh_u,R_lh_v)=\cos(h_u,h_v).$$ Thus, depth encoding neither erases nor fabricates within-layer node differences. Second, RoPE encodes relative propagation depth directly inside the representation: $$\langle R_lh^{(l)},R_mh^{(m)}\rangle =(h^{(l)})^TR_{m-l}h^{(m)}.$$ Scalar layer weights only rescale each hop's overall contribution during pooling; they do not encode relative depth in feature interactions. Additive depth embeddings introduce extra parameters and translation bias, changing norms and cosine geometry, while concatenation increases the output dimension. RoPE is parameter-free, dimension-preserving, isometric, and explicitly relative. We compare four controlled variants with all other settings fixed: |Depth treatment|Relative Recall@20 vs. no encoding|Relative NDCG@20 vs. no encoding| |---|---:|---:| |No depth encoding|0.00%|0.00%| |Learnable scalar layer weights|-6.40%|-7.31%| |Additive depth embeddings|-0.56%|-0.68%| |**RoPE**|**+0.44%**|**+0.37%**| RoPE is best on both metrics: it outperforms no encoding and additive embeddings, while scalar layer reweighting is substantially weaker. The maximum measured pre-/post-rotation error in pairwise cosine, Euclidean distance, and norm is only $1.2$--$4.8\times10^{-7}$; layer-4 within-layer user/item cosine is $0.032/0.025$, far from the collapse value 1. We therefore state the conclusion precisely: RoPE exactly preserves within-layer geometry and makes cross-layer interactions depend on relative depth. Controlled ablations support its value for depth-aware pooling, but isometry alone does not prove that depth signals can never alias. Empirically, from 2 to 8 layers DAME's Recall decreases by only $0.56\%$, compared with $1.28\%$ for pure GCN.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.