- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MarRec, a two-stage recommendation framework by semantic extraction. In the first stage, semantic embeddings are projected through a MoE module and optimized by the margin loss. In the second stage, the learned semantic representations are used to initialize trainable ID embeddings and trained by standard recommendation loss. The paper also introduces adaptive and soft margin variants and theoretically demonstrates that MarRec embodies the information bottleneck. Experiments on four datasets and five recommendation backbones demonstrate improvements over existing semantic fusion and alignment methods.
Critical strengths
- MarRec introduces a lightweight margin loss without requiring complex additional modules. The framework can be applied to different recommendation backbones. This simplicity makes MarRec useful for practical recommendation systems.
- The paper evaluates MarRec on four datasets and five recommendation backbones. The reported results support the practical effectiveness of the proposed framework.
Critical weaknesses
- The motivation and semantic extraction mechanism are insufficiently justified. First, the paper claims that Stage I compresses irrelevant semantic information. However, the margin loss operates on the output of a collaborative recommendation model, rather than directly constraining semantic information. Although gradients propagate to the MoE, there is no guarantee that the margin constraint selectively removes irrelevant semantics instead of merely limiting collaborative optimization. Second, the claimed limitations of fusion and alignment methods are oversimplified. For example, AlphaFuse (SIGIR 2025) preserves semantic information while incorporating collaborative signals, and alignment methods can also employ fine-grained constraints. Since MarRec similarly optimizes semantic representations using collaborative signals, its fundamental advantage over existing fusion and alignment approaches remains unclear.
- The information bottleneck analysis lacks sufficient theoretical justification. First, theorem 4.2 assumes that the margin constraint implies . However, the margin loss constrains ranking score differences, and no theoretical justification is provided for this mutual information bound. In fact, satisfying the margin does not generally imply such an upper bound. Second, even accepting this assumption, the derived bound also depends on and . Since the margin loss jointly optimizes the MoE and the collaborative model, both conditional entropy terms may change during training. The authors do not establish how these changes are controlled by margin constraint. Therefore, the derived bound does not guarantee semantic information compression.
Questions for authors
- How does the margin constraint guarantee the proposed semantic extraction rather than merely limiting collaborative optimization?
- What are the fundamental advantages of MarRec over existing controllable fusion and fine-grained alignment methods, such as AlphaFuse?
- How do the authors justify the bound and guarantee semantic compression when the remaining conditional entropy terms may change during joint optimization?
4 comments
Weakness 2 and Question 3
Building on Appendix A.1, we model the representations using Gaussian distributions, a common distributional assumption in representation modeling, and establish the mutual-information bound under explicit shared-covariance and moment conditions. To address the concern about joint optimization, we further state uniform covariance bounds that allow both conditional entropies to change while preserving the same information-capacity bound.
Let and fix . Consider a fixed population distribution of semantic inputs and targets, with finite relevant entropies. Suppose that, at each parameter state considered,
\qquad\Sigma_\theta\succ0, \qquad\operatorname{tr}(\Sigma_\theta^{-1}B_\theta)\leq2m,$$ where the covariance is shared across labels, the conditional means have finite second moments, and $B_\theta=\operatorname{Cov}_Y(\mu_{\theta,Y})$. The moment condition bounds between-label mean variation relative to within-label variation. Taking $Q_\theta=\mathcal N(\mathbb E_Y\mu_{\theta,Y},\Sigma_\theta)$, the KL decomposition gives $$\begin{aligned} I(H_\theta;Y) &=\mathbb E_Y D_{\mathrm{KL}}(P_{H_\theta\mid Y}\|Q_\theta) -D_{\mathrm{KL}}(P_{H_\theta}\|Q_\theta)\\ &\leq\tfrac12\operatorname{tr}(\Sigma_\theta^{-1}B_\theta)\leq m. \end{aligned}$$ Equation 5 gives $[m-\Delta]_+=m-\min(\Delta,m)$, so a triplet's direct main-loss gradient vanishes for $\Delta>m$. The moment restriction above is an explicit additional modeling condition. Hinge saturation alone does not enforce it. The mutual-information bound therefore follows under the stated distributional conditions, without assuming that a small hinge loss verifies those conditions. To control the conditional-entropy difference during joint optimization, additionally assume that $H_\theta\mid E^s=e$ is Gaussian with covariance $\Sigma_{\theta,e}$, and require $$\Sigma_\theta\preceq\sigma_y^2 I_d, \qquad\Sigma_{\theta,e}\succeq\sigma_e^2 I_d, \qquad 0<\sigma_e^2,\sigma_y^2<\infty,$$ for every parameter state considered and almost every $e$, with the same constants across parameter states. The Gaussian entropy bounds then yield $$\begin{aligned} I(E^s;H_\theta) &=I(H_\theta;Y)+h(H_\theta\mid Y)-h(H_\theta\mid E^s)\\ &\leq m+\frac d2\log\frac{\sigma_y^2}{\sigma_e^2}, \end{aligned}$$ where $h$ denotes differential entropy and logarithms are natural. Under the stated Gaussian assumptions, the covariance bounds place a fixed upper limit on $h(H_\theta\mid Y)$ and a fixed lower limit on $h(H_\theta\mid E^s)$. Both conditional entropies may change during optimization, but their difference remains bounded. This is why the same information bound holds across the parameter states satisfying these conditions.Request for Further Feedback
Dear Reviewer,
Thank you again for your time and feedback. We would be grateful if you could let us know whether our response has addressed your concerns and whether there are any remaining questions we could help clarify.
Thank you for the detailed response.
W1: The fundamental advantage of MarRec over AlphaFuse remains unclear, especially since the authors acknowledge that "the reported experiments do not establish empirical superiority over AlphaFuse."
W2: The additional theoretical analysis still does not demonstrate that margin-based training reduces semantic mutual information more effectively than BPR, nor does it establish that the retained semantic information is preference-relevant.
Therefore, I will maintain my original rating.
We are grateful to the reviewer for the detailed feedback and insightful questions.
Weakness 1 (semantic extraction) and Question 1
MarRec uses collaborative supervision to learn a preference-useful projection of frozen semantic inputs, then transfers that projection into standard recommendation training. The margin controls how this supervision updates the semantic mapping.
Let Z=gθ(Es) denote the MoE-projected semantic embeddings, and let Δt=s(u,i+)−s(u,i−) for triplet t. For the fixed-margin loss in Equation 5, ℓt=[m−Δt]+, the gradient away from the hinge boundary is
=-\mathbf1\{\Delta_t<m\} \left(\frac{\partial Z}{\partial\theta}\right)^\top\nabla_Z\Delta_t.$$ Thus, although the loss is evaluated at the recommender output, it directly regulates the preference supervision received by the semantic projector. Once $\Delta_t>m$, that triplet contributes no direct main-loss gradient, whereas BPR continues to reward a larger gap at every finite $\Delta_t$. This removes the incentive from already-satisfied triplets to keep adapting the shared semantic mapping merely to increase their score gaps. Other triplets and auxiliary losses can still update shared parameters. The margin does not impose an upper bound on the score gaps themselves. This mechanism is particularly clear with LightGCN: its graph propagation has no trainable transformation weights, and the original semantic embeddings are frozen in Stage I. In the fixed-margin variant, the MoE projector is therefore the only trainable representation mapping. The margin directly controls the learning of this semantic mapping, rather than acting only on a separate set of collaborative-model weights. Stage II then transfers the projected vectors into independently trainable ID embeddings and resumes standard recommendation training (Section 3.4). The existing controls examine both the training objective and the semantic inputs. w/o Margin (Section 5.2 and Appendix D.2) keeps the semantic inputs, projection architecture, and two-stage transfer procedure, while replacing the Stage-I main loss with BPR. It is among the variants with the largest performance degradation. Appendix C.2 further compares MarRec with generic ID warm-start and frozen random-input controls. The reported LightGCN Recall@20 results are: | Method | Beauty | Sports | Steam | Yelp | |---|---:|---:|---:|---:| | Base | 0.1551 | 0.0845 | 0.1357 | 0.1061 | | ID-Standard | 0.1544 | 0.0846 | 0.1359 | 0.1065 | | ID-Margin | 0.1594 | 0.0863 | 0.1367 | 0.1037 | | Rand-Margin-MoE-Frozen | 0.1506 | 0.0809 | 0.1342 | 0.1088 | | **MarRec** | **0.1829** | **0.1142** | **0.1463** | **0.1273** | Together, these controls support the contribution of margin-based adaptation under the same semantic-transfer pipeline and the value of semantic content beyond generic warm-start. The scope of the information-theoretic interpretation is addressed in our response to Weakness 2 and Question 3 below. # Weakness 1 (fusion and alignment) and Question 2 We agree that controllable fusion and fine-grained alignment can explicitly regulate semantic adaptation. Our discussion of potential information distortion concerns specific designs and settings. Both AlphaFuse and MarRec use collaborative supervision, but they differ in how adaptation is controlled and which embedding coordinates remain trainable. AlphaFuse uses the language-embedding spectrum to partition the input item embedding space, keeping the standardized semantic-rich coordinates fixed while learning collaborative ID updates in approximate null-space coordinates. MarRec instead learns a semantic projection from preference supervision, with the margin controlling each triplet's direct score-gap gradient reaching the projector during Stage I. The learned projections then initialize ordinary ID embeddings in Stage II, where all transferred coordinates remain trainable under the backbone's standard objective. This provides a task-adapted initialization followed by optimization of the entire transferred representation, without a prescribed frozen semantic subspace. Stage II and inference use the standard backbone without the semantic projector or an additional semantic-alignment objective. Fine-grained alignment can also regulate individual examples or relationships. MarRec's contribution lies in combining margin-controlled semantic adaptation with subsequent transfer into fully trainable ID embeddings. The comparisons with KAR, RLMRec, and PAD across five backbones, together with the MF results in Appendix C.4, support the effectiveness and architectural flexibility of this design. The comparison with AlphaFuse above concerns the mechanism. The reported experiments do not establish empirical superiority over AlphaFuse. We will add a citation to AlphaFuse and discuss these methodological differences in the Related Work of the revised manuscript.