acceptodds
Under review as a conference paper at ICLR 2027

MGSPRec: Multimodal Sequential Recommendation with Multi-Granularity Shared and Private Representation Learning

Abstract

Multimodal sequential recommendation uses visual and textual content to enrich the modeling of users' dynamic preferences. However, raw modality features may contain information that is irrelevant to recommendation. Meanwhile, different modalities have distinct representation spaces and semantic structures. Direct parameter sharing or forced alignment across modalities may therefore introduce cross modal interference and weaken valuable modality specific semantics. To address these issues, we propose MGSPRec, a multimodal sequential recommendation method based on multi granularity shared and private representation learning. MGSPRec first uses behavior guided signals to refine visual and textual representations. This process suppresses recommendation irrelevant noise while preserving modality specific semantics. It then employs separate Transformer encoders for ID, visual, and textual sequences to reduce interference across modalities. Furthermore, ID based collaborative information is used as a semantic anchor. Shared and modality specific representations are learned at both the item and sequence levels. Experiments on three real world datasets, Baby, Pantry, and Office, show that MGSPRec consistently outperforms representative baselines. Compared with SGP4SR, MGSPRec achieves average relative improvements of 4.80%, 3.69%, and 2.29% over HR and NDCG at different cutoff values on Baby, Pantry, and Office, respectively. Ablation studies and parameter sensitivity analyses further validate the effectiveness of the proposed components.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
multimodal sequential recommendation, behavior-guided modality representation refinement, independent modality modeling, multi-granularity representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_djFHOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MGSPRec, a multimodal sequential recommendation framework based on multi-granularity shared and private representation learning. Specifically, the model first introduces a behavior-guided modality representation optimization mechanism to refine multimodal features and construct modality relation graphs using collaborative signals. It then employs independent Transformer encoders to capture modality-specific sequential patterns and incorporates interest prototypes to enhance user preference representations.

Critical strengths
  1. The paper is well organized, and the overall framework is clearly presented with a reasonable motivation for addressing multimodal interference and information redundancy.

  2. The proposed method achieves consistent performance improvements over representative baselines across three datasets, demonstrating its potential effectiveness in multimodal sequential recommendation.

Critical weaknesses

1.The computation of the co-occurrence graph and its use in guiding modality relation construction are highly similar to the operations in SGP4SR. However, the methodology section does not clearly demonstrate the advantages of the proposed approach over SGP4SR, and the experimental section lacks targeted comparisons to validate these advantages.

2.The mathematical formulations are not sufficiently clear. For example, the computation of the individual loss terms in Equation (12) is not explicitly defined.

3.The ablation study is relatively coarse-grained. In particular, it lacks a variant that removes co-occurrence guidance while retaining the modality relation graphs, making it difficult to isolate the contribution of the proposed guidance mechanism.

4.The experimental evaluation lacks direct evidence demonstrating that the proposed model effectively addresses the three challenges identified in the motivation. Relying solely on improvements in overall recommendation performance is insufficient to substantiate these claims.

Questions for authors

I suggest adding more targeted experiments to directly validate the proposed motivations and dedicating more discussion to clarifying the differences between the proposed method and existing multimodal sequential recommendation approaches.

Reviewer_kCNeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MGSPRec for multimodal sequential recommendation. It combines behavior-guided feature and graph refinement, separate sequence encoders, and ID-anchored shared–private representation learning at item and sequence levels. Experiments on three Amazon datasets show improvements over the evaluated baselines.

Critical strengths
  1. The paper addresses a relevant problem: multimodal content can contain information unrelated to user preferences. Using behavioral evidence to guide content modeling is a reasonable approach. Preserving complementary modality information is also useful for recommendation.

  2. The framework is reasonably easy to follow. The main components are described through equations and an overall training algorithm. The separation between graph construction, sequential modeling, and auxiliary representation learning helps explain the pipeline.

  3. The model achieves the best reported results across the three evaluated datasets. The component ablations provide some evidence that the proposed modules contribute to performance. The sensitivity analyses also examine the balance between behavioral refinement and auxiliary regularization.

Critical weaknesses
  1. The overall methodological contribution appears incremental. Much of the pipeline is already present in SGP4SR, including co-occurrence-guided modality graphs, separated-modality modeling, clustered interest centers, Gumbel-Softmax selection, and multi-branch prediction. The paper does not clearly establish which changes constitute a substantial advance beyond this existing architecture.

  2. The positioning of shared–private learning overlooks directly relevant recommendation research. MGCN already combines behavior-guided purification with shared and modality-specific feature modeling, while DHMRec explicitly studies collaboration-guided multimodal disentanglement. MENTOR also uses ID-guided multilevel alignment, so the paper needs a much sharper comparison with these methods to justify its claimed research gap.

  3. The role of the shared–private decomposition is not convincingly established. The decomposed representations enter auxiliary losses, while prediction uses the original branch representations. Consequently, the reported gains could come from additional alignment and regularization rather than a meaningful separation of shared and private information.

  4. The ablations do not isolate the claimed new contributions sufficiently. Removing the entire shared–private module does not show whether both granularities are necessary or whether private representations add value beyond ID alignment. Likewise, replacing three independent encoders with a shared encoder changes model capacity, making the explanation based on reduced modality interference difficult to assess.

  5. The empirical gains provide limited support for the additional complexity. Average relative improvements over SGP4SR range from 2.29% to 4.80%, and evaluation is restricted to three Amazon categories. No variability estimates or significance tests are reported, and the manuscript provides no code link for independent verification.

Questions for authors
  1. Which architectural changes distinguish MGSPRec from SGP4SR, and how much improvement does each change provide?

  2. How does the proposed auxiliary objective compare with existing shared–specific decomposition and ID-guided alignment methods on the same sequential backbone?

  3. Does shared–private learning outperform a simpler ID-alignment objective without private projection heads?

  4. Can you report item-only, sequence-only, and joint regularization results, together with a parameter-matched comparison of shared and independent encoders?

  5. Can you provide results across multiple random seeds, significance tests, and the implementation with experiment configurations?

Reviewer_CRxEOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MGSPRec is a multimodal sequential recommender with three components: (1) behavior-guided modality refinement, using a distance-decayed item co-occurrence graph with reliability-weighted residual injection and behavior-enhanced modality relation graphs; (2) separate Transformer encoders for ID, visual, and textual sequences with interest prototypes per modality; (3) multi-granularity (item- and sequence-level) shared/private decomposition of visual and textual representations, anchored to ID collaborative signals with stop-gradient. On Amazon Baby, Pantry, and Office, MGSPRec outperforms seven baselines including SGP4SR, with average relative improvements of 4.80%, 3.69%, and 2.29% respectively.

Critical strengths
  1. The behavior-guided refinement module is the best-motivated part: normalizing behavioral evidence by neighbor count (Eq. 2) before residual injection limits unreliable correction for sparse items, and the ablation shows it contributing most on Pantry, consistent with the design intent.
  2. The collaborative anchoring loss (Eq. 11) with stop-gradient on the ID branch is a clean way to supervise the shared space with task-relevant signal without letting the auxiliary objective corrupt the anchor, and carrying the decomposition to both item and sequence granularity is executed consistently.
Critical weaknesses
  1. Experimental evidence does not support the claims’ strength: three small Amazon subsets, absolute improvements over SGP4SR of 0.001–0.005 at HR@10 (2.3–4.8% relative), no statistical tests, no repeated runs, and no statement of whether baseline numbers are reproduced or copied, or whether tuning budgets were matched.
  2. Novelty is combinatorial rather than conceptual: behavior signals refining modality graphs (SGP4SR), shared/private decomposition with orthogonality/variance/reconstruction regularizers (MISA, DecAlign), and per-modality encoders are all established; the paper does not isolate which combination yields non-additive benefit (no two-module ablations against the strongest baseline).
  3. Cost accounting is missing: three Transformer encoders plus dual-granularity shared/private projections roughly triple parameter count versus single-encoder baselines, yet no training/inference overhead comparison is given, weakening the practical case.
Questions for authors
  1. Please provide significance tests against SGP4SR (multiple seeds, paired test) and clarify the origin and tuning budget of baseline numbers.
  2. What are the parameter count, training time, and inference latency of MGSPRec versus baselines? How were γ and ρ (Eqs. 9, 15) set?
  3. Can you share dataset statistics (users/items/interactions, sequence lengths, feature extractors for visual/textual embeddings) in the main text, and results on at least one larger or non-Amazon dataset to support generality?
Reviewer_nxQvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MGSPRec, a multimodal sequential recommendation method that aims to reduce modality noise and interference across modalities. It uses user behavior information to refine visual and textual representations. Separate encoders and interest prototypes are then used to model user preferences. The method also learns shared and private representations at both the item and sequence levels, using collaborative information as an anchor. Finally, it combines predictions from different branches to make recommendations.

Critical strengths
  1. The paper establishes a clear connection between the identified problems and the proposed solutions. The designs directly address modality noise, interference across modalities, and the modeling of shared and complementary information.

  2. The overall design follows a clear direction, and the motivation for each main module is reasonable. The modules connect naturally, from modality representation refinement to user preference modeling and shared and private representation learning. Together, they form a coherent framework.

  3. The experiments provide good support for the main design choices and align well with the research motivation. Overall performance comparisons, ablation studies, and parameter sensitivity analyses complement each other. They provide experimental evidence from different perspectives.

Critical weaknesses
  1. The loss functions are not fully defined. Equation (12) does not provide the specific mathematical formulas for the orthogonality, variance, and reconstruction losses. This makes the method harder to understand.

  2. The role of shared and private representations in prediction is unclear. Based on Equations (14) to (16), the decomposed representations seem to be used only through the auxiliary loss during training. They do not appear to be directly used for prediction.

  3. The role of the stop gradient operation needs further explanation. All three branches share a trainable ID embedding matrix. The auxiliary loss may still update this matrix through the visual and textual branches.

  4. The update mechanism for interest prototypes is unclear. The paper states that the prototypes are initialized using K-means. However, it does not clarify whether or how they are updated during subsequent training.

  5. The ablation study for multi-granularity shared and private representation learning lacks detail. This module operates at both the item and sequence levels and includes several auxiliary losses. However, the experiments do not separately evaluate the contribution of each loss. This makes it difficult to identify the main source of performance improvements.

  6. The interest prototype mechanism lacks a separate ablation study. The paper does not independently evaluate its contribution to recommendation performance. An additional experiment would help clarify its actual effect.

Questions for authors
  1. Could the authors provide the specific mathematical formulas for the orthogonality, variance, and reconstruction losses in Equation (12)? Could they also explain how these constraints help separate shared and private information?

  2. According to Equations (14) to (16), do the decomposed shared and private representations affect the final predictions only indirectly through the auxiliary loss, rather than directly contributing to the prediction scores?

  3. Since all three branches share the same trainable ID embedding matrix, how does the stop gradient operation in Equation (11) prevent the auxiliary loss from affecting ID representations through the visual and textual branches?

  4. Could the authors provide further ablation studies to evaluate the individual contribution of each operation in the behavior guided modality representation optimization module?

  5. For shared and private representation learning, could the authors compare item level learning alone, sequence level learning alone, and joint learning at both levels?

  6. Could the authors separately evaluate the contribution of each auxiliary loss to recommendation performance?

  7. Could the authors clarify whether and how the interest prototypes are updated during subsequent training? Could they also provide an ablation study to evaluate their actual contribution?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.