acceptodds
Withdrawn from ICLR 2027

Prototype-Guided Recalibration and Inter-Modal Signal Reshaping for Multi-Modal Sequential Recommendation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-modal sequential recommendation incorporates textual and visual features to alleviate interaction sparsity. However, while uncalibrated multi-modal integration may improve tail-item recommendations, it often degrades head-item performance by distorting their robust collaborative representations. To address this limitation, we propose Prototype-Guided Recalibration and Inter-Modal Signal Reshaping for Multi-Modal Sequential Recommendation (PRISM), which recalibrates multi-modal features before ID-based sequence modeling. Cross-Modal Semantic Reshaping separates shared semantics from modality-specific cues, aligning the former while using prototypes to preserve recurring patterns in the latter. Frequency-Gated Sequence Recalibration jointly analyzes textual and visual amplitude spectra to adaptively regulate each modality's sequence variations without uniformly smoothing informative dynamics. Cross-attention then integrates the recalibrated sequences with the ID-based sequence for prediction. Experiments on four datasets show that PRISM outperforms representative baselines and improves performance on both head and tail items.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Sequential Recommendation, Multi-Modal Recommendation, Feature Recalibration, Prototype Learning, Frequency Domain Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NjypOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the challenge of feature fusion in multimodal sequential recommendation. The authors argue that directly fusing textual, visual, and ID-based representations may improve recommendations for long-tail items, yet it could simultaneously compromise performance for popular items.

To address this, the authors propose the PRISM framework. It begins by decomposing multimodal features into shared semantic components and modality-specific components, which are then refined through distribution alignment and prototype-level optimal transport. Subsequently, the multimodal sequences are processed using FFT and a frequency-domain gating mechanism. Finally, these enhanced features are integrated with ID sequences via cross-attention to generate recommendations.

Experiments conducted on datasets such as Fitness, Game, and Dogs demonstrate that PRISM consistently outperforms the baseline methods. Furthermore, ablation studies show the effectiveness of the framework's core modules.

Critical strengths
  1. The authors analyze the performance disparity between popular items and long-tail items in multimodal sequential recommendation, and propose adjusting features at both the item and sequence levels. In particular, they employ distinct processing strategies for shared semantic information versus modality-specific semantic information.
  2. The experimental analysis in the paper is quite comprehensive. Beyond achieving strong results across multiple datasets, the authors further validate the effectiveness of their approach through ablation studies, performance comparisons across varying interaction frequencies, and an analysis of gained hits versus lost hits.
Critical weaknesses

1.The authors argue that their method can reduce the interference of multimodal information on recommendation performance. However, as shown in Equation (9) in the main text and Equations (20–23) in the appendix, this module primarily functions by adaptively enhancing different frequency components, rather than directly suppressing noise. Furthermore, Equations (21–23) indicate that the effective gain of the frequency-domain gating mechanism consistently falls between 1 and 2, implying that the original frequency components are never directly attenuated. Consequently, it remains unclear whether and how this enhancement mechanism effectively mitigates interference from multimodal information while simultaneously preserving recommendation performance for popular items. The experimental comparison of frequency domain modules is not sufficient. The ablation experiment is just replaced by ordinary MLP. However, there are already many sequence recommendation methods based on frequency domain modeling. It is recommended to compare with other frequency domain filtering or adaptive frequency modeling methods.

  1. The paper needs to clarify the novelty of its proposed modules. For instance, the separation of shared semantics from modality-specific semantics within the Cross-Modal Semantic Reshaping component, as well as the cross-modal alignment approach based on Gaussian prototypes and Optimal Transport, bears a strong resemblance to the methodology used in DecAlign. Since DecAlign already employs similar mechanisms for semantic disentanglement, prototype-based optimal transport, and distribution alignment, this paper must explicitly delineate its own distinct contributions relative to existing approaches.
Questions for authors

please refer to the weaknesses

Reviewer_dkPDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a multimodal sequential recommendation framework designed to exploit textual and visual item features while preserving reliable collaborative signals. The method consists of three main components: Cross-Modal Semantic Reshaping, which separates shared and modality-specific semantics and aligns them through distribution matching and prototype-level optimal transport; Frequency-Gated Sequence Recalibration, which uses joint cross-modal amplitude spectra to adjust modality-specific temporal representations; and Collaboration-Guided Cross-Attention, which integrates recalibrated multimodal features into ID-based sequential modeling. Experiments on three main datasets, with two additional datasets in the appendix, demonstrate improvements over representative baselines. The reported results suggest that PRISM can improve overall recommendation accuracy while balancing head-item preservation and tail-item recovery.

Critical strengths
  • The paper goes beyond aggregate recommendation accuracy by investigating how multimodal information affects items with different interaction frequencies. The hit preservation and recovery analyses help explain why naive multimodal integration can be harmful and provide a useful motivation for recalibrating auxiliary information before fusion.

  • PRISM consistently outperforms the evaluated baselines on the three main datasets, with a reported average relative improvement of 6.58%. The authors also provide ablation studies, hyperparameter sensitivity analyses, spectral-gate visualizations, and additional evaluations on two datasets in the appendix. These experiments offer preliminary evidence that the proposed components are useful in combination.

Critical weaknesses
  • The Cross-Modal Semantic Reshaping module largely builds upon established techniques, including representation disentanglement, distribution alignment, and prototype-level optimal transport, closely resembling DecAlign. Although PRISM introduces frequency-gated recalibration and collaboration-guided fusion, their methodological novelty remains unclear. Moreover, replacing the frequency module with an MLP does not isolate the benefits of cross-modal spectral conditioning or adaptive frequency gating. Comparisons with simpler spectral filters and modality-independent gates would better substantiate the claimed contributions.

  • All evaluated datasets originate from the Pixel benchmark, limiting evidence of cross-domain generalization. Furthermore, the main datasets have average sequence lengths of only 7–8 interactions, making the effectiveness of frequency-domain modeling on longer sequences unclear. The FFT-based representations are also sensitive to sequence padding, yet prediction consistency under different padding lengths or batch configurations is not evaluated. Finally, although statistical significance is reported, per-seed variability and detailed test statistics are missing, weakening the robustness claims.

Questions for authors
  • What is the precise methodological distinction between PRISM's Cross-Modal Semantic Reshaping module and DecAlign? Can the authors provide controlled comparisons that retain the same common-unique disentanglement and prototype alignment components while adding the proposed sequence-level mechanisms incrementally?

  • Can the authors isolate the contribution of cross-modal frequency gating by comparing joint-modality gates, independent modality-specific gates, fixed spectral filters, and generic adaptive fusion under the same backbone and training protocol? Such experiments would help determine whether the frequency-domain design itself is responsible for the improvements.

  • How sensitive are the predictions to the FFT length and batch composition? Can the authors evaluate the same interaction history under different padding lengths and inference batch configurations, and report the resulting prediction consistency and latency? Additional evaluations on an external benchmark with longer interaction histories would also strengthen the generalization claims.

  • Can the authors provide per-seed results, standard deviations or confidence intervals, and the relevant paired-test statistics for the main comparisons? This would help verify whether the reported improvements are robust given the use of five random seeds.

Reviewer_F1W7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PRISM, a multimodal sequential recommendation framework addressing the trade-off between improving sparse-item recall and preserving performance on frequently interacted items. PRISM first aligns collaborative, textual, and visual representations through Cross-Modal Semantic Reshaping, which separates shared and modality-specific information using distribution alignment and prototype-based transport. It then applies Frequency-Gated Sequence Recalibration to enhance modality-specific sequential signals through Fourier-based residual adjustments. Finally, ID-guided cross-attention integrates multimodal features for next-item prediction. Experiments across multiple datasets demonstrate improvements over ID-only and multimodal baselines, supported by ablations, frequency-based analyses, and robustness evaluations. However, whether PRISM genuinely prevents collaborative representation pollution, rather than benefiting from increased model capacity or alternative fusion mechanisms, remains insufficiently established.

Critical strengths
  1. A relevant and well-defined head–tail trade-off. Figures 1 and 3 use frequency-stratified Recall@10 to motivate a failure mode not visible from aggregate recommendation scores alone. This is useful because gains on sparsely observed items are less compelling if they systematically degrade frequently interacted items.

  2. A structured attempt to protect collaborative signals. The architecture keeps an explicit ID-based sequence as the query for cross-attention and uses collaborative item embeddings for scoring (Eqs. 11–12). This gives the proposed design a recognizable inductive bias, rather than treating all feature channels as interchangeable.

  3. Distinct item- and sequence-level treatments of multimodal noise. Prototype/OT alignment, common–unique decomposition, and residual frequency modulation target different representational scales (Sections 4.2–4.3). Whether every piece is necessary remains debatable, but this organization yields testable modules and a coherent engineering approach.

  4. Competitive and relatively careful ranking evaluation. Table 2 includes both standard ID-only and contemporary multimodal baselines, and states that multimodal systems use the same pretrained BERT/ViT features and a full-set ranking protocol. PRISM leads on the reported Recall/NDCG metrics for Fitness, Game, and Dogs; Appendix Table 6 also reports positive results on Song and Mix, which broadens the empirical setting.

Critical weaknesses
  1. The proposed representation-pollution mechanism is not directly identified. The Introduction describes uncalibrated modalities as distorting already reliable head-item collaborative embeddings, but Figures 1/3 and Table 3 primarily measure ranking outcomes. PRISM changes several things at once: graph-based ID representation learning, alignment losses, prototype regularization, spectral processing, and cross-attention. The observed head-item improvement therefore does not by itself establish preservation of the original collaborative geometry. Direct embedding-drift, nearest-neighbor stability, or head-item score-change analyses under identical backbones would be needed to substantiate the causal explanation.

  2. The spectral interpretation should be narrower than noise suppression. Equation 9 specifies X_flt = X_item + Re[IFFT(Z ⊙ G)], with sigmoid G in (0,1). As the appendix itself formalizes (Proposition 2), the effective Fourier multiplier is 1+G, lying between 1 and 2; the module does not attenuate the original spectral coefficients before subsequent fusion. Thus frequency-specific relative amplification is supported, but claims that it filters, removes, or suppresses volatile noise should be made cautiously. Figure 4 plots learned gates, not a measured separation between signal and noise. The paper needs task-level spectral interventions or a clearly operational definition of informative volatility.

  3. Matched-capacity alternatives are needed to establish a frequency/prototype contribution. Table 3 shows that the complete model beats w/o FG and w/o alignment variants, yet the w/o FG alternative is described as an MLP transformation rather than a parameter-, depth-, and receptive-field-matched replacement for the FFT branch. Similarly, the prototype mechanism should be compared with simpler learned centroids, contrastive alignment, and standard gated fusion. Without matched alternatives, the improvements may derive from added nonlinearity, normalization, parameters, or optimization regularization rather than the distinctive spectral/transport hypotheses.

  4. Item-popularity conclusions need more rigorous conditional reporting. Head and tail are defined by training frequency, and the principal aggregate results are reported on five-core-filtered category datasets with mean user sequence lengths around 7–8 (Table 1). Frequency-bin recalls can be sensitive to the distribution of held-out positives, user cohorts, and whether head-item gains are driven by exposure frequency rather than preservation. The reported paired t-tests and frequency plots would be easier to assess with per-bin denominators, confidence intervals across user/item groups and seeds, and absolute rather than only relative gains. This affects the central “improves both head and tail” claim.

Questions for authors
  1. Under the same ID/LightGCN backbone and parameter budget, does PRISM preserve head-item collaborative embeddings more faithfully than naive fusion, as measured by embedding drift, neighborhood overlap, and head-item rank inversions?

  2. How does Eq. 9 compare with an equally expressive residual MLP, a temporal convolution, a learned time-domain gate, and a learned spectral multiplier that can both attenuate and amplify frequencies? What evidence distinguishes meaningful spectral patterns from gate scaling?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.