- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies modality competition in multimodal learning from a frequency-domain perspective. The authors observe that modality preference is not necessarily consistent across different frequency bands, which may be overlooked by existing methods that assess and regulate modalities as a whole. Based on this observation, they propose SpecGuide, which estimates modality preference separately for different frequency regions and uses it to guide spectral modulation. The method also introduces a complementarity loss to encourage different modalities to capture complementary frequency information.
The authors provide theoretical analysis to motivate frequency-wise regulation and evaluate the method on six multimodal benchmarks. The results show improvements over existing methods across all six datasets. Ablation studies examine the effects of the proposed components, and additional experiments investigate different fusion strategies, modality corruption, and missing modalities.
Critical strengths
- The observation that modality preference can vary or even reverse across frequency bands is interesting. Most existing methods treat modality competition as a global issue, while this paper shows that the situation can be quite different within the same representation. This gives a clear motivation for studying modality competition at a finer granularity.
- The experimental evaluation is fairly convincing. SpecGuide outperforms the compared methods on all six datasets, and the ablation results show how the individual components contribute to the final performance. The comparisons with different modulation strategies are particularly useful, as they suggest that the gains come from preference-guided regulation rather than simply applying frequency-domain processing.
Critical weaknesses
- The assumption behind Eq. (3) needs further justification. The decomposition of global discriminability assumes weak correlations between frequency bands. However, the spectral descriptors in Eq. (50) are normalized using the total energy across all bands, which introduces dependencies between them. Although the theoretical and implemented descriptors are not explicitly identical, the paper does not establish whether the approximation holds for the actual FMP features.
- The formulation of PGSS is somewhat restrictive. In Eqs. (12)–(13), the sigmoid produces modulation weights between 0 and 1, meaning that spectral amplitudes can only be suppressed rather than enhanced. It is unclear why this design is sufficient for balancing modality competition. The authors should explain this choice and compare it with modulation functions that allow both amplification and suppression.
- The motivation for frequency-wise modality competition is not fully supported. The analysis in Section 2 uses Fisher discriminability, while FMP estimates modality preference through class-conditioned spectral prototypes. Both measure class separability, but neither directly shows whether one modality suppresses the learning of another. The observed preference reversals could simply reflect differences in discriminative feature learning. Moreover, the fixed radial partition with thresholds of 0.25 and 0.55 is not sufficiently justified. Other partitioning strategies, such as directional or sector-based regions, may produce different preference patterns. These issues make it difficult to determine whether the proposed frequency-wise competition is a general phenomenon or partly a result of the chosen spectral representation.
- The contribution and adaptability of frequency-wise preference guidance remain unclear. In Table 5, PGSS alone improves accuracy from 62.90% to 75.27%, while adding FMP only increases it to 76.21%. The full model reaches 77.96%, but the authors do not include a PGSS + FCG variant to isolate the contribution of FMP. Furthermore, FMP uses sample-wise preferences derived from ground-truth labels during training, whereas inference relies on averaged preference statistics. This means the preference guidance cannot adapt to individual test samples, even when modality quality changes. Together, these concerns make it difficult to assess whether frequency-wise preference estimation provides a substantial benefit beyond spectral modulation itself.
Questions for authors
- Eq. (3) assumes weak cross-band correlations, but the normalized spectral descriptors in Eq. (50) depend on the energy of all frequency bands. Can the authors clarify the relationship between the theoretical descriptors and those used in FMP? Have the cross-band covariance assumptions been verified on the actual learned representations?
- In Eqs. (12)–(13), PGSS uses sigmoid-based weights that can only attenuate spectral amplitudes. Why is suppression-only modulation preferred when the goal is to mitigate modality competition? Would allowing both amplification and suppression change the results, particularly for weaker modalities?
- How do the authors distinguish frequency-wise modality competition from differences in class discriminability across frequency bands? The current preference estimates do not directly measure whether one modality suppresses another during optimization. Also, how sensitive are the observed preference reversals to the fixed radial partition in Eqs. (47)–(48)? Would directional or alternative frequency partitions produce similar observations?
- Could the authors provide the PGSS + FCG result without FMP to better isolate the contribution of frequency-wise preference estimation? Additionally, how much performance is lost when sample-wise training preferences are replaced by averaged preference statistics at inference? This comparison would help determine whether FMP provides useful adaptive guidance beyond the spectral shaping performed by PGSS.
Dear Reviewer oKqx,
Thank you again for your valuable comments and suggestions.
As the discussion period will close in approximately 48 hours, we would sincerely appreciate it if you could review our rebuttal and the additional experiments and clarifications provided in the revised manuscript.
We would be particularly grateful to know whether our responses have adequately addressed your concerns or if any issues remain that we could clarify before the discussion deadline.
We highly value your feedback and would welcome the opportunity for further discussion.
Thank you very much for your time and consideration.
Best regards,
The Authors