- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes HyDiff, a retrieval-augmented framework for incomplete multimodal learning that aims to recover missing modalities by leveraging relevant information from training instances. Motivated by recent retrieval-augmented approaches, HyDiff constructs semantic anchors from the available modalities to guide the retrieval of relevant information from a global memory bank, followed by diffusion-based missing-modality reconstruction. Although the core idea is potentially valuable and achieves promising performance on three MSA benchmarks, the paper contains numerous issues in methodological presentation, notation, experimental design, and result reporting, which substantially undermine the clarity, soundness and credibility of the work.
Critical strengths
To tackle the insufficient information caused by missing modalities, motivated by recent retrieval-augmented learning approaches, the paper leverages relevant instances from the training set to provide additional contextual information for missing modality reconstruction. Specifically, the proposed HyDiff first derives semantic candidates from the available modalities and constructs semantic anchors to guide retrieval from a global memory bank, which enables the model to access more relevant examples for subsequent modality reconstruction.
Critical weaknesses
1.The overall methodological narrative and presentation are weak, with poor consistency and insufficient standardization. More seriously, numerous variables in the methodology section are not briefly defined or accompanied by explanations of their roles, significantly reducing the overall readability and requiring considerable effort from readers to follow the proposed framework. To be more convincing, it is advisable to coherently introduce the motivation at the beginning of each module, such as highlighting the advantages of certain approaches, drawing inspiration from prior work, or explaining how traditional methods often fall short and thus motivating the proposed approach. Furthermore, I consider that it is necessary to clearly and concisely highlight several specific issues that undermine the clarity and rigor of the paper: 1)In Section 3.1, the modality set is defined as ‘M={v,a,txt}’, whereas the experimental section (e.g., Table 1) adopts the notation ‘[L,V,A]’. This inconsistency in modality notation should be clarified, especially since the meanings of these symbols are not explicitly introduced. Moreover, these modality symbols should be briefly explained when they are first introduced. Since HyDiff is mainly evaluated on MSA benchmarks, the lack of a focused discussion on missing-modality research in MSA in the Related Work section may implicitly assume that readers are familiar with this modality convention. 2)The frequently used variables J and \gamma lack clear and consistent definitions, which makes the methodology difficult to follow. Moreover, although Section 3.3 is described as target-specific, the variables in Equations (7)–(8) do not explicitly include the missing target modality index ‘m’. 3)Tthe experimental section directly refers to the ‘pre-rejection \mu=0.7’ setting, while the meaning of this setting and its sampling procedure are mainly explained in the Appendix. This makes the construction of the missing-modality protocol less transparent when reading the experimental setup. Moreover, the construction of the memory bank, which plays a central role in the retrieval component, should be briefly described in the main methodology. Specifically, the paper should clarify what the keys and values represent, how they are obtained from training samples, and whether the memory bank remains fixed or is updated during training and inference. 4)Although the paper compares against 11 representative baselines, more recent competitive approaches in 2026 are not included (such as TMDC: A Two-Stage Modality Denoising and Complementation Framework for Multimodal Sentiment Analysis with Missing and Noisy Modalities). More importantly, it should more clearly explain how the proposed HyDiff differs from the most relevant SOTA approaches. 5)Regarding the query construction in Section 3.3, the main text does not explain that the retrieval space is optimized using a contrastive alignment loss. Furthermore, this loss is mentioned in the appendix but omitted from the formulation of the overall objective function, leading to a discrepancy between the methodological description and the actual optimization process.
2.With respect to experiments, the reported results raise several credibility concerns summarized as follows: 1)Most notably, in Table 1 and several appendix tables, it’s pretty unusual and abnormal that most reported values have final digits of 0, 2, or 5 when averaged over five random seeds. 2)As reported in Table 1, under the full-modality setting, HyDiff with BERT features reaches an Acc-2 of 89.62. Even if Acc-2 is computed by excluding neutral samples (Note that Acc-2 is generally evaluated under two settings: Negative/Non-negative and Negative/Positive), it still seems unreasonable that this result approaches or even surpasses recent works using the stronger DeBERTa backbone. 3)Figure 3 suffers from insufficient resolution, while the information needed to interpret it is deferred to the Appendix. There is also an inconsistency in that Appendix D specifies a sweep over \mu in [0.1, 0.7], whereas figure 3 includes \mu=0. More notably, MissRAG, the strongest baseline in Table 1, is omitted from Figure 3, making it unclear whether HyDiff consistently outperforms the strongest competing method across different missingness levels.
3.This paper uses too much \vspace, and I am unsure whether this complies with the conference requirements.
Questions for authors
1.The total loss in Equation (13) appears to omit the L_{ret} term mentioned in the Appendix. Is this retrieval loss actually involved in the training process?
2.The paper states that the memory bank is constructed from complete training samples, but its construction and update mechanism remain unclear. How is the memory bank initialized, and is it updated during training?
3.The definition of \mu appears inconsistent between Table 1 and Fig. 3. Since Table 1 uses \mu as the pre-rejection modality-drop probability while Fig. 3 refers to it as the ‘missing rate’, how is \mu precisely defined and implemented throughout the experiments?
4.Are all 11 selected baselines strictly evaluated under the same missing-modality protocol as HyDiff and fairly reimplemented by the authors? Are the results reported in Table 1 averaged over multiple random seeds? More notably, why do most Acc-2 values reported in Table 1 and Tables E.3, E.4, and E.6 in the Appendix exhibit decimal endings of 0, 2, or 5?
5.In Table 1, HyDiff with a BERT text encoder achieves an Acc-2 score of 89.62 under the full-modality setting. Even if Acc-2 is computed by excluding neutral samples (Note that Acc-2 is generally evaluated under two settings: Negative/Non-negative and Negative/Positive), it still seems unreasonable that this result approaches or even surpasses recent works using the stronger DeBERTa backbone. Is this performance consistent with the employed text encoder and experimental setting?
6.Since HyDiff introduces a frozen Qwen2.5-Omni-7B model, memory bank construction, and multi-step diffusion sampling during the recovery stage, its overall computational pipeline appears substantially more complex than most baselines. How does the computational complexity and efficiency of HyDiff compare with existing methods, such as number of parameters, training and inference time, FLOPs, and peak GPU memory?