- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the challenges of few-shot Multimodal Sentiment Regression (MSR)—namely, continuous distribution shift under restricted training budgets (e.g., a 7% sampling ratio), the gap between pretrained vision-language representations and numerical outputs, and the limitations of point-wise losses—by proposing CoRe, a coarse-to-fine learning framework. CoRe integrates four core components: Continuous Distribution Sampling (CDS) to select a representative training subset using Kernel Density Estimation and Wasserstein-1 distance; a context-aware heterogeneous multimodal graph powered by R-GCN to capture cross-modal and adjacent temporal dependencies; a coarse-to-fine prediction module that guides InstructBLIP with graph features to compute continuous scores via soft expectations over semantic anchors; and a distance-aware supervision objective combining smoothed Wasserstein-1 loss with MAE. Extensive experiments on CMU-MOSI, CMU-MOSEI, CH-SIMS, and CH-SIMS v2 demonstrate that CoRe consistently outperforms existing multimodal regression models and adapted few-shot baselines.
Critical strengths
-
Systematic Formulation of Few-Shot MSR: While few-shot multimodal sentiment analysis has previously focused almost entirely on discrete classification or aspect-based sentiment tasks, this work provides a timely, well-motivated formalization of continuous multimodal sentiment regression under limited supervision.
-
Empirical Performance and Benchmark Coverage: The empirical evaluation is extensive, spanning four standard English and Chinese benchmarks (CMU-MOSI, CMU-MOSEI, CH-SIMS, CH-SIMS v2) across both continuous regression metrics (MAE, Corr) and classification metrics (Acc, F1). Remarkably, CoRe trained on 7% of the data remains highly competitive with, and in several cases surpasses, full-shot (100% of the data) baselines.
Critical weaknesses
- Task Boundary: Few-Shot Learning vs. Pool-Based Subset Selection The proposed framework deviates from the conventional few-shot learning paradigm—where a model is strictly limited to an isolated, small support set—because its Continuous Distribution Sampling (CDS) requires zero-shot inference over the entire unlabeled training pool to estimate distribution density via KDE. Although this process avoids ground-truth label leakage, it conceptually mirrors pool-based active subset selection under an annotation budget rather than standard few-shot learning, a distinction the authors should explicitly clarify in their problem formulation.
- Methodological Novelty and Component Lineage The methodological novelty is somewhat limited, as the core components are largely adapted from established techniques in related literature rather than introducing fundamentally new primitives. Specifically, mapping semantic anchors to continuous predictions via soft expectation resembles language-guided ordinal regression (e.g., OrdinalCLIP), the R-GCN with similarity-based context gating is standard in multimodal conversation modeling, and the discrete Wasserstein-1 loss is a known formulation in ordinal classification; thus, the contribution lies primarily in the task-specific synthesis and tailoring of these modules for MSR.
- Disparity in Backbone Capacity vs. Classical Baselines There is a substantial model capacity disparity between CoRe and the classical multimodal baselines, as CoRe employs massive instruction-tuned backbones (InstructBLIP with Flan-T5 at 4.0B and 12.3B scales), whereas baselines like TFN, MulT, and MISA rely on small encoders (100M–300M parameters, such as BERT or DeBERTa). While the inclusion of scaled baselines like VERO and MSE-Adapter helps contextualize results, a major portion of the reported performance advantages over standard multimodal architectures is inherently driven by the sheer scale of foundation-model pretraining.
- Marginal Empirical Impact of the Heterogeneous Graph The empirical gains provided by the context-aware heterogeneous graph appear disproportionately small relative to its architectural and computational complexity. Ablation experiments reveal that replacing the R-GCN with a simple MLP degrades the MAE by only 0.007 on CMU-MOSI and 0.009 on CH-SIMS v2, while omitting context altogether causes a negligible 0.003 MAE drop on CMU-MOSI. Given the asymptotic and empirical overhead associated with contextual neighbor retrieval and graph propagation, the necessity of this intricate component remains questionable.
Questions for authors
Q1. Clarification on the Few-Shot vs. Subset Selection Setup This question asks the authors to clarify whether the baseline models in Tables 1 and 2 were trained on the identical 7% subset selected by the Continuous Distribution Sampling (CDS) module or on standard random subsets. The reviewer points out that if baselines were trained on random samples, a substantial portion of CoRe's performance advantage might stem solely from the active subset selection strategy rather than the model's architectural design, necessitating confirmation of whether a controlled and fair data comparison was conducted.
Q2. Baseline Feature Parity and Backbone Disparity This query seeks clarification on the feature extraction pipeline, specifically asking whether classical baselines (e.g., TFN, MulT, MISA) were ever evaluated using the high-capacity representations from InstructBLIP and Flan-T5, or if they were strictly confined to smaller 100M–300M parameter encoders like DeBERTa-large and wav2vec-large. The reviewer emphasizes that without evaluating baselines on comparable feature backbones, it is difficult to disentangle CoRe’s genuine architectural innovations from the substantial pretraining advantage provided by multi-billion-parameter foundation models.
Q3. Justification for the Heterogeneous Graph Complexity The reviewer challenges the necessity of the context-aware heterogeneous graph, asking how its substantial computational and memory overhead can be justified when ablation results show only marginal MAE drops (0.003 to 0.009) upon removing context or replacing the R-GCN with a simple MLP. To rationalize this architectural complexity, the authors are asked to provide evidence showing whether the graph module offers statistically significant performance improvements on specific challenging scenarios, such as ambiguous utterances or videos characterized by rapid emotional shifts.
Q4. Adapting Ordinal Regression Baselines Given the strong conceptual alignment between CoRe’s coarse-to-fine soft-expectation module and recent vision-language ordinal regression methods like OrdinalCLIP, this question asks whether the authors attempted to benchmark against or adapt any modern ordinal regression frameworks. The reviewer notes that directly comparing CoRe against models utilizing similar architectural primitives is essential to isolate and validate its specific methodological contributions to the multimodal sentiment regression task.
5 comments
Weakness 2:
“The methodological novelty is somewhat limited, as CoRe primarily integrates and adapts established techniques, including semantic anchor-based soft expectation, R-GCN-based context modeling, and Wasserstein-1 loss.”
Question 4:
“Have the authors attempted to benchmark against or adapt any modern ordinal regression frameworks, such as OrdinalCLIP?”
Response:
We would like to clarify that the core methodological contributions of CoRe lie in two main aspects:
(1) CDS Sampling. Existing few-shot sentiment learning methods typically adopt K-shot sampling, whereas CoRe introduces CDS, a distribution-preserving sampling strategy tailored to continuous sentiment regression. K-shot sampling typically relies on predefined class boundaries and is therefore well suited to classification tasks but cannot be directly applied to continuous regression. Moreover, under extremely limited annotation budgets, random sampling may leave certain sentiment intensity ranges underrepresented or entirely uncovered. In contrast, CoRe proposes a distribution-preserving sampling mechanism that does not require ground-truth labels during sample selection, enabling limited labeled samples to better cover different sentiment polarities and intensity ranges. The methodological contribution lies in designing a training subset selection strategy specifically for continuous regression, rather than in the underlying mathematical tools for distribution estimation or distance measurement.
(2) Coarse-to-Fine Prediction Mechanism. Language-guided ordinal regression primarily focuses on modeling ordered relationships among different levels, whereas CoRe targets fine-grained continuous sentiment intensity prediction. Methods such as OrdinalCLIP mainly learn ordered language prototypes and perform ordinal prediction through contrastive vision–language matching. In contrast, CoRe leverages the pretrained model's generative sentiment semantics, guided by structured multimodal and contextual information, to transform semantic judgments across sentiment intensities into a probability distribution over semantic anchors. This distribution is then used to produce continuous predictions, while ground-truth continuous labels further supervise both the semantic distribution and the predicted values. The methodological novelty lies in reformulating continuous sentiment regression as structure-guided generative semantic distribution learning, rather than in the individual use of established techniques such as soft expectation, Wasserstein-1 loss, or R-GCN.
Furthermore, we provide direct comparisons with two representative language-guided ordinal regression methods, OrdinalCLIP and NumCLIP, in Appendix A.16. All methods use identical CDS-selected 7% training subsets, validation and test splits, and five random seeds. As shown in Table 1 (Reviewer-rVnM), CoRe (xl) consistently achieves lower MAE and higher Corr across all four datasets. Together with our ablation studies on coarse-to-fine prediction, graph modeling, and distance-aware supervision, these results further support the effectiveness of the proposed learning mechanism.
Table 1 (Reviewer-rVnM): Performance comparison with ordinal regression baselines.
| Method | CMU-MOSI | CMU-MOSEI | CH-SIMS | CH-SIMS v2 | ||||
|---|---|---|---|---|---|---|---|---|
| MAE↓ | Corr↑ | MAE↓ | Corr↑ | MAE↓ | Corr↑ | MAE↓ | Corr↑ | |
| OrdinalCLIP | 1.447 | 0.155 | 0.813 | 0.286 | 0.560 | 0.252 | 0.499 | 0.360 |
| NumCLIP | 1.374 | 0.383 | 0.778 | 0.424 | 0.565 | 0.348 | 0.460 | 0.466 |
| CoRe (xl) | 0.619 | 0.855 | 0.560 | 0.757 | 0.375 | 0.695 | 0.314 | 0.760 |
Weakness 3:
“CoRe's performance gains over classical multimodal baselines may largely stem from its substantially larger pretrained backbones rather than the proposed method.”
Question 2:
“Were classical baselines evaluated using InstructBLIP/Flan-T5 representations, or only smaller encoders? Without comparable feature backbones, it is difficult to isolate CoRe's architectural contributions from pretraining advantages.”
Response:
We thank the reviewer for this insightful comment. In our original experiments, classical multimodal baselines use their original feature extraction pipelines rather than InstructBLIP/Flan-T5 representations, preserving their standard settings and avoiding potential effects of feature modifications. We acknowledge the backbone capacity disparity and have included comparisons with large-scale baselines, including MSE-Adapter (6B) and VERO (7B/13B), as well as zero-shot and few-shot regression-head baselines using the same InstructBLIP backbone. Moreover, Appendix A.9 compares CoRe using only 7% of the training data with fully supervised baselines using 100%, demonstrating its effectiveness under limited supervision.
To further assess the impact of pretrained representations, we have added comparisons using InstructBLIP features for TFN, MulT, and TMDC in Appendix A.15. Specifically, we replace their original input features with InstructBLIP representations while retaining their core architectures and training procedures. Results on CMU-MOSI, CH-SIMS, and CH-SIMS v2 are presented in Table 2 (Reviewer-rVnM).
The results show that InstructBLIP features yield modest improvements for some baselines but limited or negative effects for others. Nevertheless, CoRe consistently achieves lower MAE and higher Corr than these baselines, including their InstructBLIP-feature variants. Together with the large-scale and same-backbone comparisons, these findings suggest that CoRe's performance advantages cannot be explained solely by pretrained model capacity.
Table 2 (Reviewer-rVnM): Performance comparison using original and InstructBLIP features.
| Dataset | Method | Original MAE↓ | Original Corr↑ | InstructBLIP MAE↓ | InstructBLIP Corr↑ |
|---|---|---|---|---|---|
| CMU-MOSI | TFN | 1.145 | 0.560 | 1.104 | 0.599 |
| MulT | 1.313 | 0.439 | 1.246 | 0.425 | |
| TMDC | 0.947 | 0.693 | 0.958 | 0.678 | |
| CoRe (xl) | 0.619 | 0.855 | — | — | |
| CH-SIMS | TFN | 0.535 | 0.358 | 0.524 | 0.413 |
| MulT | 0.701 | 0.004 | 0.711 | 0.014 | |
| TMDC | 0.442 | 0.622 | 0.448 | 0.612 | |
| CoRe (xl) | 0.375 | 0.695 | — | — | |
| CH-SIMS v2 | TFN | 0.443 | 0.402 | 0.433 | 0.477 |
| MulT | 0.375 | 0.594 | 0.371 | 0.597 | |
| TMDC | 0.375 | 0.683 | 0.394 | 0.684 | |
| CoRe (xl) | 0.314 | 0.760 | — | — |
Thank you very much for your insightful and constructive feedback on our submission. The following are our detailed responses.
Weakness 1:
Response:
The core of few-shot learning lies in the limited availability of labeled data, rather than the size of the entire data pool. Conventional active learning typically selects samples based on their estimated value under a limited annotation budget and often involves iterative cycles of sample selection, annotation, and model updating. In contrast, CoRe does not employ such an iterative selection mechanism. Instead, CDS performs one-time selection before training to identify distributionally representative samples, after which only the selected 7% labeled samples are used for supervised training. The remaining unlabeled samples are used exclusively for zero-shot distribution estimation and subset selection, without participating in supervised training.
Therefore, CoRe remains a few-shot multimodal sentiment regression framework under a limited annotation budget, while CDS introduces a pool-based, distribution-aware subset selection mechanism. We have clarified this distinction in the problem formulation in Section 3.1 (Problem Formulation).
Question 1:
Response:
We thank the reviewer for this insightful comment. As stated at the end of the Baselines paragraph in Section 4 of the main text, "All trainable baselines use the same few-shot training subsets and the same validation and test splits." Specifically, all trainable baselines in Tables 1 and 2 use the same CDS-selected 7% training subsets as CoRe, ensuring fair comparisons.