acceptodds
Under review as a conference paper at ICLR 2027

CoRe: Coarse-to-Fine Learning for Few-Shot Multimodal Sentiment Regression

Abstract

Multimodal sentiment regression (MSR) integrates complementary information across modalities to model sentiment polarity and intensity in a continuous sentiment space. However, existing MSR methods typically rely on large-scale annotated datasets, which are costly and difficult to obtain. While few-shot MSR offers a promising solution, it still faces three key challenges: (i) limited training sets may fail to preserve the underlying continuous sentiment distribution; (ii) pretrained models are typically better at semantic understanding than at numerical regression; and (iii) conventional regression losses focus on point-wise errors, overlooking distributional discrepancies. To address these challenges, we make the first attempt to investigate few-shot MSR and propose CoRe, a coarse-to-fine learning framework. Specifically, CoRe introduces a continuous distribution sampling strategy to construct a representative few-shot training set that closely approximates the global sentiment distribution. It further reformulates sentiment regression as a coarse-to-fine prediction process, progressively refining pretrained affective semantics into continuous sentiment predictions, with a heterogeneous multimodal graph capturing cross-modal and contextual dependencies. Moreover, CoRe employs a distance-aware objective to align predicted sentiment distributions with soft targets, thereby reducing distributional discrepancies. Extensive experiments on four datasets demonstrate that CoRe outperforms state-of-the-art baselines, reducing MAE by 6.13% and improving Corr by 5.03% on average.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Learning, Multimodal Graph Modeling, Few-Shot Learning, Multimodal Sentiment Analysis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rVnMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenges of few-shot Multimodal Sentiment Regression (MSR)—namely, continuous distribution shift under restricted training budgets (e.g., a 7% sampling ratio), the gap between pretrained vision-language representations and numerical outputs, and the limitations of point-wise losses—by proposing CoRe, a coarse-to-fine learning framework. CoRe integrates four core components: Continuous Distribution Sampling (CDS) to select a representative training subset using Kernel Density Estimation and Wasserstein-1 distance; a context-aware heterogeneous multimodal graph powered by R-GCN to capture cross-modal and adjacent temporal dependencies; a coarse-to-fine prediction module that guides InstructBLIP with graph features to compute continuous scores via soft expectations over semantic anchors; and a distance-aware supervision objective combining smoothed Wasserstein-1 loss with MAE. Extensive experiments on CMU-MOSI, CMU-MOSEI, CH-SIMS, and CH-SIMS v2 demonstrate that CoRe consistently outperforms existing multimodal regression models and adapted few-shot baselines.

Critical strengths
  1. Systematic Formulation of Few-Shot MSR: While few-shot multimodal sentiment analysis has previously focused almost entirely on discrete classification or aspect-based sentiment tasks, this work provides a timely, well-motivated formalization of continuous multimodal sentiment regression under limited supervision.

  2. Empirical Performance and Benchmark Coverage: The empirical evaluation is extensive, spanning four standard English and Chinese benchmarks (CMU-MOSI, CMU-MOSEI, CH-SIMS, CH-SIMS v2) across both continuous regression metrics (MAE, Corr) and classification metrics (Acc, F1). Remarkably, CoRe trained on 7% of the data remains highly competitive with, and in several cases surpasses, full-shot (100% of the data) baselines.

Critical weaknesses
  1. Task Boundary: Few-Shot Learning vs. Pool-Based Subset Selection The proposed framework deviates from the conventional few-shot learning paradigm—where a model is strictly limited to an isolated, small support set—because its Continuous Distribution Sampling (CDS) requires zero-shot inference over the entire unlabeled training pool to estimate distribution density via KDE. Although this process avoids ground-truth label leakage, it conceptually mirrors pool-based active subset selection under an annotation budget rather than standard few-shot learning, a distinction the authors should explicitly clarify in their problem formulation.
  2. Methodological Novelty and Component Lineage The methodological novelty is somewhat limited, as the core components are largely adapted from established techniques in related literature rather than introducing fundamentally new primitives. Specifically, mapping semantic anchors to continuous predictions via soft expectation resembles language-guided ordinal regression (e.g., OrdinalCLIP), the R-GCN with similarity-based context gating is standard in multimodal conversation modeling, and the discrete Wasserstein-1 loss is a known formulation in ordinal classification; thus, the contribution lies primarily in the task-specific synthesis and tailoring of these modules for MSR.
  3. Disparity in Backbone Capacity vs. Classical Baselines There is a substantial model capacity disparity between CoRe and the classical multimodal baselines, as CoRe employs massive instruction-tuned backbones (InstructBLIP with Flan-T5 at 4.0B and 12.3B scales), whereas baselines like TFN, MulT, and MISA rely on small encoders (100M–300M parameters, such as BERT or DeBERTa). While the inclusion of scaled baselines like VERO and MSE-Adapter helps contextualize results, a major portion of the reported performance advantages over standard multimodal architectures is inherently driven by the sheer scale of foundation-model pretraining.
  4. Marginal Empirical Impact of the Heterogeneous Graph The empirical gains provided by the context-aware heterogeneous graph appear disproportionately small relative to its architectural and computational complexity. Ablation experiments reveal that replacing the R-GCN with a simple MLP degrades the MAE by only 0.007 on CMU-MOSI and 0.009 on CH-SIMS v2, while omitting context altogether causes a negligible 0.003 MAE drop on CMU-MOSI. Given the asymptotic and empirical overhead associated with contextual neighbor retrieval and graph propagation, the necessity of this intricate component remains questionable.
Questions for authors

Q1. Clarification on the Few-Shot vs. Subset Selection Setup This question asks the authors to clarify whether the baseline models in Tables 1 and 2 were trained on the identical 7% subset selected by the Continuous Distribution Sampling (CDS) module or on standard random subsets. The reviewer points out that if baselines were trained on random samples, a substantial portion of CoRe's performance advantage might stem solely from the active subset selection strategy rather than the model's architectural design, necessitating confirmation of whether a controlled and fair data comparison was conducted.

Q2. Baseline Feature Parity and Backbone Disparity This query seeks clarification on the feature extraction pipeline, specifically asking whether classical baselines (e.g., TFN, MulT, MISA) were ever evaluated using the high-capacity representations from InstructBLIP and Flan-T5, or if they were strictly confined to smaller 100M–300M parameter encoders like DeBERTa-large and wav2vec-large. The reviewer emphasizes that without evaluating baselines on comparable feature backbones, it is difficult to disentangle CoRe’s genuine architectural innovations from the substantial pretraining advantage provided by multi-billion-parameter foundation models.

Q3. Justification for the Heterogeneous Graph Complexity The reviewer challenges the necessity of the context-aware heterogeneous graph, asking how its substantial computational and memory overhead can be justified when ablation results show only marginal MAE drops (0.003 to 0.009) upon removing context or replacing the R-GCN with a simple MLP. To rationalize this architectural complexity, the authors are asked to provide evidence showing whether the graph module offers statistically significant performance improvements on specific challenging scenarios, such as ambiguous utterances or videos characterized by rapid emotional shifts.

Q4. Adapting Ordinal Regression Baselines Given the strong conceptual alignment between CoRe’s coarse-to-fine soft-expectation module and recent vision-language ordinal regression methods like OrdinalCLIP, this question asks whether the authors attempted to benchmark against or adapt any modern ordinal regression frameworks. The reviewer notes that directly comparing CoRe against models utilizing similar architectural primitives is essential to isolate and validate its specific methodological contributions to the multimodal sentiment regression task.

5 comments

Thank you very much for your insightful and constructive feedback on our submission. The following are our detailed responses.

Weakness 1:

“The proposed framework more closely resembles pool-based subset selection than conventional few-shot learning, and this distinction should be clarified in the problem formulation.”

Response:

The core of few-shot learning lies in the limited availability of labeled data, rather than the size of the entire data pool. Conventional active learning typically selects samples based on their estimated value under a limited annotation budget and often involves iterative cycles of sample selection, annotation, and model updating. In contrast, CoRe does not employ such an iterative selection mechanism. Instead, CDS performs one-time selection before training to identify distributionally representative samples, after which only the selected 7% labeled samples are used for supervised training. The remaining unlabeled samples are used exclusively for zero-shot distribution estimation and subset selection, without participating in supervised training.

Therefore, CoRe remains a few-shot multimodal sentiment regression framework under a limited annotation budget, while CDS introduces a pool-based, distribution-aware subset selection mechanism. We have clarified this distinction in the problem formulation in Section 3.1 (Problem Formulation).

Question 1:

“Were the baseline models in Tables 1 and 2 trained on the identical 7% subset selected by CDS or on standard random subsets?”

Response:

We thank the reviewer for this insightful comment. As stated at the end of the Baselines paragraph in Section 4 of the main text, "All trainable baselines use the same few-shot training subsets and the same validation and test splits." Specifically, all trainable baselines in Tables 1 and 2 use the same CDS-selected 7% training subsets as CoRe, ensuring fair comparisons.

Weakness 2:

“The methodological novelty is somewhat limited, as CoRe primarily integrates and adapts established techniques, including semantic anchor-based soft expectation, R-GCN-based context modeling, and Wasserstein-1 loss.”

Question 4:

“Have the authors attempted to benchmark against or adapt any modern ordinal regression frameworks, such as OrdinalCLIP?”

Response:

We would like to clarify that the core methodological contributions of CoRe lie in two main aspects:

(1) CDS Sampling. Existing few-shot sentiment learning methods typically adopt K-shot sampling, whereas CoRe introduces CDS, a distribution-preserving sampling strategy tailored to continuous sentiment regression. K-shot sampling typically relies on predefined class boundaries and is therefore well suited to classification tasks but cannot be directly applied to continuous regression. Moreover, under extremely limited annotation budgets, random sampling may leave certain sentiment intensity ranges underrepresented or entirely uncovered. In contrast, CoRe proposes a distribution-preserving sampling mechanism that does not require ground-truth labels during sample selection, enabling limited labeled samples to better cover different sentiment polarities and intensity ranges. The methodological contribution lies in designing a training subset selection strategy specifically for continuous regression, rather than in the underlying mathematical tools for distribution estimation or distance measurement.

(2) Coarse-to-Fine Prediction Mechanism. Language-guided ordinal regression primarily focuses on modeling ordered relationships among different levels, whereas CoRe targets fine-grained continuous sentiment intensity prediction. Methods such as OrdinalCLIP mainly learn ordered language prototypes and perform ordinal prediction through contrastive vision–language matching. In contrast, CoRe leverages the pretrained model's generative sentiment semantics, guided by structured multimodal and contextual information, to transform semantic judgments across sentiment intensities into a probability distribution over semantic anchors. This distribution is then used to produce continuous predictions, while ground-truth continuous labels further supervise both the semantic distribution and the predicted values. The methodological novelty lies in reformulating continuous sentiment regression as structure-guided generative semantic distribution learning, rather than in the individual use of established techniques such as soft expectation, Wasserstein-1 loss, or R-GCN.

Furthermore, we provide direct comparisons with two representative language-guided ordinal regression methods, OrdinalCLIP and NumCLIP, in Appendix A.16. All methods use identical CDS-selected 7% training subsets, validation and test splits, and five random seeds. As shown in Table 1 (Reviewer-rVnM), CoRe (xl) consistently achieves lower MAE and higher Corr across all four datasets. Together with our ablation studies on coarse-to-fine prediction, graph modeling, and distance-aware supervision, these results further support the effectiveness of the proposed learning mechanism.

Table 1 (Reviewer-rVnM): Performance comparison with ordinal regression baselines.

MethodCMU-MOSICMU-MOSEICH-SIMSCH-SIMS v2
MAE↓Corr↑MAE↓Corr↑MAE↓Corr↑MAE↓Corr↑
OrdinalCLIP1.4470.1550.8130.2860.5600.2520.4990.360
NumCLIP1.3740.3830.7780.4240.5650.3480.4600.466
CoRe (xl)0.6190.8550.5600.7570.3750.6950.3140.760

Weakness 3:

“CoRe's performance gains over classical multimodal baselines may largely stem from its substantially larger pretrained backbones rather than the proposed method.”

Question 2:

“Were classical baselines evaluated using InstructBLIP/Flan-T5 representations, or only smaller encoders? Without comparable feature backbones, it is difficult to isolate CoRe's architectural contributions from pretraining advantages.”

Response:

We thank the reviewer for this insightful comment. In our original experiments, classical multimodal baselines use their original feature extraction pipelines rather than InstructBLIP/Flan-T5 representations, preserving their standard settings and avoiding potential effects of feature modifications. We acknowledge the backbone capacity disparity and have included comparisons with large-scale baselines, including MSE-Adapter (6B) and VERO (7B/13B), as well as zero-shot and few-shot regression-head baselines using the same InstructBLIP backbone. Moreover, Appendix A.9 compares CoRe using only 7% of the training data with fully supervised baselines using 100%, demonstrating its effectiveness under limited supervision.

To further assess the impact of pretrained representations, we have added comparisons using InstructBLIP features for TFN, MulT, and TMDC in Appendix A.15. Specifically, we replace their original input features with InstructBLIP representations while retaining their core architectures and training procedures. Results on CMU-MOSI, CH-SIMS, and CH-SIMS v2 are presented in Table 2 (Reviewer-rVnM).

The results show that InstructBLIP features yield modest improvements for some baselines but limited or negative effects for others. Nevertheless, CoRe consistently achieves lower MAE and higher Corr than these baselines, including their InstructBLIP-feature variants. Together with the large-scale and same-backbone comparisons, these findings suggest that CoRe's performance advantages cannot be explained solely by pretrained model capacity.

Table 2 (Reviewer-rVnM): Performance comparison using original and InstructBLIP features.

DatasetMethodOriginal MAE↓Original Corr↑InstructBLIP MAE↓InstructBLIP Corr↑
CMU-MOSITFN1.1450.5601.1040.599
MulT1.3130.4391.2460.425
TMDC0.9470.6930.9580.678
CoRe (xl)0.6190.855——
CH-SIMSTFN0.5350.3580.5240.413
MulT0.7010.0040.7110.014
TMDC0.4420.6220.4480.612
CoRe (xl)0.3750.695——
CH-SIMS v2TFN0.4430.4020.4330.477
MulT0.3750.5940.3710.597
TMDC0.3750.6830.3940.684
CoRe (xl)0.3140.760——
Reviewer_fHBcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multimodal sentiment regression under limited labeled supervision. The proposed CoRe framework combines distribution-preserving subset selection, heterogeneous graph modeling, and a coarse-to-fine prediction mechanism. It uses zero-shot sentiment estimates to select a representative training subset, integrates multimodal and contextual information through a heterogeneous graph, and injects the resulting representation into a frozen pretrained multimodal model. Sentiment predictions are obtained by scoring textual descriptions associated with ordered numerical anchors and computing their probability-weighted expectation. Training combines MAE with a Wasserstein-1 loss against soft target distributions. Experiments show improvements over the evaluated baselines. Ablation studies support the coarse-to-fine formulation, soft expectation, distribution-aware supervision, and representative sampling.

Critical strengths
  1. The method proposed in this paper has a pretty complete and coherent pipeline. CDS is used to decide which training samples to select; heterogeneous multimodal graph modeling handles cross-modal and contextual dependencies; coarse-to-fine sentiment prediction addresses the shift from pretrained semantics to continuous regression; and distance-aware sentiment supervision tackles the issue that point-wise losses do not constrain the predicted distribution.

  2. The experimental design is fairly comprehensive. It covers four datasets, multiple strong baselines, and five random seeds, and also includes ablation studies, hyperparameter sensitivity analysis, efficiency analysis, comparisons with fully supervised methods, and distribution-preservation analysis. CoRe shows clear improvements on the main regression metrics.

Critical weaknesses
  1. CDS leans on zero-shot predictions as a stand-in for the global distribution. If those predictions have systematic bias, the sampling can drift away from the true label distribution. And Figure 3 compares against the true label distribution, so there’s a mismatch: the selection uses a proxy, while the evaluation uses ground truth.
  2. The context graph could also bring in transduction or leakage. Its context candidates are segments from the “same video.” If that pool includes unlabeled validation/test segments from the same video, or uses same-video information beyond the target sample, then this isn’t a strictly inductive few-shot setup.
  3. The ablation study provides only partial support for the proposed heterogeneous graph design. Table 3 includes useful comparisons against an MLP replacement, similarity-only relation weighting, and a target-only variant. However, these comparisons do not isolate the benefit of relation heterogeneity or the specific graph construction strategy. In particular, a homogeneous graph would directly test whether distinguishing cross-modal and contextual relations is necessary. These distinctions maybe matter because some reported improvements are small: on CMU-MOSI, the full model achieves an MAE of 0.619, compared with 0.626 for the MLP replacement and 0.622 without context.
Questions for authors
  1. During training, are contextual candidates drawn only from the selected few-shot subset, or can they include unselected examples from the training split? Please clarify context availability during validation and testing, and whether contextual inputs are available to the relevant baselines.
  2. It would be helpful for the authors to explain, from a theoretical perspective, how CDS sampling error affects the generalization bound, what sufficient conditions would make the W1 loss outperform MAE under finite anchors, and to quantify the soft-target expectation bias near the boundaries.
  3. Some baselines show fairly large standard deviations—so is CoRe’s advantage statistically significant?
6 comments

Thank you very much for your insightful and constructive feedback on our submission. The following are our detailed responses.

Weakness 1:

“(1) CDS leans on zero-shot predictions as a stand-in for the global distribution. If those predictions have systematic bias, the sampling can drift away from the true label distribution.

(2) And Figure 3 compares against the true label distribution, so there’s a mismatch: the selection uses a proxy, while the evaluation uses ground truth.”

Response:

(1) Sensitivity to Zero-Shot Prediction Biases. We have added a sensitivity analysis of CDS to errors and biases in zero-shot predictions in Appendix A.19. Specifically, we evaluate three perturbation types on CMU-MOSI: Gaussian Noise (σ ∈ {0.05, 0.10, 0.20, 0.30}), Shift Bias (δ ∈ {−0.20, −0.10, +0.10, +0.20}), and Intensity Compression (α ∈ {0.40, 0.60, 0.80}). Noise and shift magnitudes are scaled by the label range. Perturbations are applied only to the zero-shot predictions used by CDS, while downstream training settings remain unchanged.

As shown in Table 1 (Reviewer-fHBc), CDS maintains stable downstream performance under Gaussian noise, while systematic shifts cause only minor degradation. Stronger intensity compression has a relatively greater impact, but the overall performance remains close to the original setting. These results suggest that CDS is reasonably robust to errors and biases in zero-shot predictions within the tested perturbation ranges.

(2) Clarification of Figure 3. The ultimate goal of CDS is to select a limited training subset whose sentiment distribution approximates the true distribution of the full training set. Since ground-truth labels are unavailable during selection, zero-shot predictions are used to estimate a proxy distribution to guide sampling. Importantly, evaluating against the proxy distribution alone would only measure how well the selected subset matches the distribution used for selection, rather than whether it preserves the true sentiment distribution. Therefore, Figure 3 uses ground-truth labels solely for post-selection evaluation to independently assess the distributional representativeness of the selected subset.

Table 1 (Reviewer-fHBc): Sensitivity analysis of CDS to perturbations in zero-shot predictions on CMU-MOSI.

PerturbationStrengthMAE↓Corr↑
Original—0.6190.855
Gaussian Noise0.050.6140.855
0.100.6150.863
0.200.6240.860
0.300.6290.862
Shift Bias−0.100.6210.855
−0.200.6290.855
+0.100.6250.853
+0.200.6320.851
Intensity Compression0.800.6210.853
0.600.6260.855
0.400.6390.842

Weakness 2:

“The context graph could introduce transduction or data leakage if contextual candidates include validation/test segments or same-video information beyond the target sample, potentially violating the strictly inductive few-shot setting.”

Question 1:

“It is unclear whether contextual candidates are drawn only from the selected few-shot subset or also from unselected training examples, how context is accessed during validation/testing, and whether relevant baselines have access to the same contextual inputs.”

Response:

We would like to clarify that, as stated in Appendix A.11, CoRe restricts contextual candidates to their respective data splits. During training, only the selected few-shot training subset is used, excluding unselected training samples and any validation or test data. During validation and testing, contextual information is drawn exclusively from segments of the same video within the corresponding split, without accessing other splits or ground-truth sentiment labels. Therefore, CoRe does not involve transductive training or cross-split information leakage. Furthermore, contextual baselines such as MoMKE and TMDC also leverage information from segments of the same video under the same data-splitting protocol.

Weakness 3:

“The ablation study does not sufficiently isolate the benefits of relation heterogeneity or the specific graph construction strategy.”

Response:

To further evaluate relation heterogeneity and the graph construction strategy, we have added two ablation studies in Section 4.2 and Table 3 of the main text: (1) w/o Heterogeneity replaces relation-specific message passing with homogeneous message passing; (2) w/o Cross-Modal Relations removes cross-modal edges to assess the contribution of cross-modal interactions. The additional results are presented in Table 2 (Reviewer-fHBc).

As shown in Table 2 (Reviewer-fHBc), both ablation variants yield higher MAE than CoRe (xl), demonstrating the contributions of relation heterogeneity and cross-modal interactions. Together with the existing ablations on relation weighting and contextual modeling, these results provide more comprehensive evidence for the effectiveness of the proposed heterogeneous graph design.

Table 2 (Reviewer-fHBc): Ablation results of relation heterogeneity and cross-modal relations on CMU-MOSI and CH-SIMS v2.

MethodCMU-MOSICH-SIMS v2
MAE↓Corr↑Acc-2↑MAE↓Corr↑Acc-2↑
w/o Heterogeneity0.6330.85388.190.3370.73181.61
w/o Cross-Modal Relations0.6370.84988.070.3300.74181.74
CoRe (xl)0.6190.85588.570.3140.76083.09

Question 2:

“The theoretical analysis could be strengthened by clarifying how CDS sampling error affects generalization bounds, under what conditions W1 outperforms MAE with finite anchors, and how soft-target expectation bias behaves near the boundaries.”

Response:

We have added further theoretical analyses in Appendices A.10 and A.13 of the revised manuscript, as detailed below.

(1) Appendix A.10 analyzes how CDS sampling error affects the generalization risk bound. Specifically, the subset distribution-matching error W1(p,qS)W_1(\mathbf p,\mathbf q_S) contributes additively to the population risk bound with coefficient Mℓ/Δmin⁡M_\ell/\Delta_{\min}. Holding the other terms fixed, reducing this error tightens the bound.

(2) We clarify that we do not claim Wasserstein-1 loss universally outperforms MAE under finite anchors; rather, we emphasize their complementary supervision. Wasserstein-1 loss measures the discrepancy between the predicted distribution and the soft target, whereas MAE directly penalizes continuous prediction errors. Thus, reducing Wasserstein-1 loss does not necessarily reduce MAE. In Appendix A.13, we construct two anchor distributions with identical predicted means and hence identical MAE, but different Wasserstein-1 distances to the same soft target. This demonstrates that Wasserstein-1 loss captures distributional differences that MAE cannot distinguish, providing complementary distance-aware supervision.

(3) Appendix A.13 derives the soft-target expectation bias B(y)=∣μτ(y)−y∣B(y)=|\mu^\tau(y)-y| as a function of the distance to the boundary. When γ=Δ\gamma=\Delta, we obtain

\Delta\left| \frac{\sum_{j=0}^{C-1}(d-j)e^{-(j-d)^2/2}} {\sum_{j=0}^{C-1}e^{-(j-d)^2/2}} \right|$$ Here, $C$ is the number of anchors, $\Delta$ is the spacing between adjacent anchors, $d=(a_C-y)/\Delta$ is the normalized distance to the upper boundary, and $j$ is the summation index. With seven anchors, the bias is approximately $0.5201\Delta$ at the boundary ($d=0$) and decreases to $0.1294\Delta$ one anchor spacing away ($d=1$). By symmetry, the same absolute biases apply near the lower boundary.

Question 3:

“Some baselines show fairly large standard deviations—so is CoRe’s advantage statistically significant?”

Response:

To assess the statistical significance of CoRe's improvements, we have added a statistical significance analysis in Appendix A.14 of the revised manuscript. Specifically, we select eight representative baselines per dataset with competitive performance or relatively large standard deviations on CMU-MOSI and CH-SIMS. We conduct two-sided paired t-tests between CoRe (xl) and each selected baseline on MAE and Acc-2 across five matched random seeds. The results are presented in Tables 3 and 4 (Reviewer-fHBc).

As shown in Tables 3 and 4 (Reviewer-fHBc), CoRe (xl) achieves statistically significant improvements over all selected baselines on both datasets in the individual paired t-tests for MAE and Acc-2 (p < 0.05).

Table 3 (Reviewer-fHBc): Paired t-test results on CMU-MOSI (p-values).

MetricInstructBLIP (reg)VERO (7B)PVLMDLFKAN-MCPLMFMulTEF-LSTM
MAE1.95e-75.03e-49.28e-51.16e-62.05e-43.67e-41.01e-42.61e-4
Acc-21.93e-60.00702.10e-44.84e-40.00220.00730.00310.0208

Table 4 (Reviewer-fHBc): Paired t-test results on CH-SIMS (p-values).

MetricInstructBLIP (reg)VERO (7B)DLFMSE-Adapter (6B)LMFTETFNSelf-MMMulT
MAE2.37e-40.01200.00130.01117.33e-44.07e-62.12e-49.34e-4
Acc-20.02520.02400.01160.04940.02310.03180.01800.0065
Reviewer_dWhGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoRe, a coarse-to-fine framework for few-shot multimodal sentiment regression (MSR). It addresses three key challenges: distribution mismatch caused by limited training samples, the gap between pretrained models' semantic understanding and numerical regression ability, and the lack of distribution-level supervision in conventional regression losses. CoRe first selects a representative few-shot subset that preserves the global sentiment distribution and uses a heterogeneous multimodal graph to capture cross-modal and contextual dependencies. It then maps samples to ordered semantic anchors and converts the resulting semantic distribution into continuous predictions through soft expectation. Finally, a Wasserstein-based distance-aware loss aligns the predicted and target sentiment distributions. Experiments on four benchmarks show consistent improvements, reducing MAE by 6.13% and improving correlation by 5.03% on average.

Critical strengths
  1. Well-motivated problem formulation.

  2. Effective coarse-to-fine prediction strategy.

  3. Distribution-aware design for few-shot regression.

  4. Strong and comprehensive empirical results.

  5. Solid empirical validation.

Critical weaknesses
  1. The empirical evidence does not fully substantiate the claimed advantage in the few-shot regime. The main experiments use a fixed 7% training split, while the paper does not provide sufficiently systematic comparisons against strong baselines across different low-resource budgets. Consequently, it remains unclear whether CoRe is specifically effective for few-shot learning or simply provides a generally stronger regression framework. This is consequential because few-shot multimodal sentiment regression is the central problem setting and motivation of the paper.

  2. The methodological novelty is somewhat limited beyond the overall integration. Several key components, including KDE-based distribution estimation, Wasserstein supervision, semantic anchors with soft expectation, and heterogeneous graph modeling, build on established techniques. While their combination is coherent and empirically effective, the paper does not yet clearly establish which new technical principle, beyond the coarse-to-fine formulation, constitutes a substantial methodological advance. This limits the significance of the contribution for a venue such as ICLR.

Questions for authors
  1. Does CoRe consistently outperform strong baselines under more restrictive few-shot budgets (e.g., 1%, 3%, and 5%)?

  2. How sensitive is Continuous Distribution Sampling to errors or biases in the zero-shot predictions used to estimate the sentiment distribution?

  3. Given the relatively modest gains from contextual graph modeling, does this module provide statistically significant improvements across all datasets?

3 comments

Thank you very much for your insightful and constructive feedback on our submission. The following are our detailed responses.

Weakness 1:

“Insufficient systematic comparisons with strong baselines across different low-resource budgets to substantiate CoRe's advantages in the few-shot setting.”

Question 1:

“Does CoRe consistently outperform strong baselines under more restrictive few-shot budgets (e.g., 1%, 3%, and 5%)?”

Response:

We thank the reviewer for this insightful comment. To evaluate CoRe under more restrictive few-shot settings, we have added comparisons at 1%, 3%, and 5% training budgets across four datasets in Section 4.5 of the main text. We compare CoRe (xl) against six representative baselines, including Self-MM, MMIM, TMDC, PVLM, VERO (7B), and InstructBLIP (reg), using identical CDS-selected training subsets. As shown in Table 1 (Reviewer-dWhG), CoRe (xl) consistently achieves the lowest MAE and highest Corr across all four datasets and three training budgets, providing further empirical evidence of its effectiveness under more restrictive few-shot conditions.

Table 1 (Reviewer-dWhG): Performance comparison under different few-shot training budgets.

RatioMethodCMU-MOSICMU-MOSEICH-SIMSCH-SIMS v2
MAE↓Corr↑MAE↓Corr↑MAE↓Corr↑MAE↓Corr↑
1%Self-MM1.3400.1700.7360.4210.6430.0420.4270.388
MMIM1.3650.3590.7320.5140.5730.2130.4130.502
TMDC1.3770.2290.7790.4510.5240.5020.4220.600
PVLM1.4660.1260.7370.4300.5870.1070.5040.329
VERO (7B)1.1090.6960.6870.6110.6200.0960.4840.412
InstructBLIP (reg)0.9650.7380.6240.7010.5470.3820.4060.624
CoRe (xl)0.7280.8500.5630.7630.4170.6450.3490.700
3%Self-MM1.1570.4770.7090.5430.6320.0210.4030.515
MMIM1.0620.6480.6600.6320.6230.2040.4040.515
TMDC1.0630.6010.7160.5710.4770.5600.3980.634
PVLM1.0980.5920.6690.6230.5720.2010.4500.491
VERO (7B)0.7700.7910.6670.6510.5230.4230.4250.583
InstructBLIP (reg)0.8090.7860.6150.7230.4770.6470.3730.665
CoRe (xl)0.6590.8530.5610.7590.3870.6830.3210.744
5%Self-MM1.1990.4510.7470.3970.6470.0130.3860.590
MMIM0.9990.6790.7200.4750.5450.3550.4240.474
TMDC1.0010.6650.7150.6370.4700.6000.3950.664
PVLM1.0550.6260.6300.6680.5510.2870.4240.559
VERO (7B)0.7460.8020.6440.6800.5190.4190.4010.616
InstructBLIP (reg)0.7450.8020.6020.7260.4550.6400.3630.683
CoRe (xl)0.6270.8580.5570.7570.3820.6860.3170.759

Weakness 2:

“The methodological novelty is somewhat limited beyond the overall integration.”

Response:

We would like to clarify that the core methodological contributions of CoRe lie in two main aspects:

(1) CDS Sampling. Existing few-shot sentiment learning methods typically adopt K-shot sampling, whereas CoRe introduces CDS, a distribution-preserving sampling strategy tailored to continuous sentiment regression. K-shot sampling typically relies on predefined class boundaries and is therefore well suited to classification tasks but cannot be directly applied to continuous regression. Moreover, under extremely limited annotation budgets, random sampling may leave certain sentiment intensity ranges underrepresented or entirely uncovered. In contrast, CoRe proposes a distribution-preserving sampling mechanism that does not require ground-truth labels during sample selection, enabling limited labeled samples to better cover different sentiment polarities and intensity ranges. The methodological contribution lies in designing a training subset selection strategy specifically for continuous regression, rather than in the underlying mathematical tools for distribution estimation or distance measurement.

(2) Coarse-to-Fine Prediction Mechanism. Beyond conventional direct regression, CoRe leverages the pretrained model's generative sentiment semantics, guided by structured multimodal and contextual information, to transform semantic judgments across sentiment intensities into a probability distribution over semantic anchors. This distribution is then used to produce continuous predictions, while ground-truth continuous labels further supervise both the semantic distribution and the predicted values. The methodological novelty lies in reformulating continuous sentiment regression as structure-guided generative semantic distribution learning, rather than in the individual use of established techniques such as soft expectation, Wasserstein-1 loss, or R-GCN.

Question 2:

“How sensitive is Continuous Distribution Sampling to errors or biases in the zero-shot predictions used to estimate the sentiment distribution?”

Response:

We have added a sensitivity analysis of CDS to errors and biases in zero-shot predictions in Appendix A.19. Specifically, we evaluate three perturbation types on CMU-MOSI: Gaussian Noise (σ ∈ {0.05, 0.10, 0.20, 0.30}), Shift Bias (δ ∈ {−0.20, −0.10, +0.10, +0.20}), and Intensity Compression (α ∈ {0.40, 0.60, 0.80}). Noise and shift magnitudes are scaled by the label range. Perturbations are applied only to the zero-shot predictions used by CDS, while downstream training settings remain unchanged.

As shown in Table 2 (Reviewer-dWhG), CDS maintains stable downstream performance under Gaussian noise, while systematic shifts cause only minor degradation. Stronger intensity compression has a relatively greater impact, but the overall performance remains close to the original setting. These results suggest that CDS is reasonably robust to errors and biases in zero-shot predictions within the tested perturbation ranges.

Table 2 (Reviewer-dWhG): Sensitivity analysis of CDS to perturbations in zero-shot predictions on CMU-MOSI.

PerturbationStrengthMAE↓Corr↑
Original—0.6190.855
Gaussian Noise0.050.6140.855
0.100.6150.863
0.200.6240.860
0.300.6290.862
Shift Bias−0.100.6210.855
−0.200.6290.855
+0.100.6250.853
+0.200.6320.851
Intensity Compression0.800.6210.853
0.600.6260.855
0.400.6390.842

Question 3:

“Given the relatively modest gains from contextual graph modeling, does this module provide statistically significant improvements across all datasets?”

Response:

We have added a statistical significance analysis across four datasets in Appendix A.20, using two-sided paired t-tests over five matched random seeds. As shown in Table 3 (Reviewer-dWhG), CoRe (xl) achieves lower average MAE than both ablation variants across all four datasets, with statistically significant improvements over w/o Graph (MLP) on three datasets.

Importantly, w/o Context removes contextual information and its associated edges while retaining intra-utterance cross-modal graph modeling. Compared with this variant, CoRe (xl) achieves lower MAE on all four datasets, with statistically significant improvements on CMU-MOSEI at the 0.05 level and CH-SIMS v2 at the 0.1 level (p = 0.0539). Notably, as dataset size increases from CMU-MOSI and CH-SIMS to CH-SIMS v2 and CMU-MOSEI, the corresponding p-values consistently decrease (0.7382, 0.2245, 0.0539, and 0.0007), suggesting that contextual modeling may provide greater benefits with larger datasets.

These results support the effectiveness of graph-based modeling and demonstrate the additional contribution of contextual relations to multimodal sentiment regression.

Table 3 (Reviewer-dWhG): Statistical significance analysis of graph modeling on four datasets. MAE is reported as mean ± standard deviation over five runs. p-values are obtained from two-sided paired t-tests against CoRe (xl).

MethodCMU-MOSICMU-MOSEICH-SIMSCH-SIMS v2
MAE↓pMAE↓pMAE↓pMAE↓p
w/o Graph (MLP)0.626 ± 0.0110.30520.573 ± 0.0020.00040.388 ± 0.0140.02300.323 ± 0.0120.0177
w/o Context0.622 ± 0.0150.73820.571 ± 0.0060.00070.384 ± 0.0240.22450.320 ± 0.0050.0539
CoRe (xl)0.619 ± 0.008--0.560 ± 0.004--0.375 ± 0.014--0.314 ± 0.009--

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.