- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
HiCTE addresses incomplete multimodal learning by selectively transferring an observed modality's latent dimensions to different target experts. It constructs probabilistic latent representations, uses latent variance as a reliability proxy, and predicts target-conditioned, sample-dependent gates. During training, these gates undergo supervised local refinement, with gradients propagated through the updates to train the global estimator. Self-expert, cross-expert, and residual representations are adaptively combined for prediction. At inference, the method uses the global estimator without label-dependent local optimization. Experiments cover MOSI, MOSEI, and IEMOCAP under fixed modality absence and random missing-rate protocols, with an additional UR-FUNNY evaluation. The results support improvements from selective transfer; local refinement provides smaller additional gains.
Critical strengths
-
Selective cross-expert transfer has concrete empirical support. The method makes a meaningful distinction between weighting expert outputs and selecting source dimensions before they enter each target expert. Relative to Whole Transfer, fixed-missing average accuracy improves by 1.32, 0.51, and 1.10 percentage points on MOSI, MOSEI, and IEMOCAP. Relative to a target-agnostic shared selector, the corresponding gains are 0.93, 0.44, and 0.53 pp. These controls support the usefulness of selection and its target dependence, beyond comparisons of complete systems alone.
-
The evaluation separates deployable gains from training-time computation. The full system improves fixed-missing average accuracy over the strongest listed baseline by 1.42, 1.07, and 2.20 pp on the three main datasets. The additional UR-FUNNY comparison explicitly matches features, splits, and missing conditions and reports 63.50 accuracy versus 62.31 for MoMKE. The K=0 and stop-gradient controls help separate the contribution of training-time refinement and gradient propagation to the global estimator used at inference. The paper also reports the extra training cost of local refinement. This provides a useful and transparent assessment of practical behavior.
Critical weaknesses
-
The added value of the proposed transferability formulation beyond simpler target-conditioned gating remains only partially isolated. MoMKE already processes available inputs through different modality experts and combines their outputs using input-dependent routing. HiCTE's target-specific pre-expert dimension selection is a concrete extension, and the Whole Transfer and Shared Selector controls support selective transfer and target conditioning. The reliability ablations also provide useful component-level evidence. However, these comparisons do not directly establish the full formulation's advantage over a simpler target-conditioned dimension-wise selector trained through the downstream prediction objective, while retaining the same expert and fusion architecture. Such a control would help distinguish the benefit of learned feature selection from the additional value of probabilistic reliability modeling and transferability-specific objectives. K=0 does not provide this comparison: it removes local updates while retaining the reliability-aware formulation and initialization objective. On top of this attribution gap, the benefit attributable specifically to local refinement is 0.18–0.98 pp across the six dataset/protocol average-accuracy comparisons. The consistent positive direction is encouraging, but uncertainty of the paired differences across the three seeds is not reported. The reported refinement cost increases training time by 45.8%, without adding inference-time iteration. Overall, the experiments support a useful selective-transfer method more clearly than the added value of its particular transferability formulation.
-
The main evaluation and model-selection protocols are insufficiently specified. For the three main datasets, the manuscript should identify which baselines were rerun, their input features, and whether complete modalities were available during training. The matched UR-FUNNY protocol addresses that experiment but does not resolve the main-table questions. In addition, the IEMOCAP description holds out one session for testing and uses the other four for training without specifying validation selection. The data used for the hyperparameter choices in Figure 3 and the checkpoint-selection criterion are also unclear. These omissions limit reproducibility and confidence in the reported differences; they do not establish that test data were used for tuning.
Questions for authors
-
Clarification using existing runs: Could you report paired per-seed differences, with their variability, for the full model versus K=0, stop-gradient refinement, and the target-agnostic selector? The existing three seeds would be sufficient for this request. These results would clarify the magnitude and stability of the gains from target conditioning and differentiable refinement.
-
Clarification of evaluation protocol: Please specify the main baselines' features, training-time modality availability, and whether results were rerun or taken from prior reports. How are validation data defined for each IEMOCAP fold, which data and metric determine the Figure 3 hyperparameters and final checkpoints, and are checkpoints shared across missing conditions? A fixed-final-epoch procedure, if used, should simply be stated. The UR-FUNNY matching already described need not be repeated.