- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies pseudo-label reliability in semi-supervised multimodal learning. The authors observe that samples with similar fusion confidence can have substantially different correctness depending on whether modality-specific heads support the fused prediction. Based on this observation, they propose Adaptive Three-Way Calibration (ATC), which partitions unlabeled samples into Accept, Boundary, and Reject regions according to modality-head support. These regions control pseudo-label admission, weighting, and target construction. The method further estimates region-level reliability by replaying labeled training examples and uses the resulting targets for pseudo-label shaping and confidence alignment. Experiments are conducted on five multimodal benchmarks with several SSL and multimodal baselines, together with calibration analyses and ablation studies.
Critical strengths
-
The paper identifies a meaningful limitation of fusion confidence in multimodal SSL: even at similar fusion confidence, modality-head support provides additional information about pseudo-label correctness.
-
The evaluation goes beyond final classification accuracy and includes calibration metrics, confidence-conditioned reliability analysis, regional diagnostics, and ablation studies.
Critical weaknesses
-
The necessity of the full three-way formulation is not convincingly established. In the CUB ablation, binary support partitions already recover nearly all of the classification gain, while Full ATC provides only a very small additional improvement in ACC/F1. Moreover, several gains over the strongest baseline are small, and the paper reports only three seeds without significance tests or confidence intervals.
-
I have concerns about the validity and reproducibility of the baseline comparisons. In the released supplementary code, the MixMatch, CoMatch, and SimMatch branches appear to omit important components of the original methods, and some of these branches use averaged modality-head predictions rather than the EMA fusion-head prediction described in the paper. If the main results in Table 1 were produced by these implementations, this would materially affect the interpretation of the empirical comparison.
Questions for authors
-
Were the MixMatch, CoMatch, and SimMatch results in Table 1 generated using the corresponding branches in the released supplementary code? If yes, please explain how these implementations preserve the key mechanisms of the original methods. If not, please provide the exact implementations and configurations used for the reported results.
-
Can the authors provide results comparing the three-way partition against simpler alternatives, such as binary support partitioning or a continuous support-ratio formulation, across multiple datasets? This would help establish whether the Boundary region is genuinely necessary.
-
Can the authors evaluate regional reliability using cross-fitting or a held-out calibration subset rather than replaying labeled examples that were already used for model training? I am concerned that the current replay procedure may yield optimistic in-sample reliability estimates.
-
How were the dataset-specific values of τg, wg, bg, λu, and λcal selected? Please report the search space, validation criterion, tuning budget, and whether comparable tuning resources were provided to the baselines.