Beyond Fusion Confidence: Adaptive Three-Way Calibration for Semi-Supervised Multimodal Learning

Semi-supervised multimodal learning often uses fusion confidence to select or weight pseudo-labels. Yet predictions with similar confidence can have different correctness depending on whether modality-specific heads support the predicted class. We propose Adaptive Three-Way Calibration (ATC), a teacher–student framework that distinguishes full, partial, and absent support as Accept, Boundary, and Reject regions. These regions condition pseudo-label admission, weighting, and target construction. Reliability estimated by replaying labeled training examples further controls target concentration and confidence alignment, separating the teacher's class proposal from its training-time reliability. Experiments on five heterogeneous benchmarks compare ATC with nine semi-supervised and multimodal baselines under a common feature and optimization protocol. ATC ranks first or second in 19 of 20 primary dataset–metric comparisons. Confidence-bin diagnostics reveal both regional correctness stratification and distinct sample-count trends. CUB ablations show that regional supervision accounts for most of the classification gain, while retaining Boundary yields lower mean ECE than either binary partition. Together, these results support using modality-head evidence to condition pseudo-label reliability rather than relying on fusion confidence alone.