Why Weak-to-Strong Consistency Regularization Enables FixMatch to Outperform Pseudo-Labeling: A Feature-Learning Perspective

FixMatch, a representative semi-supervised learning (SSL) method, has achieved strong performance and become a widely adopted baseline for subsequent SSL research. Its success is commonly attributed to the combination of pseudo-labeling and weak-to-strong consistency regularization. However, the interaction between these two components remains theoretically underexplored. To address this gap, we explain FixMatch’s advantage over Pseudo-Labeling in theory by connecting feature learning with pseudo-label selection during training. With limited labeled data, a model initially relies on features that fit the labeled samples but generalize poorly, producing conffdent yet incorrect pseudo-labels. Pseudo-Labeling trains on the same weakly augmented images used to select these targets, leaving small prediction errors that can limit further feature learning. Incorrect targets can also reinforce poorly generalizing features and delay the selection of additional correct samples. FixMatch instead trains on strongly augmented versions of the selected images. Strong augmentation can weaken the features supporting current predictions and help the model learn shared features that improve predictions across more samples. More samples can then enter training with correct pseudo-labels, providing supervision for further feature learning. Under the conditions of our theory, this feedback can give FixMatch better representations, higher pseudo-label precision, and lower classiffcation error than Pseudo-Labeling. However, weak-tostrong consistency regularization can also strengthen parameter updates that push the model toward predicting incorrect pseudo-labels. Experiments on CIFAR-10 and CIFAR-100 support key parts of this explanation: strong views increase learning signals toward both correct and incorrect pseudo-labels, FixMatch learns more discriminative features, and replacing the selected pseudo-labeled dataset changes subsequent feature learning and pseudo-label selection.