- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The manuscript proposes HAMPL, a self-supervised vertical federated learning framework that uses shared trainable prototypes to transfer knowledge across participants while preserving participant-specific information under scarce labels and limited sample overlap. It combines unbalanced optimal transport, entropy-guided consensus fusion, and consensus-guided assignment refinement within alternating cross-participant and local training stages, reporting improved downstream performance over the evaluated baselines.
Critical strengths
This work proposes a new self-supervised VFL under scarce labels and limited sample overlap.
Critical weaknesses
An intensive comparison study with the recent SOTA model is not provided, limiting the evidence for the advantages of HAMPL over existing models. Moreover, it does not systematically evaluate sensitivity to prototype count, initialization, or geometric diversity, leaving the robustness of this design unclear.
Questions for authors
-
One recent VFL model, FALSE-VFL[1], is not considered as a baseline in the comparison analysis of this work. FALSE-VFL is relevant because it also combines participant-specific information through a shared latent structure while addressing scarce labels and limited alignment. Although the two methods communicate different quantities—latent-distribution parameters versus prototype assignments and updates—a direct comparison would clarify HAMPL’s advantages. Moreover, HAMPL’s stated training procedure does not exploit labels attached to unaligned samples, whereas FALSE-VFL can incorporate such supervision. [1] K. Hong et. al, Deep Latent Variable Model Based Vertical Federated Learning with Flexible Alignment and Labeling Scenarios, In International Conference on Learning Representation 2026.
-
The OT objectives in equations (2) and (5) include positive entropy regularization terms, and , which encourage softer assignments. In contrast, equation (3) assigns greater influence to participants with lower-entropy assignments through the confidence weight . Could the authors explain how these two effects interact and how the entropy regularization coefficients affect consensus weighting?
-
Please clarify the inference procedure, including whether the projector, prototype bank, or OT solver is used after fine-tuning. Does inference require features from every participant for each test sample? If some participants do not have that sample, how does HAMPL perform prediction?
-
Since the prototype bank is central to HAMPL, please evaluate sensitivity to its size and initialization, and characterize its geometry before and after training. Useful diagnostics include pairwise distances and effective rank. A comparison between fixed and trainable banks would also help isolate the benefit of learning the prototypes.
16 comments
Question 1 (Continued). Third, our benchmarks combine larger training populations and scarce labels with heterogeneous feature blocks, long-tailed distributions, and limited cross-party co-observations. These conditions can complicate latent-distribution modeling, missing-feature generation, and the association between learned latent variables and task labels. We interpret FALSE-VFL as emphasizing inference of a common latent description, while HAMPL aims to acquire shared knowledge and preserve distinct participant-specific information. This difference may contribute to HAMPL’s stronger performance in the evaluated settings.
In addition, FALSE-VFL’s original pre-training budgets are at least 100 epochs, whereas our standard comparison uses 40 epochs for all methods. The shorter budget may therefore contribute to its weaker results. We additionally extend FALSE-VFL-II to 100 epochs on NUS-WIDE, Avazu, and BHI, reported as FALSE-VFL-II* in Table 1. Although this improves its results on Avazu and BHI, the gap to HAMPL remains on all three datasets. Its sampling-based training also requires repeated exchanges of posterior parameters, latent samples, likelihoods, and gradients. Appendix B.2.2 and Table 9 show that even FALSE-VFL-I’s aligned-sample communication subtotal exceeds HAMPL’s complete pre-training communication payload For convenience, we reproduce the relevant part of Table 9 below.
**Table 9 (partial): Communication counts and costs per participant per pre-training epoch. **
| NUS-WIDE | Avazu | BHI | ModelNet | |||||
|---|---|---|---|---|---|---|---|---|
| Method | Times | Cost (MB) | Times | Cost (MB) | Times | Cost (MB) | Times | Cost (MB) |
| FALSE-VFL-I$^{\dagger}$ | 1,134.31 | 15,754.24 | 2,167.78 | 277.27 | ||||
| HAMPL | 237.50 | 819.59 | 113.12 | 59.24 |
Question 2. Entropy regularization and confidence weighting are compatible: the former regulates assignment construction, and the latter evaluates the resulting uncertainty when forming consensus. For cross UOT in Eq. (2), scales representation–prototype similarities through . A smaller amplifies similarity differences and typically produces sharper assignments, whereas a larger value promotes smoother assignments. Together, , the marginal regularization coefficients, and the current representation–prototype similarities affect the sharpness of local assignment matrices and thus their assignment confidence, which we measure using assignment entropy. Thus, although all participants use the same , their assignment entropies can differ because their actual matching results differ for each sample. We therefore use confidence weighting to evaluate assignment uncertainty when forming consensus, giving higher weights to more certain assignments to prevent the consensus from becoming unstable or having low confidence.
For consensus-guided OT in Eq. (5), expanding the KL term gives an effective entropy coefficient of . At fixed , increasing generally smooths the refined assignments. Since this step follows consensus fusion, it affects future weights indirectly through model and prototype updates, while the current weights are determined by the first-pass assignments.
Question 3. First, HAMPL follows the same forward procedure during inference as during downstream supervised fine-tuning. Each participant encodes its local features, and the active participant combines these representations and feeds them into the joint predictor. Following the representation-learning paradigm of methods such as SwAV [1], the projector, prototype bank, and OT solver are used only during pre-training to improve encoder representations and generalization. Downstream fine-tuning and inference use the local encoders and the task-specific joint predictor. We have clarified the relationship between fine-tuning and inference in the downstream fine-tuning description.
Second, conventional VFL methods, including supervised VFL and SSVFL, generally give limited consideration to joint inference with missing feature blocks [2]. Accordingly, HAMPL primarily targets collaborative inference in which all participants provide their respective features for each test sample. Its strong empirical performance under complete feature availability supports the effectiveness of its pre-training in improving the joint model’s generalization. During pre-training, HAMPL can also accommodate missing feature subsets by restricting collaboration to the subset of participants holding aligned observations. At inference time, if a participant lacks the features for a test sample, a simple fallback would be to use a zero vector in place of its representation before applying the joint predictor. This permits a prediction, but accuracy may deteriorate substantially. Possible adaptations include masking participant representations during supervised fine-tuning to improve robustness to missing feature blocks, or estimating missing representations using an approach inspired by FedCVT [3]. Exploring how the missing-data modeling in FALSE-VFL can complement HAMPL to improve prediction under incomplete observations is an interesting direction for future research, but it is beyond the scope of this paper.
Last but not least, our privacy analysis in Appendix B.1 indicates that label and attribute inference attacks based on a participant’s local model depend strongly on the predictive information acquired by that model during collaboration. When an adversary’s local model absorbs semantic information from other participants, its representations may become more similar to theirs, increasing its ability to infer their private information. HAMPL mediates knowledge exchange through multiple meta-prototypes while preserving participant-specific information. By avoiding rigid alignment to averaged representations, it can reduce encoder coupling and limit the acquisition of other participants’ private predictive information. The label-inference experiments in Appendix D.2 and Table 10 show that, under the evaluated protocol, collaboration adds less label-predictive information to a single pretrained HAMPL encoder than to its DACLSC counterpart. These results are consistent with HAMPL’s cooperative inference design, in which each participant contributes complementary information that may be difficult to recover from the other participants alone. Accordingly, HAMPL’s joint inference relies on the distinct information contributed by each participant.
[1] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems, volume 33, pp. 9912–9924, 2020.
[2] Yang Liu, Yan Kang, Tianyuan Zou, Yanhong Pu, Yuanqin He, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang, and Qiang Yang. Vertical federated learning: Concepts, advances, and challenges. IEEE Transactions on Knowledge and Data Engineering, 36(7):3615–3634, 2024. doi: 10.1109/TKDE.2024.3352628.
[3] Yan Kang, Yang Liu, and Xinle Liang. FedCVT: Semi-supervised vertical federated learning with cross-view training. ACM Transactions on Intelligent Systems and Technology, 13(4):1–16, 2022.
We sincerely thank the reviewer for their careful evaluation and constructive feedback. We have thoroughly revised the manuscript and submitted the updated version. Below, we address the reviewer's concerns and questions point by point.
Weakness 1. We thank the reviewer for this valuable comment. We have added an comparison with recent SOTA methods, as detailed in our response to Question 1. Furthermore, we have systematically evaluated the sensitivity of HAMPL to prototype count, initialization, and geometric diversity to further validate its robustness, as detailed in our response to Question 4.
Question 1. Thank you for highlighting this relevant work. We have carefully revisited FALSE-VFL and studied its mechanisms, and have added FALSE-VFL-I and FALSE-VFL-II to the comparison. Firstly, when feature-space expansion is accompanied by a change in the prediction target or label semantics, labels from the original task may become inapplicable. Therefore, in our experimental setting, we use labeled samples with complete feature blocks across all participants to supervise the joint prediction task. In Question 3 below, we discuss missing feature blocks in labeled samples during downstream supervised fine-tuning and acknowledge the corresponding limitations of our framework. Here, we focus on adapting FALSE-VFL to the datasets investigated in this work, using the encoder architectures and data partitions adopted in our homogeneous setting, while tuning its hyperparameters as extensively as possible. Revised Table 1 reports the resulting comparisons, showing that both FALSE-VFL variants achieve lower mean performance than HAMPL. For convenience, we reproduce the relevant part of Table 1 below.
Table 1 (partial): Performance evaluation of various methods.
Note: For each dataset, the best-performing value in each column is marked in bold. ∗ indicates that pre-training was extended to 100 epochs due to the larger dataset size.
We discussed possible explanations for this gap in Appendix D.8, which we summarize below together with the training-budget comparison.
First, the original missing-mask protocol in FALSE-VFL retains many examples observed by several participants. For its independent MCAR configurations, our calculation gives a single-party proportion of approximately 0.008%–9.5% within the randomly masked subset, conditional on at least one observed party. In our construction, 60% of all training examples are distributed into disjoint single-party subsets, which is common in two-participant setting. This provides fewer co-observations for estimating cross-participant dependencies and may make joint latent-distribution learning more challenging.
Second, FALSE-VFL aggregates posterior uncertainty through the inverse of summed local precisions. Holding local uncertainty estimates fixed, fewer observed parties provide less uncertainty reduction, and a single-party observation receives no benefit from combining multiple precision estimates. This may be particularly relevant to our two-party NUS-WIDE, Avazu, and BHI settings, where most training examples are observed by only one participant.