acceptodds
Under review as a conference paper at ICLR 2027

PIGR-VFL: Partition-Induced-Gap Reduction Vertical Federated Learning

Abstract

Vertical Federated Learning (VFL) enables clients to collaboratively exploit partitioned feature blocks without sharing their raw features. However, when each client holds an increasingly fine-grained partitioned feature block, aggregating local representations obtained by local extraction models may cause information loss and degrade predictive performance. To address this issue, we first derive a worst-case bound on the population risk to identify the factors responsible for this VFL-specific partition-induced performance degradation. Decomposing this bound suggests that combining model architectural designs (coordinate-preserving embedding, participant-mean aggregation) with optional training objectives (extraction-curvature regularization, proximal extraction-output regularization) reduces the identified factors without unduly restricting the extraction model's expressiveness; we refer to the resulting method as Partition-Induced-Gap Reduction VFL (PIGR-VFL). Through numerical experiments, we confirm the effectiveness of PIGR-VFL, particularly on image classification tasks.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
vertical federated learning, LASER-VFL, split learning, partition-induced-gap

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6FbFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper study the performance degradation problem in VFL caused by increasingly feature-partitioning feature block. The author first derive the worst-case bound on the population risk, reduces the identified factors without unduly restricting the extraction model’s expressiveness. Experiments are conducted on CIFAR10, CIFAR100 and HAPT.

Critical strengths
  1. This paper identified an interesting issue in VFL that become relevant as feature partition become fine-grained.
  2. The proposed method is simple and yet yielding large improvement, particularly at large KK.
  3. I also appreciate the authors explicitly acknowledge that the population-risk bound is two-sided, and reducing the bound does not necessary improve the actual risk.
  4. The runtime analysis and aggregation ablation are useful.
Critical weaknesses
  1. The implementations appear inconsistent, i.e. main text state Adam but CIFAR appendix state AdamW.

  2. all experiments use full participation �0��0�. This substantially limits the empirical validation of participant-mean aggregation, since its claimed advantage is most relevant when the number of participating clients varies. Under fixed full participation, mean versus sum differs largely by a fixed scaling, and the paper's own ablation consequently finds little difference.

  3. the theoretical motivation for T3 appears incomplete. Proposition 1 uses the Jacobian-Lipschitz constant �1��1� of an inaccessible reference extractor �2��2�, whereas T3 penalizes finite-difference curvature of the local models �3��3�. I do not see an argument connecting these quantities. The experimental extractors additionally use ReLU, while Assumption 1 assumes continuous differentiability and a Lipschitz Jacobian.

Questions for authors
  1. See weakness
5 comments

Thank you for taking the time to review our paper and for the constructive comments. We address the three weaknesses below. In addition, the revised paper is available, in which the updated phrases are highlighted in blue. We would greatly appreciate it if the reviewer could share whether our responses mitigate the concerns. We are happy to continue the discussion until the concerns are resolved.

W1. Typo regarding Adam/AdamW.

Thank you for pointing this out. This is a typo in the main text. We used AdamW for CIFAR-10/100 and Adam for HAPT, as stated in Appendix C. See the revised version, lines 446--447 in Section 4.1.

W2. Limitation to full participation cases.

Thank you for pointing this out. First, we explain why experiments under partial participation are difficult (Point 1). Second, we report the results of a partial-participation experiment that we conducted during the rebuttal period in a limited scenario (Point 2).

(Point 1) Our source code follows the LASER-VFL implementation (https://github.com/Valdeira/LASER-VFL), which evaluates the training objective as an exact weighted sum over all subsets instead of a sampled estimate. It therefore requires enumerating all 2K−12^K-1 client-availability patterns. Consequently, in the large-KK regime (e.g., K=8K{=}8 for 255255 patterns and K=16K{=}16 for 65,53565{,}535 patterns), the training time increases exponentially with KK because of this enumeration. We tested this, but could not run it within a realistic amount of time. This limitation is already stated in the original version (see lines 406--412).

However, we agree with the reviewer's concern regarding the evaluation under partial participation. We therefore addressed it during the rebuttal period, following a suggestion from another reviewer (7B81). Instead of enumerating all 2K−12^K-1 client-availability patterns, we fix the number of missing clients to ν\nu and enumerate only (Kν)\binom{K}{\nu} patterns, with a small ν\nu (e.g., (K,ν)=(8,1)(K,\nu)=(8,1) for 88 patterns, (K,ν)=(8,2)(K,\nu)=(8,2) for 2828 patterns). Under this limitation, the evaluation under partial participation becomes feasible.

(Point 2) Following the above policy, we additionally conducted a partial-participation experiment during the rebuttal period. As explained above, we made this feasible by restricting the client-availability patterns to those in which the number of missing clients is a small number ν\nu, even in the large-KK regime (e.g., (K,ν)=(8,1),(8,2)(K,\nu)=(8,1), (8,2)), on the two image classification benchmarks (CIFAR-10/100). Our method has four variants, depending on the presence or absence of the two optional regularizations; because of limited computational resources, we tested only the variant without either regularization (P1).

Test accuracy under partial participation in limited scenarios. Since each method is extended so that each client holds its own fusion model, KK prediction results are obtained; their mean and standard deviation are reported.

Method(B1) CIFAR-10, (K,ν)=(8,1)(K,\nu)=(8,1)(B1) CIFAR-10, (K,ν)=(8,2)(K,\nu)=(8,2)(B2) CIFAR-100, (K,ν)=(8,1)(K,\nu)=(8,1)(B2) CIFAR-100, (K,ν)=(8,2)(K,\nu)=(8,2)
(M1) Vanilla VFL0.8496 ± 0.00500.8478 ± 0.00380.4930 ± 0.00920.4559 ± 0.0064
(M2) LASER-VFL0.8464 ± 0.00260.8396 ± 0.00460.4772 ± 0.01180.4380 ± 0.0096
(M3) PlugVFL0.8852 ± 0.00570.8445 ± 0.00630.4910 ± 0.00800.4843 ± 0.0081
(M4) Local0.6352 ± 0.04950.6357 ± 0.05150.3398 ± 0.04150.3395 ± 0.0407
(P1) PIGR-VFL (γcurv=0\gamma_{\text{curv}}=0, γprox=0\gamma_{\text{prox}}=0)0.8906 ± 0.01000.8918 ± 0.00360.5601 ± 0.01330.5068 ± 0.0091

Although this was conducted under limited partial-participation scenarios, the effectiveness of PIGR-VFL is maintained. We have added these experimental results to Appendix C.4 in the revised version. In a further revised version, we will extend this experiment, for example, with the other variants of our method. We hope that this addresses the reviewer's concern.

W3. Theoretical motivation of (T3) and concerns about using ReLU.

Thanks to the reviewer's insightful suggestion, we noticed a deficiency in our explanation. As the reviewer recognizes, the reference extractor ff is a conceptual function introduced for the theoretical analysis and is not accessible in practice. Therefore, in principle, we cannot establish a method that guarantees the reduction of Δcurv\Delta_{\mathrm{curv}}.

However, we believe that our idea of reducing the curvature of the accessible local extractors fif_i, instead of that of the inaccessible reference extractor ff, is reasonable. As explained in (T2), under (T1) and (T2), Δcurv\Delta_{\mathrm{curv}} is proportional to the variance of the participating clients' feature blocks scaled by the curvature βf\beta_f. Hence, Δcurv\Delta_{\mathrm{curv}} can be reduced only by reducing βf\beta_f. In the limit βf→0\beta_f \to 0, this direction coincides with reducing the curvature of the local extraction models fif_i. This can be proven rigorously under a restriction such as expressing ff as a convex combination of the fif_i. Our original idea, however, did not rely on such a restriction: as described above, we designed (T3) because, in the limit βf→0\beta_f \to 0, the direction of reducing βf\beta_f coincides with that of reducing the curvature of the local extraction models fif_i.

We have added this explanation to Section~3.2 of the revised version (blue-colored sentences, lines 266-274, 316-333), and we would be grateful if the reviewer could read it. If the concern has not been mitigated, we would appreciate it if the reviewer could point it out again.

On the other hand, we understand the reviewer's point regarding ReLU. However, Lipschitz smoothness is assumed in many theoretical analyses (e.g., in the optimization literature [1, 2] below), while deep neural networks containing ReLU are commonly used in the corresponding empirical evaluations. We do not think this should be a basis for a negative score.

[1] S. J. Reddi et al., ``Adaptive Federated Optimization,'' ICLR 2021.

[2] S. P. Karimireddy et al., ``SCAFFOLD: Stochastic Controlled Averaging for Federated Learning,'' ICML 2020.

We understand that the reviewer is busy. However, the discussion period is running short, and about four days have passed since we posted our rebuttal. We have tried to respond to the reviewer's constructive comments sincerely and carefully, and we sincerely hope for a constructive exchange of views with each other. We would greatly appreciate it if the reviewer could let us know whether our responses have mitigated the concerns or point out any remaining issues, while there is still time to address them.

Reviewer_KdNFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PIGR-VFL, a VFL framework designed to reduce the performance gap induced by partitioned feature representations. Specifically, the authors introduce two model architectural designs that eliminate the embedding-reconstruction gap and zero-input offset: coordinate-preserving embedding, which retains each feature block’s original coordinates via zero-padding, and participant-mean aggregation, which also provides a theoretical justification for the aggregation rule LASER-VFL has so far used only empirically. The authors evaluate PIGR-VFL on image and tabular classification tasks. PIGR-VFL achieves an improvement, particularly on the image classification tasks, which critically require the global position/coordinate information of fine-grained feature blocks for accurate classification.

Critical strengths

S1: The paper attempts to connect partition-induced error with feature heterogeneity and extraction curvature rather than relying only on empirical observations.

S2: The evaluations compare different numbers of participants and include comparisons with existing VFL baselines.

Critical weaknesses

W1: The paper does not clearly show that the proposed regularizers can reduce the partition-induced gap described in the theoretical analysis.

W2: Aligning participants’ representations may cause the loss of useful information because different participants may have different feature sets. The paper does not sufficiently justify why their representations should be similar.

Questions for authors

Q1: Under what assumptions can minimizing the objective in Eq. (17) reduce the gap characterized by Proposition 2? Can this connection be formally established?

Q2: Why should local extraction outputs represent compatible quantities that can be meaningfully averaged? Under what conditions might the regularizer suppress complementary information?

Q3: How sensitive are the results to the finite-difference step size \mu and the number of probe pairs J, given the use of J=1 in the CIFAR experiments?

Q4: How much of the performance gain is attributable to coordinate-preserving embedding and participant-mean aggregation, and how much additional benefit comes from the two regularizers?

Q5: Are the reported means and standard deviations computed across independent training runs or across client-level prediction results? How many independent runs were conducted?

Q6: Does PIGR-VFL remain effective under imbalanced feature blocks and different feature-assignment strategies?

Q7: How do the theoretical assumptions relate to the actual neural-network architectures and activation functions used in the experiments?

7 comments

Thank you for taking the time to review our paper and for the constructive comments. We address the three weaknesses below. In addition, the revised paper is available, in which the updated phrases are highlighted in blue. We would greatly appreciate it if the reviewer could share whether our responses mitigate the concerns. We are happy to continue the discussion until the concerns are mitigated.

W1: The paper does not clearly show that the proposed regularizers can reduce the partition-induced gap.

This is a good question. Proposition~2 shows that PIGR-VFL retains three gap terms (Δmiss\Delta_{\mathrm{miss}}, Δcurv\Delta_{\mathrm{curv}}, and Δhet\Delta_{\mathrm{het}}); the reviewer may be concerned about how large these gaps, particularly Δcurv\Delta_{\mathrm{curv}} and Δhet\Delta_{\mathrm{het}}, remain in our method.

However, Δcurv\Delta_{\mathrm{curv}} and Δhet\Delta_{\mathrm{het}} cannot be measured directly, since they involve the reference extraction model ff, which is a conceptual function that is not available (see their definitions in Proposition 1). We understand the reviewer's wish for empirical evidence that supports the theory. Since a quantity defined through a function that does not exist cannot be computed, we instead measured the effect of the two optional regularizations in Section 4 through the test accuracy, by an ablation with and without them ((P1)--(P4) in Table 1).

To prevent this question from arising, we have added to the revised version a statement in lines 268--269 that Δcurv\Delta_{\mathrm{curv}} and Δhet\Delta_{\mathrm{het}} cannot be observed since ff is inaccessible. We hope that this mitigates the reviewer's concern.

W2 / Q2: Why local representations should be similar, and why local outputs can be meaningfully averaged.

This is a good question. We clarify it in the following points.

Point 1: ``Aligning participants' representations may cause the loss of useful information [...].''

We agree with this point. Aligning the local outputs may suppress complementary information when clients hold very different features. We therefore do not argue that local representations should be similar for their own sake.

Point 2: The paper does not sufficiently justify why their representations should be similar,''__ __Why should local extraction outputs represent compatible quantities that can be meaningfully averaged? Under what conditions might the regularizer suppress complementary information?''

We feel that our claim in the original version may not have been fully conveyed. We first state our claims, and then explain them in order.

Claim 1: (T4) does not aim at making the local outputs similar to each other; its intent is to bring each local output closer to the reference ff. As stated around (T4) in the original paper, ``the extraction-output heterogeneity gap measures how far each local extraction model's output fi(Zixi;θfi)f_i(Z_ix_i;\theta_{f_i}) deviates from a common reference extraction output f(Zixi;θf)f(Z_ix_i;\theta_f)'' (lines 365--366). That is, Δhet\Delta_{\mathrm{het}} in Proposition 1 asks each local output to be close to the output of the conceptual reference extractor ff, not to be similar to the other clients' outputs.

Claim 2: Since ff is inaccessible, we use the average of the local extraction outputs as a heuristic proxy for ff. We also stated that "ff is introduced conceptually for theoretical analysis and is inaccessible, and thus cannot be directly used in a training regularization" (lines 367--368). We therefore anchor each local output at the average of the local outputs, motivated by the view that "each local extraction model is a noisy perturbation of ff," so that "averaging the local extraction models' outputs across clients cancels out the independent noise terms and thus approaches ff itself" (lines 371--373). We acknowledge that this is a heuristic and not a rigorous justification: the anchor averages outputs evaluated at different inputs, so it is not equivalent to f(Zixi)f(Z_i x_i), and the local outputs need not be compatible quantities in general. Accordingly, the regularizer may suppress complementary information when clients hold very different features.

Claim 3: (T4) is optional and its effect depends on the setting, and Δhet\Delta_{\mathrm{het}} remains in Proposition 2. This is why (T4) is an optional regularization, and why Δhet\Delta_{\mathrm{het}} remains in Proposition 2. We evaluated the effect of (T4) empirically through the test accuracy, by ablation with and without it ((P1)--(P4) in Table 1). In some settings, introducing (T4) improved the accuracy, and in others it did not.

We suspect that your questions on "compatible quantities that can be meaningfully averaged" and on "complementary information" arise from reading (T4) as a regularizer that makes the local outputs similar to each other. This is not the intent of (T4). In Δhet\Delta_{\mathrm{het}}, each local output fi(Zixi)f_i(Z_ix_i) is compared with f(Zixi)f(Z_ix_i), i.e., the output of a single common extractor ff on the same input, and never with the output of another client's extractor. Hence, Δhet\Delta_{\mathrm{het}} does not ask the local outputs of different clients to carry the same information, and it does not require the outputs of clients with different features to be averaged into a common meaning. The average in (T4) is merely a heuristic proxy for the inaccessible ff, which is why we acknowledge that it is not equivalent to f(Zixi)f(Z_ix_i).

To reflect the reviewer's insightful comment, we added sentences in lines 365--369 of the revised version. We would be happy if the reviewer could let us know whether the concerns have been mitigated.

Q1: Under what assumptions can minimizing the objective in Eq. (17) reduce the gap characterized by Proposition~2?

We agree that the relationship between Eq. (17) and Proposition 2 should be clarified as below.

Eq. (17). Eq. (17) is the training problem of PIGR-VFL: it minimizes the main objective LPIGR(θ)\mathcal L_{\mathrm{PIGR}}(\theta) together with the optional regularizers (T3) and (T4). It determines how the parameters θ\theta are trained for the model in Eq. (15), which incorporates (T1) and (T2).

Proposition 2. Proposition 2 bounds the discrepancy ∣RPIGR(θ)−RREF(θREF)∣|R_{\mathrm{PIGR}}(\theta)-R_{\mathrm{REF}}(\theta_{\mathrm{REF}})| by Δmiss+Δcurv+Δhet\Delta_{\mathrm{miss}}+\Delta_{\mathrm{curv}}+\Delta_{\mathrm{het}}. It is derived from Proposition 1 by substituting the model design in Eq. (15): (T1) and (T2) eliminate gaps (b) and (c) by construction, for any θ\theta.

Relation. Proposition 2 is therefore connected to the model design in Eq. (15), not to the training problem in Eq. (17). We do not claim that training θ\theta by minimizing Eq. (17) reduces the remaining gaps, i.e., the right-hand-side terms of Proposition 2.

To clarify this point, we add a sentence in lines 398--399 in the revised version. We hope that this clarification mitigates the reviewer's concern.

Q3: How sensitive are the results to the finite-difference step size μ\mu and the number of probe pairs JJ?

Thank you for the comment. Because of limited computational resources, we did not investigate this in the original version. During the rebuttal period, we conducted additional experiments in a limited scenario (CIFAR-10 with K=8K{=}8) using the full version of our method ((P4) in the paper). We varied the step size over μ∈{0.01,0.1}\mu \in \{0.01, 0.1\} and the number of probe pairs over J∈{1,2,4}J \in \{1, 2, 4\}.

Sensitivity of (P4) PIGR-VFL (γcurv>0\gamma_{\text{curv}}>0, γprox>0\gamma_{\text{prox}}>0) to the finite-difference step size μ\mu and the number of probe pairs JJ on (B1) CIFAR-10 with K=8K=8.

(1) μ=0.01\mu=0.01, J=1J=1(2) μ=0.01\mu=0.01, J=2J=2(3) μ=0.01\mu=0.01, J=4J=4(4) μ=0.1\mu=0.1, J=1J=1(5) μ=0.1\mu=0.1, J=2J=2(6) μ=0.1\mu=0.1, J=4J=4
0.8815 ± 0.00150.8806 ± 0.00170.8805 ± 0.00250.8806 ± 0.00240.8777 ± 0.00240.8779 ± 0.0015

Although the investigated range is limited, the results are not sensitive to JJ or μ\mu: the test accuracy varies only within about 0.40.4 points across all six settings, and a larger μ\mu slightly lowered the accuracy. These additional experimental results are summarized in Appendix C.6 in the revised version.

Q4: How much of the performance gain is attributable to four techniques?

Thank you. The number of possible combinations of the four techniques is too large to evaluate exhaustively. However, if we restrict ourselves to the six patterns in the table below, the results are already shown in Tables 1 and 5.

[Our investigated patterns]

Method(T1)(T2)(T3)(T4)
(M2) LASER-VFL--✓----
(A1) Ablation 1 (Appendix C.3)✓------
(P1) PIGR-VFL (γcurv=0\gamma_{\text{curv}}=0, γprox=0\gamma_{\text{prox}}=0)✓✓----
(P2) PIGR-VFL (γcurv=0\gamma_{\text{curv}}=0, γprox>0\gamma_{\text{prox}}>0)✓✓--✓
(P3) PIGR-VFL (γcurv>0\gamma_{\text{curv}}>0, γprox=0\gamma_{\text{prox}}=0)✓✓✓--
(P4) PIGR-VFL (γcurv>0\gamma_{\text{curv}}>0, γprox>0\gamma_{\text{prox}}>0)✓✓✓✓

(T1) coordinate-preserving embedding, (T2) participant-mean aggregation, (T3) extraction-curvature regularization, and (T4) proximal extraction-output regularization. A check mark indicates that the component is used.

The simplest variant, (P1), contains only the required components and already outperforms the baselines on the image classification tasks. For example, (M2) LASER-VFL and (P1) differ only in whether (T1) is used, and (P1) improves from 0.81130.8113 to 0.88300.8830 on CIFAR-10 (K=16K{=}16) and from 0.52680.5268 to 0.63290.6329 on CIFAR-100 (K=4K{=}4). In contrast, the additional benefit of (T3) and (T4) is small and varies across settings, as stated in our response to Weakness 1. This is consistent with Proposition 1: (T1) and (T2) remove gaps (b) and (c) by construction, whereas gaps (d) and (e) remain even with (T3) and (T4).

If the reviewer feels that any other ablation patterns are needed, we would be glad to consider them.

Reviewer_7B81OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies vertical federated learning (VFL) in the regime where features are split into increasingly fine-grained blocks across K clients, and asks why predictive performance degrades as K grows. Proposition 1 derives a worst-case bound on the population risk gap relative to a reference model trained on the complete, unpartitioned feature vector, decomposed into five interpretable factors: missing-feature gap, embedding-reconstruction gap, zero-input offset, extraction-curvature-induced gap, and extraction-output heterogeneity gap. Building on this, PIGR-VFL introduces (T1) coordinate-preserving embedding via zero-padding, (T2) participant-mean aggregation — which also gives the first theoretical justification for the rule LASER-VFL used empirically — and two optional regularizers (T3) extraction-curvature and (T4) proximal extraction-output regularization, with Proposition 2 giving the resulting bound. Experiments partition CIFAR-10/100 into spatial grids (K in {4,8,16}) and HAPT features into blocks (K in {20,40}), comparing against Vanilla VFL, LASER-VFL, PlugVFL, and local-only training under full participation. Gains are substantial on the image tasks (e.g., 0.811 to 0.883 on CIFAR-10 at K=16 over LASER-VFL) and essentially nil on tabular HAPT; the regularizers give marginal improvements at non-trivial cost, which the authors report honestly.

Critical strengths
  • A principled theory-to-design pipeline. Each method component maps to a specific term in the worst-case bound, and the five-factor decomposition is interpretable and, to my knowledge, the first quantitative account of partition-induced degradation in VFL. The theoretical justification of participant-mean aggregation — previously used by LASER-VFL purely empirically — is a genuinely useful contribution to the VFL literature on its own. This is the main reason I lean positive.
  • Clean attribution in the experiments. Comparing (P1) against LASER-VFL, which differ only in (T1), isolates the effect of coordinate-preserving embedding, and the summation-vs-mean ablation connects back to the zero-input offset term, validating the theory's practical relevance. The honest reporting of (T3)'s unfavorable cost-benefit trade-off is commendable.
Critical weaknesses
  • The motivation and the evaluation are misaligned. The introduction motivates the work by partial participation and scalability to large federations, yet all experiments use full participation because evaluating partial participation requires enumerating 2^K-1 availability patterns. Sampling a manageable subset of patterns instead of enumerating would already provide evidence; without it, a central claim remains experimentally unsupported. Moreover, the gains are concentrated where coordinates matter: on tabular HAPT, PIGR-VFL ties Vanilla VFL and LASER-VFL, and the authors themselves note coordinate information would not substantially affect tabular prediction. Since the motivating applications (credit evaluation across hospitals, insurers, banks) are tabular cross-silo scenarios, this tension deserves prominent discussion.
  • The setup deviates from the standard VFL formulation without discussion. In most VFL literature, a single active party holds the labels and the fusion model; here every client receives the same supervisor label and holds its own fusion model. This changes the threat model and the privacy/utility trade-offs substantially — including representations mapped back into the full d0-dimensional space via zero-padding — and the privacy implications are never discussed.
Questions for authors
  • [Clarification] Can you evaluate partial participation by subsampling availability patterns rather than enumerating all 2^K-1, at least for moderate K?
  • [Clarification] In your decentralized setup all clients hold labels and their own fusion model; what is the assumed threat model, and what are the privacy implications of coordinate-preserving embedding placing representations back into the full feature space?
  • [Clarification] How does (T1) compare against learned positional embeddings applied to local representations? Zero-padding into the full coordinate system is architecturally close to positional encoding, and a small experiment would clarify the novelty boundary.
  • [Suggestion] An empirical estimate of the five bound terms on a small task would tell readers whether the bound is descriptive or vacuous in practice.
  • [Minor] Sentences in Section 3 frequently run very long and would benefit from splitting; Table 1 packs eight column groups and is hard to parse; please state earlier (not only in Sec. 4.1) that all experiments are full-participation.
3 comments

We sincerely appreciate your positive assessment of our work. Below, we provide our responses to your feedback. In addition, the revised paper is available, in which the updated phrases are highlighted in blue. We would be happy to address any additional questions or concerns you may have.

W1, Q1. The motivation and the evaluation are misaligned. Can you evaluate partial participation by subsampling?

Thank you for this comment. As the reviewer recognizes (thank you for the careful reading), evaluating under partial participation following the LASER-VFL implementation requires enumerating all 2K−12^K-1 client-availability patterns, since this implementation computes the training objective as an exact weighted sum over all subsets instead of a sampled estimate. This is particularly serious in the large-KK regime we target (e.g., 255255 patterns for K=8K{=}8 and 65,53565{,}535 patterns for K=16K{=}16).

However, we found the reviewer's suggestion that ``sampling a manageable subset of patterns instead of enumerating would already provide evidence'' valuable. Following it, instead of enumerating all 2K−12^K-1 client-availability patterns, we fix the number of missing clients to ν\nu and enumerate only the (Kν)\binom{K}{\nu} patterns, with a small ν\nu (e.g., (K,ν)=(8,1)(K,\nu)=(8,1) for 88 patterns and (K,ν)=(8,2)(K,\nu)=(8,2) for 2828 patterns). In this limited scenario, the evaluation under partial participation becomes feasible.

Following this policy, we conducted a partial-participation experiment during the rebuttal period. As explained above, we made this feasible by restricting the client-availability patterns to those in which the number of missing clients is a small number ν\nu, even in the large-KK regime (e.g., (K,ν)=(8,1)(K,\nu)=(8,1) and (8,2)(8,2)), on the two image classification benchmarks (CIFAR-10 and CIFAR-100). Our method has four variants, depending on the presence or absence of the two optional regularizations; because of limited computational resources, we tested only the variant without either regularization (P1).

Test accuracy under partial participation in limited scenarios. Since each method is extended so that each client holds its own fusion model, KK prediction results are obtained; their mean and standard deviation are reported.

Method(B1) CIFAR-10, (K,ν)=(8,1)(K,\nu)=(8,1)(B1) CIFAR-10, (K,ν)=(8,2)(K,\nu)=(8,2)(B2) CIFAR-100, (K,ν)=(8,1)(K,\nu)=(8,1)(B2) CIFAR-100, (K,ν)=(8,2)(K,\nu)=(8,2)
(M1) Vanilla VFL0.8496 ± 0.00500.8478 ± 0.00380.4930 ± 0.00920.4559 ± 0.0064
(M2) LASER-VFL0.8464 ± 0.00260.8396 ± 0.00460.4772 ± 0.01180.4380 ± 0.0096
(M3) PlugVFL0.8852 ± 0.00570.8445 ± 0.00630.4910 ± 0.00800.4843 ± 0.0081
(M4) Local0.6352 ± 0.04950.6357 ± 0.05150.3398 ± 0.04150.3395 ± 0.0407
(P1) PIGR-VFL (γcurv=0\gamma_{\text{curv}}=0, γprox=0\gamma_{\text{prox}}=0)0.8906 ± 0.01000.8918 ± 0.00360.5601 ± 0.01330.5068 ± 0.0091

Although this was conducted under limited partial-participation scenarios with limited benchmark tests, the effectiveness of PIGR-VFL is maintained. We have added these experimental results to Appendix C.4 in the revised version. In a further revised version, we will extend this experiment, for example, with the other variants of our method. We hope that this addresses the reviewer’s concern.

Furthermore, we agree with the reviewer that the motivating applications primarily involve tabular cross-silo scenarios, where our method may offer limited advantages. To better reflect the applications in which our method could be beneficial, we have added an example of multi-sensor VFL scenarios to the Introduction of the revised manuscript (lines 034–036). We thank the reviewer for pointing this out.

W2, Q2. Deviation from the standard VFL formulation, threat model, and privacy implications.

Thank you for this question. We agree that in standard VFL a single active party holds the labels and the fusion model, and that the most fundamental privacy analysis then concerns how much information about the data can be inferred from the received information (e.g., in the differential privacy formalism). More advanced threats can also be considered, such as whether the other clients can infer the active party's labels (label inference) and whether the active party or the server can reconstruct each client's raw features from the transmitted representations (feature reconstruction). In the decentralized VFL setting, malicious nodes may also be present, so we partially agree with the reviewer's point.

However, we would like to highlight that our method (PIGR-VFL) can also be used in the centralized VFL setting, where only the central server holds the fusion model: (T1) and (T2) only modify the embedding and the aggregation of the local representations, and the analysis in Section~3.1 does not depend on how many fusion models exist. Our problem setup (decentralized VFL) is inherited from LASER-VFL (Valdeira et al., 2025), which formulates decentralized VFL, because our source code is built on their implementation (https://github.com/Valdeira/LASER-VFL), as described in Section 2.2. Given the privacy concerns the reviewer raises, evaluating our method in the standard centralized VFL setting with a central server might have been preferable. We would appreciate the reviewer's understanding that neither our method nor our analysis is specific to decentralized VFL settings.

We added addressing privacy concerns, particularly in decentralized settings, as future work in Appendix D in the revised version.

Q3. [Clarification] How does (T1) compare against learned positional embeddings applied to local representations?

This is a good question. Positional embedding was in fact the first approach we tried in this research. It did not work well, so we turned to a theoretical approach, which led to the current method. The code of that early attempt was still available, so we ran it for this response, in a limited test (CIFAR-10). In this experiment, we implemented positional embedding (PE) as a one-dimensional position index (0,…,K−1)(0,\ldots,K-1) given as an input, and integrated it into (M2) LASER-VFL, which employs (T2) participant-mean aggregation without (T1) coordinate-preserving embedding. The experimental results are shown below.

Comparison with positional encoding (PE) on (B1) CIFAR-10. A check mark indicates that the component is used.

MethodPE(T1)(T2)K=4K=4K=8K=8K=16K=16
(M2) LASER-VFL----✓0.8611 ± 0.00130.8398 ± 0.00140.8113 ± 0.0014
(M2') LASER-VFL + PE✓--✓0.8596 ± 0.00150.8371 ± 0.00200.8139 ± 0.0013
(P1) PIGR-VFL (γcurv=0\gamma_{\text{curv}}=0, γprox=0\gamma_{\text{prox}}=0)--✓✓0.8994 ± 0.00290.8806 ± 0.00290.8830 ± 0.0023

As shown in the table, the performance of (M2) LASER-VFL and (M2') LASER-VFL + PE is almost the same. In contrast, (P1) PIGR-VFL, which introduces (T1) coordinate-preserving embedding, improves the performance clearly, by about 44--77 points in test accuracy over (M2') for K=4,8,16K=4, 8, 16. These additional experiments are summarized in Appendix C.5 in the revised version.

Q4. [Suggestion] An empirical estimate of the five bound terms [...].

We note that in Appendix C.3, we report the zero-input offset Δoffset\Delta_{\mathrm{offset}}, which depends only on the extractor's output at the zero input and can therefore be measured directly. The other gap factors, however, are defined with respect to the reference extractor ff, which is a conceptual function for the analysis and is not available in practice. They therefore cannot be measured.

Q5. [Minor] Sentences in Section 3 frequently run very long [...].

Thank you for this suggestion. Following Weakness 1 (W1), we have added a partial-participation experiment. In response to the comment on Section 3, we have split several long sentences, as you point out. We also agree that ``Table 1 packs eight column groups and is hard to parse,'' and we have therefore split the table so that (B1)--(B2) and (B3) are displayed separately. We hope that this alleviates the concern.

We sincerely appreciate your positive assessment of our work. Thanks to your valuable comments, we were able to update the paper to a revised version. If you have time, we would be grateful if you could briefly let us know whether our feedback and the revised version have helped to mitigate your concerns. Thank you again.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.