acceptodds
Desk rejected at ICLR 2027

OrbiVFL: System-Algorithm Co-Design for Asynchronous Vertical Federated Learning

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Vertical Federated Learning (VFL) enables multiple parties with vertically partitioned features to collaboratively train models without directly sharing private data. However, existing VFL systems often suffer from substantial communication and synchronization overhead under heterogeneous computation and wide-area networks, while existing asynchronous approaches insufficiently exploit worker-level parallelism within each party. We present OrbiVFL, a system-algorithm co-design framework for practical asynchronous VFL. OrbiVFL builds on Ray and the parameter-server architecture to decouple computation and communication across workers and parties, while explicitly aligning embeddings and gradients through unique identifiers. To reduce communication-induced stalls, we derive inter-party local update intervals and design a semi-asynchronous synchronization mechanism for heterogeneous workers within each party. We further provide convergence guarantees for the proposed training scheme. We evaluate OrbiVFL on four representative datasets, including UCIHAR, Mini-ImageNet, VisionTouch, and MuJoCo. Compared with state-of-the-art VFL approaches, OrbiVFL reduces end-to-end training time by up to 76.40% while maintaining comparable or higher model accuracy, and improves computational resource utilization by 76.35%. Extensive ablation studies further demonstrate the effectiveness and robustness of our design. Finally, we deploy OrbiVFL in a production-oriented embodied-intelligence scenario, where it achieves 26.30% training-time reduction without sacrificing accuracy under realistic computation.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Vertical Federated Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NyzTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on two issues in the practical deployment of vertical federated learning (VFL) systems under asynchronous conditions: inefficiency in the set intersection stage due to heterogeneous system latencies, and resource scheduling difficulties within the system arising from asynchronous execution. To address these challenges, the paper proposes a training framework aimed at improving coordination across multiple parties.

The first contribution uses Ray and message buffering to support flexible parallel execution and improve coordination efficiency. The second introduces semi-asynchronous mechanisms both across parties and within each party to improve resource utilization.

Critical strengths
  1. A simple but effective approach.
Critical weaknesses
  1. Limited novelty in the main contributions.

The paper’s two main contributions are primarily engineering-level implementations rather than advances in general optimization theory. Furthermore, components such as Ray, buffering mechanisms, and semi-asynchronous coordination are basic techniques widely used in asynchronous federated learning and scheduling methods, rather than contributions specific to this work.

  1. Inconsistency between the main motivation and the actual contributions.

The first challenge identified in the paper concerns the inefficiency of the private set intersection (PSI) procedure required for ID alignment (lines 61–63). However, the proposed design for this stage neither describes a corresponding privacy-preserving algorithm nor provides a relevant evaluation.

Questions for authors

Please see the Critical Weaknesses section.

Reviewer_94GBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents OrbiVFL, a method intended to make Vertical Federated Learning more efficient, particularly when multiple parties with different computational resources collaborate. To this end, a Ray-based architecture is combined with asynchronous communication and various synchronization mechanisms to reduce waiting times and make better use of available resources. Evaluations on four datasets and in an industrial robotics scenario show substantially shorter training times with largely comparable model quality.

Critical strengths
  • Good results compared to the baseline methods, with particularly large improvements in training time, CPU utilization, and GPU utilization across four datasets (Table 1).

  • The evaluation of the method in a real-world application provides an indication that it can also be used under real-world conditions (Table 3).

  • The use of multiple parties and workers with semi-asynchronous approaches addresses relevant challenges in VFL.

Critical weaknesses
  • The algorithmic novelty is currently not sufficiently convincing, as several already established concepts, such as local updates, bounded staleness, and parameter servers, are combined. The dynamic synchronization mechanism (Equation (3)) also follows only a predefined schedule and does not account for actual worker heterogeneity. A comparison with fixed synchronization intervals would be helpful to clarify the added value of the proposed mechanism.

  • Although supporting heterogeneous workers is a central motivation of the paper (lines 63–66), I could not find an evaluation involving different computational speeds within a party that demonstrates the advantages of OrbiVFL in this context. Evaluations with different worker speeds are necessary. I acknowledge that experiments with varying numbers of workers (Figure 5) and network latencies are already included in the appendix.

  • Although evaluations regarding more efficient communication and CPU and GPU utilization are presented, the reported metrics do not reveal where or how these savings arise. While Table 2 already examines individual components, it could be extended to cover additional components. A more granular analysis would have been necessary, particularly regarding communication time itself, the amount of data transmitted, and workers' waiting times for embeddings, gradients, or synchronization.

Minor Issues:

  • Appendix F lists model types, such as MLP, LSTM, and MobileNetV2. However, specific architectural details are still partially missing. In my view, reproducibility is therefore only possible to a limited extent. The source code appears to have been submitted as supplementary material. However, for the long-term value of the paper, it would still be beneficial if all information required for reproducibility were also included in the paper.

  • In section 4.1, it is stated that OrbiVFL detects failed workers, reassigns tasks, and discards late results. It would have been desirable to evaluate this approach as well, rather than considering only a failure-free setting.

  • A timing diagram comparing OrbiVFL with the baseline methods would be desirable to illustrate more clearly where exactly OrbiVFL provides an advantage over other approaches and how this is achieved methodologically.

  • Since the use of early stopping is mentioned in the appendix, a more in-depth analysis of its effects would be useful. At present, it is not entirely clear to me how many epochs, optimization steps, and communication operations the individual methods actually require. Since OrbiVFL performs multiple local updates, the training effort could differ between methods. Such an analysis would help to better understand the reasons for the shorter training times.

  • The observed loss in accuracy is linked to the convergence analysis (Section 5.3, Lines 408 and 409), a more detailed justification would be desirable.

Questions for authors
  • What advantage does the dynamic synchronization strategy offer over a fixed interval? An additional ablation study would be desirable here.

  • Since a semi-asynchronous approach was chosen for synchronization, there is still a dependency on all passive parties for a batch. Can a slow participant therefore still delay the processing of the batch?

  • What value of QQ was used in the evaluations in Section 5.2? Was it Q=4Q=4? A description of this value in the experimental setup would be necessary. Furthermore, what happens if passive parties are slower than the active party with respect to Equation (2)?

  • Why is a channel capacity of 1515 used?

  • Is the higher resource utilization in Section 5.2 attributable to an algorithmic improvement or to different execution architectures?

  • Why was D=5D=5 chosen?

  • Since early stopping was used, do the methods require different numbers of epochs or optimization steps to reach their final model quality?

Reviewer_sPSMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

OrbiVFL combines Ray-based embedding/gradient routing, repeated local updates, and worker synchronization for asynchronous vertical federated learning. The paper derives a local-update count from a communication-overlap model and gives a conditional convergence bound, with evaluation on four datasets and a production-oriented robotic task.

Critical strengths

The reported training-time reductions across four tasks and the production-oriented experiment indicate practical potential, with generally modest predictive-performance differences from the baselines. Appendix G also includes performance-versus-time curves and latency sensitivity.

Critical weaknesses
  1. The convergence guarantee is not connected to the proposed freshness control (Sec. 4.3; Appendix D, Eq. 14). The counter D bounds outstanding batches per worker, whereas the theorem assumes a uniform bound on full parameter-context displacement, including replica and aggregation differences. A slow worker can retain one pending batch while other workers make many updates; the outstanding-count bound alone does not bound this version lag. The paper does not derive the assumed tau from D, the synchronization schedule, and an explicit aggregation/freshness rule. Thus the conditional theorem does not yet establish convergence for the stated scheduler.

  2. The supplied implementation differs at mechanisms central to the claimed speedup (Algorithm 1; Sec. 4.3). In active_party.py, process_embeddings returns gradients only after all local updates; trainer.py waits for this return before applying passive updates. This differs from the immediate gradient release needed for the modeled overlap. The worker synchronization code also uses staged epoch periods rather than Eq. 3. The release documents acknowledge incomplete correspondence with the manuscript, but the implementation that produced Tables 1–3 is not identified. This prevents attribution of the reported gains to the specified co-design.

Questions for authors
  1. What enforced freshness and aggregation rules imply a uniform tau in Eq. 14? Please give the relation to D and E, including slow workers.
  2. Which implementation produced Tables 1–3? Please reconcile its gradient-release timing and synchronization schedule with both Algorithm 1 and the supplied code. Does the cosine schedule improve time to a common predictive-quality target over fixed synchronization intervals on the same execution backend?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.