- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies RFL-DFDC, a federated primal–dual learning algorithm designed to address dual variable drift caused by partial client participation. Building on A-FedPD, which compensates for missing dual updates using the average update of participating clients, the proposed RFL-DFDC introduces client-specific compensation based on relationships among clients. Specifically, it constructs a dynamic graph using cosine similarities between client model updates and uses the resulting weights to reconstruct missing dual updates for inactive clients. This method also introduces a corresponding server-side correction to maintain consistency between the global dual variable and the average of the virtual client dual variables. Theoretical convergence analyses provide bounds on reconstruction errors on nonconvex optimization under certain assumptions. Experiments on heterogeneous federated learning benchmarks demonstrate improved prediction accuracy and communication efficiency compared with existing methods, including A-FedPD.
Critical strengths
S1. Simple and intuitive algorithmic extension.
The proposed method provides a simple and intuitive extension of A-FedPD by replacing uniform mean-based dual compensation with client-specific compensation based on relationships among clients. The overall algorithmic design is straightforward and easy to understand.
S2. Theoretical analysis of the proposed algorithm.
This paper provides theoretical analyses of the proposed algorithm, including bounds on dual reconstruction errors and their effects on nonconvex optimization.
S3. Empirical improvements over existing methods.
The proposed method demonstrates improvements in test accuracy and communication efficiency over existing FL methods, including A-FedPD and A-FedPDSAM, on the evaluated benchmarks.
Critical weaknesses
W1. Limited algorithmic novelty.
The problem of dual-variable drift under partial client participation has already been extensively investigated in federated primal–dual learning, particularly in A-FedPD. The main algorithmic contribution of this paper appears to be replacing the uniform mean-based dual compensation in A-FedPD with a weighted average based on cosine similarities between client model updates. Although the proposed extension is reasonable, I find its algorithmic novelty somewhat limited.
W2. Insufficient theoretical justification for the proposed dynamic graph-based reconstruction.
Although this paper provides a theoretical analysis of reconstruction errors and their effects on nonconvex optimization, I am not fully convinced that the analysis establishes the effectiveness of the proposed dynamic graph-based reconstruction mechanism. In particular, Theorem 1 provides a stationarity bound in terms of general reconstruction-error quantities, and the analysis appears to be applicable to various reconstruction schemes, including uniform mean-based compensation. Therefore, it remains unclear whether the theoretical results demonstrate that the proposed graph-based reconstruction achieves smaller reconstruction errors or improved convergence rates than existing methods.
Moreover, several key components of the proposed method, including cosine-similarity-based graph construction, EMA-based dynamic graph updates, and the limited observability of client relationships under sparse participation, do not appear to be explicitly characterized in the main convergence analysis. Although their effects may be implicitly captured by the general reconstruction-error terms, the theoretical implications of these specific design choices remain unclear.
W3. Limited empirical evidence supporting the generality of the proposed method.
Although the proposed method outperforms the considered baselines, I have some concerns about the experimental evaluation scores. In particular, the absolute test accuracies appear relatively low. This may be partly attributed to the use of relatively small LeNet-style architectures for CIFAR-10/100, whereas ResNet-18 is evaluated only on Tiny-ImageNet.
Questions for authors
Q1. Theoretical characterization of the reconstruction-error bound.
Regarding W2, I have some concerns about Assumptions 1 and 2, which play central roles in the convergence analysis. In particular, Assumption 2 introduces a general upper bound on the graph reconstruction error, but it remains unclear how this bound can be theoretically characterized for the proposed method. If possible, could the authors provide a theoretical characterization of this bound? If such an analysis is difficult, could the authors empirically investigate the magnitude and evolution of the reconstruction-error bound during training, and compare it with that of uniform mean-based compensation?
2 comments
Thank you for the detailed responses. However, I must say that my main concerns remain unresolved.
Regarding W1 and W2, I understand that the proposed method jointly considers client-specific dual reconstruction and the corresponding server-side correction, and I acknowledge its empirical effectiveness. However, my main concern remains the limited methodological and theoretical novelty. The proposed method appears to be a natural extension of the uniform mean-based compensation in A-FedPD, while the server-side correction follows directly from the average-dual identity. Furthermore, although the theoretical analysis characterizes how reconstruction errors affect optimization, it does not establish why the proposed cosine-similarity-based dynamic graph should provide more accurate reconstruction or better convergence guarantees than existing approaches. Regarding W3, my concern is primarily about the relatively low test accuracies on CIFAR-10/100, which may be partly attributed to the use of relatively small LeNet-style architectures.
I will maintain my original rating at this stage.
Thank you for recognizing the algorithm's clarity and empirical gains. The advance is client-specific evolution of persistent dual states, with a matching server update derived from the same reconstruction operator and an analysis of imperfect completion.
1. The contribution goes beyond choosing cosine weights (W1). A-FedPD supplies the mean-compensation framework. RFL-DFDC changes how each inactive client is represented within it. Rows of the reconstruction operator determine individual virtual-memory updates; column sums determine active clients' influence on the server dual. Consequently, the same inferred increment affects the returning client's next augmented local problem and the current global update. Equation (11) makes the need to coordinate these two paths explicit.
The existing interventions demonstrate the value of this construction:
These controls support relationship-aware correction, coordination, and persistent application within the evaluated host.
2. The theory characterizes how reconstruction quality affects optimization (W2; Q1). The analysis establishes three complementary results. First, Eq. (9) preserves the global dual as the average of virtual client duals exactly, regardless of prediction accuracy. Second, Lemma 1 bounds the perturbation caused by incomplete reconstruction:
∥ξt∥2≤N2N−PtEβt.
Third, Theorem 1 propagates this perturbation into stationarity under explicit host and graph budgets. Appendix B.1 additionally characterizes how one-step errors accumulate into a returning client's dual input under its local-transition conditions. This connects missing-update reconstruction to both current server optimization and subsequent client optimization.
Assumption 1 has a concrete sufficient characterization in Appendix A.2. Conditional uniform sampling, local-displacement budget ct, and local-stationarity-residual budget rt yield
vt=3(χt−2+L2)ct+3rt.
Finite-step local optimization, including SAM, enters through the residual budget. Assumption 2 controls the conditional expected scaled graph energy. Importantly, Eq. (12) includes (1−skt)∥zkt∥2, explicitly accounting for unsupported clients, and the analysis permits dependence between learned relationships and current updates.
The general kernel formulation is useful: mean completion is a special case, and Appendix A.4 specifies how a smaller matched-state energy budget improves the graph-error term with the host term fixed. Cosine, EMA, and co-participation history affect that term through the learned kernel. The theorem provides conditional error propagation, rather than a universal ranking of similarity metrics or an unconditional faster rate for cosine–EMA. Numerical energy trajectories are not reported; the rule's practical benefit is assessed through the controlled comparisons in Table 3.
The equations establish consistency and error propagation; the interventions test the relationship rule. Appendix A.5 distinguishes reconstruction error from neighbor dispersion, so energy serves as an analytical bound.
3. The gains extend beyond LeNet and across task types (W3). The evaluation includes LeNet-style CIFAR models, ResNet-18 with GroupNorm on Tiny-ImageNet, and a two-layer LSTM on naturally partitioned Shakespeare clients (Section 5.1; Appendix C.2).
On Tiny-ImageNet, RFL-DFDC achieves 39.27% versus 34.81% for A-FedPDSAM, a 4.46-point gain, and reaches 30% in 542 rather than 822 rounds. It also exceeds FedSpeed's 35.24%, the strongest baseline accuracy in that table. Thus the benefit is observed with ResNet-18, alongside the 1.48- and 3.97-point gains over A-FedPDSAM on CIFAR-10/100. Shakespeare further evaluates the method on a different architecture and naturally partitioned language data.
The relevant comparison holds the architecture and federated protocol fixed within each task. With 500 heterogeneous vision clients and nominal 2% participation, the results show improved accuracy and fewer communication rounds under the same evaluated setting. The empirical contribution is consistent gains across these sparse-participation tasks.