- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes a new method for federated domain generalization, a setting in which each client holds a different domain. In this setting, clients hold different data that follow different probability distributions, resulting in a degradation of performance when performing federated learning. The authors tackle this problem a slightly different and harder problem, where the domain labels (i.e., to which domain each client's data belong) is not known by the server. The authors' method combines (i) learning a set of prototypes from each client (e.g., class mean vectors per domain) and (ii) a matrix factorization method that discovers/models the latent structure of the different domains. The paper validates on 3 image benchmarks: Office-10, PACS and DomainNet demonstrating superior performance to previous prototypical FL methods w/o domain labels, and competitive performance to prototypical FL with domain labels.
Critical strengths
- The paper is overall well written and easy to follow
- If demonstrated (see weaknesses) the competitiveness against FedDAP is a relevant finding
Critical weaknesses
There are several major problems with this paper.
Empirical Comparison Issues
Issue 1: The reported DomainNet results isn't really DomainNet
The original DomainNet dataset proposed by (Peng et al., 2019) contains 345 classes. The authors remark, in the appendix, that they use a subset of 10 classes, following (Zhou, 2024). There are a couple of serious problems here,
-
Using a subset of 10 classes severely downscales the DomainNet dataset (the authors don't even report which classes were selected, nor how many samples that represent) which is the precise setting it is supposed to test (in the DA/DG/FedDG community, this benchmark serves to demonstrate scalability).
-
By looking at (Zhou, 2024), these authors don't seem to define this 10 classes subset (they don't even experiment with DomainNet), so there's an attribution error here.
-
Even if we would overlook the previous 2 issues, reporting the results as "DomainNet" is misleading in the main text, as the only mention of subsampling comes in the appendix.
Issue 2: Missing Statistical Analysis of the reported results
The authors claim in Appendix A.2, lines 585-586,
Each experiment is independently repeated 3 times, and we report the mean accuracy averaged over the final 5 communication rounds.
The authors should,
-
Report which seeds were used in these experiments,
-
Report mean std-dev so a clearer comparison can be established against the baselines.
Methodological Issues
Looking at Table 3, the over-complete row (e.g., ) shows a relevant performance degradation for Office-10 (70.22 best vs. 64.47 R=8, worst) and PACS (85.25 best vs. 83.46 R=8 worse). Therefore the claim,
This prune-and-merge refinement enables the factorisation to begin with an over-complete set of latent components while progressively eliminating weak or redundant ones. Consequently, the effective number of latent domains is determined adaptively by the structure of the learnt client representations, rather than being explicitly prescribed or constrained by the number of observed domains.
is too strong. If the prune-and-merge mechanism was effective, we would not see a degradation at high . The reported results in Table 1, by the way, seem to come from the best rows in Table 3 which indicate a "select-on-test", which is apparently unfair to other methods.
Missing Analysis
-
The authors do not include a complexity analysis of Algorithm 2 nor a running time evaluation of how it affects Algorithm 1
-
It would be nice to have the prune-and-merge strategies included somewhere in Algorithms 1 and 2.
Questions for authors
I am going to use this section for requesting changes from the authors. If they manage to answer these in their rebuttal, I would be happy to increase my score,
-
The authors did their experiments on an fairly heterodox setting. They should: (i) Run their method on the full DomainNet dataset (see [1]) and report performance against the competitors. (ii) Experiment with Office Home (see [2]) besides the other presented benchmarks, and desirably (iii; somewhat optional) use a more standard setting for the backbone, such as ResNet-50 for Office Home, ResNet-101 for DomainNet and Office 10 (see [1]) and ResNet-18 for PACS.
-
The authors should include the mean +- std-dev in all results.
-
The authors should include a practical way of selecting in their algorithm, desirably without the use of labeled information from the target domain
-
The authors should report running times for Algorithm 2 and their algorithmic complexity
-
The prune-and-merge strategies deserve more analysis and a dedicated algorithm.
References
[1] Peng, Xingchao, et al. "Moment matching for multi-source domain adaptation." 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2019.
[2] Venkateswara, Hemanth, et al. "Deep hashing network for unsupervised domain adaptation." 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, 2017.
3 comments
Response to Reviewer G6E3 (Part 2/3)
Methodological issue / Q3. Sensitivity to , the prune-and-merge claim and selection of .
The reviewer's statement is correct. We agree that the quoted sentence overstates the effect of prune-and-merge. Prune-and-merge is a refinement mechanism that reduces redundant latent components; it does not guarantee invariance to the initial rank. To characterise the behaviour fully, we have extended Table 3 of the revised manuscript to initial ranks up to . Accuracy (%) for each initial (Table 3 of the revised manuscript; best per dataset in bold):
| 2 | 3 | 4 | 5 | 6 | 8 | 10 | 12 | 14 | |
|---|---|---|---|---|---|---|---|---|---|
| DomainNet | 64.49 | 64.72 | 64.49 | 64.16 | 64.98 | 64.68 | 64.27 | 64.27 | 65.24 |
| Office-10 | 68.81 | 67.72 | 70.10 | 66.66 | 70.22 | 64.47 | 67.72 | 68.68 | 66.99 |
| PACS | 85.02 | 84.20 | 85.25 | 84.26 | 85.01 | 83.46 | 84.55 | 84.96 | 85.37 |
The extended results show two distinct behaviours:
- DomainNet and PACS are robust to . For all , accuracy stays within 64.16–65.24% and 83.46–85.37%, respectively, and every tested outperforms the best label-free prototypical baseline (FedPLVM: 62.22% / 82.06%). On these benchmarks, performance is insensitive to the initial capacity.
- Office-10 shows larger fluctuations across (64.47–70.22%), and several values of fall below FedPLVM (68.77%). These fluctuations are not monotonic in : is the lowest, while and recover to 67.72% and 68.68%. This pattern points to variance rather than a systematic over-capacity effect. Office-10 is the smallest benchmark, so client prototypes, and hence the factorisation input , are noisier than on the other datasets. We acknowledge that prune-and-merge does not remove this sensitivity on small benchmarks.
Revision: We will rewrite the quoted sentence to state that (i) the initial rank R remains a hyperparameter; (ii) prune-and-merge refines the latent components but does not fully remove the need to specify R; (iii) performance is robust to R on DomainNet and PACS, while the small Office-10 benchmark shows larger variability. The extended Table 3 is included in the revised manuscript, and mean ± std for every R will follow.
Select-on-test and choice of .
We acknowledge that the submitted Table 1 reported the best R per dataset. The revised Table 1 now uses a single fixed for all datasets, which yields 64.49 / 70.10 / 85.25 (DomainNet / Office-10 / PACS). This still outperforms FedPLVM (62.22 / 68.77 / 82.06) on all three datasets and exceeds FedDAP on PACS (84.63), so our conclusions do not rely on per-dataset tuning. In practice, we recommend a small shared (we use throughout), since a larger initial capacity brings no consistent benefit on any benchmark (Table 3). No labelled information from the test domains is used to set in the revised results.
(continued in Part 3/3)
Response to Reviewer G6E3 (Part 3/3)
Missing analysis / Q4. Complexity and running time of Algorithm 2.
Let be the number of participating clients, the number of classes, the feature dimension, , the initial rank, and the number of optimisation steps.
- Initialisation (FINCH): on client vectors.
- Joint optimisation: each Adam step computes and its gradients in , and the simplex projection costs , giving in total.
- Prune-and-merge (Algorithm 3): .
For DomainNet (, , ), , i.e. roughly FLOPs per round including the backward pass, independent of the number of local samples. The additional download per client is and ( floats), at most 120 KB for ; after prune-and-merge, the measured download is 28–40 KB per round, below 0.05% of the model payload. Overall, FedLDD's per-client communication is on par with FedDAP's (78.97 vs. 79.05 MB on DomainNet).
Table 9 of the revised manuscript reports measured wall-clock costs on a single NVIDIA TITAN RTX (clients run sequentially; mean over steady-state rounds):
| Dataset | Alg. 2 / round (ms) | Local training / round (s) | Alg. 2 / total | FedDAP total / round (s) | FedLDD total / round (s) |
|---|---|---|---|---|---|
| DomainNet | 59.0 | 144.86 | 0.04% | 179.46 | 144.99 (−19.2%) |
| Office-10 | 57.9 | 19.72 | 0.29% | 28.62 | 19.81 (−30.8%) |
| PACS | 54.4 | 120.93 | 0.04% | 145.70 | 121.05 (−16.9%) |
Algorithm 2 takes 54–59 ms per round, below 0.3% of the round time when clients run sequentially. Moreover, because FedLDD replaces FedDAP's heavier per-batch client computation (1.35 vs. 22.32 ms per batch on DomainNet), a full FedLDD round is 17–31% faster than a FedDAP round.
Q5. A dedicated algorithm for prune-and-merge.
We agree. We have added Algorithm 3 (Appendix A.1), which details the prune-and-merge steps. It is invoked at the end of Algorithm 2 (line 9).
We thank the reviewer again for the concrete requests, which have improved the rigour and reproducibility of the paper. In summary, the revised manuscript now includes a fixed-R main result (Table 1), results up to (Table 3), measured running time (Table 9) and a dedicated prune-and-merge algorithm (Algorithm 3). We will further add a corrected and transparent DomainNet description, mean ± std with seeds for FedLDD. We hope these address the reviewer's concerns and would be grateful if the reviewer would consider increasing their score.
Response to Reviewer G6E3 (Part 1/3)
We sincerely thank the reviewer for the detailed and actionable feedback, and for explicitly listing the changes that would address their concerns. We have uploaded a revised manuscript and respond to each request below.
Issue 1 / Q1. DomainNet subset, attribution, benchmarks and backbones.
Clarification of the subset. We thank the reviewer for pointing out the insufficient description of our DomainNet setting. Our experiments use a 10-class subset of DomainNet, not the full 345-class benchmark. This subset follows the experimental setting of FedDAP (Le et al., 2026), and we adopted it to keep our comparison consistent with the baseline results reported there. The 10 classes are: bird, feather, headphones, ice cream, teapot, tiger, whale, windmill, wine glass and zebra, used across all six domains (Clipart, Infograph, Painting, Quickdraw, Real, Sketch). We agree that the original presentation was insufficiently clear and that the attribution to Zhou et al. (2024) is incorrect.
Backbones. Following the protocol of FedDAP, we use ResNet-10 for DomainNet and Office-10, and ResNet-18 for PACS (Appendix A.2). PACS therefore already uses the backbone suggested by the reviewer. All baselines in Table 1 use the same backbones, so the comparison is like-for-like.
Additional benchmarks. We agree that Office-Home and the full DomainNet would strengthen the evaluation. The full 345-class DomainNet with 20 clients exceeds our compute budget within the rebuttal period; we will state this explicitly as a limitation.
Revision:
Issue 2 / Q2. Statistical analysis: seeds and mean ± std.
We agree that reporting seeds and standard deviations improves transparency and reproducibility. Each FedLDD experiment is repeated with 3 seeds; for each seed, we average test accuracy over the final 5 communication rounds, and we will report mean ± std across seeds. Baseline results are taken from FedDAP, which follows an identical protocol. To make the comparison with the closest competitors statistically grounded, we are additionally re-running FedDAP with the same seeds and data partitions, and will report their mean ± std as well.
Revision: All FedLDD results will be reported as mean ± std, the seeds will be stated, and results we re-ran will be clearly distinguished from results taken from prior work.
(continued in Part 2/3)