- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Fed-MIAC, which aims to reduce the uplink communication overhead in multimodal federated learning. Clients first estimate modal importance using SHAPE scores based on Shapley values, then combine this with the singular values of model updates to allocate the compression level for each modality within a shared rank budget. Experimental results demonstrate the effectiveness of this method.
Strengths The paper is clearly written, and the logic behind the method is intuitive and easy to understand. The experiments are robust and demonstrate the method’s effectiveness.
Weaknesses
- The method is not particularly novel in essence; it primarily involves assessing modal importance and then allocating a budget to truncated SVD. Therefore, I would like the authors to explain what the paper’s greatest contribution is and which aspects represent new contributions.
- In the experimental section, I would like the authors to provide experimental results under the same communication costs (including both uplink and downlink) and to include metrics such as accuracy, client-side training time, and the number of communication rounds.
- Could the authors provide experiments under non-IID conditions? The current experiments are primarily conducted under IID conditions, but in real-world scenarios, non-IID conditions are more common.
Critical strengths
The paper is clearly written, and the logic behind the method is intuitive and easy to understand. The experiments are robust and demonstrate the method’s effectiveness.
Critical weaknesses
- The method is not particularly novel in essence; it primarily involves assessing modal importance and then allocating a budget to truncated SVD. Therefore, I would like the authors to explain what the paper’s greatest contribution is and which aspects represent new contributions.
- In the experimental section, I would like the authors to provide experimental results under the same communication costs (including both uplink and downlink) and to include metrics such as accuracy, client-side training time, and the number of communication rounds.
- Could the authors provide experiments under non-IID conditions? The current experiments are primarily conducted under IID conditions, but in real-world scenarios, non-IID conditions are more common.
Questions for authors
- The method is not particularly novel in essence; it primarily involves assessing modal importance and then allocating a budget to truncated SVD. Therefore, I would like the authors to explain what the paper’s greatest contribution is and which aspects represent new contributions.
- In the experimental section, I would like the authors to provide experimental results under the same communication costs (including both uplink and downlink) and to include metrics such as accuracy, client-side training time, and the number of communication rounds.
- Could the authors provide experiments under non-IID conditions? The current experiments are primarily conducted under IID conditions, but in real-world scenarios, non-IID conditions are more common.
6 comments
Response to Reviewer uHSZ - 2
Q2: Additional experiments regarding communication cost
We evaluate communication efficiency using two measures: the number of communication rounds and the cumulative communication cost (in gigabits), including both uplink and downlink transmissions, required to reach a target F1 score or accuracy.
Fed-MIAC requires fewer rounds and lower cumulative communication cost to achieve the same performance as the baselines.
As shown in Table 2 of the main manuscript, the per-round training times are 0.88, 0.66, and 2.97 seconds for CrisisMMD, KU-HAR, and PTB-XL, respectively. Since the compared methods follow the same local training procedure, we report the number of communication rounds as a measure of training efficiency.
F1 vs Round
| Dataset | CrisisMMD | KU-HAR | PTB-XL | |||
|---|---|---|---|---|---|---|
| Target F1 | 30 | 40 | 40 | 50 | 40 | 50 |
| SVDFed | 38 | — | 109 | 180 | 71 | 116 |
| MCB | 35 | 150 | 87 | 127 | 41 | 81 |
| MFedMC | 40 | 191 | 79 | 138 | 41 | 86 |
| fedMIAC | 35 | 134 | 77 | 102 | 36 | 71 |
F1 vs Comm. Cost
| Dataset | CrisisMMD | KU-HAR | PTB-XL | |||
|---|---|---|---|---|---|---|
| Target F1 | 30 | 40 | 40 | 50 | 40 | 50 |
| SVDFed | 10.63 | — | 14.70 | 24.26 | 11.01 | 17.95 |
| MCB | 8.70 | 37.28 | 10.80 | 15.77 | 5.82 | 11.51 |
| MFedMC | 9.94 | 47.47 | 9.81 | 17.14 | 5.82 | 12.22 |
| fedMIAC | 8.70 | 33.33 | 9.57 | 12.68 | 5.12 | 10.11 |
Acc vs Round
| Dataset | CrisisMMD | KU-HAR | PTB-XL | |||
|---|---|---|---|---|---|---|
| Target Acc (%) | 65 | 70 | 60 | 70 | 45 | 50 |
| SVDFed | 35 | 93 | 150 | — | 56 | 131 |
| MCB | 35 | 64 | 148 | — | 26 | 111 |
| MFedMC | 37 | 61 | 138 | — | 36 | 91 |
| fedMIAC | 34 | 50 | 129 | 199 | 21 | 61 |
Acc vs Comm. Cost
| Dataset | CrisisMMD | KU-HAR | PTB-XL | |||
|---|---|---|---|---|---|---|
| Target Acc (%) | 65 | 70 | 60 | 70 | 45 | 50 |
| SVDFed | 9.73 | 25.92 | 20.21 | — | 8.69 | 20.26 |
| MCB | 8.70 | 15.90 | 18.38 | — | 3.69 | 15.78 |
| MFedMC | 9.19 | 15.16 | 17.14 | — | 5.11 | 12.93 |
| fedMIAC | 8.45 | 12.43 | 16.04 | 24.74 | 2.99 | 8.68 |
Response to Reviewer uHSZ - 3
Q3: Additional experiments on non-IID conditions
We constructed the datasets following the existing benchmark. Specifically, CrisisMMD was partitioned in an IID manner, while PTB-XL, KU-HAR, and CMU-MOSEI were partitioned using natural splits based on Site, Person ID, and Video Index, respectively.
These ID- or index-based natural splits reflect the inherent heterogeneity of the datasets and therefore should not be assumed to be IID. Artificially repartitioning the data using methods such as a Dirichlet distribution could distort their inherent distributional characteristics. Therefore, we preserved the natural splits of the existing benchmark.
We also conducted experiments using non-IID partitions of CrisisMMD generated from a Dirichlet distribution with , where is the concentration parameter and smaller values indicate greater data heterogeneity across clients. The results show that Fed-MIAC consistently achieves the best performance across all evaluated rank configurations under both non-IID settings.
| Algorithm | F1 | Rank 32 | Rank 16 | Rank 8 | Rank 4 |
|---|---|---|---|---|---|
| FedAvg | 43.95 | — | — | — | — |
| SVDFed | 33.41 | — | — | — | — |
| MCB | — | 34.18 | 35.08 | 33.44 | 33.60 |
| MFedMC | — | 36.08 | 35.39 | 34.53 | 33.65 |
| fedMIAC | — | 44.56 | 40.82 | 40.46 | 35.05 |
| Algorithm | F1 | Rank 32 | Rank 16 | Rank 8 | Rank 4 |
|---|---|---|---|---|---|
| FedAvg | 33.62 | — | — | — | — |
| SVDFed | 25.95 | — | — | — | — |
| MCB | — | 32.40 | 30.86 | 27.83 | 29.16 |
| MFedMC | — | 32.11 | 29.84 | 31.28 | 31.92 |
| fedMIAC | — | 33.54 | 33.74 | 33.64 | 33.49 |
Response to Reviewer uHSZ - 1
We sincerely appreciate your feedback, which encouraged us to further explain the novelty of Fed-MIAC and broaden our experimental settings. We have provided detailed responses to Q1, Q2, and Q3 below and revised the main manuscript accordingly to address each question (Q2 in Appendix E.9 and Q3 in Appendix E.10; revisions are highlighted in blue).
Q1: Concern Regarding Novelty
The greatest contribution of this work is the development of an importance-aware compression framework for communication-efficient multimodal federated learning (MFL). Our framework addresses the limitations of directly applying importance-agnostic compression approaches developed for unimodal FL to MFL, where the heterogeneous importance of different modalities is overlooked. To operationalize the proposed framework, we formulate an importance-weighted distortion minimization problem that jointly accounts for modality importance and compression distortion under a shared communication budget.
Instantiating the proposed framework in practical MFL settings requires a modality importance measure and a compression scheme. We adopt SHAPE as a practical proxy for capturing the heterogeneous importance of different modalities, while truncated SVD is employed because its rank-dependent compression distortion can be explicitly characterized, enabling tractable optimization of rank allocation. We consider this combination well suited to illustrating and evaluating the proposed importance-aware resource allocation principle. Nevertheless, we agree with the reviewer that the novelty of our work does not lie in these individual methods or their straightforward combination. Instead, their use represents a concrete instantiation of the proposed framework, which is not inherently restricted to either technique. In Section 6.2 of the revised manuscript (previously Appendix E.2 - Applying modality importance to quantization-based compression), we also extend the importance-aware resource allocation principle to quantization-based compression and observe performance improvements, providing further evidence that the effectiveness of the proposed approach is not limited to SVD-based compression.
Finally, we empirically validate the proposed framework across diverse MFL settings and communication budgets. In addition to comparisons with existing modality-selection frameworks, we evaluate uniform rank allocation and importance-agnostic rank allocation to examine the specific benefits of incorporating modality importance into compression resource allocation. The observed performance improvements indicate that the benefits of the proposed method are not solely attributable to SVD-based compression, but also to the importance-aware allocation of communication resources. These results provide empirical support for the effectiveness of the proposed optimization framework in communication-constrained MFL.