acceptodds
Under review as a conference paper at ICLR 2027

FedMAGE: Architecture-Aware Graph Hypernetwork for Heterogeneous Multimodal Federated Learning

Abstract

Multimodal federated learning aims to collaboratively train multimodal models across decentralized clients while preserving data privacy. However, existing methods struggle with highly heterogeneous client tasks and architectures, often rely on public data for knowledge distillation, and incur substantial communication overhead. To address these challenges, we propose FedMAGE, a graph hypernetwork-based multimodal federated learning framework. FedMAGE represents client models built from a predefined operator vocabulary as computational graphs and employs a server-side hypernetwork to generate architecture-specific parameters, enabling implicit parameter sharing across heterogeneous clients without requiring a shared parameter space. To improve the generalization of a shared decoder across diverse operator types, we further introduce Type-aware Decoding, which conditions the parameter generation process on operator-specific information. We conduct extensive experiments on three multimodal tasks: image-text retrieval, sentiment recognition, and scene recognition. Compared with a broad range of federated learning baselines, FedMAGE consistently achieves superior performance, improving accuracy by an average of 3.8 % over the strongest baseline, while substantially reducing communication time compared with existing multimodal federated learning methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
federated learning, mutimodal learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PpCcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedMAGE for multimodal federated learning, where clients may differ in their modalities, tasks, and model architectures. Rather than aggregating incompatible model parameters, FedMAGE represents each client model as a typed computational graph and uses a graph hypernetwork to generate its parameters. The authors also introduce Type-aware Decoding (TAD) to account for differences in operator types and parameter structures, along with a learnable client embedding to capture client-specific information.

The method is evaluated on three benchmarks for image-text retrieval, multimodal sentiment recognition, and RGB-D scene recognition. The experiments compare FedMAGE with naive, personalized, and multimodal federated learning baselines, and examine the effect of TAD, varying levels of heterogeneity, communication time, and generalization to deeper variants within the same architecture families, specifically CNN-3 to CNN-5 and ViT-3 to ViT-5.

Critical strengths
  1. Practical relevance. The paper considers architecture heterogeneity in multimodal federated learning, where clients may use different modalities, tasks, and model architectures. FedMAGE represents client models as typed computational graphs and generates client-specific parameters without requiring a shared parameter layout. This avoids the need to directly aggregate parameters from structurally different models. The method is evaluated on three benchmarks covering image-text retrieval, sentiment recognition, and RGB-D scene recognition.

  2. TAD ablation results. Full TAD outperforms the shared MLP by 4.0, 5.8, and 4.6 points on the three tasks, respectively, and performs better than the other decoder variants. The comparisons also suggest that operator-aware decoding offers benefits beyond using separate decoders or type-specific heads, and that normalization further improves performance.

Critical weaknesses
  1. The incremental contribution over related hypernetwork-based methods is unclear. FedMAGE uses a graph hypernetwork to generate parameters for heterogeneous client models. Related work has already explored architecture heterogeneity in federated learning (HAFL-GHN) and hypernetwork-based approaches to multimodal federated learning (FedMBridge). TAD appears to be the main methodological addition. While the ablations show that it performs better than the tested alternatives, they do not rule out the possibility that some of the gains come from the decoder's increased capacity. A comparison with a parameter-matched, operator-agnostic decoder would help isolate the effect of type-aware decoding. The lack of direct comparisons with HAFL-GHN and FedMBridge also makes it difficult to assess the advance over prior work.

  2. Some of the paper's main claims are not directly tested. The paper aims to handle task, modality, and architecture heterogeneity jointly, but the three benchmarks appear to be evaluated separately. I could not find an experiment where clients with different task objectives participate in the same federation. The stress test also changes several aspects of heterogeneity at once, making it difficult to separate the effects of modality and architecture differences. For architecture generalization, the experiments move from CNN-3 to CNN-5 and from ViT-3 to ViT-5. This primarily tests depth extrapolation within known architecture families, providing limited evidence of generalization across architecture families or to substantially different operator configurations. Xavier initialization provides only a limited reference point for this claim; comparison with a trained GHN or another parameter-generation method would be more informative.

Questions for authors
  1. Difference from existing hypernetwork-based methods. I would like to better understand what distinguishes FedMAGE from HAFL-GHN and FedMBridge beyond the use of TAD. For example, does FedMAGE introduce a different way of representing client architectures, conditioning the hypernetwork on client-specific information, or training the model across clients? Or is the main contribution the decoder design itself? A more direct comparison with these methods would help answer this question. I am also unsure how much of the improvement can be attributed specifically to type-aware decoding, since the current ablation does not appear to control for decoder capacity. Comparing TAD with a standard decoder of similar parameter count would help determine whether the gains come from using operator-type information or from the additional capacity.

  2. How client embeddings are used in Table 4. The architecture generalization experiment changes CNN-3 to CNN-5 and ViT-3 to ViT-5 while keeping the trained generator fixed, but I could not find a clear explanation of how the client embedding is handled in this process. Is the embedding learned for the original client retained when its architecture is changed? If so, does the experiment test whether the generator can produce useful parameters for a new architecture given an already learned client representation? Alternatively, is a new embedding initialized for the changed architecture, and if so, is it updated in any way? Since the generated parameters depend on both the architecture graph and the client embedding, this detail seems important for interpreting what Table 4 actually demonstrates.

  3. Whether task heterogeneity is evaluated jointly. I could not tell from the experiments whether clients performing different tasks participate in the same federation. Are the image-text retrieval, sentiment recognition, and RGB-D tasks trained separately, with a different federation for each benchmark, or does a single hypernetwork support clients with different task objectives at the same time? The distinction matters because evaluating several tasks separately does not, by itself, show that the method can accommodate task heterogeneity within one federation. If the paper does include such an experiment, could the authors point to the relevant results? If not, what evidence supports the claim that FedMAGE handles task heterogeneity jointly with modality and architecture heterogeneity?

8 comments
AuthorsOpenReview ↗

Response to Reviewer PpCc - Part 1

We thank the reviewer PpCc for taking the time to carefully evaluate our paper and for the constructive comments. Your suggestions have helped us clarify the methodological contribution, strengthen the comparison with related methods, and improve the interpretation of our experimental evidence.

Question 1. Differences from Existing Hypernetwork Methods and the Role of Decoder Capacity

Your comments are highly professional. To address it, we elaborate on the advantages of our method below and supplement comparative experiments with HAFL-GHN [1] and FedMBridge [2]. Please find our response below, which we hope addresses your concerns regarding our manuscript.

1.1 Methodological Distinctions and Advantages

The main methodological contribution of FedMAGE is the decoding design in Section 3.2, together with client conditioning during both architecture encoding and parameter generation. Representing client architectures as computational graphs follows the graph-hypernetwork framework. The distinctions concern how client information enters this representation and how the resulting features are converted into model parameters:

  • 1. Shared decoding with operator-specific specialization. HAFL-GHN decodes graph representations through separate MLPs for different layer types, whereas FedMBridgeuses an MLP node decoder conditioned on layer-role representations and client embeddings. FedMAGE combines a shared decoder trunk with operator-specific output heads, expressed as Hov(D0(qv))H_{o_v}(D_0(q_v)) in Eq. (9). Relative to HAFL-GHN, this introduces sharing across operator types within the decoder. Relative to FedMBridge, it provides explicitly specialized output mappings. Operators in modality encoders, fusion modules, and prediction heads can therefore share intermediate generation features while retaining output mappings adapted to their respective parameter structures.
  • 2. Client conditioning during both graph encoding and weight generation. FedMAGE incorporates the client embedding into node initialization, zv(0)=ξ(xv,ck)z_v^{(0)}=\xi(x_v,c_k), and parameter-query construction, qv=Qψ(zv(L),eov,esv,ck)q_v=Q_\psi(z_v^{(L)},e_{o_v},e_{s_v},c_k). Client information can thus influence both the propagation of architectural context and the subsequent decoding of parameters. In FedMBridge, architecture encoding produces layer-role representations before the client embedding enters the decoder. HAFL-GHN conditions its shared generator on architecture without the explicit learned client embedding used in FedMAGE. Our formulation allows clients with the same architecture to develop different graph representations according to their learned client embeddings, introducing personalization before the final decoding stage.
  • 3. Operator-specific normalization integrated into parameter generation. FedMAGE applies NovN_{o_v} to the generated tensor after chunk concatenation and trimming, before reshaping it to the required dimensions. This integrates operator-specific control of parameter statistics with the shared trunk and specialized heads. Table 2 (main-text) directly evaluates this component: adding normalization to “Query + Heads, w/o Norm” improves Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1 by 1.8%, 2.6%, and 2.4%, respectively.

Regarding federated training, FedMAGE and FedMBridge share the broad procedure of generating client parameters on the server, training client models locally, and using returned updates to train the generator. HAFL-GHN instead trains local copies of the graph hypernetwork and aggregates their parameters. Therefore, FedMAGE’s contribution is concentrated in its client-conditioned encoding and structured decoding within the server-side generation framework.

1.2 Direct Comparison under the Same Client Architectures and Training Budgets

We have completed direct comparisons with HAFL-GHN and FedMBridge using our reimplementations of their published methods, as their code is not publicly available. All methods use the same per-client architectures, data partitions, modality assignments, participation schedules, communication rounds, and local minibatch-update budgets. Each baseline retains its original federated optimization procedure.

Table R1. Comparison with graph-hypernetwork-based federated learning methods. All scores are reported as percentages.

MethodFlickr30K mR ↑CMU-MOSEI Acc-7 ↑CMU-MOSEI Acc-2 ↑CMU-MOSEI F1 ↑NYU-Depth-v2 Top-1 ↑NYU-Depth-v2 F1 ↑
HAFL-GHN (our reimplementation)72.350.882.784.878.276.4
FedMBridge (our reimplementation)78.553.085.487.178.879.2
FedMAGE (ours)81.456.889.590.282.481.0
Gain over FedMBridge (pp)+2.9+3.8+4.1+3.1+3.6+1.8
AuthorsOpenReview ↗

Response to Reviewer PpCc - Part 2

FedMAGE outperforms both FedMBridge and HAFL-GHN on all six evaluation metrics**.** Compared with FedMBridge:

  • Image–text retrieval: Flickr30K mR increases from 78.5% to 81.4%, an improvement of 2.9%.
  • Sentiment recognition: CMU-MOSEI Acc-7, Acc-2, and F1 improve by 3.8%, 4.1%, and 3.1%, respectively, covering both fine-grained and binary sentiment classification.
  • Scene recognition: NYU-Depth-v2 Top-1 and F1 improve by 3.6% and 1.8%, respectively.

These above comparisons establish the predictive advantage of the complete FedMAGE method across the three benchmarks under the same client architectures and training schedules.

1.3 Empirical Evidence and Theoretical Rationale for Type-aware Decoding

The existing ablations from Table 2 examine complementary aspects of the decoding design. Full TAD outperforms Shared MLP by 4.0%, 5.8%, and 4.6%, and Separate Decoders by 2.3%, 3.8%, and 2.9%, on Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1, respectively. Furthermore, adding normalization to “Query + Heads, w/o Norm” improves these metrics by 1.8%, 2.6%, and 2.4%. These results consistently support the complete design, which combines shared intermediate generation features, operator-specific output mappings, and normalization.

Structural distinction at a matched parameter count. Matching parameter counts controls the number of trainable coefficients; operator-specific routing determines how those coefficients are shared and updated. To analyze this distinction, consider a shared decoder with the same total parameter count, the same query inputs (including operator-type information) and the same normalization. TAD explicitly assigns each operator type to its corresponding output head, while retaining a shared trunk D0D_0. This assignment imposes a specific parameter-sharing structure that does not follow from parameter count alone.

This property can be expressed through the server objective in Eq. (12). Let Lsrv,oL_{\mathrm{srv},o} denote the component associated with generated parameters of operator type oo, with the local-training targets held fixed by the stop-gradient operator. The objective decomposes as:

Lsrv=∑o∈OLsrv,oL_{\mathrm{srv}}=\sum_{o\in\mathcal O}L_{\mathrm{srv},o}.

Writing par⁡(Ho)\operatorname{par}(H_o) for the trainable parameters of output head HoH_o, the decoding structure in Eq. (9) gives ∇par⁡(Ho)Lsrv=∇par⁡(Ho)Lsrv,o\nabla_{\operatorname{par}(H_o)}L_{\mathrm{srv}}=\nabla_{\operatorname{par}(H_o)}L_{\mathrm{srv},o}, and ∇par⁡(Ho′)Lsrv,o=0\nabla_{\operatorname{par}(H_{o'})}L_{\mathrm{srv},o}=0, o′≠oo'\ne o. Thus, the reconstruction loss associated with one operator type directly updates its corresponding output head without directly changing the heads assigned to other types. For a shared output head HsharedH_{\mathrm{shared}}, the gradient instead combines contributions from all operator types:

∇par⁡(Hshared)Lsrvshared=∑o∈O∇par⁡(Hshared)Lsrv,oshared\nabla_{\operatorname{par}(H_{\mathrm{shared}})}L_{\mathrm{srv}}^{\mathrm{shared}}=\sum_{o\in\mathcal O}\nabla_{\operatorname{par}(H_{\mathrm{shared}})}L_{\mathrm{srv},o}^{\mathrm{shared}}.

When different operator types favor opposing changes to shared output parameters, these contributions can interfere. TAD places their direct output-head updates in separate parameter blocks. Increasing the width of a shared decoder changes its parameter budget, but does not itself impose this operator-specific separation.

Complementary roles of specialization, sharing, and normalization. The graph encoder and D0D_0 continue to receive training signals from different operator types, supporting shared intermediate representations. The specialized heads provide dedicated output adaptation for operators with different parameter structures. Applying the same operator-specific normalization to both decoder designs controls generated parameter statistics while preserving this difference in output-head dependencies.

A shared MLP can learn type-dependent mappings from its inputs; TAD builds the corresponding specialization directly into the decoder architecture. This provides an explicit inductive bias for heterogeneous parameter generation. Table 2 supports the effectiveness of the complete design, while the analysis identifies its structural contribution under matched parameter count and normalization: shared intermediate generation combined with operator-specific output adaptation.

Table 2 in main-text provides the following evidence for this design within FedMAGE:

Decoder variantFlickr30K (mR↑)MOSEI (Acc-7↑)NYU-Depth-v2 (Top-1↑)
Shared MLP77.451.077.8
Query + Decoder79.553.479.8
Separate Decoders79.153.079.5
Type-specific Heads78.052.479.1
Query + Heads, without normalization79.654.280.0
Full TAD81.456.882.4
AuthorsOpenReview ↗

Response to Reviewer PpCc - Part 3

Question 2. Client Embeddings in the Architecture Generalization Experiment

We thank you for highlighting this important detail. The embedding learned for the original client is retained and frozen when its architecture changes. The trained generator is also frozen, and the generated model is evaluated directly without additional local training or fine-tuning. Table 4 (main-text) therefore evaluates parameter generation for an unseen, deeper architecture of an existing client, conditioned on that client’s already learned representation.

2.1 How the Client Embedding is Handled

Let ϕT\phi^T and ckTc_k^T denote the generator parameters and client embedding obtained after federated training. When changing CNN-3 to CNN-5 or ViT-3 to ViT-5, we replace the original architecture graph GkG_k with the corresponding deeper graph Gk′G_k'. The parameters of the new model are generated as θknew=hϕT(Gk′,ckT)\theta_k^{\mathrm{new}}=h_{\phi^T}(G_k',c_k^T). The evaluation protocol is as follows:

ComponentTreatment in Table 4
Architecture graphReplace with the deeper architecture graph Gk′G_k'.
Generator parameters ϕT\phi^TKeep the trained parameters fixed.
Client embedding ckTc_k^TRetain the original client’s learned embedding and keep it fixed.
New model parameters θknew\theta_k^{\mathrm{new}}Generate from (Gk′,ckT)(G_k',c_k^T) and evaluate directly on the test set.

No new client embedding is initialized, and the embedding-update rule in Eq. (14) is not applied during this evaluation. Thus, “zero-shot” refers specifically to generating and evaluating parameters for the changed architecture without adaptation to that architecture.

2.2 Why a Fixed Client Embedding Supports a Changed Architecture

The client embedding and architecture graph provide complementary conditioning inputs. The embedding supplies the client-specific context learned during federated training, while the graph explicitly specifies the target model’s operators, computational dependencies, and parameter shapes. The dimensionality of ckc_k is independent of the number of layers in the client model.

Keeping ckTc_k^T fixed therefore preserves the learned client conditioning while allowing parameter generation to respond to the new architecture. In particular, node initialization remains zv(0)=ξ(xv,ckT)z_v^{(0)}=\xi(x_v,c_k^T), but the node attributes and message-passing structure are now determined by Gk′G_k'. The resulting graph representations enter the parameter query in Eq. (8):

qv=Qψ(zv(L),eov,esv,ckT)q_v=Q_\psi(z_v^{(L)},e_{o_v},e_{s_v},c_k^T).

Although the client embedding is unchanged, the graph-dependent terms can change with the added layers and modified topology. The shared trunk and type-specific heads then generate parameters for the parametric nodes of Gk′,G_k', including the newly introduced layers. This is possible because the graph encoder and decoding functions are shared across nodes and operate within the predefined operator vocabulary. Increasing the depth introduces additional instances and connections of supported operators; it does not require a new client embedding or a separate decoder for the deeper architecture.

2.3 What Table 4 Demonstrates

Table 4 (main-text) evaluates within-family depth generalization under a fixed, learned client representation. It tests whether the trained client-conditioned generator can produce useful parameters when an existing client changes its architecture from three to five layers.

The existing results support this interpretation:

  • CNN-3 → CNN-5: FedMAGE achieves 59.5%, 35.6%, and 60.2% on Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1, respectively. These exceed Xavier initialization by 32.7%, 17.2%, and 31.2%.
  • ViT-3 → ViT-5: FedMAGE achieves 56.4%, 34.0%, and 64.3%, exceeding Xavier initialization by 31.1%, 14.5%, and 36.4%, respectively.

These scores are obtained with both ϕT\phi^T and ckTc_k^T frozen and without fine-tuning the generated models. They show that the learned generator and client representation can be reused together to produce useful parameters for a previously unseen deeper architecture. The architectural generalization demonstrated in Table 4 is conditional on an existing client’s learned representation, which remains fixed throughout the architecture change.

Table 4 (main-text). Zero-shot generalization to unseen deeper client architectures.

ScenarioInitializationFlickr30K mR ↑CMU-MOSEI Acc-7 ↑NYU-Depth-v2 Top-1 ↑
No switchFedMAGE81.456.882.4
CNN-3 → CNN-5Xavier init26.818.429.0
CNN-3 → CNN-5FedMAGE59.535.660.2
ViT-3 → ViT-5Xavier init25.319.527.9
ViT-3 → ViT-5FedMAGE56.434.064.3
Reviewer_YxjfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FedMAGE uses a graph hypernetwork to generate parameters for clients with different multimodal architectures. Each client model is represented as a typed computation graph, and the server conditions parameter generation on this graph and a client embedding. Clients update their models locally and return parameter changes to train the shared generator. The proposed Type-aware Decoding module uses parameter queries, a shared decoder trunk, operator-specific heads, and normalization to accommodate different kinds of layers. Experiments cover image-text retrieval, multimodal sentiment recognition, and RGB-D scene recognition. FedMAGE improves the main task metrics over the evaluated baselines. The paper also compares decoder variants and examines performance under greater client heterogeneity and changes in architecture.

Critical strengths
  1. Allowing clients to use different architectures makes sense for this setting. The graph representation gives the server a way to share information across these models without requiring their parameter tensors to align.

  2. The gains appear across all three tasks. The decoder study compares several designs, including variants with and without normalization, so it gives some insight into which choices affect parameter generation.

Critical weaknesses
  1. The closest comparison is FedMBridge [1], which already uses a topology-aware hypernetwork for architecture-personalized multimodal federated learning. The main difference here appears to be the decoder, but FedMBridge is not evaluated. The decoder ablation does not control for capacity and normalization together. I therefore cannot tell how much the type-specific structure adds over the existing approach with a comparably sized and normalized decoder.

  2. The paper motivates clients that differ in modalities, architectures, and task objectives, but reports results separately for three downstream tasks. It is unclear whether clients with different task objectives ever learn together. The handling of incompatible architectures in the baselines also needs clarification. These choices affect whether the comparison tests the proposed collaboration mechanism under the heterogeneity the paper describes.

  3. The efficiency comparison reports time per round, while the method still exchanges generated parameters and local updates. This leaves open how much communication is needed to reach a given performance level. A comparison at matched accuracy or total communication budget would better support the efficiency claim.

[1] FedMBridge: Bridgeable Multimodal Federated Learning, ICML 2024.

Questions for authors
  1. How does FedMAGE compare with FedMBridge using the same client architectures and training budget?

  2. Does type-aware decoding still help against a shared decoder matched in capacity and normalization?

  3. Are clients with different task objectives trained together in any experiment? How are baselines that assume aligned parameters applied to clients with different architectures?

7 comments
AuthorsOpenReview ↗

Response to Reviewer Yxjf - Part 1

We thank the reviewer Yxjf for taking the time to carefully evaluate our manuscript. Your constructive comments have helped us clarify the methodological contribution and strengthen the experimental evaluation, improving the quality of this paper.

Question 1. Comparison with FedMBridge under the Same Client Architectures and Training Budgets

Thank you for your insightful comments. We have compared our method with FedMBridge [1] and HAFL-GHN [2] (another hypernetwork-based method), and conducted additional comparative experiments using the same client architectures and training budget. We hope these revisions address your concerns.

1.1 Advantages of FedMAGE

The main methodological contribution of FedMAGE is the decoding design in Section 3.2, which combines a shared decoder trunk, operator-specific heads, and type-specific normalization. Three aspects distinguish its parameter-generation design:

  • 1. Shared decoding with operator-specific specialization. HAFL-GHN uses separate MLPs for different layer types, while FedMBridge uses an MLP node decoder conditioned on layer-role and client embeddings. FedMAGE combines a shared decoder trunk with operator-specific heads, Hov(D0(qv))H_{o_v}(D_0(q_v)), allowing different operators to share intermediate decoding features while retaining specialized output mappings. Table 2 (main-text) supports the complete TAD design: Full TAD improves over Shared MLP by 4.0%, 5.8%, and 4.6% on Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1 metrics, respectively.
  • 2. Client conditioning during both architecture encoding and parameter generation. FedMAGE introduces the client embedding into node initialization, zv(0)=ξ(xv,ck)z_v^{(0)}=\xi(x_v,c_k), and parameter-query construction in Eq. (8). Client context can therefore influence both graph message passing and subsequent weight generation. FedMBridge incorporates the client embedding after its layer-role encoding stage, while HAFL-GHN conditions its shared generator on the architecture. FedMAGE thus supports client-dependent graph representations even when clients use the same architecture.
  • 3. Operator-specific normalization within the decoder. FedMAGE applies type-specific normalization to the assembled and trimmed generated parameters before their final reshape, integrating parameter-scale control with shared decoding and specialized output heads. In Table 2 (main-text), Full TAD outperforms “Query + Heads, w/o Norm” by 1.8%, 2.6%, and 2.4% on the three primary metrics, respectively, providing direct evidence for the contribution of normalization within our framework.

Table 2 in main-text provides the following evidence for this design within FedMAGE:

Decoder variantFlickr30K (mR↑)MOSEI (Acc-7↑)NYU-Depth-v2 (Top-1↑)
Shared MLP77.451.077.8
Query + Decoder79.553.479.8
Separate Decoders79.153.079.5
Type-specific Heads78.052.479.1
Query + Heads, without normalization79.654.280.0
Full TAD81.456.882.4

1.2 Comparison with FedMBridge under the Same Experimental Conditions

Following your suggestion, we have completed reimplementations of FedMBridge (including HAFL-GHN) based on the methodological descriptions in their original papers, as the algorithm is not publicly available, so the official implementations are unavailable to us.

All three methods use the same per-client architectures and training budgets. Specifically, we match the client architecture assignments, data partitions, modality assignments, participation schedules, communication rounds, and number of local minibatch updates per participating client. We preserve each method’s original federated optimization procedure. The results are reported in Table R1.

Table R1. Direct comparison with graph-hypernetwork-based federated learning methods under the same client architectures and training budgets. All scores are percentages.

MethodFlickr30K mR ↑CMU-MOSEI Acc-7 ↑CMU-MOSEI Acc-2 ↑CMU-MOSEI F1 ↑NYU-Depth-v2 Top-1 ↑NYU-Depth-v2 F1 ↑
HAFL-GHN (our reimplementation)72.350.882.784.878.276.4
FedMBridge (our reimplementation)78.553.085.487.178.879.2
FedMAGE (ours)81.456.889.590.282.481.0
Gain over FedMBridge (pp)+2.9+3.8+4.1+3.1+3.6+1.8

FedMAGE outperforms both FedMBridge and HAFL-GHN on all six evaluation metrics.

AuthorsOpenReview ↗

Response to Reviewer Yxjf - Part 2

Relative to FedMBridge:

  • Image–text retrieval: Flickr30K mR increases from 78.5% to 81.4%, a 2.9% improvement.
  • Sentiment recognition: CMU-MOSEI Acc-7, Acc-2, and F1 improve by 3.8%, 4.1%, and 3.1%, respectively.
  • Scene recognition: NYU-Depth-v2 Top-1 accuracy increases from 78.8% to 82.4%, while F1 increases from 79.2% to 81.0%, yielding gains of 3.6% and 1.8%.

These comparisons establish higher predictive performance across retrieval, sentiment recognition, and scene recognition under the same client architectures and training budgets. Together with the decoder ablations in Table 2 (main-text), they support the effectiveness of FedMAGE’s structured parameter-generation design.

Question 2. Type-aware Decoding under Matched Capacity and Normalization

Thank you for highlighting the importance of separating the effect of type-aware decoding from decoder capacity and normalization. Even when these factors are matched, TAD retains a distinct structural property: it separates operator-specific output mappings and their direct update paths while preserving a shared decoding trunk.

2.1 Matching Capacity and Normalization Preserves the Distinction in Parameter Sharing

Following the notation in the paper, TAD uses the shared decoder trunk D0D_0, operator-specific heads HovH_{o_v}, and type-specific normalization NovN_{o_v}. Its query is $Q_\psi \left( z_v^{(L)},e_{o_v},e_{s_v},c_k \right)$. For the theoretical comparison, the shared decoder receives the same query information and block-index conditioning. Both designs use the same block assembly and trimming procedure, followed by the same normalization NovN_{o_v}.

We denote the comparison model’s trunk and shared output head by D0sharedD_0^{\mathrm{shared}} and HsharedH_{\mathrm{shared}}, respectively. Let par⁡(M)\operatorname{par}(M) denote the parameter vector of module MM, and let P(M)P(M) count its trainable scalar parameters. Matching the decoder parameter budget means P(D0shared)+P(Hshared)P\left(D_0^{\mathrm{shared}}\right) + P(H_{\mathrm{shared}}). This equality controls the total number of trainable decoder parameters. The remaining distinction is their allocation: TAD assigns separate output-head parameters to different operator types, whereas a shared head permits different types to update overlapping output-mapping parameters. Both models have access to operator information; TAD makes the corresponding parameter routing explicit.

2.2 Type-specific Heads induce Separable Output Mappings in the Server Objective

This distinction follows directly from the server objective in Eq. (12). Using the paper’s residual definition rk(ϕ)=hϕ(Gk,ck)−sg⁡(θkt+Δθkt)r_k(\phi)=h_\phi(G_k,c_k)-\operatorname{sg}\left(\theta_k^t+\Delta\theta_k^t\right). Here, sg⁡(⋅)\operatorname{sg}(\cdot) holds the locally updated target parameters fixed during server optimization. Let rk,v(ϕ)r_{k,v}(\phi) denote the residual block corresponding to parametric node vv. The server objective can then be decomposed by operator type:

Lsrv(ϕ,{ck})=12∑k∈Stpk∥rk(ϕ)∥22=∑o∈OLsrv,o,L_{\mathrm{srv}}(\phi,\{c_k\})=\frac{1}{2}\sum_{k\in S_t}p_k\lVert r_k(\phi)\rVert_2^2=\sum_{o\in\mathcal O}L_{\mathrm{srv},o},

where Lsrv,o=12∑k∈Stpk∑v∈Vkp,ov=o∣∣rk,v(ϕ)∣∣22L_{\mathrm{srv},o}=\frac12\sum_{k\in S_t}p_k\sum_{v\in V_k^p,\\ o_v=o}||r_{k,v}(\phi)||_2^2

Since a node of type oo generates its parameters through HoH_o, we obtain ∇par(Ho)Lsrv=∇par(Ho)Lsrv,o\nabla_{\mathrm{par}(H_o)} L_{\mathrm{srv}} = \nabla_{\mathrm{par}(H_o)} L_{\mathrm{srv},o} , and ∇par⁡(Ho′)Lsrv,o=0\nabla_{\operatorname{par}(H_{o'})}L_{\mathrm{srv},o}=0, for o′≠oo'\neq o. Thus, reconstruction requirements for one operator type do not directly modify another type’s output-head parameters. This dependency structure follows from type-specific routing and remains unchanged when the total parameter budget is matched.

2.3 The Structural Difference Changes Head-level Gradient Interference

Let Lsrv,osharedL_{\mathrm{srv},o}^{\mathrm{shared}} denote the corresponding reconstruction loss for operator type oo in the shared-decoder model. Define go=∇par⁡(Hshared)Lsrv,osharedg_o=\nabla_{\operatorname{par}(H_{\mathrm{shared}})}L_{\mathrm{srv},o}^{\mathrm{shared}} , we examine the contribution of an output-head gradient step while holding all other parameters fixed. For the shared decoder, the update uses ∑ogo\sum_o g_o. To first order, its effect on the reconstruction loss of type oo is:

ΔLsrv,oshared≈−η(go,go)−η∑o′≠o(go,go′)\Delta L_{\mathrm{srv},o}^{\mathrm{shared}} \approx -\eta (g_o,g_o) - \eta \sum_{o'\neq o}(g_o,g_{o'})

When an inner product is negative, the corresponding cross-type term opposes progress on Lsrv,osharedL_{\mathrm{srv},o}^{\mathrm{shared}} [3]. For TAD, the corresponding contribution from updating HoH_o is:

ΔLsrv,o∣Ho≈−η(go,go)\Delta L_{srv,o}|_{H_o} \approx -\eta (g_o,g_o).

AuthorsOpenReview ↗

Response to Reviewer Yxjf - Part 3

The direct cross-type terms are absent from this head update. Cross-type sharing continues through the graph encoder and shared trunk D0D_0, while operators of the same type across clients continue to share HoH_o. TAD therefore introduces specialization at the output mapping while retaining collaboration in the shared representation.

2.4 Identical Normalization Preserves This Routing Property

Consistent with Eq. (9) and our implementation clarification, NovN_{o_v} is applied after block concatenation and trimming, before the final reshape. Applying the same NovN_{o_v} in both designs can modify the gradients passing through each generated tensor, but it does not change which output-head parameters that tensor depends on. The zero cross-head derivatives above therefore remain valid with identical normalization.

A shared decoder supplied with the operator information in qvq_v can also learn type-dependent mappings. TAD expresses this specialization directly through HovH_{o_v}, making the sharing pattern available throughout optimization. Its theoretical motivation is strongest when different operator types require distinct final mappings and their updates would otherwise interfere, while their intermediate decoding features remain useful to share through D0D_0.

Overall, the resulting contribution of TAD is an explicit inductive bias for allocating capacity and controlling direct cross-type interference at the output heads, which remains present after matching parameter count and normalization.

Question 3. Task Heterogeneity and the Adaptation of Parameter-alignment Baselines

Thank you for highlighting the distinction between evaluating different tasks separately and training them within the same federation, and for requesting clarification of the baseline adaptations. At submission, the three tasks were evaluated independently. We have since completed an additional experiment in which clients performing all three tasks jointly train one FedMAGE hypernetwork. Baselines requiring parameter alignment perform model aggregation within compatible client groups, as detailed below.

3.1 Scope of the Original Experiments

In the submitted evaluation, each benchmark uses a separate federation and an independently trained hypernetwork. Clients within each federation perform the same downstream task while adopting heterogeneous network architectures. Accordingly, Tables 1 and 3 (main-text) evaluate architectural heterogeneity within individual tasks, rather than task heterogeneity within one federation.

3.2 Task Heterogeneity Experiment

The additional experiment includes 60 clients participating in every communication round: 20 perform image–text retrieval on Flickr30K, 20 perform sentiment recognition on CMU-MOSEI, and 20 perform scene recognition on NYU-Depth-v2.

All 60 clients share one FedMAGE hypernetwork. Updates from all three tasks jointly optimize its shared parameters. Each client retains its architecture graph, client embedding, task-specific output structure, and local training objective. Thus, the experiment directly evaluates parameter generation for heterogeneous task objectives and architectures within a single federation.

3.3 Applying Parameter-alignment Baselines to Heterogeneous Architectures

For baselines requiring aligned parameters, we partition clients into groups according to the compatibility of the parameters being aggregated. Each group maintains its own aggregated model and independently performs parameter uploading, aggregation, and downloading according to the baseline’s original update rule. In Table R2, this applies to FedAvg and Ditto, as well as the model-parameter aggregation components of CAR-MFL and PmcmFL.

Methods supporting heterogeneous architectures, including CreamFL, HAFL-GHN, and FedMBridge, retain their respective mechanisms for sharing across architectures. All methods are evaluated on the same client population, data partitions, architecture assignments, communication rounds, and per-client local training budgets.

3.4 Evaluation Metrics

We compute Top-1 accuracy and F1 on each client’s test set and then average the client-level scores equally. With equal client counts, the overall score also equals the arithmetic mean of the three task-specific scores. For Flickr30K, Top-1 accuracy is the average of image-to-text and text-to-image Recall@1. For CMU-MOSEI, it denotes binary sentiment classification accuracy (Acc-2) over positive and negative samples, excluding neutral samples. For NYU-Depth-v2, it denotes scene classification accuracy.

Reviewer_LwKUOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FedMAGE, a graph hypernetwork framework for heterogeneous multimodal federated learning. It represents client architectures as computational graphs and generates personalized model parameters through a shared hypernetwork. A Type-aware Decoding module is introduced to better handle different operator types.

Experiments on image-text retrieval, sentiment recognition, and scene recognition demonstrate improved performance over several federated learning baselines. The authors also evaluate communication efficiency, architectural heterogeneity, and generalization to unseen architectures.

Critical strengths
  1. Using graph hypernetworks to support clients with different architectures is a promising alternative to conventional parameter aggregation.

  2. FedMAGE performs consistently well across three multimodal tasks, and the ablation study in Table 2 provides useful evidence for the effectiveness of Type-aware Decoding.

Critical weaknesses
  1. The paper discusses existing graph hypernetwork approaches, including HAFL-GHN and FedMBridge, but does not include them in the main experiments. Since architecture-conditioned parameter generation is central to FedMAGE, comparisons with these methods would better establish its novelty and advantages.

  2. Table 1 mainly reports communication time per round, while FedMAGE still requires downloading full model parameters and uploading parameter updates. The authors should clarify the total communication volume and whether server-side parameter generation overhead is included in the reported efficiency results.

Questions for authors
  1. Why are existing graph hypernetwork-based FL methods, particularly HAFL-GHN and FedMBridge, not included in the experimental comparison? What are the main advantages of FedMAGE over these approaches?

  2. Does the reported communication time include server-side hypernetwork computation? Could the authors clarify the total parameter transmission cost per round?

  3. Are the three tasks trained independently, or does the evaluation include clients performing different tasks within the same federated system? This distinction seems important for the claimed task heterogeneity.

  4. In Eq. (9), type-specific decoder heads generate parameters for operators with potentially different tensor shapes. However, a fixed decoder head cannot directly produce arbitrary output dimensions. The authors should clarify how variable-sized parameter tensors are generated and whether padding, chunking, or shape-specific decoding is employed.

  5. FedMAGE aggregates client update directions into a shared hypernetwork through Eq. (12)–(14). However, clients performing different tasks may produce conflicting update directions. The authors should clarify whether the client embeddings and type-aware decoding are sufficient to mitigate negative transfer, or whether additional mechanisms are needed to handle task conflicts.

8 comments
AuthorsOpenReview ↗

Response to Reviewer LwKU - Part 1

We appreciate Reviewer LwKU for the time and effort in reviewing our paper and offering constructive feedback. Thank you for your careful review and technically insightful questions. We appreciate your recognition of the potential of graph hypernetworks for heterogeneous client architectures and of the consistent results across our three benchmarks. We highly value your valuable feedback. Please find our responses to the comments below.

Question 1. Advantages of FedMAGE and Experimental Comparison

Your question is highly pertinent. To address it, we elaborate on the advantages of our method below and supplement comparative experiments with HAFL-GHN [1] and FedMBridge [2].

1.1 Advantages of FedMAGE

The main methodological contribution of FedMAGE is the decoding design in Section 3.2.

Its parameter query combines the graph representation, operator type, target shape, and client embedding in Eq. (8). A shared trunk then maps this query to generation features, while type-specific heads and normalization produce the parameters in Eq. (9).

This design shares the intermediate generation mapping across operators while specializing the output mapping for their different parameter structures and scales.

The comparison with the two prior methods involves different aspect:

1. Shared decoding with operator-specific specialization. HAFL-GHN maps graph representations to weights through separate MLPs for different layer types, while FedMBridge uses an MLP node decoder conditioned on layer-role and client embeddings. FedMAGE introduces a shared decoder trunk followed by operator-specific heads, Hov(D0(qv))H_{o_v}(D_0(q_v)) , as described in Eq. (9) of main-text. Relative to HAFL-GHN, this introduces sharing across operator types within the decoder itself. Relative to FedMBridge, it introduces explicitly specialized output mappings alongside the operator information encoded in the graph. This structure allows modality encoders, fusion modules, and prediction heads to share intermediate decoding features while retaining mappings suited to their different parameter requirements. Our current ablations support the complete TAD design: compared with the Shared MLP variant, Full TAD improves Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1 accuracy by 4.0%, 5.8%, and 4.6%, respectively. It also exceeds the Separate Decoders variant by 2.3%, 3.8%, and 2.9%, respectively.

2. Client conditioning throughout architecture encoding and weight generation. FedMAGE incorporates the client embedding at two stages: node initialization, zv(0)=ξ(xv,ck)z_v^{(0)}=\xi(x_v,c_k), and parameter-query construction, qv=Qψ(zv(L),eov,esv,ck)q_v=Q_\psi(z_v^{(L)},e_{o_v},e_{s_v},c_k) . Thus, client information can influence both the propagation of architectural context and the subsequent generation of parameters. This differs from FedMBridge’s formulation, where the architecture first determines the layer-role representations and the client embedding is subsequently combined with those representations during decoding. HAFL-GHN’s shared generator is conditioned on the architecture, whereas FedMAGE additionally maintains an explicit, learned client embedding. The resulting capability is client-dependent graph representation learning: even when two clients use the same architecture, their graph representations can adapt to their respective local objectives and data through ckc_k. This provides a mechanism for personalization before the final weight-decoding stage.

3. Operator-specific normalization of generated parameters. FedMAGE applies the type-specific normalization after the corresponding output head, integrating control of generated parameter statistics into the shared-trunk and specialized-head decoder. This addresses the different parameter statistics associated with modality encoders, fusion operators, and prediction layers. Table 2 in main-text directly evaluates this component. Full TAD achieves 81.4%, 56.8%, and 82.4% on the three primary metrics, compared with 79.6%, 54.2%, and 80.0% for “Query + Heads, w/o Norm.” Adding normalization therefore improves the corresponding scores by 1.8%, 2.6%, and 2.4% within our framework.

Table 2 in main-text provides the following evidence for this design within FedMAGE:

Decoder variantFlickr30K (mR↑)MOSEI (Acc-7↑)NYU-Depth-v2 (Top-1↑)
Shared MLP77.451.077.8
Query + Decoder79.553.479.8
Separate Decoders79.153.079.5
Type-specific Heads78.052.479.1
Query + Heads, without normalization79.654.280.0
Full TAD81.456.882.4
AuthorsOpenReview ↗

Response to Reviewer LwKU - Part 2

1.2 Experimental Comparison with HAFL-GHN and FedMBridge

HAFL-GHN and FedMBridge are directly relevant baselines for evaluating architecture-conditioned parameter generation. They are omitted from the submitted experiments because official implementations are unavailable to us and we had not completed our own reproductions when the manuscript was submitted.

Following your suggestion, we have reproduced methods HAFL-GHN and FedMBridge from the methodological descriptions in their original papers, and compare them with FedMAGE on all three benchmarks. To evaluate these design choices against the complete prior methods, the additional comparison will use the same data partitions, modality assignments, participation schedules, communication rounds, and local minibatch-update budgets. Besides, we also preserve each method’s original federated optimization procedure. The results are shown in Table R1 below.

Table R1. Direct comparison with graph-hypernetwork-based federated learning methods.

MethodFlickr30K mR ↑CMU-MOSEI Acc-7 ↑CMU-MOSEI Acc-2 ↑CMU-MOSEI F1 ↑NYU-Depth-v2 Top-1 ↑NYU-Depth-v2 F1 ↑
HAFL-GHN (our reimplementation)72.350.882.784.878.276.4
FedMBridge (our reimplementation)78.553.085.487.178.879.2
FedMAGE (ours)81.456.889.590.282.481.0

It can be observed from Table R1 that our method achieves better performance than HAFL-GHN and FedMBridge. These direct comparisons provide empirical evidence that FedMAGE improves upon both graph-hypernetwork-based FL and architecture-personalized multimodal FL baselines in our benchmark settings. Together with the decoder ablations in Table 2 (main-text), the results support the effectiveness of FedMAGE’s structured parameter-generation design.

Question 2. Communication Time and Overhead

Thank you for highlighting this issue. We clarify the timing scope and provide explicit communication accounting below.

2.1 The scope of Communication Time Measurement

Yes. The reported CoT includes server-side hypernetwork computation. CoT includes parameter transmission, server-side generation of client model parameters, and the backward computation and updates of the hypernetwork. Therefore, the reported values of 16.3, 25.5, and 30.2 seconds per round on Flickr30K, CMU-MOSEI, and NYU-Depth-v2 already include this server-side overhead. We will label this metric as “communication and server computation time” and make its scope explicit in the metric definition and Table 1 (main-text).

2.2 Communication Overhead per Round

FedMAGE exchanges a complete client model and a dense parameter update for each participating client in every federated round:

  • Downlink: The server generates the client-specific parameters θkt=hϕt(Gk,ckt)\theta_k^t=h_{\phi^t}(G_k,c_k^t) and sends them to client kk.
  • Uplink: After local training, client kk sends the parameter displacement Δθkt=θ~kt−θkt\Delta\theta_k^t=\widetilde{\theta}_k^t-\theta_k^t to the server. This dense update has the same number of elements as the downloaded model.

The shared hypernetwork parameters and client embeddings are maintained and updated on the server. Architecture graphs are registered before training and retransmitted only when an architecture changes; their metadata transmission is accounted for separately from model parameters.

The total parameter transmission cost per round, summed over all participating clients and both communication directions, is:

\qquad \mathrm{M/round}.$$ Here, $P_{k,t}^{\mathrm{down}}$ and $P_{k,t}^{\mathrm{up}}$ denote the numbers of scalar parameters downloaded and uploaded by client $k$, respectively, and $S_t$ is the set of participating clients. Thus, each participating client contributes one full model download and one equally sized update upload.
AuthorsOpenReview ↗

Response to Reviewer LwKU - Part 3

Question 3. Evaluating Task Heterogeneity within a Single Federation

Thank you for highlighting the distinction between evaluating multiple tasks independently and training clients with different task objectives in the same federation. This distinction is essential for interpreting our experimental evidence.

3.1 Scope of the Original Experiments

The three tasks in the original evaluation are trained independently, with a separate federation and an independently trained hypernetwork for each benchmark. Within each federation, clients perform the same downstream task while using heterogeneous network architectures. Accordingly, the results in Tables 1 and 3 evaluate architectural heterogeneity within individual tasks, they do not evaluate task heterogeneity within a single federation.

3.2 Additional Evaluation with Heterogeneous Tasks in One Federation

Evaluation with heterogeneous tasks. To directly evaluate task heterogeneity, we are adding an experiment with 60 clients participating in every communication round: 20 clients perform image–text retrieval (Flickr30K), 20 perform sentiment recognition (CMU-MOSEI), and 20 perform scene recognition (NYU-Depth-v2). All 60 clients share one FedMAGE hypernetwork, whose parameters are jointly updated using the local updates from all three tasks. Each client retains its own architecture graph, client embedding, task-specific output structure, and local training objective. Therefore, the shared hypernetwork learns to generate client-specific parameters from the combined training signals of retrieval, sentiment recognition, and scene recognition.

Adaptation of parameter-alignment baselines. For methods that require aligned parameters, such as FedAvg [3]、MOON [4] and Ditto [5], we partition clients into groups with compatible architectures. Compatibility is determined by the complete parameter layout, including task heads. Each group independently performs parameter uploading, aggregation, and downloading according to the baseline’s original update rule. Methods that support heterogeneous architectures retain their respective sharing mechanisms. All methods use the same client data partitions, architecture assignments, communication rounds, and local training budgets.

Evaluation metrics. We compute Top-1 accuracy and F1-score on each client’s own test set and then take an equally weighted average across clients. For each task, we also report the corresponding averages over its 20 clients. Because each task has the same number of clients, each overall score equals the arithmetic mean of the three task-level scores. For Flickr30K task. We define Top-1 accuracy as the average of image-to-text and text-to-image Recall@1. For CMU-MOSEI task, Top-1 accuracy refers to binary sentiment classification accuracy (Acc-2) over positive and negative samples.

Results. Table R2 presents both task-specific and overall performance. FedMAGE achieves the highest performance on all six task-specific metrics, with an Overall Top-1 accuracy of 66.7% and Overall F1 of 67.5%. These exceed FedMBridge, the strongest baseline on both overall metrics, by 3.4% and 3.7%, respectively.

Table R2. Evaluation of task heterogeneity.

MethodFlickr30K Top-1 ↑Flickr30K F1 ↑CMU-MOSEI Top-1 ↑CMU-MOSEI F1 ↑NYU-Depth-v2 Top-1 ↑NYU-Depth-v2 F1 ↑Overall Top-1 ↑Overall F1 ↑
FedAvg43.245.665.164.258.859.555.756.4
Ditto47.346.868.868.562.463.059.559.4
CreamFL48.349.172.373.662.661.861.161.5
CAR-MFL47.747.873.672.863.165.461.562.0
PmcmFL47.546.974.274.862.762.061.561.2
HAFL-GHN (our reimplementation)47.047.270.472.161.862.459.760.6
FedMBridge (our reimplementation)49.350.575.074.665.566.263.363.8
FedMAGE (ours)52.454.377.378.570.469.666.767.5

These results provide direct empirical support for FedMAGE’s ability to accommodate heterogeneous task objectives and client architectures within a single federation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and ACs,

We sincerely thank all reviewers for their thoughtful evaluations, constructive suggestions, and continued engagement throughout the discussion. Your feedback has helped us substantially strengthen the experimental evidence and clarify the contributions of FedMAGE.

Guided by your suggestions, we have made several important improvements to the revised manuscript, summarized below.

  • Clarified the methodological contribution and strengthened comparisons with closely related methods. We clarified FedMAGE’s methodological distinctions and added comparisons under the same architectures and training budgets. Gains over FedMBridge are 2.9%, 3.8%, and 3.6% on Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1, respectively (Section 4.2, Table 1; Appendix A).
  • Strengthened the theoretical explanation of type-aware decoding and its optimization behavior. We analyzed operator-specific gradient routing under matched capacity and normalization. Existing ablations show gains over Shared MLP of 3.6%, 5.6%, and 4.5% on Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1, respectively (Section 4.4, Table 3; Appendix B.4).
  • Added experimental evidence for task heterogeneity within a single federation. We added task heterogeneity experiments using one shared hypernetwork and clarified baseline adaptations. FedMAGE achieves 66.7% Overall Top-1 and 67.5% Overall F1, exceeding the strongest baseline by 3.4% and 3.7%, respectively (Section 4.3, Table 2; Appendix B.7).
  • Made parameter generation and computational reporting more explicit and reproducible. We specified block-index conditioning, concatenation, trimming, normalization, and reshaping. (Section 4.1, Table 1; Appendices B.2–B.3).
  • Defined the architecture-generalization setting and the interpretation of its results precisely. We clarified evaluation with frozen generators and client embeddings, without fine-tuning. Flickr30K mR, CMU-MOSEI Acc-7, and NYU-Depth-v2 Top-1 reach 59.5%, 35.6%, and 60.2% for CNN-3→CNN-5, and 56.4%, 34.0%, and 64.3% for ViT-3→ViT-5, supporting within-family depth generalization (Section 4.5, Table 5; Appendix B.6).

We sincerely appreciate the time and care that all reviewers have devoted to our work. Your insightful suggestions have motivated meaningful improvements to both our experimental analysis and the revised manuscript. We believe these additions have made the work substantially clearer, more rigorous, and more convincing.

Thank you once again for your valuable guidance, support, and encouragement.

Warm regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.