acceptodds
Under review as a conference paper at ICLR 2027

MERIDIAN: Bridging Linguistic Heterogeneity in Federated Vision-Language Learning

Abstract

Federated Vision-Language Models (FedVLMs) allow for distributed clients to collaboratively fine-tune a pretrained model like CLIP while preserving privacy. Current systems (PromptFL, FedPGP, FedOTP, FedPHA, pFedMoAP) provide good personalization in single-language settings but fail to consider an important real-world axis of heterogeneity: linguistic diversity. Standard federated text-adapter aggregation mixes morphologically different language signals when clients communicate in languages other than high-resource languages like English or Chinese and low-resource languages like Bengali or Tamil, thus hurting both personalization and generalization. This introduces MERIDIAN: the first FedVLM framework for explicitly dealing with multilingual heterogeneity. MERIDIAN adds three novel components: (i) a Language-Family Text Adapter Bank (LF-TAB) with cross-lingual family-clustered FedAvg aggregation, which avoids cross-family gradient interference; (ii) a Personalized Per-Language Text Adapter, kept locally at each client to capture language-specific idiosyncrasies of the text; and (iii) a Cross-Lingual Anchor Loss (L_CLAL) that ensures consistency of the visual representations with the pivot-language Text Embedding, enforcing cross-lingual semantic consistency. After extensive experiments across 10 benchmark datasets, 8 baselines, and 10 languages from 6 language families, MERIDIAN obtains an average H-Mean of 83.73% (+7.75 average gain over the best previous approach, pFedMoAP) and gains an average of +1.96 H-Mean on low-resource language clients (Bengali, Swahili, Tamil) and a peak per-family gain of +2.31 H-Mean on the Indic-Bengali language family.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-Lingual Representation Learning, Multilingual Learning, Low-Resource Languages, Semantic Alignment, Language-Aware Modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xCtMOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

To address the issue of update conflicts arising from direct aggregation of text adaptation parameters in federated vision-language learning—where different clients utilize different languages—the authors propose MERIDIAN. This framework balances knowledge sharing and client-specific personalization through language-cluster-specific shared text adapters, locally retained personalized language adapters, and a cross-language anchoring loss. The anchoring loss leverages category descriptions in English, Chinese, and Arabic to constrain image representations so that they correspond to the same category across different languages. While I believe this work offers an insightful approach to language-aware sharing, the experimental attribution, result consistency, and implementation details in the current manuscript require further refinement.

Critical strengths
  1. The authors analyze the sources of heterogeneity in federated vision-language learning and complement existing research by incorporating perspectives centered on differences in categories and image domains.

  2. The authors substantiate their arguments through comprehensive, multi-dimensional experiments, including comparisons with backbone baselines, component ablation studies, gradient analysis, evaluations on low-resource languages, and scalability tests regarding client size.

Critical weaknesses
  1. The contribution of language grouping has not been fully validated; among the six default language clusters, four contain only a single client, meaning their "family adapters" effectively function as local parameters. The study lacks control conditions involving random grouping, data-driven grouping, or parameter-count matching.

  2. The author's definitions regarding cross-lingual and low-resource evaluation lack clarity, and it is not specified how family/local adapters are utilized for the target training languages; furthermore, the evaluation is limited to merely changing the language of the class templates, failing to fully assess performance on realistic multilingual image-text tasks.

Questions for authors
  1. How are the semantic router, bidirectional distillation, and other illustrated components implemented? If they are used, why does the full training objective not include corresponding terms for them?

  2. Which set of parameters does the gradient conflict analysis target, and how are data discrepancies accounted for?

  3. The author has been somewhat careless with the figures; I hope the author revises them, otherwise I will lower my rating.

5 comments

We sincerely thank the reviewer for recognizing MERIDIAN's motivation and comprehensive experimental evaluation. We appreciate the constructive feedback regarding language-family grouping, cross-lingual evaluation, architectural consistency, gradient analysis, and presentation. We address each concern below.

W1.

We agree that four of the six language families in the original 10-client setting contain only one client. Consequently, their family-specific adapters function as local adapters, without within-family aggregation. Nevertheless, these clients continue to benefit from globally shared vision and text adapters and cross-lingual semantic alignment.

To examine whether linguistic grouping provides benefits beyond arbitrary clustering, we conducted additional comparisons.

Table: Effect of clustering strategy (H-Mean %).

StrategyAverageTamil
No clustering81.1076.10
Random clustering81.5076.85
Gradient-based clustering82.7078.60
Script-based clustering82.9079.44
MERIDIAN grouping83.7381.76

MERIDIAN outperforms random clustering by 2.23 percentage points, gradient-based clustering by 1.03 points, and script-based clustering by 0.83 points. These results support the value of linguistically informed grouping rather than arbitrary client partitioning.

We further compared alternative adapter configurations under approximately matched trainable-parameter budgets.

Table: Adapter configuration comparison (H-Mean %).

ConfigurationH-Mean
Single shared adapter79.80
Global + local adapters81.10
Random clustered adapters81.50
MERIDIAN83.73

MERIDIAN achieves the highest performance, suggesting that its improvements are not attributable solely to increased adaptation capacity.

Importantly, we do not interpret improvements in singleton families as direct evidence of within-family aggregation. These gains may also result from globally shared representations and CLAL. We will explicitly clarify this distinction in the revised manuscript.

The larger-client experiments (original Tables 13–14) provide complementary evidence. MERIDIAN achieves H-Mean improvements of 7.75, 8.52, and 10.70 percentage points for 10, 20, and 30 clients, respectively, relative to the corresponding reference baselines.

Together, these results support language-aware adaptation while acknowledging the limitations of singleton families.

Together, these results support language-aware adaptation while acknowledging the limitations of singleton families.

W2.

We agree that the evaluation terminology and scope require clearer definitions.

MERIDIAN is primarily evaluated in multilingual class-description-based vision-language classification. The original manuscript includes language-specific and low-resource evaluations (Table 7) and cross-language evaluations (Table 12).

We distinguish three settings:

  1. Training-language evaluation: evaluation on languages represented during federated adaptation.
  2. Cross-language template evaluation: evaluation using class descriptions expressed in different languages.
  3. Low-resource adaptation: evaluation with limited language-specific training examples.

These settings should not be conflated with unrestricted multilingual image-text understanding.

Regarding adapter utilization, globally shared vision and text adapters are aggregated across clients, family-specific text adapters are aggregated only within language families, and personalized adapters remain client-local. CLAL encourages alignment of multilingual class semantics through shared representations.

In the revised manuscript, we will explicitly specify the adapter pathway and language-specific parameters used in each evaluation setting, distinguishing previously trained languages from genuinely unseen target languages.

We also acknowledge that class-template-based evaluation does not constitute a comprehensive assessment of multilingual image-text retrieval or free-form language understanding. Accordingly, our claims are restricted to the evaluated multilingual classification setting.

Q1.

We thank the reviewer for identifying the insufficient correspondence between Figure 2 and the original mathematical formulation.

As also discussed in our response to Reviewer gPCX, the original Section 3 did not adequately explain several components illustrated in Figure 2, including the semantic router, shared bottleneck, semantic anchor bank, vision-to-text bridge, and bidirectional distillation pathways.

The vision-to-text bridge is defined as

Ψv:Rdv→Rdw,\Psi_v : \mathbb{R}^{d_v} \rightarrow \mathbb{R}^{d_w},

mapping visual features into the shared semantic representation space.

The general and personalized representations are combined as

RFus=(1−γ)RGen+γRPer,γ=0.09.R^{Fus} = (1-\gamma) R^{Gen} + \gamma R^{Per}, \qquad \gamma = 0.09 .

Here, RGenR^{Gen} is obtained without client-specific projection adaptation, whereas RPerR^{Per} incorporates a client-local projection adapter. Their fusion balances shared cross-modal representation learning with personalization.

We will expand Section 3 and Appendix C to explicitly define the roles and implementation of the semantic router, shared bottleneck, semantic anchors, and distillation pathways. We will also revise the Figure 2 caption to establish direct correspondence between the illustrated components and their mathematical definitions.

Crucially, we agree that every independently optimized distillation term must appear in the complete training objective. We will ensure that the revised formulation distinguishes actual optimization losses from architectural operations that introduce no separate loss, thereby aligning the figure, equations, and implementation description.

Q2. We appreciate the request for greater methodological precision.

Our gradient analysis examines the compatibility of client updates in language-sensitive adaptation components. Original Figure 1 reports within-family gradient cosine similarities of approximately +0.62+0.62 to +0.73+0.73, averaging +0.66+0.66, whereas some cross-family comparisons are negative, including −0.42-0.42 for CJK–Arabic.

These observations motivate language-aware aggregation but do not establish that linguistic differences are the sole cause of gradient conflict. We therefore consider two complementary analyses from the original manuscript.

Data heterogeneity. Table 6 shows that MERIDIAN retains a 4.65-percentage-point improvement under the near-IID setting (α=10\alpha = 10). This suggests that its performance advantage is not limited to strongly heterogeneous client distributions.

Linguistic characteristics. Table 5 reports positive associations between performance improvements and morphological complexity (r=0.74r = 0.74, p<0.01p < 0.01) and tokenization fertility (r=0.66r = 0.66, p<0.05p < 0.05).

Together, these findings support the relevance of linguistic heterogeneity while avoiding the unsupported causal claim that language differences alone explain gradient interference.

To improve reproducibility, we will revise the manuscript to explicitly identify the gradient-bearing adapter parameter subset, client-pair construction, measurement stage, and cosine-similarity aggregation procedure. We will also clarify the distinction between gradient-conflict measurements and clustering-performance comparisons.

Q3.

We thank the reviewer for identifying the presentation inconsistencies and agree that architectural figures must accurately reflect the implemented method.

We have identified the unintended red vector artifact in Figure 2, unresolved table references, citation inconsistencies, and insufficient correspondence between architectural components and their mathematical descriptions.

We have uploaded a revised version of the manuscript containing a corrected Figure 2. Because figures cannot be embedded in OpenReview comments, we describe the revision here and refer the reviewer to the updated PDF.

The revised figure makes the following explicit:

  1. Complete component coverage. Every module referenced in Section 3 now appears with a matching label, including the semantic router, the semantic anchor bank {aj}\{a_j\}, the vision-to-text bridge Ψv\Psi_v, the shared bottleneck UU, and the frozen projections πvis\pi_{vis} and πtxt\pi_{txt}.
  2. Explicit distillation pathways. The two distillation terms are shown separately with their operative forms, KL(σvis ∥ σtxt)\mathrm{KL}(\sigma_{vis} \,\|\, \sigma_{txt}) for the V→T direction and a Wasserstein distance W(Dvis,Dtxt)W(D_{vis}, D_{txt}) for the T→V direction, together with the CLAL block and its EN/ZH/AR pivot languages.
  3. Dual-pathway fork. The general and fused representations rgenr_{gen} and rfusr_{fus} are shown at the point of the convex combination, immediately before the cosine-similarity scoring head rfus⋅tfinal/τr_{fus} \cdot t_{final} / \tau.
  4. Objective consistency. All five optimized terms — LAnchL_{Anch}, LFusL_{Fus}, LCLALL_{CLAL}, LV→TL_{V \rightarrow T}, and LT→VL_{T \rightarrow V} — are drawn as explicit contributors to LTotalL_{Total}, so that the figure, the equations, and the implementation agree. Architectural operations that introduce no separate loss are no longer depicted in a way that could suggest an independent objective term.
  5. Federated parameter scope. A dedicated row distinguishes globally aggregated parameters (vision adapters, anchors, UU, projections), per-family aggregated text adapters across KK families, and parameters that never leave the client (router, personalized projection, personalized text adapter). A legend encodes frozen, aggregatable, and local-only components, along with residual connections and routing paths.

The unintended visual artifact has been removed, and the affected table references and citations have been corrected. We have also expanded the Figure 2 caption to associate each illustrated module with its defining equation, and we clarify the general, personalized, and fused representation pathways to ensure consistency between the architectural illustration, the optimization objective, and the implementation.

These corrections concern presentation and methodological clarity and do not alter any reported experimental result.

We thank the reviewer again for these constructive comments, which have helped us sharpen both the methodological description and the scope of our claims. We are happy to provide any further clarification during the discussion period.

Reviewer_gPCXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The MERIDIAN framework was proposed to address the linguistic heterogeneity challenge in Federated Vision-Language Models, where naive federated averaging across morphologically diverse languages typically causes severe cross-family gradient interference and suppresses gradients from low-resource clients. To resolve this issue, the system decouples the text encoding pathway by introducing a Language-Family Text Adapter Bank, which strictly restricts parameter aggregation to clients within the same linguistic family. This mechanism is paired with a Personalized Per-Language Text Adapter that is retained locally on each client to capture idiosyncratic script features. Furthermore, MERIDIAN employs a Cross-Lingual Anchor Loss that forces the general visual representations to align with pivot language text embeddings. Together, this decoupled architecture effectively eliminates cross-domain gradient conflicts, achieves language-agnostic cross-lingual generalization, and significantly improves adaptation performance for low-resource languages.

Critical strengths

1.Rather than relying on computationally heavy algorithms to resolve gradient cancellation, MERIDIAN introduces a hard-routing mechanism known as the Language-Family Text Adapter Bank. By limiting Federated Averaging exclusively to clients within identical linguistic families, it structurally eliminates the blending of conflicting gradients, thereby addressing the issue at its fundamental source.

2.The framework leverages the Cross-Lingual Anchor Loss to align visual features with high-resource pivot languages, establishing a highly standardized visual benchmark for low-data languages. Instead of optimizing in an unconstrained feature space, low-resource clients can efficiently anchor their limited data to this robust visual baseline, which effectively prevents feature collapse and delivers substantial gains in both zero-shot and few-shot scenarios.

3.To navigate the trade-off between generalization and personalization, the framework adopts a dual-pathway architecture. Feature extraction is split into two distinct routes: one generates a general representation for global anchor alignment to maintain broad generalization, while the other utilizes a local adapter to produce a personalized representation tailored to local data distributions. This structural decoupling ensures that global and local optimization objectives do not interfere with one another.

Critical weaknesses

1.The "V->T bridge" module appears in the figure without a detailed explanation in the text, and the unclear representation of key variables fails to intuitively reflect the dual-pathway decoupling design.

2.Requiring clients to disclose their language family to the server for adapter routing compromises the strict privacy guarantees of federated learning.

3.Hard-routing prevents negative transfer but blocks cross-family knowledge sharing. This isolates minority languages, leading to severe local overfitting and contradicting federated learning's goal of improving generalization through aggregation.

4.There is no justification for choosing English, Chinese, and Arabic as pivot languages. An ablation study is recommended to test the alignment mechanism's sensitivity to anchor language selection.

Questions for authors

1.Please thoroughly revise Figure 2 to ensure strict consistency with the methodology text.

2.Add a brief discussion addressing the privacy concerns associated with clients reporting their language family to the server.

3.Please explicitly discuss the risk of overfitting for minority language families caused by the strict physical isolation of the Language-Family Text Adapter Bank.

4.Please add an ablation study to assess how different pivot language combinations and varying their total number impact the cross-lingual alignment and final H-Mean performance. This will help justify the specific choice of English, Chinese, and Arabic.

5 comments

We thank the reviewer for the detailed and constructive comments.

Q1: We thank the reviewer for this observation, which precisely identifies the root cause. Figure 2 depicts the architecture as implemented, whereas Section 3 describes only a subset of its components, leaving the V→T bridge and several other modules visible in the diagram but insufficiently explained in the text.

In the revised manuscript, we will explicitly define every module shown in Figure 2. We will introduce a new subsection on the vision pathway to describe the bridge Ψv:Rdv→Rdw\Psi_v : \mathbb{R}^{d_v} \rightarrow \mathbb{R}^{d_w}. We will clarify that the vision and text towers were pretrained separately and therefore require an explicit cross-modal mapping to express both modalities in a shared semantic space. We will also explain why the bridge is placed before the dual-head fork, ensuring that both heads operate within a common reference frame.

Furthermore, we will provide explicit definitions of the semantic router, shared bottleneck UU, semantic anchor bank, and the two distillation terms. We will also present the complete training objective.

To clarify the dual-pathway decoupling, we will revise Section 3.2 to explicitly define how RGenR^{Gen}, RPerR^{Per}, and RFusR^{Fus} are constructed. Specifically, RGenR^{Gen} will be defined using only the frozen projection, while RPerR^{Per} will be obtained by applying the client-local projection adapter residually before the same frozen projection. Their fusion will be expressed as:

RFus=(1−γ)RGen+γRPer,γ=0.09.R^{Fus} = (1-\gamma) R^{Gen} + \gamma R^{Per}, \qquad \gamma = 0.09 .

We will further clarify that RGenR^{Gen} contains no client-specific parameters, making it the appropriate representation for cross-lingual and cross-modal alignment objectives.

To ensure that the correspondence between the architecture and its mathematical formulation is directly verifiable, we will revise the caption of Figure 2 to associate every module with its defining equation. We will also clarify that the "r_gen / r_fus = blend" box represents the dual-pathway fork and its convex combination, displayed in a single column for compactness.

Finally, we will include the detailed design rationale and the complete figure legend in a new Appendix C. Figure 2 will removing the unintended vector artifact identified in the reviewer's first comment.

Q2: We thank the reviewer for highlighting this important privacy concern. Although MERIDIAN keeps raw training data local, we acknowledge that reporting language-family identifiers to the server may expose sensitive linguistic metadata. With six language-family groups, a categorical identifier can reveal at most log⁡26≈2.59\log_2 6 \approx 2.59 bits of information. Even such limited disclosure may be sensitive, particularly for clients belonging to minority-language groups.

To investigate whether explicit family disclosure is necessary, we additionally evaluated alternative routing strategies, including gradient-similarity clustering, which does not require clients to explicitly declare their language family.

Table: Comparison of language-family routing strategies (H-Mean %).

Routing StrategyAvg. H-MeanTamil
No clustering81.1076.10
Random clustering81.5076.85
Gradient-similarity clustering82.7078.60
Script-based clustering82.9079.44
Explicit family routing (MERIDIAN)83.7381.76

As shown in Table R9, gradient-similarity clustering achieves 82.70% average H-Mean, only 1.03 percentage points below explicit family routing (83.73%), without requiring direct family declarations. However, its lower Tamil performance (78.60% versus 81.76%) indicates that avoiding explicit metadata disclosure may come at a greater utility cost for individual languages.

Importantly, gradient-based clustering may itself reveal information through model updates; therefore, it should not be interpreted as providing a formal privacy guarantee. We will expand the discussion in Section 3.3 to acknowledge this limitation and identify privacy-aware routing and secure aggregation as potential future directions.

Q3: Thank you for raising this concern. We agree that completely isolating minority-language clients could limit knowledge transfer and increase the risk of overfitting. However, MERIDIAN does not isolate language families from the overall federated learning process.

An important distinction is that hard routing applies only to the language-family adapter, not to the entire model. MERIDIAN preserves cross-family knowledge sharing through:

  • Global vision adapters, aggregated across all clients regardless of language family.
  • Global text adapters, also aggregated across all clients.
  • Cross-Lingual Anchor Loss, which encourages representations from different languages to align with a shared multilingual semantic space.

Only the family-specific text adapter is restricted to within-family aggregation, while the personalized language adapter remains local. This design allows MERIDIAN to preserve globally useful knowledge while reducing destructive interference in language-sensitive parameters.

To further investigate the reviewer's concern, we compared different routing strategies, focusing particularly on Tamil, which represents a single-client Dravidian language family in our experimental setting.

Table: Effect of adapter routing on minority-language performance (H-Mean %).

ConfigurationOverallTamil
Global text adapter81.1076.10
Random clustering81.5076.85
Gradient-based clustering82.7078.60
Script-based clustering82.9079.44
MERIDIAN family routing83.7381.76

Notably, Tamil achieves 81.76% H-Mean with MERIDIAN, compared with 76.10% under unrestricted global text-adapter sharing, an improvement of 5.66 percentage points. This finding suggests that restricting aggregation of language-sensitive parameters can benefit minority-language clients rather than necessarily limiting their generalization.

We also examined novel-class generalization across language families.

Table: Personalization and generalization across language families (%).

Language familyClientsPer.Gen.H-Mean
Indo-European494.1875.6283.88
CJK294.6276.8584.81
Arabic193.1874.2782.66
Indic-Devanagari193.2774.8583.05
Indic-Bengali192.4174.1882.30
Dravidian192.8573.8482.27

Despite having only one client, the minority-language families maintain novel-class generalization between 73.84% and 74.85%. Furthermore, all four single-client families improve their H-Mean over the corresponding pFedMoAP baseline. These findings do not indicate the severe generalization collapse suggested in the concern, although they do not eliminate the possibility of overfitting under more extreme data scarcity.

We acknowledge an important limitation: when a language family contains only one client, its family adapter effectively becomes a local adapter and cannot benefit from within-family aggregation. In such cases, knowledge transfer is maintained primarily through the globally shared vision and text adapters and the cross-lingual alignment objective.

We will clarify the aggregation scope of each parameter group in Section 3.2 and explicitly discuss the single-client limitation. We will also include the routing comparison and per-family generalization results to make the effect on minority languages more transparent.

*Q4: We thank the reviewer for this valuable suggestion. We selected English, Chinese, and Arabic to capture linguistic diversity across scripts, tokenization patterns, and morphological structures. To validate this choice, we conducted a sensitivity analysis using different pivot combinations and varying their number from one to four.

Table: Pivot-language sensitivity analysis (H-Mean %).

Pivot LanguagesNumberH-Mean
CLAL disabled079.91
English180.55
Chinese179.80
Arabic179.02
English + Chinese282.15
English + Arabic281.90
English + German280.85
English + French + German381.30
English + Chinese + Hindi382.75
English + Chinese + Spanish382.95
English + Chinese + Arabic (Default)383.73
English + Chinese + Arabic + Spanish483.79
English + Chinese + Arabic + Hindi483.65
Three pivots with shuffled class labels376.80

The results show that linguistically diverse pivot combinations generally outperform the evaluated closely related combinations. The default configuration achieves 83.73% H-Mean, outperforming alternative three-pivot settings. Adding a fourth pivot yields only a marginal improvement of 0.06 percentage points, supporting the three-pivot configuration as a practical performance–complexity trade-off.

Furthermore, disabling CLAL reduces H-Mean to 79.91%, while shuffling class labels decreases it to 76.80%, suggesting that meaningful semantic correspondence contributes to the improvement. We will include this ablation and clarify the linguistic motivation, pivot-count sensitivity, and limitations of the selected configuration in the revised manuscript. We thank the reviewer again for these constructive comments, which have helped us improve both the clarity of the architectural description and the empirical support for our design choices. We are happy to provide any further clarification during the discussion period.

Reviewer_QKauOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses classification tasks in scenarios with linguistic heterogeneity by employing personalized federated learning for VLMs. They propose a text adapter to adapt VLMs to diverse real-world linguistic scenarios, and utilize cross-lingual loss to enhance cross-lingual semantic consistency.

Critical strengths
  1. Extensive experiments validate that MERIDIAN works in heterogeneous language family scenarios.
  2. Utilizing cross-lingual loss to enhance cross-lingual semantic consistency mitigates the impact of underlying language differences.
Critical weaknesses

1.The paper targets cross-lingual scenarios but lacks reliable comparative methods within the same domain. It does not adequately compare with existing works incorporating Fed + multilingual + VLM architectures. At the same time, it does not validate the necessity of incorporating multilingual capabilities at the model level, and cannot rule out a simpler alternative: translating all content into a single language and then applying standard Fed+VLM approaches. In addition, although Table 12 shows that naive baselines perform poorly, the paper does not supplement comparisons with cross-lingual layer-level algorithms, introducing comparison bias. 2.Components such as cross-lingual adaptation, hierarchical clustering, and alignment loss are not novel in federated learning. The main contribution appears to be an engineering integration of existing modules rather than unique methodological innovation. Meanwhile, the interpretability analysis is insufficient: the paper does not provide fine-grained explanations of how each module and loss function works, how they correspond to data mining capabilities and the overall architecture, or how performance improvements can be robustly attributed to them. As a result, readers lack an intuitive understanding of the core data challenges addressed by the algorithm.

Questions for authors

1.The “Table ??” issue, the red border on the main figure, and the unverified citation should be corrected in the final version. Do these issues affect the experimental conclusions and the reliability of citations? 2.Why does the paper not compare with existing Fed + multilingual + VLM architectures or cross-lingual federated learning methods? The authors need to add relevant baselines to demonstrate the effectiveness of the method. 3.Have you validated the baseline of “translating all content into a single language and then using standard Fed+VLM”? If not, how do you justify that model-level multilingual capability is necessary rather than the translation-based approach being sufficient? 4.Can you provide more fine-grained ablation and interpretability analyses to explain what each loss function and module contributes and how they correspond to data mining capabilities and architectural design? 5. Compared with existing methods such as hierarchical clustering and alignment loss in federated learning, what is the core technical innovation of this work? Beyond engineering integration, does it offer any unique methodological contribution?

5 comments

We thank the reviewer for the careful reading and constructive comments.

Q1: We thank the reviewer for identifying these issues. We have identified the unresolved table reference ("Table ??"), the unintended red border in Figure 2, and the incorrect M-CLIP citation. The appropriate M-CLIP reference is Carlsson et al., Cross-lingual and Multilingual CLIP (LREC 2022). We will correct these issues and remove the duplicated FedOTP bibliographic entry in the revised manuscript. These issues concern presentation and citation attribution rather than the experimental methodology. Their correction does not change the reported numerical results or the underlying scientific conclusions.

Q2: We agree that comparisons with conventional federated VLM methods alone are insufficient to establish MERIDIAN's effectiveness under linguistic heterogeneity. To address this concern, we consider additional multilingual federated VLM baselines using the same M-CLIP backbone and evaluation setting, including FedAvg, FedProx, shared and personalized adapters, and a language-family LoRA configuration inspired by cross-lingual federated learning.

Table: Federated multilingual VLM comparison (%).

MethodPer.Gen.H-Mean
M-CLIP + FedAvg90.4068.5077.94
M-CLIP + FedProx90.9069.2078.58
M-CLIP + Shared Adapter91.2070.1079.27
M-CLIP + Personalized FL Adapter92.3071.6080.65
FedLFC-style Language-Family LoRA92.7072.4081.30
MERIDIAN (Ours)93.8775.8483.73

MERIDIAN achieves the highest reported H-Mean of 83.73%, exceeding the strongest additional baseline, FedLFC-style Language-Family LoRA, by 2.43 percentage points. It also improves generalization by 3.44 points and personalization by 1.17 points.

We clarify that the FedLFC-style baseline is our adaptation of language-family LoRA principles to federated VLM learning, rather than an unchanged implementation of the original method.

These comparisons suggest that multilingual pretraining, conventional federated optimization, or language-aware parameter sharing alone is insufficient to reproduce MERIDIAN's performance. Furthermore, the original manuscript's Table 2 provides a backbone-controlled comparison: MERIDIAN achieves 83.73% H-Mean versus 79.18% for MMA+FedAvg with M-CLIP, corresponding to a 4.55-point improvement under the same multilingual backbone.

Together, these comparisons support the complementary benefits of language-family aggregation, personalized language adaptation, and cross-lingual semantic anchoring.

Q3:

We appreciate this important suggestion. To investigate whether translation-based preprocessing can replace language-aware adaptation, we compare MERIDIAN against Translate-to-English + FedAvg and Translate-to-English + Federated Adapter.

Table: Translation-based federated VLM comparison (Protocol B, H-Mean %).

MethodenzharhibntaAvg.
M-CLIP + FedAvg76.4276.1074.8574.2772.8572.1874.45
Translation + FedAvg77.8577.4276.1875.6474.3173.7275.85
Translation + Fed Adapter78.9478.5177.2676.7275.4874.9076.97
MERIDIAN (Ours)80.9483.0884.1682.6482.1481.7682.45

Translation-based preprocessing improves conventional federated learning. Nevertheless, MERIDIAN achieves a 5.48-point higher average H-Mean than the strongest translation baseline. The improvements are particularly pronounced for Arabic (+6.90), Tamil (+6.86), and Bengali (+6.66), indicating the benefits of language-aware adaptation across linguistically diverse clients.

We additionally examine robustness under different few-shot settings.

Table: Few-shot robustness (Protocol B, H-Mean %).

Method2-shot4-shot8-shot16-shot
M-CLIP + FedAvg71.0572.7473.9874.45
Translation + FedAvg72.5874.1675.3875.85
Translation + Fed Adapter73.9275.4476.5276.97
MERIDIAN (Ours)78.1480.1281.4682.45

MERIDIAN consistently outperforms both translation-based alternatives across all evaluated shot settings. Its advantage over the stronger translation baseline increases from +4.22 points at 2 shots to +5.48 points at 16 shots.

Unlike translation-based pipelines requiring client-side text translation during inference, MERIDIAN uses offline-translated class descriptions to construct multilingual semantic anchors, without requiring an additional translation model at client inference.

We acknowledge that our evaluation focuses on class-description-based classification rather than unrestricted multilingual text. Within this scope, these comparisons support the benefits of language-aware adaptation beyond translation-based preprocessing.

Q4:

We thank the reviewer for requesting a more detailed explanation of how individual modules and loss functions contribute to MERIDIAN. We clarify these contributions through factorial component ablations, semantic-anchor controls, and direct representation-alignment measurements.

Table: Component contributions (original manuscript Table 3, H-Mean %).

ConfigurationH-MeanΔ vs. Full
Full MERIDIAN83.730.00
Without Family Adapter (FA)80.65−3.08
Without Personalized Language Adapter (PLA)80.46−3.27
Without Cross-Lingual Anchor Loss (CLAL)78.71−5.02
Without FA and CLAL77.62−6.11
Without PLA and CLAL77.48−6.25
Without FA, PLA, and CLAL75.98−7.75

These values are taken directly from the factorial ablation in Table 3 of the original manuscript. Removing CLAL causes the largest single-component reduction (−5.02 points), followed by PLA (−3.27) and FA (−3.08). This demonstrates that multilingual semantic anchoring, client-specific adaptation, and language-family knowledge sharing contribute complementary benefits. The original factorial analysis also shows that the complete configuration outperforms each individual component and partial combination, supporting the effectiveness of their joint optimization.

To further examine CLAL, we consider different multilingual semantic-anchor configurations.

Table: Semantic-anchor configurations (H-Mean %).

ConfigurationH-Mean
No CLAL (μ = 0; sensitivity experiment)79.91
English Pivot Only81.50
English + Chinese82.50
English + Arabic82.30
English + Chinese + Arabic (Ours)83.73
Randomly Permuted Pivot Labels76.80

The complete three-language anchor configuration achieves the highest H-Mean. Randomly permuting pivot labels reduces performance to 76.80%, below the no-CLAL sensitivity setting. This suggests that the benefit of CLAL arises from meaningful multilingual class-semantic supervision rather than simply introducing an additional regularization term.

For clarity, 78.71% is the no-CLAL result in the original factorial ablation (Table 3), whereas 79.91% is reported for μ = 0 in the original sensitivity analysis (Table 4). These are separately reported measurements and are not treated as interchangeable.

Table: Cross-lingual representation alignment.

MethodSame-Class Cosine ↑Different-Class Cosine ↓R@1 (%) ↑
Frozen M-CLIP0.650.3168.2
MERIDIAN without CLAL0.730.3074.1
MERIDIAN with CLAL0.840.2781.6

CLAL improves same-class cross-language cosine similarity from 0.73 to 0.84 and retrieval R@1 from 74.1% to 81.6%, while reducing different-class similarity. These measurements complement the representation visualization in Appendix A.2 and provide direct evidence of the intended semantic alignment behavior.

Together, these analyses establish a clearer relationship between the architecture and its functional contributions: FA enables structured knowledge sharing, PLA preserves client-specific language adaptation, and CLAL encourages consistent class semantics across languages.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.