acceptodds
Under review as a conference paper at ICLR 2027

Collaborative Inference for Federated Noisy Labels: Convergence under Dual Heterogeneity

Abstract

Heterogeneous label noise across clients poses critical challenges to robust training and stable convergence in federated learning. Existing methods largely rely on local label correction, limiting their ability to leverage global information. Moreover, theoretical convergence analysis under dual data-noise heterogeneity remains under-explored. To address this, we propose FedCoIn, a federated noisy label learning framework based on generative collaborative inference. This method treats ground-truth labels as latent variables and performs label inference through global generative modeling, enabling cross-client collaborative noise correction under privacy constraints. Since heterogeneity may disrupt generative inference and undermine global correction consistency, FedCoIn introduces a distribution-aware dynamic prior and a noise-aware weighted aggregation strategy to stabilize label inference and harmonize corrections across clients. Theoretically, we provide convergence analysis under three heterogeneous scenarios, including data homogeneity with noise heterogeneity, data heterogeneity with noise homogeneity, and dual data-noise heterogeneity. This analysis characterizes the impact of both types of heterogeneity on convergence, thereby establishing a new theoretical foundation for federated noisy label learning and providing theoretical support for FedCoIn. Experimental results demonstrate that FedCoIn consistently outperforms state-of-the-art methods in federated noisy label learning tasks.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Learning, Label Noise, Heterogeneity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wTi7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the authors propose FedCoIn, a federated learning framework designed to address noisy labels. FedCoIn consists primarily of two modules: the first is a global generative modeling approach that enables collaborative noise correction across clients, while the second incorporates distribution-aware dynamic priors and a noise-aware weighted aggregation strategy. The authors provide a convergence analysis for three different heterogeneity settings. Experimental results demonstrate that FedCoIn consistently outperforms state-of-the-art methods across various scenarios involving heterogeneous noise.

Critical strengths
  1. The structure of this paper is clear and reasonable.

  2. The paper addresses both the discrepancies in client data distribution and the variations in label noise, which is of practical significance.

  3. The paper provides comprehensive theoretical analysis, ensuring the robustness of the proposed method.

Critical weaknesses
  1. Equation (4) shows that minimizing the posterior KL divergence is equivalent to maximizing the ELBO. However, Equation (5) instead features the term y^log⁡y^∗+(1−y^)log⁡(1−y^∗)\hat y\log \hat y^*+(1-\hat y)\log(1-\hat y^*). Need to clarify the negative ELBO objective.

  2. Is the initial classifier a local model or a global model? While Figure 1 clearly illustrates the aggregation of global parameters for the initial classifier, the main text and the algorithm description only state that each client trains its own classifier and subsequently freezes its parameters, without explicitly mentioning this aggregation step.

  3. Why is the non-IID performance of FedClean not shown in Table 2?

  4. Does w/o CoIn mean removing all generated inferences, or only removing the collaborative steps?

  5. In the legends for Figures 3(a,b), the method is labeled as FedCoGen.

Questions for authors

See in weakness.

1 comment

Thank you for your rebuttal. I will keep my positive rating.

Reviewer_Y2A6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

FedCoIn addresses federated learning with noisy labels under simultaneous data and noise heterogeneity. It fixes a trained classifier and learns a Dirichlet-based variational calibration model, combining an adaptive prior, local-global posterior alignment, noise-aware aggregation, and server-side optimization. The paper reports improvements on CIFAR-10, CIFAR-100, and Clothing1M and claims O(1/T)O(1/T) convergence guarantees under three heterogeneity settings.

Critical strengths
  1. FedCoIn separates classifier training from label-space correction. Keeping the initial classifier fixed allows the calibration stage to reuse pretrained features and classifier predictions, while the conditional correction matrix can express sample-dependent changes across classes rather than only a clean/noisy decision. Local-global posterior alignment provides a mechanism for sharing corrective information without exchanging raw examples. This modular architecture is a useful feature, although post-hoc calibration itself is not a new paradigm.

  2. The combination of sample-level prior construction and client-level aggregation is the most promising methodological contribution. DADP uses class-aware confidence selection and adaptive, distance-weighted neighborhoods, while NADA combines correction gains with a sublinear dependence on client sample size. These choices explicitly target local class imbalance, neighborhood reliability, and unequal client contributions. Table 4 shows consistent degradation when DADP or NADA is removed, supporting the utility of these components within the evaluated configurations, alongside the same-backbone improvements in Table 2.

Critical weaknesses
  1. The paper does not sufficiently establish what the proposed collaboration contributes beyond existing calibration. The Dirichlet model, confidence/KNN prior, posterior-mode mapping, and prediction marginalization closely match NPC, which is not cited. The potential novelty therefore lies primarily in the federated adaptation. However, "w/o CoIn" is insufficiently defined to distinguish local calibration from cross-client collaboration. A matched comparison between local NPC, a straightforward federated NPC variant, and FedCoIn, using the same initial classifier and features, is needed to establish whether the added alignment and aggregation mechanisms provide additional corrective information. The current results support the complete system but do not adequately identify the source of its algorithmic advantage.

  2. The central optimization and aggregation mechanisms remain insufficiently specified or validated. Equation 5 is described as minimizing a positive reconstruction log-likelihood despite the ELBO-maximization objective in Equation 4, and Algorithm 1 does not explain how uploaded quantities suffice to evaluate the feature-dependent server objective in Equation 7. These omissions prevent reconstruction of the actual collaborative update. NADA also treats gains on noisy validation sets as comparable measures of correction quality. Under the appendix's uniform-relabeling model, expected gains for fixed predictors evaluated independently of validation-label corruption are scaled by one minus the client's relabeling probability. Thus, the weights reflect both correction gains and noise levels, not correction quality alone. This does not prove that NADA fails, but its central interpretation requires a cross-client diagnostic. The convergence analysis cannot resolve these concerns: it does not cover the changing aggregation weights and server optimization, and its contraction argument in Equations 107 is invalid as written.

Questions for authors
  1. What exactly is new relative to NPC, and which operations remain in "w/o CoIn"? Can a matched local-NPC/federated-NPC/FedCoIn comparison isolate the contribution of posterior alignment and aggregation in one representative setting?

  2. What objective is actually optimized locally, and how is Equation 7 evaluated from the communicated information? Please reconcile the local objective with its variational derivation and specify the complete server update. Does the theoretical analysis apply to that implemented update or only to a simplified fixed-weight variant?

  3. How well do noisy-validation gains track genuine correction gains across clients with different noise rates? A diagnostic using known synthetic-noise ground truth only for evaluation would clarify how NADA scores relate to actual correction ability.

Reference:

[1] Heesun Bae et al. From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Model. ICML, PMLR 162:1277-1297, 2022.

Reviewer_PYMrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes FedCoIn to handle label noise that varies across federated clients. It uses Dirichlet-based inference to estimate clean labels, builds local priors from confident predictions and nearby samples, and weights clients based on their correction gains on noisy validation data. The authors also provide convergence results under data and noise heterogeneity. Experiments cover CIFAR-10, CIFAR-100, and Clothing1M, using ResNet models and ViT features.

Critical strengths
  1. The paper addresses a real problem. Clients can have different data distributions and different levels of label noise, so using information across clients to improve label correction is a useful idea.
Critical weaknesses
  1. Equation (107) puts the same parameter error on both sides, but multiplies the right-hand side by a factor smaller than one. This cannot hold unless the error is already zero. Before Equation (104), the proof also assumes that the global optimum makes every client’s gradient zero, which generally does not hold when clients have different objectives.

  2. The paper names the ResNet architectures but does not specify the ViT checkpoint, MLP structure, or variational-network architecture. It also does not report how many parameters are trained or communicated. The classifier pretraining budget and number of independent runs are unclear. For Table 7, I would also like to know whether feature extraction and classifier pretraining are included, and whether competing methods use comparable frozen or trainable components.

Questions for authors
  1. Can you explain how Equation (38) holds for the two-client example above?

  2. What exact loss was used in the code? Is the reconstruction term in Equation (5) minimized or maximized?

  3. How do you ensure that Equation (8) always gives valid probabilities? Are all posterior Dirichlet parameters constrained to exceed one?

  4. Which ViT checkpoint and network architectures were used? How many parameters were trained and communicated, and does Table 7 include feature extraction and classifier pretraining?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.