acceptodds
Under review as a conference paper at ICLR 2027

Temporal Loss Signatures for Data Pruning in Federated Learning with Noisy Labels

Abstract

Federated learning enables collaborative model training without sharing raw data, but heterogeneity and label noise can degrade performance. Data pruning is difficult because high loss conflates mislabeled examples with hard but informative ones, while static selections become stale as the global model evolves. We introduce the Loss Signature Score (LSS), a dynamic, client-local criterion that combines sliding-window mean loss with loss gain. Mean loss captures persistent difficulty, whereas gain favors hard examples that show learning progress. Periodic re-scoring, class-aware retention quotas, and FedProx enable adaptive pruning without estimating the noise rate. The framework adds one full-local-data inference pass per round but requires no scoring-related backward pass or sample-level communication. Experiments on CIFAR-10/100 and CIFAR-10N/100N cover non-IID partitions, synthetic and human annotation noise, and nominal pruning rates from 10% to 70%, with the largest benefits under stronger noise and aggressive pruning. On CIFAR-100 with 20% symmetric noise and a 50% pruning target, LSS achieves 65.38% test accuracy, exceeding the next-best baseline by 4.29 percentage points. On CIFAR-100N with 40.2% human-annotated label noise, it outperforms all evaluated data-pruning baselines at every tested pruning rate.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Learning, Learning with Noisy Labels, Data Pruning, Sample Selection, Training Dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9cUvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates a data-pruning federated learning framework designed to address the challenge of noisy labels. The authors propose a Loss Signature Score (LSS) to characterize the learning difficulty of samples based on their consistent performance, calculated by combining the average loss within a sliding window with the loss gain. By integrating the LSS into a dynamic federated pruning framework, the approach enables adaptive sample selection in scenarios involving heterogeneous data. Experimental results on several datasets show that LSS outperforms existing methods in terms of accuracy.

Critical strengths
  1. Overall, the paper is well-written with a clear presentation.

  2. Investigating data pruning in federated noisy label learning is a significant problem and direction.

  3. Extensive experiments have been conducted to evaluate the performance of the proposed framework.

Critical weaknesses
  1. The paper primarily presents overall accuracy results, but it fails to provide sufficient evidence that it can effectively distinguish between noisy samples and hard clean samples. It would be beneficial to include metrics such as the actual noise rate within the retained set (before and after pruning), as well as the precision, recall, or AUROC for the identification of noisy samples.

  2. While positioned as federated learning with noisy labels, this work lacks comparisons with methods that directly address this problem.

  3. In Table 3, the mean-only baseline also performs very strongly in the CIFAR-100N setting; the overall effectiveness of the entire module therefore requires further clarification.

  4. Enforcing category quotas based on observation labels may result in the retention of incorrect labels, and could even increase the noise rate within the retained set.

  5. The time window and refresh interval are core parameters; corresponding sensitivity analyses should be conducted.

  6. The synthetic noise consists of only 10% and 20% symmetric noise; such noise levels are too low to be sufficiently challenging or practically meaningful.

  7. The model definition allows for different noise rates for each client, but the synthetic experiments primarily employ a uniform noise rate.

Questions for authors

Refers to the weakness

2 comments

Dear Reviewer,

Please take a moment to review the authors' rebuttal and provide your feedback on whether your concerns have been adequately addressed. Thank you for your time and effort.

I still maintain that, for a study on noisy-label FL, comparing with relevant FNLL baselines is fundamental. Without these comparisons, the work is difficult to find convincing.

Although some human-annotated or naturally occurring datasets contain a larger proportion of noise, the maximum level is only around 40%. In contrast, these FNLL baselines typically evaluated performance against synthetic noise at all levels, from low to high, reaching 70% or 80%. Consequently, the noise ratios currently used in synthetic settings are still not solid enough.

These concerns are not difficult to clarify, and baselines like FedCorr have open-source code. Running experiments like CIFAR-10/100 doesn't take much time. Therefore, I will still retain my existing negative score.

Reviewer_NfJUOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a sample selection metric called the Loss Signature Score in federated learning, arguing that sample selection should consider not only how high a sample’s loss is, but also how much the loss decreases over time.

Critical strengths

The paper presents a useful approach to addressing label noise in federated learning by considering both the loss magnitude of each sample and how its loss evolves over time, helping avoid mistakenly discarding useful samples that have high losses but are still being learned by the model.

Critical weaknesses

LSS cannot reliably distinguish hard samples from mislabeled ones, and the results are not consistently better across different noise levels and pruning settings.

Questions for authors

See Critical Weaknesses.

1 comment

Dear Reviewer,

Please take a moment to review the authors' rebuttal and provide your feedback on whether your concerns have been adequately addressed. Thank you for your time and effort.

Reviewer_sB5gOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Loss Signature Score (LSS), a dynamic data pruning method for federated learning with noisy labels. LSS combines the average loss and loss reduction over a sliding window to distinguish potentially noisy samples from hard but informative samples. The method performs client-side data selection through periodic rescoring, class-aware pruning, and FedProx, without requiring additional backward passes for scoring or sample-level communication. Experiments on CIFAR-10/100 and CIFAR-10N/100N demonstrate its effectiveness, particularly under high label noise levels and large pruning ratios.

Critical strengths
  1. Strong Empirical Performance. LSS achieves competitive results across multiple datasets with noisy labels, particularly under high noise levels on CIFAR-100. The ablation studies also demonstrate that Loss Gain effectively complements Mean Loss at higher pruning ratios, supporting the practical value of the proposed method.
  2. Simple and Practical Framework. The proposed method is simple and easy to integrate into existing federated learning frameworks. It requires neither prior estimation of noise rates nor additional backward passes for scoring or sample-level communication, making it practically useful.
Critical weaknesses
  1. Limited Methodological Novelty. LSS is essentially a simple combination of mean loss and loss reduction, and its core scoring mechanism does not explicitly leverage client heterogeneity or cross-client information specific to federated learning. Its integration into federated learning mainly relies on existing techniques, including dynamic pruning, class-aware selection, and FedProx, resulting in relatively limited methodological novelty.
  2. Insufficient Validation of the Selection Mechanism. The paper primarily evaluates the effectiveness of LSS through final accuracy, but lacks direct analyses of the noise rate among retained samples and the retention of hard but clean samples. Therefore, it remains unclear to what extent the performance improvements can be attributed to the proposed mechanism for distinguishing noisy samples from hard but informative samples.
Questions for authors
  1. LSS uses Min-Max statistics from all local samples for normalization, but performs ranking independently within each class. Why do the authors choose client-wide normalization instead of class-wise normalization? Could this design affect scoring stability under highly imbalanced class distributions?
  2. The paper fixes (W = \Delta = 10), meaning that pruning is updated every 10 rounds. How do different history window lengths and refresh intervals affect sample selection stability and final performance? Have the authors observed cases where more frequent updates lead to performance degradation?
1 comment

Dear Reviewer,

Please take a moment to review the authors' rebuttal and provide your feedback on whether your concerns have been adequately addressed. Thank you for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.