- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TIMBRE, a server-side defense against backdoor attacks in federated learning based on layer-wise normalized spectral energy distributions. The key observation is that malicious and benign updates may exhibit different intra-layer energy allocation patterns, even when their overall magnitudes are similar. TIMBRE uses Hellinger distance and robust statistics to compute spectral anomaly scores, accumulates evidence across communication rounds to identify persistent anomalies, and combines coordinate-level filtering with dynamic aggregation weight adjustment to suppress suspicious updates.
Critical strengths
-
Rather than relying on update norms, distances, or overall directions, TIMBRE characterizes the internal structure of model updates through layer-wise normalized spectral energy distributions, providing a different perspective for detecting malicious updates that may be difficult to distinguish using conventional geometric statistics.
-
TIMBRE operates entirely on the server side, without requiring access to clients' local data or modifications to their local training procedures.
-
The paper is clearly structured and easy to follow.
Critical weaknesses
-
The paper argues that backdoor training tends to concentrate update energy in a few singular components. However, benign clients with highly skewed class distributions may exhibit similar behavior. The paper therefore does not establish whether this feature remains reliably discriminative under Non-IID conditions.
-
MARS (NeurIPS 2025) has already explored singular-value-related backdoor energy for malicious client detection in federated learning. Although TIMBRE uses the full normalized spectral energy distribution, the paper does not demonstrate its additional advantages over simpler spectral statistics, such as the largest singular value, spectral entropy, or effective rank.
-
TIMBRE assigns a strongly restricted aggregation weight coefficient of 0.01 to suspicious clients, which may itself substantially suppress backdoor updates. Although Table D7 provides a component ablation study, it does not compare different anomaly scoring methods under the same suppression mechanism. It is therefore unclear whether the performance gains primarily come from spectral energy-based detection or the subsequent aggressive update suppression.
-
Section 2.2 claims that deployment requires neither trusted reference data nor an attack-free training phase. However, Appendix C.4 uses a separate 50-round attack-free experiment to determine the thresholds, with different thresholds adopted for different datasets and models. Without experiments on threshold transferability across data distributions, the practical applicability of TIMBRE remains unclear.
-
PSM-BadNets only attempts to evade detection by mimicking benign spectral energy distributions. It does not jointly optimize against TIMBRE's historical anomaly accumulation and coordinate-level same-sign support mechanisms.
-
The proposed method involves several hyperparameters, including spectral scoring parameters, anomaly detection thresholds, coordinate filtering proportions, and aggregation weight coefficients. Although Appendix C.4 provides parameter settings and threshold calibration rules, a systematic sensitivity analysis is lacking.
-
Appendix C.1 indicates that the main experiments continue training from a global model that has already undergone many training rounds, rather than starting from random initialization. Since the spectral structure of updates may differ substantially between early and late training stages, the current results do not establish TIMBRE's effectiveness throughout the complete federated learning process.
Questions for authors
-
Could the authors report the spectral anomaly score distributions of benign and malicious clients, together with AUROC and false positive rates, under different levels of data heterogeneity?
-
MARS has already used singular-value-related backdoor energy for malicious client detection. What additional advantages does TIMBRE's full normalized spectral energy distribution offer over simpler spectral statistics, such as the largest singular value, spectral entropy, or effective rank? Could the authors provide comparisons within a unified detection framework?
-
Could the authors provide controlled experiments to demonstrate that the performance improvements are indeed attributable to the spectral energy features, rather than the subsequent update suppression mechanism?
-
Appendix C.4 uses a separate 50-round attack-free experiment to determine thresholds, with different thresholds for different datasets and models. Does TIMBRE require dataset- and model-specific attack-free calibration?
-
The authors should consider stronger adaptive attacks that jointly optimize spectral distribution similarity and coordinate-level sign consistency, or distribute malicious updates across communication rounds to evade historical anomaly accumulation.
-
The authors should provide sensitivity analyses for key hyperparameters (e.g., , , , , , and ) to evaluate whether the method maintains stable performance across different parameter settings.
-
The authors should include complete federated learning experiments starting from random initialization and report ASR, ACC, and spectral anomaly detection performance at different training stages.
13 comments
Response to Reviewer JBmo (2/8)
W1 / Q1 (continued). The AUROC is at least 0.921 at all three levels. The distribution of benign evidence hardly changes with heterogeneity because of how the score is computed: each client's score is standardized against the other clients of the same round, that is, the median of the round's scores is subtracted and the result is divided by their spread (MAD), and this spread grows as the data become more heterogeneous; the fraction of benign participations exceeding the coordinate-processing threshold does not rise with heterogeneity either. When heterogeneity increases, the change is mainly on the malicious side: at α = 0.2, the median malicious evidence drops to 2.11–5.22, and fewer malicious participations exceed the thresholds. TIMBRE does not need to identify the attacker in every round: the evidence of malicious participations that do not exceed the thresholds still enters the evidence accumulated across rounds (called debt in the main text), so a persistently poisoning client is processed as rounds accumulate. For example, under Batman at α = 0.2, only 36% of the malicious participations exceed the coordinate-processing threshold in a single round, yet the final ASR is 0.42%, rising to 88.89% without evidence accumulation (Appendix D.5).
Regarding the benign clients with highly skewed class distributions that you mentioned, we also split the benign participations into four groups by the quartiles of the client's largest class share, from group 1 (most balanced) to group 4 (most skewed). If skewed benign clients looked more like attackers, the Spearman rank correlation between the largest class share and the evidence should be positive and statistically significant, and the fraction exceeding the coordinate-processing threshold should increase from group 1 to group 4. The fractions (%) exceeding the coordinate-processing threshold in each group are:
| α | Attack | Spearman () | Group 1 (most balanced) | Group 2 | Group 3 | Group 4 (most skewed) |
|---|---|---|---|---|---|---|
| 0.2 | BadNets | 0.06 (0.24) | 0.9 | 0.9 | 0.0 | 6.0 |
| Batman | 0.05 (0.31) | 2.7 | 0.9 | 0.9 | 7.0 | |
| Neurotoxin | −0.02 (0.74) | 0.9 | 0.0 | 0.0 | 4.0 | |
| DBA | 0.05 (0.31) | 0.0 | 0.9 | 1.9 | 4.0 | |
| 0.5 | BadNets | −0.00 (0.99) | 2.6 | 1.0 | 0.9 | 1.0 |
| Batman | −0.02 (0.71) | 7.9 | 1.9 | 0.9 | 2.9 | |
| Neurotoxin | −0.04 (0.47) | 3.5 | 1.0 | 0.9 | 1.0 | |
| DBA | −0.04 (0.43) | 3.5 | 1.9 | 0.9 | 1.9 | |
| 1.0 | BadNets | −0.06 (0.25) | 7.0 | 5.9 | 4.5 | 4.9 |
| Batman | 0.02 (0.76) | 5.0 | 2.0 | 3.6 | 5.9 | |
| Neurotoxin | −0.03 (0.57) | 3.0 | 5.0 | 2.7 | 2.0 | |
| DBA | −0.06 (0.21) | 6.0 | 6.9 | 2.7 | 2.0 |
At all three heterogeneity levels, is not statistically significant (, ), and the fractions do not increase with skew; only at α = 0.2 is the most skewed group higher (4.0%–7.0%, versus 0–2.7% for the other three groups), still far below the 36%–76% of malicious participations in the same setting (the "TPR: coordinate" column above). Overall, skewed benign clients are not more likely to be misjudged.
Response to Reviewer JBmo (3/8)
W2 / Q2: MARS (NeurIPS 2025) has already used singular-value-related backdoor energy for malicious client detection. Although TIMBRE uses the full normalized spectral energy distribution, the paper does not explain its additional advantages over simpler spectral statistics such as the largest singular value, spectral entropy, or effective rank. Could the authors compare them within a unified detection framework?
The new Appendix D.6 replaces the per-matrix Hellinger distance with these three statistics in the same pipeline, keeping everything else unchanged and re-determining the thresholds with the same procedure; the results for the main settings are also in Table 2 of the main text. Worst final ASR (%) over four attacks (BadNets, Batman, Neurotoxin, and DBA) in the eight settings of Appendix D.3:
| Setting | TIMBRE | Spectral entropy | Effective rank | (unnormalized) | FedAvg without defense |
|---|---|---|---|---|---|
| α = 0.5, (default) | 1.38 | 1.78 | 2.04 | 42.04 | 89.03 |
| α = 0.2 | 0.79 | 7.70 | 5.50 | 78.40 | 89.24 |
| α = 0.3 | 0.72 | 0.47 | 1.46 | 65.34 | 94.09 |
| α = 0.7 | 1.50 | 1.32 | 1.01 | 60.52 | 87.28 |
| α = 1.0 | 0.99 | 1.10 | 0.74 | 0.50 | 93.68 |
| 1.53 | 1.28 | 1.19 | 1.47 | 70.88 | |
| 0.56 | 14.96 | 33.10 | 73.72 | 89.82 | |
| 2.23 | 76.50 | 38.01 | 90.79 | 91.08 |
The unnormalized exceeds 40% in six of the eight settings, which is exactly the difference between TIMBRE and MARS: TIMBRE reads how the energy is distributed across components rather than how large it is. Spectral entropy and effective rank are also based on the normalized spectrum; they are close to TIMBRE in the default setting but rise markedly at α = 0.2 and with 3–4 malicious clients per round. For detection, the AUROC of each evidence in separating malicious from benign participations in the default setting and at α = 0.2 is:
| Setting | Attack | TIMBRE | Spectral entropy | Effective rank | (unnormalized) |
|---|---|---|---|---|---|
| α = 0.5 (default) | BadNets | 0.992 | 0.977 | 0.960 | 0.809 |
| Batman | 0.971 | 0.937 | 0.861 | 0.973 | |
| Neurotoxin | 0.996 | 0.990 | 0.983 | 0.631 | |
| DBA | 0.980 | 0.956 | 0.934 | 0.818 | |
| α = 0.2 | BadNets | 0.972 | 0.956 | 0.910 | 0.565 |
| Batman | 0.921 | 0.748 | 0.737 | 0.435 | |
| Neurotoxin | 0.963 | 0.915 | 0.896 | 0.563 | |
| DBA | 0.954 | 0.958 | 0.945 | 0.483 |
In the default setting, the three evidences based on the normalized spectrum all reach a high AUROC (0.861–0.996), and is comparable to TIMBRE only under Batman. At α = 0.2, falls to 0.435–0.565 under the four attacks, comparable to the chance level of 0.5; spectral entropy and effective rank fall to 0.748 and 0.737 under Batman, with final ASRs of 7.70% and 5.50%, whereas TIMBRE has an AUROC of 0.921 and a final ASR of 0.42%. The gap widens with 3–4 malicious clients per round: under Batman (3 or 4 malicious clients per round) and Neurotoxin (4 per round), the AUROC of spectral entropy and effective rank is only 0.618–0.705 and their final ASR is 14.96%–76.50%, while TIMBRE does not exceed 2.23% in these three cases. This is because spectral entropy and effective rank each summarize the whole distribution by one number, so distributions of different shapes may receive similar values, whereas the Hellinger distance compares the distributions themselves and simultaneously bounds the differences in the cumulative energy shares at every rank (Theorem A.2), without having to choose a particular summary statistic in advance.
Response to Reviewer JBmo (1/8)
We thank the reviewer for the detailed comments and have added all the suggested experiments. An overview by question comes first:
W1 / Q1: Benign clients with highly skewed class distributions may also concentrate update energy in a few singular components, so it is unclear whether this feature remains reliably discriminative under non-IID conditions. Could the authors report the spectral anomaly score distributions of benign and malicious clients, with AUROC and false positive rates, under different levels of data heterogeneity?
The new Appendix D.4 reports the separation achieved by the current-round evidence (the spectral anomaly score that TIMBRE computes for each client in each round) at three heterogeneity levels; the conclusion is also stated in Section 3.2. The experiments use CIFAR-10 × ResNet18; the attack lasts 50 rounds, and each round has 2 malicious and 8 benign clients among its 10 participants; we treat one participation of one client in one round as one sample, so each attack and heterogeneity level has 100 malicious and 400 benign participations. Each of the two thresholds corresponds to one kind of processing: a client whose current-round evidence exceeds the coordinate-processing threshold undergoes coordinate filtering in that round (only coordinates with same-sign support from other clients are kept), and a client whose evidence exceeds the strong-processing threshold enters the strong tier (its aggregation weight is multiplied by ms=0.01). TPR and FPR are the percentages of malicious and benign participations whose evidence exceeds the corresponding threshold, so the FPR is exactly the fraction of benign participations that receive the corresponding processing because of their current-round evidence; the last two columns are the median evidence, with the 10% and 90% quantiles in brackets: