- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies post-hoc out-of-distribution detection after federated training has ended. A deployment client has access only to the trained global classifier and its own in-distribution data, but should also accept legitimate ID inputs originating from other clients, including classes absent from its local data. FedOPhD constructs a diagnostic copy of the model through a short sequence of client-conditioned semantic-attenuation updates. It measures representation changes across intermediate layers, fits local geometric response modes, and uses prediction-margin changes to construct functional routing directions. The resulting response-alignment support is combined with energy-based recognition support from the unchanged global classifier through a locally calibrated, response-dependent fusion rule. The evaluation covers near- and far-OOD detection on CIFAR-10/100, transferred-ID preservation, DomainNet10 under FedAvg and FedBN, additional backbones and federation configurations, and extensive intervention, readout, and fusion controls.
Critical strengths
-
The deployment problem is well motivated and precisely formulated.
-
The principal empirical gains are substantial under shared-checkpoint comparisons.
-
The intervention and readout controls provide meaningful evidence for the proposed mechanism.
Critical weaknesses
W1. Federation-level ID preservation remains conditional on local response coverage, and this boundary is not fully characterized under domain shift: The fusion rule gives global recognition a response-dependent role: sufficiently weak response support can force rejection regardless of the global-recognition score. Appendix A.4 explicitly establishes this property in Proposition A.3/Eq. (23). Consequently, a globally recognizable T-ID input whose response is weakly coupled to the local intervention, or poorly represented by the locally fitted modes, may still be rejected. Proposition A.4 appropriately makes transfer conditional on response coverage and route agreement, but does not establish how frequently these conditions hold in deployment. The existing T-ID and class-coverage experiments in Tables 6–10 provide useful evidence. However, the DomainNet10 experiments in Tables 25–27 use FPR95 and T-FPR95 thresholds selected to detect at least 95% of the evaluation OOD population. They do not extend the Table 6 analysis to cross-domain T-ID at the threshold that a receiving client can actually calibrate using only local ID. Thus, the evidence for cross-domain ranking improvements is stronger than the evidence for the preservation–detection trade-off at the deployed local threshold. A targeted evaluation under both FedAvg and receiving-client FedBN, together with an analysis of T-ID samples satisfying the response-only rejection condition, would clarify this important methodological boundary.
W2. The fusion comparisons do not fully isolate query-dependent weighting from a favorable average weighting of the two signals: Tables 12–13 show consistent improvements over equal-weight arithmetic and geometric averaging and the other tested combining rules. However, the equal-weight comparisons change both the query dependence of the gate and the overall allocation of weight between response and recognition support. This distinction matters because the two channels have substantially different near- and far-OOD behavior. For example, on CIFAR-100 with , near-OOD AUROC is 58.15 for FedOPhD-R, 72.05 for FedEnergy, and 76.79 for the full method. A more targeted control would freeze the arithmetic gate in Eq. (5) to its mean value over the same client-local reference set. Specifically, with , one could compare against
\frac{1}{|D_{\mathrm{ref},k}|} \sum_{x\in D_{\mathrm{ref},k}}w_k(x), \qquad B_{\mathrm{fixed}}(x)= \bar w_k s_G(x)+(1-\bar w_k)s_R(x).$$ This control requires no OOD-based tuning and preserves the same fitted component scores and average reference-set weighting. It would make the attribution of gains specifically to query-dependent fusion more precise. W3. Selected fusion-baseline and sparse-client implementation details remain underspecified: Despite the extensive appendix, the GLRT and BH fusion baselines are presented primarily through their names and numerical results. The exact input transformations, combining statistics, parameter choices, and calibration-data roles should be specified. For the BH comparator in particular, the paper should explain how the component scores are converted into the conformal p-values used by the procedure; the response support itself is an aggregate of empirical ranks. The sparse-client handling also needs a more explicit algorithmic specification. Appendix D.3 states that the nominal number of response modes is reduced to the “largest locally supported” count, but the support criterion and complete rule for choosing that count are unclear. This choice affects the large-federation and reduced-local-data experiments. Providing these details would improve the reproducibility and interpretation of the relevant comparisons.Questions for authors
-
Local-threshold behavior under domain shift: Can you apply the Table 6 deployment protocol to DomainNet10 under both FedAvg and receiving-client FedBN, reporting L-ID false rejection, cross-domain T-ID false rejection, and OOD detection at thresholds calibrated only from receiving-client ID data? A breakdown by receiving domain would be especially informative. Among rejected T-ID samples, how many satisfy the forced-rejection condition in Eq. (23), particularly when the original classifier predicts them correctly with strong global-recognition support?
-
Attribution to adaptive fusion: How does the proposed fusion compare with a query-independent gate matched to its mean local-reference weight, as defined above? Reporting the comparison across the same seeds and both OOD regimes would directly assess the contribution of query dependence.
-
Reproducibility of selected comparisons: Please provide the precise GLRT and BH implementation formulas and calibration protocol, together with the executable rule for selecting the effective number of response modes for sparse clients.
-
Routing stability as a secondary methodological clarification: The method routes with functional directions but scores against geometric directions . A small response change near a functional-routing boundary can therefore switch between different geometric scores. The finite-gain analysis in Proposition A.2 is correctly restricted to a fixed route, and Table 14 supports the average detection performance of the two-stage design. Could you additionally report routing margins or a small diagnostic of route and score stability under minor response perturbations? This would clarify the practical relevance of the fixed-route qualification.
9 comments
Response to Reviewer gQeq (Part 2)
W1 / Q1 (continued): Analysis of the response-only rejection condition under domain shift, including T-ID samples correctly classified with strong global-recognition support.
Response to W1 & Q1
Part 2/2: Analysis of the sufficient rejection condition.
We first clarify that our theoretical analysis does not claim unconditional T-ID preservation. Proposition A.4 explicitly characterizes transfer under response coverage and route agreement, while Proposition A.3 identifies a sufficient rejection condition when Response Alignment Support is weak. We therefore extend our experiments to quantify both the practical preservation–detection trade-off and the occurrence of this theoretically identified failure condition under domain shift.
Analysis of the sufficient rejection condition. To examine the boundary highlighted by the reviewer, we evaluate Eq. (23) at each client's locally calibrated deployment threshold:
where denotes the client-specific fusion scale and the deployment threshold. We measure how frequently rejected cross-domain T-ID samples satisfy this sufficient condition, including those correctly classified by the original model with strong Global Recognition Support.
Table 36: Percentage of rejected cross-domain T-ID samples satisfying Eq. (23).
| Rejected T-ID subset | FedAvg (%) | FedBN (%) |
|---|---|---|
| All rejected cross-domain T-ID | 10.36 | 17.26 |
| Correctly classified, | 67.50 | 78.70 |
| Correctly classified, | 83.20 | 90.50 |
Notes. Each percentage is conditional on the rejected T-ID subset specified in the corresponding row. denotes locally rank-calibrated Global Recognition Support, rather than a class-posterior probability.
These results confirm the failure mechanism identified in Proposition A.3 while clarifying its practical prevalence. Among rejected T-ID samples correctly classified with strong global support, the majority satisfy Eq. (23). This concentration is consistent with the response-gated fusion mechanism: as approaches one, the sufficient condition in Eq. (23) approaches the exact rejection boundary.
Importantly, these conditional proportions should not be interpreted as false-rejection rates for the corresponding T-ID populations. Across all rejected cross-domain T-ID samples, only 10.36% and 17.26% satisfy this sufficient condition under FedAvg and FedBN, respectively. The results therefore distinguish the existence of response-induced forced rejection from its frequency among the observed rejection events, without implying that the condition captures every possible response-related failure.
Overall, these additional experiments show that the conditional theoretical boundary does not preclude effective cross-domain T-ID preservation in practice. At locally calibrated deployment thresholds, FedOPhD achieves substantially stronger OOD detection than FedEnergy while maintaining comparable or lower T-ID rejection, even under receiving-client FedBN. The analysis also identifies where the response gate can override strong global recognition, providing a more precise characterization of the method's preservation–detection trade-off under domain shift.
We have incorporated both the local-threshold evaluation and the sufficient-rejection analysis into Appendix E.3.1 (Tables 35–36) of the updated Rebuttal Version of our manuscript.
Response to Reviewer gQeq (Part 3)
W2: The fusion comparisons do not fully isolate query-dependent weighting from a favorable average weighting of the two signals: ... This control requires no OOD-based tuning and preserves the same fitted component scores and average reference-set weighting. It would make the attribution of gains specifically to query-dependent fusion more precise.
Q2: Attribution to adaptive fusion: How does the proposed fusion compare with a query-independent gate matched to its mean local-reference weight, as defined above? Reporting the comparison across the same seeds and both OOD regimes would directly assess the contribution of query dependence.
Response to W2 & Q2
We thank the reviewer for this valuable suggestion. We agree that the original equal-weight comparisons do not fully isolate the contribution of query dependence from the average allocation between Response Alignment Support and Global Recognition Support. The suggested mean-matched control provides a more direct assessment of this distinction.
Mean-matched Fixed Gate. We implement the proposed control using the same model checkpoints, support scores, reference data, fusion parameters, random seeds, and evaluation protocol. Specifically, for each client k, we compute the average adaptive weight over its local reference set:
The fixed-gate support is then
This control preserves the same two support signals and their average reference-set weighting while removing only the query dependence. All weights are determined exclusively from client-local ID data, without OOD-based tuning.
Table 15: Query-dependent Adaptive Fusion vs. Mean-matched Fixed Gate (%, five-seed means).
| Dataset | α | Fusion Strategy | Near AUROC ↑ | Near FPR95 ↓ | Far AUROC ↑ | Far FPR95 ↓ |
|---|---|---|---|---|---|---|
| CIFAR-10 | 0.1 | Adaptive Fusion (Ours) | 86.03 | 43.96 | 97.13 | 13.96 |
| Mean-matched Fixed Gate | 85.31 | 45.77 | 92.27 | 33.85 | ||
| CIFAR-10 | 0.5 | Adaptive Fusion (Ours) | 89.15 | 39.30 | 98.32 | 7.66 |
| Mean-matched Fixed Gate | 88.60 | 39.63 | 95.98 | 17.73 | ||
| CIFAR-10 | 10 | Adaptive Fusion (Ours) | 89.52 | 39.57 | 98.43 | 6.72 |
| Mean-matched Fixed Gate | 87.98 | 45.61 | 95.84 | 25.27 | ||
| CIFAR-100 | 0.1 | Adaptive Fusion (Ours) | 76.79 | 67.45 | 94.06 | 23.80 |
| Mean-matched Fixed Gate | 75.03 | 69.11 | 87.69 | 52.66 | ||
| CIFAR-100 | 0.5 | Adaptive Fusion (Ours) | 78.72 | 61.72 | 94.29 | 21.87 |
| Mean-matched Fixed Gate | 76.28 | 64.38 | 89.90 | 45.60 | ||
| CIFAR-100 | 10 | Adaptive Fusion (Ours) | 79.58 | 58.71 | 93.52 | 23.63 |
| Mean-matched Fixed Gate | 78.09 | 59.88 | 89.50 | 42.29 |
Adaptive Fusion consistently outperforms the mean-matched control across all six settings and both OOD regimes, improving all 24 reported AUROC and FPR95 metrics. The gains are particularly pronounced for Far-OOD, where FPR95 decreases by 10.07–28.86 percentage points, alongside consistent improvements in AUROC. Near-OOD performance also improves across all settings, although the gains are generally smaller.
These improvements are consistent with the intended mechanism of our response-dependent fusion. As defined in Eq. (5), weak Response Alignment Support limits the influence of Global Recognition Support, preventing potentially misleading global recognition from dominating the OOD score. Conversely, stronger response alignment allows global recognition to contribute more substantially. A fixed gate cannot make this distinction across queries, even when its average weight is matched to the adaptive gate. This mechanism also helps explain the larger Far-OOD improvements, consistent with the stronger Far-OOD discrimination observed in our response-only ablation.
The mean-matched comparison therefore isolates the benefit of query-dependent weighting from the average allocation of the two support signals. The consistent improvements provide direct empirical support for Adaptive Fusion as a substantive component of FedOPhD, rather than merely a favorable choice of fixed fusion weights.
We have incorporated this controlled comparison and its implementation details into Appendix E.2.2 (Table 15) of the updated Rebuttal Version of our manuscript.
Response to Reviewer gQeq (Part 1)
Response to W1 & Q1
Part 1/2: Cross-domain T-ID preservation under local thresholds.
We thank the reviewer for this thoughtful suggestion. We agree that evaluating cross-domain T-ID preservation under locally calibrated deployment thresholds provides important evidence beyond the ranking-based results in Tables 32–34.
Local-threshold evaluation. Following the Table 6 deployment protocol, we evaluate DomainNet10 under both FedAvg and receiving-client FedBN, retaining the original checkpoints, client partitions, detector configurations, and five random seeds. Each receiving client calibrates its threshold using only local ID data, targeting 95% L-ID acceptance. We then measure L-ID false rejection, cross-domain T-ID false rejection, and OOD detection across the same eight external OOD sources as Tables 32–34, without retraining or OOD-based recalibration.
Table 35: DomainNet10 deployment results with L-ID-calibrated thresholds (%, five-seed means).
The results demonstrate a favorable preservation–detection trade-off under both aggregation protocols. Compared with FedEnergy, FedOPhD improves OOD detection by 19.15 and 31.04 percentage points under FedAvg and FedBN, respectively, while achieving lower cross-domain T-ID false-rejection rates (6.09% vs. 6.83% and 10.99% vs. 13.63%). Notably, under FedBN, FedOPhD achieves nearly the same OOD detection rate as FedViM (54.68% vs. 55.23%) while substantially reducing legitimate cross-domain T-ID rejection (10.99% vs. 42.20%). Moreover, among cross-domain T-ID samples correctly classified by the original model, FedOPhD's false-rejection rates decrease further to 2.81% and 6.13% under FedAvg and FedBN, respectively.