- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FedRALA, a federated test-time OOD detection framework based on vision-language models. To address sparse cross-client OOD evidence, conflicts with local ID semantics, and temporal mismatches in evolving test streams, FedRALA introduces Top-K negative-label aggregation, a local Q95 constraint, and bounded local adaptation. Experiments on CIFAR-100 and TinyImageNet demonstrate consistent improvements over existing OOD detection baselines under heterogeneous federated settings.
Critical strengths
- Meaningful Problem Setting: The paper explores federated test-time OOD detection, extending existing centralized negative-label adaptation methods to heterogeneous federated environments.
- Clear Motivation and Method Design: The three proposed components directly address the identified challenges, with an intuitive and well-structured framework.
- Consistent Experimental Improvements: FedRALA demonstrates improvements over existing baselines across different ID datasets, OOD distributions, and federated settings.
- Training-Free and Communication-Efficient: The method requires no additional model training at test time and communicates only prompt-level activation statistics without sharing raw images or visual features.
Critical weaknesses
- Limited Methodological Novelty: The proposed components, including Top-K aggregation, quantile-based filtering, and bounded reordering, are relatively straightforward extensions of existing techniques, particularly TANL.
- Cross-Client Activation Inconsistency: Negative-label activations are computed using client-specific ID prompts, potentially resulting in inconsistent activation scales across clients. Direct Top-K aggregation may therefore introduce biases.
- Restrictive Theoretical Assumptions: The theoretical analysis relies on simplified activation distributions and i.i.d. memory samples, which may not adequately reflect heterogeneous and noisy federated test streams.
- Limited Statistical Validation: Most experiments rely on a single random seed, and some improvements are relatively small. Multiple runs with standard deviations are needed to demonstrate statistical reliability.
- Insufficient Evaluation of Temporal Distribution Shifts: Although temporal mismatch is a key motivation, the experiments primarily focus on heterogeneous client distributions rather than explicitly evolving OOD streams.
- Limited Contribution of Bounded Local Adaptation: The ablation results suggest that most improvements come from Top-K aggregation and the Q95 constraint, while bounded local adaptation provides only marginal additional gains.
Questions for authors
- Methodological Novelty: Could the authors further clarify the key methodological novelty of FedRALA beyond combining existing aggregation and negative-label adaptation techniques?
- Cross-Client Activation Consistency: Since negative-label activations are computed using client-specific ID prompts, how do the authors ensure that activation statistics are comparable across heterogeneous clients?
- Theoretical Assumptions: How well do the theoretical guarantees hold when the client activation distributions are more complex and the pseudo-labeled memories contain noisy or correlated samples?
- Statistical Reliability: Could the authors report results over multiple random seeds with standard deviations to verify the significance of the reported improvements, particularly in the 100-client setting?
- Temporal Distribution Shifts: How does FedRALA perform under explicitly non-stationary OOD streams, such as abrupt domain shifts or gradually emerging OOD categories?
- Effectiveness of Bounded Local Adaptation: Given its relatively marginal additional improvement in the ablation study, could the authors further demonstrate when and why bounded local adaptation is necessary?
11 comments
Comment 2: Cross-Client Activation Inconsistency: Negative-label activations are computed using client-specific ID prompts, potentially resulting in inconsistent activation scales across clients. Direct Top-K aggregation may therefore introduce biases. Cross-Client Activation Consistency: Since negative-label activations are computed using client-specific ID prompts, how do the authors ensure that activation statistics are comparable across heterogeneous clients?
Response 2:
Thank you for raising cross-client activation-scale comparability. The shared encoder and negative-label bank provide common coordinates, but the softmax denominator containing local ID prompts can still introduce numerical differences. We address this through constrained prompt personalization, Top-2 averaging, and an empirical activation audit.
First, client ID prompts are not trained entirely independently. After global aggregation, each participating client blends the global and round-local prompt parameters with equal weights (50% each), providing a common global anchor. The CLIP encoders are frozen; only prompt prefixes are optimized, while class words remain fixed. All clients use the same ordered negative-label bank and temperature. These choices constrain prompt drift and maintain a common semantic reference for negative-label coordinates.
Second, upper-tail selection and averaging have distinct roles: selection retains strong OOD responses supported by a few clients, while averaging the top two rather than taking only the maximum reduces dependence on a single extreme response without diluting sparse evidence across all clients.
We audited TinyImageNet with 10 clients, , and . All clients processed the same 256 images from each of six OOD datasets while retaining their original ID classes and personalized prompts. Let be the maximum-to-minimum ratio of total negative-label probability mass across clients for the same image; means equal total mass.
Matched-probe activation scales and online contribution diagnostics.
| OOD dataset | 95th percentile of | Maximum | range |
|---|---|---|---|
| SSBHard | 1.0251 | 3.1797 | [−0.406, 0.115] |
| NINCO | 1.0275 | 10.5235 | [−0.200, 0.406] |
| iNaturalist | 1.0002 | 1.0034 | [−0.527, −0.248] |
| SUN | 1.0007 | 1.4353 | [−0.067, 0.600] |
| Places | 1.0057 | 1.3513 | [−0.006, 0.285] |
| DTD | 1.0038 | 1.1245 | [−0.079, 0.503] |
Scale statistics use these 1,536 matched OOD probes. We use Spearman's rank correlation to examine whether clients with higher probe-average negative-label probability mass are selected more often in online Top- aggregation. The online contribution audit uses three OOD partition seeds under the semantic-group-disjoint setting, excluding the first round and keeping the trained prompts fixed. The last column reports the range of correlations across these three partition seeds, not training seeds.
The 95th percentile of is below 1.028 for every dataset, and 1,520/1,536 probes (98.96%) satisfy . Thus, most tested OOD images have similar total negative-label probability mass across clients, with a few larger tail differences. In the online audit, the highest-contributing client varies with the dataset and partition. The median across 18 configurations is −0.0303, with no consistent positive association between probe scale and aggregation occupancy.
Together, these audits support the practical comparability of negative-label activation statistics across the evaluated heterogeneous clients. Most matched images exhibit similar activation scales, and clients with larger probe-average activations do not consistently receive more Top- aggregation slots. Appendix I.1 (Table 22) includes the definitions, results, and diagnostic settings.
Comment 3: Restrictive Theoretical Assumptions: The theoretical analysis relies on simplified activation distributions and i.i.d. memory samples, which may not adequately reflect heterogeneous and noisy federated test streams. Theoretical Assumptions: How well do the theoretical guarantees hold when the client activation distributions are more complex and the pseudo-labeled memories contain noisy or correlated samples?
Response 3 (Part 1/2):
Thank you for asking about the scope of the theory. Our analysis characterizes the stability of negative-label selection and bounded reordering under the stated assumptions. Detection gains are evaluated empirically.
Distributional and independence assumptions. The two-level client-mean model isolates sparse support rather than assuming Gaussian activations. The appendix already extends it to deviations: if supporting and non-supporting client means differ from their respective reference levels by at most , the error term increases from to . FIFO-mean concentration uses softmax activations bounded in and allows different client distributions. It requires conditionally i.i.d. samples within each FIFO given its admission distribution, not independence across clients or negative labels.
Structural properties conditional on estimation errors. Top- satisfies the deterministic perturbation bound
regardless of the vectors' distribution. The Q95-penalized set-recovery criterion relates the selection-boundary gap to mean and quantile estimation errors. Within each fixed-set bounded reordering, , so prevents order reversal for any normalized current-batch response. The appendix's contamination extension also quantifies how memory contamination, anomalous clients, and density near the quantile affect utility-estimation error.
For actual online memories with feedback, correlation, and non-stationarity, we treat conditional-i.i.d. concentration as analysis under the stated assumptions, not a probability guarantee for arbitrary correlated sequences. We complement it with persistent enqueue perturbations that retain online scoring, gating, and subsequent aggregation, and the true-class non-stationary streams in Response 5.
Comment 1: Limited Methodological Novelty: The proposed components, including Top-K aggregation, quantile-based filtering, and bounded reordering, are relatively straightforward extensions of existing techniques, particularly TANL. Methodological Novelty: Could the authors further clarify the key methodological novelty of FedRALA beyond combining existing aggregation and negative-label adaptation techniques?
Response 1:
Thank you for raising this important concern. We agree that Top-K aggregation, quantile estimation, and ranking are established operations. Our methodological contribution does not lie in introducing these operations individually, but in addressing a research problem that is not considered by existing test-time negative-label adaptation methods: how to enable reliable negative-label collaboration across heterogeneous clients when the usefulness of shared OOD evidence depends on the receiving client's ID semantics and evolving test distribution.
Existing methods such as TANL adapt negative labels using historical and current observations from a single test stream. In federated environments, however, evidence discovered by one client cannot be directly transferred to another. Our analysis identifies three interconnected difficulties. First, useful OOD evidence may be concentrated in only a few clients and consequently diluted by global averaging. Second, negative labels strongly activated by other clients may conflict with the receiving client's ID semantics. Third, even historically reliable and locally compatible labels may become less relevant as the client's OOD stream evolves. These difficulties mean that sharing negative-label statistics alone is insufficient; the shared evidence must also be evaluated for its reliability and applicability at each receiving client.
FedRALA addresses these difficulties through a coupled, problem-driven mechanism rather than an independent combination of existing techniques. Top-K aggregation first reduces the dilution of sparse OOD evidence under global averaging. However, strong cross-client evidence does not necessarily imply local validity. The receiving client's Q95 constraint therefore evaluates this shared evidence against its private ID semantics, producing client-specific negative-label utilities. Bounded reordering subsequently uses current-batch observations to refine the priorities of the selected labels, while restricting how strongly transient evidence can alter their reliability margins at each selection stage. Importantly, these operations act on successive representations of the same shared evidence. The output of cross-client aggregation is not directly adopted as the final negative-label ranking; it is conditioned on local ID compatibility and subsequently refined according to current-stream relevance.
This distinction is central to our contribution. Unlike Local TANL, FedRALA can exploit negative-label evidence accumulated by other clients. Unlike direct global averaging, it accounts for both sparse client support and the receiving client's semantic constraints. Unlike unconstrained batch-based adaptation, its bounded ranking limits the influence of transient observations on established reliability margins. Consequently, the same globally shared statistics can produce different negative-label priorities at different clients, according to their local ID semantics and current observations.
Our experiments provide evidence for these design decisions. The mechanism diagnostics identify sparse cross-client support, local ID conflicts, and historical-to-current stream mismatch. Component ablations show that reliable aggregation and local ID constraints account for most of the detection improvement, while bounded reordering provides a smaller, complementary benefit, particularly for early worst-client performance following OOD class changes. We have revised the Introduction and methodology overview to make this research problem, the dependencies between the components, and the scope of our methodological contribution more explicit.