acceptodds
Under review as a conference paper at ICLR 2027

FedRALA: Federated Test-Time OOD Detection via Reliable Negative-Label Aggregation and Local Adaptation

Abstract

Vision-language models enable training-free out-of-distribution (OOD) detection by comparing image representations with in-distribution (ID) class prompts and negative textual labels. Recent work dynamically activates negative labels from unlabeled test streams to capture evolving OOD semantics, but mainly considers centralized test streams. In federated settings, OOD samples vary across clients, making the utility of negative labels dependent on local ID semantics and each client's current test distribution. Naive global averaging can suppress strong negative-label activations confined to a few clients and transfer labels that conflict with local ID semantics or respond weakly to local OOD samples. To address these challenges, we formulate federated test-time OOD detection through negative-label collaboration and propose FedRALA. FedRALA explicitly addresses three federation-specific failure modes: sparse cross-client OOD evidence, conflicts between shared negative labels and client-specific ID semantics, and temporal mismatch between historical shared evidence and the current local stream. FedRALA exchanges only prompt-level activation statistics, without sharing raw images or visual features. Experiments across multiple ID benchmarks and heterogeneous federated OOD streams validate its effectiveness. Notably, with TinyImageNet as the ID benchmark, FedRALA reduces the average FPR95 across six OOD datasets from 35.6% to 29.7% under severe client heterogeneity and disjoint OOD streams.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
out-of-distribution detection, federated learning, test-time adaptation, vision-language models, negative-label learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_izZgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FedRALA, a federated test-time OOD detection framework based on vision-language models. To address sparse cross-client OOD evidence, conflicts with local ID semantics, and temporal mismatches in evolving test streams, FedRALA introduces Top-K negative-label aggregation, a local Q95 constraint, and bounded local adaptation. Experiments on CIFAR-100 and TinyImageNet demonstrate consistent improvements over existing OOD detection baselines under heterogeneous federated settings.

Critical strengths
  1. Meaningful Problem Setting: The paper explores federated test-time OOD detection, extending existing centralized negative-label adaptation methods to heterogeneous federated environments.
  2. Clear Motivation and Method Design: The three proposed components directly address the identified challenges, with an intuitive and well-structured framework.
  3. Consistent Experimental Improvements: FedRALA demonstrates improvements over existing baselines across different ID datasets, OOD distributions, and federated settings.
  4. Training-Free and Communication-Efficient: The method requires no additional model training at test time and communicates only prompt-level activation statistics without sharing raw images or visual features.
Critical weaknesses
  1. Limited Methodological Novelty: The proposed components, including Top-K aggregation, quantile-based filtering, and bounded reordering, are relatively straightforward extensions of existing techniques, particularly TANL.
  2. Cross-Client Activation Inconsistency: Negative-label activations are computed using client-specific ID prompts, potentially resulting in inconsistent activation scales across clients. Direct Top-K aggregation may therefore introduce biases.
  3. Restrictive Theoretical Assumptions: The theoretical analysis relies on simplified activation distributions and i.i.d. memory samples, which may not adequately reflect heterogeneous and noisy federated test streams.
  4. Limited Statistical Validation: Most experiments rely on a single random seed, and some improvements are relatively small. Multiple runs with standard deviations are needed to demonstrate statistical reliability.
  5. Insufficient Evaluation of Temporal Distribution Shifts: Although temporal mismatch is a key motivation, the experiments primarily focus on heterogeneous client distributions rather than explicitly evolving OOD streams.
  6. Limited Contribution of Bounded Local Adaptation: The ablation results suggest that most improvements come from Top-K aggregation and the Q95 constraint, while bounded local adaptation provides only marginal additional gains.
Questions for authors
  1. Methodological Novelty: Could the authors further clarify the key methodological novelty of FedRALA beyond combining existing aggregation and negative-label adaptation techniques?
  2. Cross-Client Activation Consistency: Since negative-label activations are computed using client-specific ID prompts, how do the authors ensure that activation statistics are comparable across heterogeneous clients?
  3. Theoretical Assumptions: How well do the theoretical guarantees hold when the client activation distributions are more complex and the pseudo-labeled memories contain noisy or correlated samples?
  4. Statistical Reliability: Could the authors report results over multiple random seeds with standard deviations to verify the significance of the reported improvements, particularly in the 100-client setting?
  5. Temporal Distribution Shifts: How does FedRALA perform under explicitly non-stationary OOD streams, such as abrupt domain shifts or gradually emerging OOD categories?
  6. Effectiveness of Bounded Local Adaptation: Given its relatively marginal additional improvement in the ablation study, could the authors further demonstrate when and why bounded local adaptation is necessary?
11 comments

Comment 1: Limited Methodological Novelty: The proposed components, including Top-K aggregation, quantile-based filtering, and bounded reordering, are relatively straightforward extensions of existing techniques, particularly TANL. Methodological Novelty: Could the authors further clarify the key methodological novelty of FedRALA beyond combining existing aggregation and negative-label adaptation techniques?

Response 1:

Thank you for raising this important concern. We agree that Top-K aggregation, quantile estimation, and ranking are established operations. Our methodological contribution does not lie in introducing these operations individually, but in addressing a research problem that is not considered by existing test-time negative-label adaptation methods: how to enable reliable negative-label collaboration across heterogeneous clients when the usefulness of shared OOD evidence depends on the receiving client's ID semantics and evolving test distribution.

Existing methods such as TANL adapt negative labels using historical and current observations from a single test stream. In federated environments, however, evidence discovered by one client cannot be directly transferred to another. Our analysis identifies three interconnected difficulties. First, useful OOD evidence may be concentrated in only a few clients and consequently diluted by global averaging. Second, negative labels strongly activated by other clients may conflict with the receiving client's ID semantics. Third, even historically reliable and locally compatible labels may become less relevant as the client's OOD stream evolves. These difficulties mean that sharing negative-label statistics alone is insufficient; the shared evidence must also be evaluated for its reliability and applicability at each receiving client.

FedRALA addresses these difficulties through a coupled, problem-driven mechanism rather than an independent combination of existing techniques. Top-K aggregation first reduces the dilution of sparse OOD evidence under global averaging. However, strong cross-client evidence does not necessarily imply local validity. The receiving client's Q95 constraint therefore evaluates this shared evidence against its private ID semantics, producing client-specific negative-label utilities. Bounded reordering subsequently uses current-batch observations to refine the priorities of the selected labels, while restricting how strongly transient evidence can alter their reliability margins at each selection stage. Importantly, these operations act on successive representations of the same shared evidence. The output of cross-client aggregation is not directly adopted as the final negative-label ranking; it is conditioned on local ID compatibility and subsequently refined according to current-stream relevance.

This distinction is central to our contribution. Unlike Local TANL, FedRALA can exploit negative-label evidence accumulated by other clients. Unlike direct global averaging, it accounts for both sparse client support and the receiving client's semantic constraints. Unlike unconstrained batch-based adaptation, its bounded ranking limits the influence of transient observations on established reliability margins. Consequently, the same globally shared statistics can produce different negative-label priorities at different clients, according to their local ID semantics and current observations.

Our experiments provide evidence for these design decisions. The mechanism diagnostics identify sparse cross-client support, local ID conflicts, and historical-to-current stream mismatch. Component ablations show that reliable aggregation and local ID constraints account for most of the detection improvement, while bounded reordering provides a smaller, complementary benefit, particularly for early worst-client performance following OOD class changes. We have revised the Introduction and methodology overview to make this research problem, the dependencies between the components, and the scope of our methodological contribution more explicit.

Comment 2: Cross-Client Activation Inconsistency: Negative-label activations are computed using client-specific ID prompts, potentially resulting in inconsistent activation scales across clients. Direct Top-K aggregation may therefore introduce biases. Cross-Client Activation Consistency: Since negative-label activations are computed using client-specific ID prompts, how do the authors ensure that activation statistics are comparable across heterogeneous clients?

Response 2:

Thank you for raising cross-client activation-scale comparability. The shared encoder and negative-label bank provide common coordinates, but the softmax denominator containing local ID prompts can still introduce numerical differences. We address this through constrained prompt personalization, Top-2 averaging, and an empirical activation audit.

First, client ID prompts are not trained entirely independently. After global aggregation, each participating client blends the global and round-local prompt parameters with equal weights (50% each), providing a common global anchor. The CLIP encoders are frozen; only prompt prefixes are optimized, while class words remain fixed. All clients use the same ordered negative-label bank and temperature. These choices constrain prompt drift and maintain a common semantic reference for negative-label coordinates.

Second, upper-tail selection and averaging have distinct roles: selection retains strong OOD responses supported by a few clients, while averaging the top two rather than taking only the maximum reduces dependence on a single extreme response without diluting sparse evidence across all clients.

We audited TinyImageNet with 10 clients, αID=0.1\alpha_{\mathrm{ID}}=0.1, and K=2K=2. All clients processed the same 256 images from each of six OOD datasets while retaining their original ID classes and personalized prompts. Let R(x)R(x) be the maximum-to-minimum ratio of total negative-label probability mass across clients for the same image; R=1R=1 means equal total mass.

Matched-probe activation scales and online contribution diagnostics.

OOD dataset95th percentile of RRMaximum RRρ\rho range
SSBHard1.02513.1797[−0.406, 0.115]
NINCO1.027510.5235[−0.200, 0.406]
iNaturalist1.00021.0034[−0.527, −0.248]
SUN1.00071.4353[−0.067, 0.600]
Places1.00571.3513[−0.006, 0.285]
DTD1.00381.1245[−0.079, 0.503]

Scale statistics use these 1,536 matched OOD probes. We use Spearman's rank correlation ρ\rho to examine whether clients with higher probe-average negative-label probability mass are selected more often in online Top-KK aggregation. The online contribution audit uses three OOD partition seeds under the semantic-group-disjoint setting, excluding the first round and keeping the trained prompts fixed. The last column reports the range of correlations across these three partition seeds, not training seeds.

The 95th percentile of RR is below 1.028 for every dataset, and 1,520/1,536 probes (98.96%) satisfy R≤1.1R\leq1.1. Thus, most tested OOD images have similar total negative-label probability mass across clients, with a few larger tail differences. In the online audit, the highest-contributing client varies with the dataset and partition. The median ρ\rho across 18 configurations is −0.0303, with no consistent positive association between probe scale and aggregation occupancy.

Together, these audits support the practical comparability of negative-label activation statistics across the evaluated heterogeneous clients. Most matched images exhibit similar activation scales, and clients with larger probe-average activations do not consistently receive more Top-KK aggregation slots. Appendix I.1 (Table 22) includes the definitions, results, and diagnostic settings.

Comment 3: Restrictive Theoretical Assumptions: The theoretical analysis relies on simplified activation distributions and i.i.d. memory samples, which may not adequately reflect heterogeneous and noisy federated test streams. Theoretical Assumptions: How well do the theoretical guarantees hold when the client activation distributions are more complex and the pseudo-labeled memories contain noisy or correlated samples?

Response 3 (Part 1/2):

Thank you for asking about the scope of the theory. Our analysis characterizes the stability of negative-label selection and bounded reordering under the stated assumptions. Detection gains are evaluated empirically.

Distributional and independence assumptions. The two-level client-mean model isolates sparse support rather than assuming Gaussian activations. The appendix already extends it to deviations: if supporting and non-supporting client means differ from their respective reference levels by at most hmh_m, the error term increases from ϵ\epsilon to ϵ+hm\epsilon+h_m. FIFO-mean concentration uses softmax activations bounded in [0,1][0,1] and allows different client distributions. It requires conditionally i.i.d. samples within each FIFO given its admission distribution, not independence across clients or negative labels.

Structural properties conditional on estimation errors. Top-KK satisfies the deterministic perturbation bound

∣TK(x)−TK(y)∣≤∥x−y∥∞,|T_K(\mathbf x)-T_K(\mathbf y)|\leq\|\mathbf x-\mathbf y\|_\infty,

regardless of the vectors' distribution. The Q95-penalized set-recovery criterion relates the selection-boundary gap to mean and quantile estimation errors. Within each fixed-set bounded reordering, dm≤rm≤2dmd_m\leq r_m\leq2d_m, so di>2djd_i>2d_j prevents order reversal for any normalized current-batch response. The appendix's contamination extension also quantifies how memory contamination, anomalous clients, and density near the quantile affect utility-estimation error.

For actual online memories with feedback, correlation, and non-stationarity, we treat conditional-i.i.d. concentration as analysis under the stated assumptions, not a probability guarantee for arbitrary correlated sequences. We complement it with persistent enqueue perturbations that retain online scoring, gating, and subsequent aggregation, and the true-class non-stationary streams in Response 5.

Reviewer_jC53OpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes FedRALA, a training-free framework for federated test-time OOD detection. It combines Top-K aggregation, local Q95 filtering, and bounded adaptation to share negative-label evidence across heterogeneous clients while preserving local ID semantics. Experiments demonstrate improvements over local and naive global aggregation baselines, although the methodological novelty and theoretical guarantees remain relatively limited.

Critical strengths
  1. The setting is interesting. The detection results from each cluster can be shared to improve one another, which is an interesting direction.

  2. The experimental results demonstrate the validity of the proposed methods.

Critical weaknesses
  1. This paper is difficult to follow. The paper is technically detailed, but its main motivation and methodological contributions are unclear. In particular, the introduction presents three federation-specific failure modes without first clearly establishing why cross-client negative-label collaboration is necessary. Furthermore, the methodology introduces numerous intermediate statistics and two-stage adaptation procedures before intuitively explaining the overall algorithm. A simplified motivating example and a more concise description of the core Top-K aggregation, local ID filtering, and bounded adaptation steps would substantially improve readability.

  2. This paper appears to overstate the novelty and theoretical significance of its contributions. While federated test-time negative-label collaboration is a potentially meaningful setting, the proposed method primarily combines established aggregation, quantile filtering, and ranking techniques. Moreover, the theoretical guarantees largely formalize elementary properties of these operations without establishing improved OOD detection performance. Some claimed federation-specific challenges, such as temporal mismatch, are also common to centralized test-time adaptation. The authors should more carefully distinguish genuinely novel contributions from extensions of existing techniques and align their claims with what the theoretical and experimental evidence actually supports.

  3. Insufficient experimental validation of the central federated motivation. Although the paper reports extensive results across datasets, client configurations, and OOD benchmarks, the experiments do not fully establish the necessity and effectiveness of federated negative-label collaboration. In particular, the Disjoint protocol randomly partitions OOD samples rather than enforcing semantically disjoint OOD categories across clients, leaving the claimed benefit of cross-client semantic transfer insufficiently tested. Furthermore, improvements become relatively modest in the 100-client setting, and the bounded adaptation component contributes only marginal additional gains over Top-K aggregation with Q95 filtering. More targeted experiments on unseen OOD semantics, temporal distribution shifts, and statistical variability would substantially strengthen the empirical evidence.

Questions for authors

Please ref to weaknesses.

7 comments

Comment 1: This paper is difficult to follow. The paper is technically detailed, but its main motivation and methodological contributions are unclear. In particular, the introduction presents three federation-specific failure modes without first clearly establishing why cross-client negative-label collaboration is necessary. Furthermore, the methodology introduces numerous intermediate statistics and two-stage adaptation procedures before intuitively explaining the overall algorithm. A simplified motivating example and a more concise description of the core Top-K aggregation, local ID filtering, and bounded adaptation steps would substantially improve readability.

Response 1:

Thank you for the suggestions on improving the motivation and organization. Our starting point is that clients operating in a shared task environment have limited, asynchronous local test observations. Relevant OOD evidence accumulated elsewhere can supplement insufficient local evidence, without requiring all clients to use identical negative labels.

For example, in multi-sensor monitoring, client A may repeatedly encounter an unknown object type before B encounters the same or a semantically related type. When B has few local observations, A's negative-label activation statistics can provide supplementary evidence without transmitting images or visual features. Sharing does not imply indiscriminate reuse: a label may conflict with B's known classes or be irrelevant to its current stream. We therefore separate "what other clients have discovered" from "whether that evidence is applicable to this client."

The core procedure can first be summarized in three steps:

Core FedRALA steps and their roles.

Core stepIntuitionImplementation
Cross-client evidence aggregationReduce the dilution caused by averaging responses over all clientsAverage the top-KK client activation means separately for each negative label.
Local ID-semantic constraintSuppress conflicts between shared negative labels and local ID classesPenalize shared evidence using Q95 activations from local ID memory.
Current-stream adaptationRefine the use of historical evidence using current observationsFuse current-batch evidence and apply bounded reordering within each selected label set.

The two-stage implementation serves this intuition: preliminary detection using historical evidence yields high-confidence ID/OOD groups in the current batch; these observations are then fused to re-mine labels and compute final scores. The intermediate statistics estimate quantities needed for this process, rather than representing separate algorithmic objectives.

The revised Introduction now gives a collaboration example before presenting the challenges, and Section 3.1 summarizes the synchronized two-stage workflow. These explanations establish why collaboration is useful and how local semantics govern its use. The three-step summary above provides a concise guide to the method; Response 3 below gives the new staggered-arrival experiment and results.

Comment 2: This paper appears to overstate the novelty and theoretical significance of its contributions. While federated test-time negative-label collaboration is a potentially meaningful setting, the proposed method primarily combines established aggregation, quantile filtering, and ranking techniques. Moreover, the theoretical guarantees largely formalize elementary properties of these operations without establishing improved OOD detection performance. Some claimed federation-specific challenges, such as temporal mismatch, are also common to centralized test-time adaptation. The authors should more carefully distinguish genuinely novel contributions from extensions of existing techniques and align their claims with what the theoretical and experimental evidence actually supports.

Response 2 (Part 1/2):

Thank you for raising this important concern. We agree that Top-K aggregation, quantile estimation, and ranking are established operations. Our methodological contribution does not lie in introducing these operations individually, but in addressing a research problem that is not considered by existing test-time negative-label adaptation methods: how to enable reliable negative-label collaboration across heterogeneous clients when the usefulness of shared OOD evidence depends on the receiving client's ID semantics and evolving test distribution.

Existing methods such as TANL adapt negative labels using historical and current observations from a single test stream. In federated environments, however, evidence discovered by one client cannot be directly transferred to another. Our analysis identifies three interconnected difficulties. First, useful OOD evidence may be concentrated in only a few clients and consequently diluted by global averaging. Second, negative labels strongly activated by other clients may conflict with the receiving client's ID semantics. Third, even historically reliable and locally compatible labels may become less relevant as the client's OOD stream evolves. These difficulties mean that sharing negative-label statistics alone is insufficient; the shared evidence must also be evaluated for its reliability and applicability at each receiving client.

FedRALA addresses these difficulties through a coupled, problem-driven mechanism rather than an independent combination of existing techniques. Top-K aggregation first reduces the dilution of sparse OOD evidence under global averaging. However, strong cross-client evidence does not necessarily imply local validity. The receiving client's Q95 constraint therefore evaluates this shared evidence against its private ID semantics, producing client-specific negative-label utilities. Bounded reordering subsequently uses current-batch observations to refine the priorities of the selected labels, while restricting how strongly transient evidence can alter their reliability margins at each selection stage. Importantly, these operations act on successive representations of the same shared evidence. The output of cross-client aggregation is not directly adopted as the final negative-label ranking; it is conditioned on local ID compatibility and subsequently refined according to current-stream relevance.

This distinction is central to our contribution. Unlike Local TANL, FedRALA can exploit negative-label evidence accumulated by other clients. Unlike direct global averaging, it accounts for both sparse client support and the receiving client's semantic constraints. Unlike unconstrained batch-based adaptation, its bounded ranking limits the influence of transient observations on established reliability margins. Consequently, the same globally shared statistics can produce different negative-label priorities at different clients, according to their local ID semantics and current observations.

Our experiments provide evidence for these design decisions. The mechanism diagnostics identify sparse cross-client support, local ID conflicts, and historical-to-current stream mismatch. Component ablations show that reliable aggregation and local ID constraints account for most of the detection improvement, while bounded reordering provides a smaller, complementary benefit, particularly for early worst-client performance following OOD class changes. We have revised the Introduction and methodology overview to make this research problem, the dependencies between the components, and the scope of our methodological contribution more explicit.

Response 2 (Part 2/2):

Regarding theory, we distinguish mechanism properties from detection gains. Under the stated assumptions, the analysis relates sparse client support, estimation errors, and utility gaps to evidence retention and selection stability. Q95 set recovery concerns the population penalized utility, not an oracle-optimal detection set. Bounded-reordering guarantees concern order changes within each selected set, not direct AUROC/FPR95 improvement. Detection gains are evaluated empirically through comparisons, ablations, and the new independent runs and temporal experiments in Response 3.

Regarding "federation-specific," the revised abstract refers to "three challenges in federated negative-label collaboration," rather than claiming all three are unique to federation. Our temporal concern is that shared histories combine clients at different exposure stages and may not match the receiving client's present stream.

Reviewer_naWbOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focus on federated test‑time out‑of‑distribution (OOD) detection using vision‑language models. Prior negative‑label based OOD detection methods adapt textual negative concepts from unlabeled test streams but operate in centralized settings. In federated scenarios, each client holds local private data; clients exhibit heterogeneous in‑distribution (ID) semantics and encounter disjoint/non‑IID OOD samples. Simple global averaging of negative‑label statistics across clients suffers three key failure modes: Sparse cross‑client OOD evidence, ID‑semantic conflicts, and Temporal mismatch. To address these issues, the authors propose FedRALA, a training‑free federated framework that only exchanges prompt‑level activation statistics. Extensive experiments are conducted on multiple benchmarks.

Critical strengths
  1. The paper provide sufficient theoretical analysis: sparse‑signal preservation for top‑K aggregation, exact recovery conditions for Q95‑filtered negative‑label sets, and stability bounds for bounded local re‑ordering. It also analyzes memory concentration and contamination robustness.
  2. It has Comprehensive empirical evaluation.
Critical weaknesses

This seems to be an application of OOD technique on federated learning field.

Questions for authors
  1. Local FIFO memories rely on pseudo‑labelling from online OOD scores. Early pseudo‑label errors can pollute local ID/OOD memory buffers and propagate into uploaded activation statistics, which may harm downstream aggregation performance. The paper provides contamination analysis but no practical mitigation for pseudo‑label drift.
  2. Federated OOD works seems to conduct evaluations using generic OOD benchmarks. I am curious why several well‑established authoritative OOD benchmarks (e.g. OpenOOD) are not utilized in the present study.
2 comments

Comment 1: Local FIFO memories rely on pseudo‑labelling from online OOD scores. Early pseudo‑label errors can pollute local ID/OOD memory buffers and propagate into uploaded activation statistics, which may harm downstream aggregation performance. The paper provides contamination analysis but no practical mitigation for pseudo‑label drift.

Response 1:

Thank you for highlighting the propagation of online pseudo-label errors. We address this concern through the existing safeguards and a new stress test with continuous errors in FIFO memory updates.

Practical safeguards. Only samples with high-confidence ID or OOD predictions from preliminary detection enter the corresponding memory. The adaptive threshold and confidence gap exclude uncertain samples. Each activation FIFO stores at most 300 entries. Once a FIFO is full, newly admitted samples replace its oldest entries. Both memories are updated only after final batch scoring, affecting subsequent batches and aggregation rounds. Bounded reordering preserves the label set selected for each reordering operation and limits the reliability-margin multiplier to [1,2][1,2], constraining the current batch's influence on the historical ordering. These safeguards require neither ground-truth labels nor additional supervision.

New noisy-memory stress test. We evaluate Disjoint and Non-IID OOD with TinyImageNet as ID, ten clients, six OOD datasets, and αID=0.1\alpha_{\mathrm{ID}}=0.1. Non-IID uses αOOD=0.1\alpha_{\mathrm{OOD}}=0.1. Both protocols use training seed 1, OOD partition seed 1, and batch size 128. Within each protocol, prompts, test images, and sample order stay fixed. Starting with the first batch, we score normally and then place each activation vector selected for a memory update in the opposite ID/OOD buffer with the chosen flip probability. Rejected samples remain excluded. Scoring and adaptation rules stay unchanged, and the modified memories affect subsequent batches and shared statistics.

For each noise seed, we first average results over ten clients and six datasets within each protocol, then equally average Disjoint and Non-IID. Nonzero probabilities report mean ± sample standard deviation over three noise seeds, which change only the flips, not training. The 0% row averages one unperturbed replay per protocol and retains the original pseudo-label errors. Both replays match their saved original scores within 1.11×10−161.11\times10^{-16}.

FIFO memory robustness averaged over Disjoint and Non-IID (%).

Flip probabilityMethodAUROC ↑FPR95 ↓
0%Local TANL91.3033.37
0%Global Mean90.2635.71
0%FedRALA92.3929.33
10%Local TANL91.39 ± 0.0333.00 ± 0.06
10%Global Mean89.69 ± 0.0336.37 ± 0.08
10%FedRALA92.38 ± 0.0229.46 ± 0.10
20%Local TANL91.35 ± 0.0432.69 ± 0.04
20%Global Mean89.53 ± 0.1036.35 ± 0.19
20%FedRALA92.26 ± 0.0229.74 ± 0.05
50%Local TANL90.61 ± 0.0434.28 ± 0.11
50%Global Mean88.56 ± 0.0237.54 ± 0.12
50%FedRALA91.77 ± 0.0531.23 ± 0.03

The flip probability is not the actual percentage of wrong memory entries. True ID/OOD labels are used only for this audit, not for detection or memory updates. Within each protocol, dataset, and buffer, we count wrong and total real-image entries after every client batch, then divide summed wrong counts by summed entry counts. Initialization entries are excluded. We average the fractions over six datasets and two protocols, then over three noise seeds for nonzero probabilities. At 20% flips, FedRALA's protocol-averaged ID-buffer error fraction rises from 4.01% to 40.36%, and its OOD-buffer error fraction from 38.15% to 39.85%. Memory errors therefore genuinely increase.

FedRALA has higher protocol-averaged AUROC and lower FPR95 than both controls at all four reported levels. Relative to 0%, 20% flips reduce its AUROC by 0.13 percentage points and increase FPR95 by 0.41 points. At 50%, AUROC is 91.77 ± 0.05% and FPR95 is 31.23 ± 0.03%, corresponding to changes of −0.62 and +1.90 points.

The VLM and prompts remain fixed. Noisy memory statistics influence which negative labels are selected and how they are ordered. FedRALA remains relatively stable under low-to-moderate memory perturbations and degrades gradually at higher flip probabilities, while retaining its protocol-averaged advantage over both controls. Appendix I.3 (Table 24) reports the experimental setup and protocol-averaged results at the four representative flip probabilities.

Comment 2: Federated OOD works seems to conduct evaluations using generic OOD benchmarks. I am curious why several well‑established authoritative OOD benchmarks (e.g. OpenOOD) are not utilized in the present study.

Response 2:

Thank you for the suggestion. We clarify that our experiments do use data resources and image lists provided by OpenOOD. The original appendix subsection "Datasets and federated streams" listed the datasets and described following OpenOOD's data organization, but this source was not sufficiently prominent in the original main text.

The configurations are as follows:

Benchmark data sources and federated evaluation protocols.

ID datasetPaired OOD datasetsRelationship to OpenOOD
CIFAR-100CIFAR-10, TinyImageNet, MNIST, SVHN, DTD, Places365Uses the corresponding OpenOOD data resources and OOD image lists; DTD is named Texture in the code.
TinyImageNet (all 200 classes)SSBHard, NINCO, iNaturalist, SUN, Places, DTDA custom TinyImageNet-ID extension combining OpenOOD data resources, distinct from its official standard ID protocols.

Our extension constructs heterogeneous client partitions and online test streams on these existing resources to evaluate cross-client collaboration over dynamically activated negative labels. Disjoint and Non-IID specify how OOD samples are allocated to clients and combined with local ID samples; they are not new dataset benchmarks replacing OpenOOD. The revised main experimental setup now highlights the OpenOOD source, and Appendix A.3 explicitly distinguishes the custom full-200-class TinyImageNet pairing from an official standard OpenOOD ID protocol.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers (naWb, jC53, and izZg) for their time and constructive comments. In this work, we present FedRALA, to our knowledge, the first framework for collaborative negative-label adaptation across heterogeneous federated test streams. FedRALA enables evidence discovered by one client to be reliably reused by another through a receiving-client-conditioned selection and adaptation mechanism that accounts for local ID semantics and current observations. We have posted point-by-point responses and uploaded the revised manuscript.

The main concerns generally fall into four aspects:

  • (1) Methodological novelty and component dependencies (jC53, izZg): Whether FedRALA provides a methodological advance beyond TANL and whether the components have a nontrivial dependency, rather than merely being applied sequentially.
  • (2) Cross-client activation comparability (izZg): Whether client-dependent activation scales bias Top-K aggregation, requiring normalization controls and actual client-selection analysis.
  • (3) Temporal adaptation and statistical reliability (jC53, izZg): Whether collaboration remains effective under explicit OOD-class changes and whether the reported gains are stable across random seeds, particularly with 100 clients.
  • (4) Noisy memories and theoretical scope (naWb, izZg): How pseudo-label errors affect FIFO memories and aggregation, and how the theoretical guarantees relate to correlated, prediction-dependent memory updates.

Regarding each concern, we summarize our response as follows:

  • (1): Shared evidence and local Q95 constraints jointly determine label utilities and selection-boundary margins. Those same margins bound current-batch reordering, so the local constraint affects both which labels are retained and how their priorities adapt. We added the requested Mean + Q95 control. In the tested TinyImageNet Non-IID setting, with Q95 fixed and bounded reordering disabled, Top-2 + Q95 improves mean AUROC by 0.44 percentage points and reduces mean FPR95 by 1.06 points. Both metrics improve in all three OOD-partition seeds.

  • (2): We applied normalization consistently to initialization proxies, both FIFOs, uploaded means, Q95, and current-batch statistics. Across both ID benchmarks and both OOD protocols, normalization produces small performance changes, while the maximum matched client-contribution change is below 0.39 percentage points. These results support that the observed performance is not primarily driven by client-dependent activation scales in the evaluated settings.

  • (3): We constructed staggered streams using verified true OOD classes. In the first two post-switch batches, full FedRALA achieves worst-client FPR95 of 41.84 ± 3.47%, compared with 43.92 ± 4.05% without bounded reordering. The 100-client evaluation now reports mean and sample standard deviation over three independent full runs, including training and ID/OOD partitioning. Separately, the new component controls use three OOD-partition seeds with fixed trained prompts. These two forms of statistical validation are explicitly distinguished.

  • (4): We evaluated continuous FIFO admission-direction perturbations in the complete online loop. The original confidence gates and finite-length FIFO updates are retained. Averaged over Disjoint and Non-IID, FedRALA's FPR95 changes from 29.33% without extra flips to 29.74% at 20% flips and 31.23% at 50% flips, retaining its advantage over Local TANL and Global Mean. We also distinguish finite-memory estimation bounds under the stated sampling assumptions, deterministic aggregation and fixed-set reordering properties, and empirical robustness under evolving streams and prediction-dependent memory updates.

The revised manuscript includes the 100-client results in Table 2 and the additional analyses in Appendix I, with revisions highlighted in red. The new matched component controls and normalization analyses appear in Appendices I.4–I.5.

We would appreciate your feedback on whether these additions and clarifications address your concerns. We are happy to clarify any remaining questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.