acceptodds
Under review as a conference paper at ICLR 2027

Label-Prior Leakage in Federated Evaluation and an Inference Management Fix

Abstract

Federated learning enables collaborative medical AI without sharing patient data, yet post-deployment inference and evaluation remain largely unmanaged. We identify a previously overlooked label-prior leakage in personalized federated evaluation: client-matched testing of private heads implicitly reveals client identity and its label distribution, inflating apparent performance even for a pure prior-only predictor. To address both training heterogeneity and reliable deployment, we introduce FedGIM, a proximal-regularized method with private heads and class-balanced local objectives, together with Federated Inference Management (FedIM). FedIM is a model-agnostic post-training layer that coordinates four stages: Adapt (entropy-minimizing test-time adaptation), Monitor (MMD and Kolmogorov-Smirnov drift detection), Act (confidence-gated computational escalation, optional differential-privacy logit release, and model fusion), and Refer (selective classification with explicit coverage control and human referral). Evaluated on non-IID endoscopic screening under a leakage-aware pooled protocol with label-free feature-space routing, FedGIM recovers near-centralized accuracy while FedIM provides controllable coverage-risk trade-offs, drift response, latency-accuracy balance, and privacy-utility curves. Our results demonstrate that evaluation protocol choice can dominate reported gains and that managed inference is essential for clinically trustworthy federated systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federated Learning, Personalized Federated Learning, Test-Time Adaptation, Selective Classification, Dataset Shift, Differential Privacy

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GHzjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies personalized federated evaluation and proposes a combined training and inference-management framework. FedGIM combines proximal regularization, private prediction heads, and class-balanced objectives. FedIM adds test-time adaptation, drift monitoring, computational escalation, output perturbation, and selective referral. The paper argues that client-matched evaluation exposes label-prior information and proposes pooled evaluation with feature-based head routing. Experiments are conducted on an endoscopic screening task. My main concerns are the justification of the evaluation setting, limited methodological novelty, and insufficient evidence that the inference-management layer provides a useful advantage.

Critical strengths

A useful evaluation sanity check. The prior-only experiment demonstrates that client identity and client-specific prevalence can contribute substantially to aggregate discrimination. This is a useful reminder to distinguish image-derived predictive information from between-client distributional information.

Attention to deployment behavior beyond accuracy. Considering drift, computational cost, abstention, and referral is relevant to medical AI deployment. Reporting results across five seeds and including component ablations are positive aspects of the evaluation.

Critical weaknesses

The central leakage claim is insufficiently justified for personalized federated deployment. In a system deployed locally at a participating hospital, the client identity is naturally known and determines which personalized model is available. Using that hospital’s training-derived class prior is not automatically test-label leakage. The prior-only control demonstrates predictive information in client identity, but does not establish that using this information is illegitimate under the intended deployment conditions.

Pooled evaluation with unknown client identity addresses a different inference problem. The paper needs to explain why this is the appropriate target setting and distinguish evaluation shortcuts from legitimate personalization. Otherwise, performance differences may reflect the removal of available deployment information and the introduction of routing errors, rather than correction of an invalid evaluation protocol.

The deployment architecture also remains unclear. Private heads are described as remaining local, whereas the router selects among client-specific heads. The paper should explain where routing occurs and how those heads are accessed. Separately, output-level differential privacy requires a specified recipient, adversary, and protected information. Its necessity is not established merely by invoking federated learning, especially when predictions remain within the originating institution.

Limited novelty and empirical evidence do not establish the value or generality of the combined framework. FedGIM combines established proximal regularization, private-head personalization, and class balancing. FedIM similarly assembles established adaptation, drift detection, quantization, model fusion, output perturbation, and selective prediction techniques. The paper does not clearly identify a new coordination principle or demonstrate an advantage attributable to that coordination.

All empirical validation concerns one endoscopic screening task. Although Algorithm 1 is written in generic notation, the experiments do not establish that its choices, thresholds, and interactions transfer to other tasks or acquisition conditions. This is insufficient support for the broad model-agnostic deployment claims.

Moreover, Table 2 shows that adding FedIM to FedGIM reduces AUROC from 0.8572 to 0.8419 and F1 from 0.7218 to 0.6984, while increasing ECE from 0.0612 to 0.0923. Such losses might be justified by a demonstrated operational benefit, but the paper does not establish that benefit against appropriate matched alternatives. Figure 5 compares FedGIM plus FedIM with FedAvg, changing both training and inference simultaneously. A comparison against the same FedGIM model with a standard confidence-based referral policy is needed to isolate the contribution of inference management.

Questions for authors

Deployment and leakage: Is client identity known at deployment? If inference occurs at the originating hospital, why is selecting its personalized head considered leakage? Please distinguish known-client personalization from inference on samples with unknown provenance.

Routing architecture: Where does the pooled router run, and how does it access private heads that are stated to remain local? What deployment scenario requires this routing mechanism?

Novelty and usefulness: What specific methodological contribution goes beyond combining established components? Does FedIM improve risk–coverage or latency–utility behavior over the same trained model with standard selective prediction, under matched privacy and computational constraints?

Privacy assumptions: Who receives the perturbed outputs, what information is protected, and how is the sensitivity bound Δ established for the released log probabilities? The stated Gaussian guarantee depends on this bound.

1 comment
AuthorsOpenReview ↗

Response to GHzj: deployment setting, novelty, routing architecture, privacy assumptions

Thank you for the thorough review. We take the concerns about deployment setting, novelty and privacy seriously.

1. Leakage and known-client deployment. We agree: if inference runs at the originating hospital, selecting its personalized head is legitimate personalization, not leakage. Our claim should have been limited to samples of unknown provenance, where the head must be inferred. We will rewrite the framing to distinguish the two settings, stop describing known-client gains as artifactual, and report pooled and within-client AUROC under known client identity, plus oracle-identity pooled evaluation to separate client information from routing error.

2. Routing architecture. [Answer from your actual design.] In our setup, "private" means the head is excluded from aggregation during training (Eqs. 4, 20); it does not mean the head is unavailable at inference. The router (Eq. 5) uses only training-set feature centroids and runs [where, e.g., at a coordinating service holding the heads, or within a consortium of participating sites]. The motivating scenario is [e.g., pooled screening or referral of cases whose originating site is unknown]. We will state this architecture explicitly and acknowledge where it conflicts with stricter notions of head locality.

3. Novelty. We agree that each component (proximal regularization, private heads, class balancing, TTA, drift detection, quantized escalation, output perturbation, selective prediction) is established, and we do not claim otherwise. Our contribution is their coordination: drift detection gates adaptation, confidence gates escalation, and referral operates on the released privacy-perturbed posterior, so decisions share state. We acknowledge the paper does not yet demonstrate an advantage attributable to this coordination. The matched comparison below tests it directly, and we will narrow our claims if it does not hold.

4. Table 2 and Figure 5. We acknowledge the losses (AUROC 0.8572 to 0.8419, F1 0.7218 to 0.6984, ECE 0.0612 to 0.0923) and that Figure 5 changes training and inference together. We will add, on the same FedGIM model, no management vs. standard calibrated confidence-based referral vs. full FedIM, at matched coverage, computational cost and privacy setting, with risk-coverage and latency-utility curves.

5. Single task. We agree that one endoscopic task does not support broad model-agnostic claims. We will [add a second dataset if feasible / otherwise] restrict our claims to this setting and list the limitation explicitly.

6. Privacy assumptions. We will specify the recipient of the perturbed outputs, the adversary and the protected information, define the neighboring relation, and establish Δ\Delta by clipping released log-probabilities to [−B,0][-B,0] (sensitivity at most BCB\sqrt{C}), then recompute σ\sigma and re-run the privacy-utility experiments. [If output-level DP cannot be justified for the intended deployment, we will present it as an optional component rather than a core claim.]

We thank you for the specific questions, which will make the paper more precise.

Reviewer_6JxUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines how client identity and heterogeneous label priors affect the evaluation of personalized federated learning. It compares client-matched evaluation with pooled testing using a feature-space router constructed from training data, and presents a prior-only control to quantify the contribution of client information. The proposed FedGIM training recipe combines proximal regularization, private prediction heads, and class-balanced losses. A separate FedIM layer coordinates drift monitoring, entropy-based test-time adaptation, computational escalation, optional output perturbation, and selective referral. On an endoscopic screening task, the paper reports performance approaching a centralized reference under pooled evaluation, substantially higher AUROC under client-matched testing, and coverage-risk and privacy-utility trade-offs across five seeds.

Critical strengths
  1. The protocol comparison and prior-only control expose an informative evaluation confound. Appendix H.2 reports FedGIM AUROC of 0.8512 under pooled routing and 0.9387 under client-matched evaluation. Appendix H.3 shows that a predictor without image information can achieve AUROC of 0.8717 at Dirichlet concentration 0.05 when client identity is available. These controls make the contribution of client information concrete and motivate reporting performance conditional on the intended deployment setting, rather than treating aggregate discrimination as evidence of image understanding.
  2. The inference-management specification provides a useful framework for evaluating deployment trade-offs. Algorithm 1 and Appendix C explicitly connect monitoring, adaptation, prediction-path selection, and referral, while Table 2 reports the associated deterioration in predictive performance and calibration rather than implying that every component improves accuracy. The training-only comparisons include centralized, local, and personalized federated references, with results summarized over five seeds in Table 1. This combination provides a reasonable starting point for a more controlled assessment of operational utility.
Critical weaknesses
  1. The scope of the leakage claim needs qualification. Appendix B.1 explicitly defines a pooled setting in which true client identity is unavailable, and H.3 shows that revealing this identity can produce high aggregate AUROC without image information. This is a useful control for that target setting. However, the conclusion characterizes client-matched gains more generally as artifactual. In a known-hospital deployment, selecting that hospital's personalized head can be legitimate. Moreover, the H.2 comparison changes both the available client information and the routing procedure, so its AUROC gap does not isolate these contributions. A prior-only score is constant within each client; its aggregate AUROC can arise from cross-client prevalence differences. Distinguishing the known-client and unknown-client targets, and reporting pooled versus within-client discrimination, would support a more precise conclusion.
  2. The evidence for the additional operational benefit of FedIM remains incomplete. Table 2 provides a same-learner comparison and Figure 4 provides component ablations; these are useful existing controls. For FedGIM, Table 2 nevertheless reports AUROC changing from 0.8572 to 0.8419, F1 from 0.7218 to 0.6984, and ECE from 0.0612 to 0.0923 after adding FedIM. As Section 4.3 acknowledges, these costs must be assessed against operational benefits. Figure 5 compares FedAvg with FedGIM plus FedIM, so it does not establish the management layer's benefit over simple uncertainty-based referral on the same FedGIM model at matched coverage.
  3. There are also unresolved differences between the described policy, supplied implementation, and reported settings. Equations (52)–(53) calibrate a threshold independently, whereas src/inference_manager.py (lines 232–248) ranks the current test outputs. Such unlabeled ranking can support a retrospective risk–coverage analysis, but it does not evaluate the stated calibrated threshold. Section 4.4 reports coverage/referral of 0.9287/0.0713. The supplied default target coverage of 0.90 caps the accepted mask at floor(0.90N), so this pair requires a different configuration or pipeline. This is a reproducibility question: an independently calibrated threshold can legitimately yield test coverage above its calibration target. Similarly, Table 1's Specificity column may use a different threshold from Sens@90%Spec; if the same operating point is intended, its values below 0.90 need reconciliation with the supplied test-ROC routine.
Questions for authors
  1. Please clarify whether the leakage conclusion is restricted to the unknown-client setting in Appendix B.1, and how it applies to deployment at a known hospital. Can you separate the contribution of client information from routing error, and report both pooled AUROC and macro-averaged within-client AUROC, including the prior-only control and stating how single-class clients are handled? This would clarify which prediction target supports the claimed evaluation correction.
  2. Please identify the exact configurations, referral calibration procedure, and metric thresholds that generated Tables 1–2 and Section 4.4, and reconcile them with the supplied code. Can you compare FedIM against uncertainty-based referral using the same FedGIM model at matched coverage, and distinguish modeled latency from measured end-to-end conditional execution? For the privacy–utility experiments, what neighboring-input relation is intended and how is the sensitivity of 1.0 established? These clarifications and controls address the current claims, rather than requesting new application domains.
1 comment
AuthorsOpenReview ↗

Response to 6JxU: leakage scope, matched referral baseline, code-paper reconciliation

Thank you for the detailed review and for engaging with the supplied code. We respond to each point below.

1. Scope of the leakage claim. We agree the conclusion over-generalizes. Appendix B.1 defines the setting as pooled evaluation where true client identity is unavailable, and we will restrict the conclusion to that setting. In a known-hospital deployment, selecting that hospital's head is legitimate. We also agree that the H.2 comparison (0.9387 vs. 0.8512 for FedGIM) changes both the client information and the routing procedure, so it does not isolate them. We will:

  • Add pooled evaluation with oracle client identity (true head, pooled metrics) to separate the effect of client information from routing error.
  • Report pooled AUROC and macro-averaged within-client AUROC for all methods, including the prior-only control. Single-class clients will be [state your rule, e.g., excluded from the macro average and counted separately].
  • Make clear that a prior-only score is constant within a client, so its aggregate AUROC arises only from cross-client prevalence differences.

2. Evidence for FedIM's benefit. We agree that Figure 5 compares FedAvg with FedGIM+FedIM, changing training and inference together. We will add a same-model comparison on FedGIM: no management vs. calibrated uncertainty-based referral vs. full FedIM, at matched coverage and computational cost. [Report the outcome honestly, including if FedIM does not improve risk-coverage.]

3. Policy, code and reported settings.

  • Threshold calibration. [Confirm the actual behavior.] Eqs. (52)-(53) describe a threshold calibrated independently, while src/inference_manager.py (lines 232-248) ranks the current test outputs, which supports only a retrospective risk-coverage analysis. We will align the implementation with the stated policy: the threshold is set on a calibration split and then applied to held-out test data. We will release the updated code.
  • Coverage 0.9287 / referral 0.0713 (Sec. 4.4). We will document the exact configuration that produced these values. As you note, a threshold calibrated independently can give test coverage above its calibration target. [Confirm this is the explanation before stating it.]
  • Table 1 thresholds. We will state whether the Specificity column and Sens@90%Spec share an operating point, and reconcile any Specificity values below 0.90 with the evaluation routine.
  • Table identification. We will list the configurations, referral calibration procedure and metric thresholds behind Tables 1-2 and Section 4.4 in the appendix.

4. Latency. We will separate modeled latency (Eq. 10) from measured end-to-end conditional-execution timing, and report both. [Only if measurement is feasible.]

5. Privacy. The intended neighboring relation was not stated clearly. We will define it, clip released log-probabilities to [−B,0][-B,0], derive the L2 sensitivity from the clipping bound (rather than assuming Δ=1.0\Delta=1.0), recompute the noise scale, and re-run the privacy-utility curves.

We appreciate that you identified the existing same-learner comparison (Table 2) and ablations (Figure 4) as useful controls, and we will build on them in the revision.

Reviewer_wNV2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes two components. FedGIM combines proximal regularization, private client-specific prediction heads, and class-balanced local training to handle heterogeneous client data. For evaluation, it pools test examples and selects prediction heads through label-free routing in feature space. FedIM then adds a post-training management layer: it adapts models at test time, monitors distribution drift, adjusts computation or combines predictions, optionally perturbs released logits for privacy, and refers uncertain cases to humans. Experiments on endoscopic screening examine predictive performance alongside calibration, coverage, drift, and privacy–utility trade-offs.

Critical strengths
  • A predictor using only client label priors achieves surprisingly high AUROC under client-matched evaluation but roughly chance performance under pooled evaluation. This clearly demonstrates how a model can appear successful without using image information.
  • FedGIM reaches approximately 0.851 AUROC against 0.860 for centralized training, suggesting it recovers much of the centralized performance in the evaluated setting.
  • The paper considers practical deployment behavior. Coverage, referral, calibration, drift detection, and computational cost are relevant concerns that accuracy alone cannot capture.
Critical weaknesses
  • Client-matched evaluation gives the model the originating hospital’s identity; pooled evaluation with routing removes that information. If hospital identity is available at deployment, using its disease prevalence can be legitimate. The prior-only experiment, 0.87 AUROC with client identity versus 0.50 without, demonstrates that prevalence explains substantial discrimination, but does not establish that the evaluation is invalid. This directly challenges the paper’s main framing: it needs to define the intended deployment setting and separate within-hospital discrimination from differences between hospitals.
  • The main experiment does not establish that the proposed inference-management layer helps. In Table 2, adding FedIM to FedGIM lowers AUROC from 0.8572 to 0.8419, lowers F1 from 0.7218 to 0.6984, and worsens calibration error from 0.0612 to 0.0923. FedAvg also deteriorates. Referral could justify these costs, but the risk–coverage comparison changes both the training method and inference management, so it cannot isolate FedIM’s benefit. The decisive missing comparison is the same FedGIM model with simple calibrated referral versus full FedIM, at matched coverage and computational cost.
  • Appendix D assumes a sensitivity bound, ∥z(x)−z(x′)∥2≤Δ\|z(x)-z(x')\|_2\leq\Delta, and uses it to set the Gaussian noise. However, the released representation is log probabilities, which can become arbitrarily negative as probabilities approach zero; the paper does not establish a global bound or explain how clipping enforces one. It also leaves the neighboring-input definition insufficiently specified. Consequently, the privacy–utility curves show the effect of adding noise, but do not by themselves establish that the implemented system achieves the advertised privacy budgets.
Questions for authors
  • When is client identity unavailable at deployment? Please define the target deployment setting and report both within-client AUROC and pooled AUROC with known client identity, to distinguish label-prior effects from invalid evaluation.

  • Does FedIM outperform simple referral on the same trained model? Please compare FedGIM with full FedIM against FedGIM with only calibrated confidence-based referral, at matched coverage and computational cost, given the deterioration in AUROC, F1, and calibration in Table 2.

  • How is the sensitivity bound used for differential privacy enforced? Please define neighboring inputs, explain how released log probabilities are bounded or clipped, and provide the resulting sensitivity �0��0� used to calibrate noise in the privacy–utility experiments.

1 comment
AuthorsOpenReview ↗

Response to wNV2: scoped leakage claim, matched-referral ablation, DP sensitivity fix

Thank you for the careful and constructive review. We agree with all three main points and will revise accordingly.

1. Deployment setting and the prior-only control. We agree that the prior-only result (AUROC about 0.87 with client identity vs. about 0.50 without, Appendix H.3) shows that prevalence carries substantial discrimination, but does not show that client-matched evaluation is invalid. If the hospital is known at deployment, using its personalized head can be legitimate. In the revision we will:

  • Define two target settings explicitly: known-client deployment and unknown-provenance deployment (Appendix B.1).
  • Rewrite the title, abstract and conclusion so the claim is scoped to the unknown-provenance setting, and remove the word "artifactual".
  • Report, for FedGIM, the baselines and the prior-only control, both pooled AUROC with known client identity (true head, pooled metrics) and macro-averaged within-client AUROC. A prior-only score is constant within a client, so its within-client AUROC should be near chance. We will state how single-class clients are handled. [Insert numbers when available.]

2. Does FedIM beat simple referral on the same model? We agree this is the decisive missing comparison, and that Table 2 shows real costs: for FedGIM, AUROC falls from 0.8572 to 0.8419, F1 from 0.7218 to 0.6984, and ECE rises from 0.0612 to 0.0923. We will add, on the same trained FedGIM model, (a) no management, (b) calibrated confidence/entropy-based referral only, and (c) full FedIM, at matched coverage and matched computational cost, with risk-coverage curves. [If FedIM does not improve on (b), we will state this plainly and reposition its contribution around drift response, latency control and privacy release rather than accuracy.]

3. Sensitivity bound for the Gaussian mechanism. We agree that assuming ∥z(x)−z(x′)∥2≤Δ\|z(x)-z(x')\|_2\le\Delta is insufficient when z=log⁡pz=\log p is unbounded below. In the revision we will:

  • Define the neighboring relation. [State it, e.g., any two inputs, giving per-query output-level protection.]
  • Clip released log-probabilities to [−B,0][-B,0] before adding noise, state BB, and derive the resulting L2 sensitivity (at most BCB\sqrt{C} for CC classes).
  • Recompute σ\sigma from that sensitivity and re-run the privacy-utility experiments. [If the original runs used Δ=1.0\Delta=1.0 without clipping, we will say so and relabel those curves as noise-injection results.]

Thank you again for the helpful and specific suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.