acceptodds
Under review as a conference paper at ICLR 2027

Asymmetric Generative Adaptation for Federated Multispectral Palmprint Recognition under Spectral Heterogeneity

Abstract

Multispectral palmprint recognition captures both surface ridge textures and subcutaneous vein patterns across multiple spectral bands, offering improved robustness to presentation attacks in high-security authentication scenarios. However, deploying such systems on distributed consumer-grade devices via Federated Learning (FL) introduces a critical practical barrier:Spectral Heterogeneity. Variations in local sensor hardware and ambient illumination produce severe Non-IID spectral distributions across clients, causing conventional personalized FL (pFL) to overfit on scarce local samples. To address these challenges, we present AsyGA (Asymmetric Generative Adaptation), a privacy-first pFL framework designed for heterogeneous deployment on consumer-grade devices. Each client trains a lightweight diffusion model entirely on-device to capture its local spectral profile, while only the downstream multispectral classifier is shared with the server for communication-efficient aggregation. After deployment, clients leverage these local generators to synthesize single-domain surrogate images for on-device personalization, eliminating raw biometric data exposure while minimizing communication overhead. To counteract Asymmetric Modality Collapse, a phenomenon where biased single-domain gradients overwrite representations of unseen bands during local optimization, we implement a hardware-friendly Fidelity-Preserving Structural Constraint consisting of four progressive mechanisms: (1)supervised contrastive optimization on synthetic identities for robust open-set separation; (2)feature-anchored cosine distillation against a frozen global teacher to stabilize the student embedding space; (3)batch normalization statistics locking to suppress synthetic distribution shift; and (4)targeted convolutional backbone freezing to shield globally learned multi-band filters from biased gradients while truncating backpropagation to slash on-device memory overhead. Evaluated under a strict cross-dataset open-set protocol spanning 287 million verification pairs across four spectral domains, AsyGA reduces the Equal Error Rate by over 50% against state-of-the-art FL baselines and more than doubles the True Acceptance Rate under a strict threshold. Furthermore, the on-device personalization stage adds only 44.6% of the FedAvg training time (61.5% including one-time generator training) and executes seamlessly on a consumer-grade laptop GPU without specialized infrastructure.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multispectral Palmprint, Federated Learning, Personalized Fine Tuning, Diffusion Models, Privacy Preserving, Cross-Spectral Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JnfvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AsyGA, a personalized federated learning framework for multispectral palmprint verification. Following federated classifier training, each client uses a locally trained diffusion model to generate single-spectrum images for personalization. Contrastive learning, feature distillation from a frozen global model, fixed batch normalization statistics, and backbone freezing are intended to preserve cross-spectrum performance. The authors train on CASIA and evaluate open-set verification on PolyU, reporting substantial reductions in equal error rate and an improvement in true acceptance rate at a low false acceptance rate.

Critical strengths
  1. The paper addresses a meaningful failure mode: personalization using data from one observed spectrum may damage performance on other spectra. The cross-spectrum results and component ablations make this question worth investigating.

  2. Evaluation on a separate dataset with open-set verification and a stringent false acceptance rate is more relevant to the proposed application than closed-set accuracy alone.

Critical weaknesses
  1. The experiments do not adequately test the deployment setting motivating the paper. Clients are simulated by partitioning one dataset by identity and spectral band. This does not establish robustness to distinct consumer sensors, lighting conditions, or hardware constraints. The reported laptop GPU timing also supports a narrower feasibility claim than the paper makes.

  2. The privacy and mechanistic claims exceed the evidence. Keeping raw images and generator weights local does not establish a “zero-trust” privacy guarantee for shared classifier updates; no explicit threat model or leakage evaluation is provided. The theoretical account of modality collapse relies on an assumed gradient relationship that is not measured, and its preservation analysis assumes a relationship between observed-band anchoring and unseen-band alignment. These assumptions limit how much the theory independently explains the empirical results.

Questions for authors
  1. What are the exact number of clients, their identity and spectral partitions, and the per-client diffusion training and sampling protocols? Please reconcile the image counts used in the method and hardware profiling.

  2. What privacy adversary and attack surface does “privacy-first” address? Do you have evidence concerning leakage from shared classifier updates or memorization by local generators?

  3. Can the authors provide a client-level analysis of unseen-spectrum embedding drift, and evidence from genuinely different devices or acquisition conditions? Either could materially change my assessment of the central claim.

Reviewer_m8qTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AsyGA, a personalized federated learning framework for multispectral palmprint recognition under spectral heterogeneity. Basically, this work can be considered as the combination of many existing methods, and the paper reports improvements in cross-spectral verification performance on a cross-dataset evaluation protocol. However, I have significant concerns regarding the assumption, motivation and the novelty.

Critical strengths
  1. The reported improvements in verification EER are substantial.
  2. The manuscript is well-organized and well-written.
Critical weaknesses
  1. The proposed deployment scenario lacks sufficient practical justification. It is unclear why distributed devices with disjoint identities and spectral modalities require cross-spectral verification. More importantly, without cross-spectral positive identity pairs during federated training, how the global model learns meaningful cross-spectral alignment remains unclear.

  2. Insufficient justification for synthetic data mitigating spectral heterogeneity. Each diffusion generator is trained exclusively on local single-spectrum data, potentially reproducing rather than reducing spectral discrepancies. Generating additional samples from the same distribution does not introduce information about missing modalities. Basically, I'm not convinced why it can alleviate this issue without introducing any other information.

  3. Inadequate evidence of client-specific personalization. The paper does not demonstrate that local generators capture meaningful client-specific characteristics or that these characteristics drive performance gains. Client-wise evaluations and comparisons with shared generators, cross-client synthetic data, conventional augmentation, and regularized fine-tuning are needed to distinguish genuine personalization from generic adaptation.

  4. Limited methodological novelty. The framework primarily combines established techniques, including local diffusion generation, feature distillation, contrastive learning, and parameter freezing. Its distinction from existing generative and personalized federated learning approaches is insufficiently established. It seems like one incremental work rather than one impressive work.

  5. Similar to Point 2. the diffusion models are trained on data already used during federated learning. It is unclear what additional task-relevant information the generation stage contributes beyond augmentation, regularization, or further optimization. Comparisons under matched training and computational budgets are needed to justify its necessity.

  6. Theoretical analysis relies on strong assumptions. The theoretical results assume gradient relationships that directly imply the claimed degradation and preservation effects, rather than deriving them from the proposed mechanism. Moreover, feature distillation on observed modalities does not necessarily preserve unobserved cross-spectral relationships.

  7. Insufficient experimental validation. The evaluation lacks strong personalized/generative FL baselines, particularly global-model-plus-regularized-fine-tuning comparisons.

  8. Ablation results question component necessity. Table 2 shows that feature anchoring alone reduces average EER from 8.72% to 4.47%, while BN and backbone freezing yield relatively modest additional improvements. The reported results suggest that conventional feature distillation may explain most gains.

  9. Unsupported privacy and deployment claims. Keeping generators and synthetic data locally does not guarantee privacy, as model updates and diffusion generators may still leak biometric information. No formal privacy analysis is provided.

  10. Missing diffusion training details and fairness concerns. The paper does not clarify whether local diffusion models are trained from scratch or fine-tuned. Given the limited local data, training from scratch seems impractical. If pretrained models are used, external training data may explain the cross-dataset improvements, raising concerns about unfair comparisons and potential privacy leakage. Therefore, the observed performance improvements may be attributed primarily to the incorporation of the external dataset, rather than to the design of the proposed federated learning framework itself.

  11. Several terms, such as "Asymmetric Modality Collapse" and "Fidelity-Preserving Structural Constraint," are introduced without sufficiently distinguishing the underlying mechanisms from established catastrophic forgetting and regularization concepts.

Questions for authors

Please check the weakness.

Reviewer_fUU7OpenReview ↗
Rating
2
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies federated multispectral palmprint recognition when each client observes only one spectral band. It argues that adapting a global model using single-spectrum synthetic data can degrade representations of locally unobserved bands. The phenomenon is named Asymmetric Modality Collapse. The authors propose AsyGA, which separates federated classifier training on real data from subsequent personalization using locally generated images. Four progressive mechanisms, supervised contrastive learning, cosine feature distillation from a frozen global teacher, fixed batch normalisation statistics, and convolutional backbone freezing, are combined. Cross-dataset evaluations demonstrate that AsyGA cuts EER by over 50% relative to SOTA FL baselines and more than doubles TAR under FAR = 10−6.

Critical strengths
  • The adaptation setting is practical. A client’s available data may cover only one spectral band, while its model must retain the cross-spectral capabilities learned during federation. Explicitly considering this mismatch provides a useful perspective: improving performance on the available synthetic distribution also requires controlling its effects on locally unobserved domains.
  • The progressive and leave-one-out ablations provide useful information about component roles. Table 2 and Appendix A consistently identify feature anchoring and fixed BN statistics as the main contributors to the reported verification gains. The backbone-freezing results also help distinguish computational benefits from recognition benefits. Together with cross-dataset evaluation and alternative-backbone experiments, these empirical analyses are useful for investigating the proposed approach.
Critical weaknesses
  • The methodological novelty remains unclear. It seems that feature anchoring closely resembles existing feature-preserving domain adaptation methods. In addition, the paper does not clearly establish what new mechanism or insight emerges from combining it with BN locking and backbone freezing.
  • It seems that the submitted implementation does not consistently reflect the stated experimental settings. Section 4.1 specifies disjoint identities and spectral domains across clients, but psfed.py loads four spectral index files containing the same 100 identities. In generater/generate.py, generation for client 0 combines domain prompts from all four clients and averages their class prompts. The differences may affect the information available to each client and the difficulty of the task. Consequently, it is unclear whether the reported gains establish the claimed ability to personalize using only private, single-spectrum synthetic data without post-FL access to real biometrics.
Questions for authors
  • Which exact data partition and personalization procedure produced the main results? Could you explain the disjoint-identity claim in Section 4.1, given the overlapping identities in the supplied spectral index files?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.