FedFields: Learning Shared Priors for Fast Implicit Neural Representations

Implicit Neural Representations (INRs) model signals as continuous coordinate-based functions, but typically require a separate network to be optimized for every signal. Amortized approaches avoid costly per-signal fitting by learning across many signals to directly predict signal-specific representations. Existing methods, however, assume centralized training on pooled data, limiting their applicability when signals are distributed across multiple clients. Federated learning provides a natural framework for learning across such decentralized data, yet existing federated neural field methods retain per-signal optimization: they either federate a single field for one scene or meta-learn an initialization that must still be optimized for each new signal. We introduce FedFields, a federated amortized neural field that learns a shared representation prior across decentralized signals. A shared encoder maps each signal to a latent representation, from which a lightweight per-point hypernetwork generates modulations for a shared INR decoder. Once trained, the model represents unseen signals in a single forward pass, with optional latent-only adaptation when higher reconstruction fidelity is desired. The shared model is learned collaboratively across clients through federated aggregation, while both the signals and their latent representations remain local to each client. Experiments show that FedFields achieves high-fidelity reconstruction and generalizes to unseen data distributions, while remaining effective across varying client configurations, data imbalance, and aggregation rules. The same formulation extends from images to videos, 3-D occupancy, and audio, demonstrating that amortized neural fields can be learned effectively across decentralized data.