OneField: A Shared Embedding Space for NeRFs and 3D Gaussian Splatting

Radiance fields are a data modality where an object is stored as a function of space. Recent encoders read that function directly to perform classification, retrieval, segmentation, and language-based tasks. A radiance field is commonly stored in one of two formats: Neural Radiance Fields (NeRFs) or 3D Gaussian Splatting (3DGS). These formats exhibit a different trade-off between memory footprint and rendering speed: NeRFs are memory-efficient but slow to render, whereas 3DGS enable fast rendering at a substantially higher memory cost. Existing encoders can process only one format, i.e., a model trained on NeRF weights does not read Gaussian primitives, and vice versa. Yet, the two formats can be trained on the same rendered views of an object. Thus, they differ in the representation used to model the object, not in the object they encode. OneField is an encoder that maps both formats into a common embedding space, aligned across formats by a contrastive objective, and with a shared decoder enforcing the object's field reconstruction. In this space, the format of the input no longer matters: in classification, part segmentation, and language tasks, a head trained on one format transfers to the other one, with little to no performance degradation. OneField remains competitive with format-specific encoders in all tasks. In addition, it enables retrieval from a unified gallery containing both formats, a task that no format-specific encoder can do, and retrieves the correct object at $88$–$90$% recall@1, $14$–$18$ points above a CLIP baseline processing rendered views.