MOSAIC-CONUS: A Multimodal, Multi-Temporally Paired dataset for Earth Sciences

Earth embeddings—vector representations of geographic locations indexed in space and time—are emerging as a unifying interface for geospatial AI. However, their quality depends not only on model design, but on how multimodal Earth observation (EO) data are spatially indexed, temporally aligned, and cross-modally associated during pretraining. We introduce MOSAIC-CONUS ($\text{M}$ultimodal $\text{O}$bservations with $\text{S}$patially $\text{A}$ligned Imagery, Urban Points of Interest, $\text{I}$n-Situ Measurements and Text $\text{C}$aptions), a large-scale EO dataset over the contiguous United States, organized around 250,000 stratified point indices that serve as stable spatial keys across seven modalities: active radar, passive optical imagery, lidar-derived elevation, land cover, functional context, hydrometeorological measurements, and textual summaries. Unlike Existing EO datasets, MOSAIC-CONUS introduces four contributions not jointly addressed in prior work: $\text{1.}$an open-source, large-scale multimodal EO corpus structured around point-indexed data designed to support Earth embedding learning; $\text{2.}$explicit radar-optical pairing tables spanning twelve temporal alignment regimes, formalizing cross-sensor alignment as a controllable variable for analyzing how temporal mismatch across modalities influences learned embeddings quality; $\text{3.}$a benchmark suite spanning cross-modal retrieval, annual nightlights regression, and basin-held-out streamflow prediction, positioning MOSAIC-CONUS as a benchmark-ready resource for multimodal AI systems; and $\text{4.}$a language-based embedding layer through co-registered textual summaries, enabling Earth embeddings to function as a queryable interface for agentic AI systems. The dataset and pairing protocols are publicly released.