ZeroSync: Learning Motion Matching from Paired Relative Timestamp Shifts

Estimating unknown time offsets between heterogeneous sensors requires matching motion observed at different rates and with varying reliability. Sparse or repeated motion can produce ambiguous alignment peaks, making fixed correlation scores unreliable. We introduce ZeroSync, which learns temporal matching through controlled timestamp shifts in a common motion representation. Sensor-specific front ends convert observations into angular-velocity sequences for a shared scoring network. Shifting only one stream in time preserves the recorded motion and supplies supervision without absolute clock labels. Shared branches refine component- and block-specific correlations, while the known shift supervises their relative offset distribution. A confidence head learns recovery success using observable and motion-erased examples. At evaluation, the difference between refined branch offsets estimates the correction that reverses the injected shift. Experiments on camera–IMU and LiDAR–IMU robot recordings show that learned residual scoring improves paired recovery without confidence selection. The improvement persists under sequence-equal averaging in each test dataset. These findings support learning a shared motion-matching rule across sensor pairs from known relative timing changes.