CalibBench: Diverse Multimodal Calibration Benchmark

Cross-modal sensor fusion is integral to modern autonomous systems for perceiving and interacting with their environments. The safe and successful deployment of these systems depends on ensuring that the distinct sensor modalities are aligned, necessitating cross-modal calibration to identify the extrinsic parameters among the sensors. Current methods seek to address the need for cross-modal calibration in various ways, including target-based and target-less calibration, as well as deep learning. However, existing methods have been evaluated on homogeneous or closed-source datasets, which makes it difficult to develop and validate follow-on methodologies. This work focuses on developing a cross-modal (camera and LiDAR) calibration dataset. We elaborate on how we make our dataset diverse and challenging, and report the performance of previous methods on our dataset. Through our benchmark, we demonstrate the research gap in cross-modal calibration, the challenges faced by existing methods, and their improved performance after training. This dataset paves the way for the community to develop more accurate, robust, and domain-transferable cross-modal calibration methods.