Multi-View Color Correction with Multi-objective Geometry-Guided Voxel Consensus

The modern image processing pipeline is often designed for one single image without any spatial and temporal information. Each component in the ISP can introduce slight view dependent differences which will cause larger visual inconsistencies, thereby impact virtual reality experience and also affecting downstream tasks like texture mapping and some 3D reconstruction tasks. We formulate the color consistency as a multi-view geometry-guided optimization problem. Utilizing depth and camera poses to back-project image into 3D space to jointly optimize all correspondence camera views in a global manner through voxelization. By optimizing a trainable per-view bilateral grid as our color transformation matrix indirectly in 3D voxel space to achieve multi-view color consistency. To further improve the color consistency while preserving the visual fidelity of the image, we also introduce a robust multi-objective color consensus method for color correction instead of operating on sRGB space directly. The proposed method shows near SOTA spatial color consistency and better performance in complex cases.