acceptodds
Under review as a conference paper at ICLR 2027

Multi-View Color Correction with Multi-objective Geometry-Guided Voxel Consensus

Abstract

The modern image processing pipeline is often designed for one single image without any spatial and temporal information. Each component in the ISP can introduce slight view dependent differences which will cause larger visual inconsistencies, thereby impact virtual reality experience and also affecting downstream tasks like texture mapping and some 3D reconstruction tasks. We formulate the color consistency as a multi-view geometry-guided optimization problem. Utilizing depth and camera poses to back-project image into 3D space to jointly optimize all correspondence camera views in a global manner through voxelization. By optimizing a trainable per-view bilateral grid as our color transformation matrix indirectly in 3D voxel space to achieve multi-view color consistency. To further improve the color consistency while preserving the visual fidelity of the image, we also introduce a robust multi-objective color consensus method for color correction instead of operating on sRGB space directly. The proposed method shows near SOTA spatial color consistency and better performance in complex cases.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Processing, Multi-view Images, Color Consistency, Bilateral Grid, Global Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CBL1OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates multi-view color correction as a geometry-guided global optimization problem, where RGB-D observations are projected into a shared voxel space to establish cross-view correspondences, and per-view bilateral grids are optimized using a multi-objective color consensus to improve color consistency while preserving image fidelity.

Critical strengths

The problem is practical and relevant to multi-view imaging and 3D reconstruction.

Critical weaknesses

(1) The comparison is not sufficiently comprehensive. Recent 3D reconstruction and Gaussian Splatting works have already explored appearance, exposure, and ISP inconsistencies across views, e.g., WildGaussians: 3D Gaussian Splatting in the Wild, Bilateral Guided Radiance Field Processing, and Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting. Some of these methods are also closely related to the proposed bilateral-grid formulation. The paper should include stronger comparisons with representative methods from this line of work, or clearly explain why they are not applicable. Otherwise, the specific advantage of the proposed voxel-consensus formulation remains unclear.

(2) The method heavily depends on accurate depth and camera poses, but this dependency is insufficiently analyzed.The paper acknowledges that errors in depth or pose can lead to incorrect observation mixing and degraded correction quality. I think the paper should further discuss whether color inconsistency itself affects depth/pose estimation in practical pipelines, and provide robustness experiments under controlled depth and pose perturbations.

(3) The presentation would benefit from a clearer overview figure.

(4) The qualitative results do not clearly demonstrate the claimed gain in multi-view consistency. The current visualization mainly shows corrected images, but does not sufficiently illustrate whether the same physical regions become more consistent across different viewpoints. Since cross-view consistency is the central claim, the paper should visualize multiple overlapping views of the same scene, preferably with corresponding regions highlighted or with difference maps before and after correction.

Questions for authors

No

Reviewer_HUMTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a geometry-guided multi-view color-correction framework for reducing exposure and color discrepancies across viewpoints. Using depth maps and camera poses, it projects image observations into shared 3D voxels and constructs approximate cross-view correspondences through pivot sampling and nearest-neighbor selection, reducing the risk of cross-surface color mixing. View-specific bilateral grids are jointly optimized to model spatially and luminance-dependent color transformations. The objective operates in linear RGB space and combines chromaticity consensus, log-exposure consistency, and saturation-preserving targets. The method performs scene-specific optimization without ground-truth color supervision. Experiments on Realsee3D and Matterport3D show improved cross-view consistency over the plain-RGB variant, while the saturation analysis on Realsee3D also demonstrates better saturation retention.

Critical strengths
  1. By constructing cross-view correspondences using depth maps and camera poses, the framework can incorporate low-texture regions where appearance-based feature matching is difficult. Its self-supervised formulation jointly optimizes the views without requiring ground-truth color references, providing a practical approach when suitable geometry is available.

  2. Decoupling chromaticity from exposure and incorporating an explicit saturation-preservation objective helps reduce washed-out colors during alignment. The comparison with the plain-RGB variant shows improved cross-view consistency and better saturation retention on Realsee3D, supporting the effectiveness of the combined formulation.

Critical weaknesses
  1. The method depends on reliable depth maps and camera poses, but its robustness to geometric errors is insufficiently evaluated. The experiments use dataset-provided geometry, leaving the effects of depth and pose inaccuracies unquantified. A sensitivity analysis with controlled depth/pose errors and different voxel sizes would help establish the method’s practical applicability.

  2. The incremental contributions of the proposed components are insufficiently isolated. View-specific bilateral grids are already used in prior work such as CHROMA, so the distinctive contribution needs to be established through the voxel-based correspondence formulation and multi-objective consensus design. However, the ablation compares the full multi-objective formulation with a plain-RGB variant without separately evaluating hierarchical sampling, confidence weighting, saturation preservation, and exposure consistency. Component-wise ablations would clarify the contribution of each design choice.

  3. The use of different test subsets limits the comparability of the Neura results. While the other methods are evaluated on all 1,000 Realsee3D scenes, Neura is evaluated only on the first 10 scenes, citing computational cost and poor performance. Differences in scene difficulty may confound the comparison, so results for all methods should also be reported on the same 10-scene subset.

  4. The reported gains appear primarily in the newly proposed Voxel-SSIM metric, whereas CHROMA achieves higher reference-based, affine-aligned PSNR and SSIM. It remains unclear whether improvements in the proposed consistency metric translate into better perceived visual quality or downstream utility. User studies assessing visual discontinuities during viewpoint transitions, or qualitative and quantitative evaluations of downstream 3D reconstruction and novel-view rendering, would help establish the practical value of the consistency–fidelity trade-off.

Questions for authors
  1. Could the authors evaluate robustness to controlled depth and pose errors, together with different voxel sizes, to clarify the method’s sensitivity to geometric inaccuracies?
  2. Could the authors provide component-wise ablations of hierarchical sampling, confidence weighting, saturation preservation, and exposure consistency to isolate their individual contributions?
  3. Could the authors validate the practical benefits of improved Voxel-SSIM through user studies of cross-view visual consistency or downstream 3D reconstruction and novel-view rendering experiments?
Reviewer_hgZjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a per-scene, multi-view color correction method that relies on RGB, depth, and camera poses. The algorithm back-projects valid pixels and divides them into 3D voxels, creating weak geometric correspondences using pivots and neighbors. It then estimates robust chroma/hue saturation/log-exposure consensus and optimizes the 3D bilateral grid color transform for each view. Experiments were conducted on Realsee3D with synthetic color perturbations and Matterport3D. The main metric is the Voxel-SSIM proposed by the authors, along with reported affine-aligned PSNR/SSIM and saturation statistics.

Critical strengths

1.The motivation and modeling of method are clear, the weak correspondences can be obtained using geometric co-visibility, which helps when there’s little texture for pure feature matching; per-view bilateral grids also provide a calibration model that’s more flexible than full-image gains and smoother than per-pixel transforms. 2. The princile of method is also easy to be analyzed, by handling chroma, hue/saturation, and exposure separately, it can explain why directly using RGB consensus often leads to desaturation issues, the algorithm components and loss functions are basically clear. 3. The experiments include open-source dataset Realsee3D with 1,000 scenes under color perturbations, and there’s also evaluation on Matterport3D real-world data, the paper also compares plain-RGB variants and reports the trade-offs between color consistency and image fidelity.

Critical weaknesses
  1. The method relies on high-quality depth and pose, and always uses 0.15m voxels. Within a voxel, it can still treat adjacent surfaces or color boundaries as weak correspondences, especially in thin structures, occlusion edges, and when the depth is noisy. The paper lists depth/pose errors as a limitation, but doesn’t report how voxel size, depth noise, or pose disturbances affect the results.
  2. The experiments on Matterport3D lack an independent color fidelity reference. This dataset has quite a bit of color variation, and the table mainly reports a custom consistency score, without a ground truth for color consistency in the same scene. This can show consensus across views, but it’s hard to tell if that consensus matches the real colors. It’s recommended to provide human/reference frame evaluation or verifiable color charts and material fidelity measurements.
  3. The main metric, Voxel-SSIM, hasn't been fully validated and is highly related to the optimization target. Voxel-SSIM is a new metric defined in this paper, and the training loss directly optimizes cross-view color consistency for corresponding voxels. In the Realsee3D table, the ground-truth Voxel-SSIM is 0.8953, while the method reaches 0.9146; this might suggest that the output is more consistent than the captured ground truth, but it also shows that this score doesn't equate to color/detail fidelity. Independent validation or standard metrics are needed to verify its correlation with actual perceptual quality and to rule out falsely high scores caused by smoothing or overcorrection.
Questions for authors
  1. Can we report paired metrics on the same subset of scenarios that run across all baselines, and provide scenario-level confidence intervals or significance tests?
  2. How does Voxel-SSIM correlate with independent human evaluations, standard color difference metrics, or held-out voxel correspondences? How can the corrected results exceed the ground truth for these metrics?
  3. When the voxel size changes or depth/pose is perturbed with real-world noise, how do Vox-SSIM, PSNR, and detail retention change?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.