- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SegDem, a demosaicing method which tries to incorporate structural information in order to improve image demosaicing performance. The proposed method consists of 2 training stages. The first stage uses MSCOCO instance segmentation dataset to pretrain the model for instance segmentation. Then, the next stage trains the demosaicing network (initialized from the 1st stage weights). Both stages use representation alignment with DINOv2 features to provide further structural supervision. Ablation experiments on different backbones show consistent gains in the evaluation metrics. In the Single- and Quad-Bayer demosaicing settings SegDem demonstrates strong results on synthetic benchmarks and competitive performance on real RAW data.
Critical strengths
-
Using structural information to improve demosaicing performance is well-motivated, as it could help to better fill the missing color information around the edges by avoiding the averaging artifacts around boundaries.
-
The proposed method is evaluated in both Single-Bayer and Quad-Bayer settings using multiple backbones and it demonstrates consistent gains across all evaluated configurations.
Critical weaknesses
-
The novelty of the paper appears limited, as the proposed approach primarily combines 2 widely used approaches in Computer Vision literature (transfer learning and representation alignment) for demosaicing. While this may still offer a value, the experiments do not show whether the gains come from segmentation pretraining or such gains could be achieved through other pre-training, such as image classification, image inpainting or denoising.
-
The writing quality is poor. For example the Method section has some unimportant details, while the details of instance segmentation loss are left to the Appendix without a cross-reference. Also Figure 2 is hard to follow and does not help with understanding of the method. For example, it is not clear how and why DINOv2 features are concatenated to the Linear RGB image given their different spatial and channel dimensions (should be a mistake). The input to the rightmost DINOv2 Encoder and the purpose of different arrowed lines are unspecified. This figure needs a major revision in order to clearly communicate the proposed method.
-
An ablation experiment is missing for the inclusion of segmentation and demosaicing embeddings in the method. As Stage 1 and Stage 2 are trained sequentially, it is unclear why such embeddings are necessary, so an ablation experiment is necessary for justification.
Questions for authors
-
Why did the authors choose to use cached Qwen2.5-VL embeddings instead of other simpler alternatives?
-
Why did authors not report PSNR, SSIM and LPIPS metrics in the real-world RAW evaluation table (Table 2)?
-
Given that the instance segmentation head predicts an RGB representation, how do authors compute the dice, focal and boundary losses in equation 10 during the pre-training and how does this help to the underlying instance segmentation task?
2 comments
Thanks for recognizing our rebuttal and score improvement. We’re working on revising the core framework diagram and method description to make them more readable. Specifically, we’re adding detailed inputs for stage 1 in the core diagram, giving specific meanings to the line colors, and including some necessary structural information and loss functions. In the methods section, we’re adding the exact loss functions to the stage 1 description and including the analysis from the discussion to further illustrate the value of our framework. Also, we’re adding the necessary ablation experiments and analyses to the experimental section, and finally, we are polishing the appendix.
Thank you for the rebuttal. It addresses my main questions. However, I still think that in the current form the paper is hard to follow and would like the authors to make a major revision to improve the readability. Assuming that authors will improve the writing, I am willing to increase the score to weak accept.