- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This manuscript completes infrared and visible light image fusion based on pre-trained representations and calculable image quality rewards. The model retains the frozen visual basic model and visual language backbone, uses trainable branches to complete feature adaptation, and introduces perturbations in training while generating multiple fusion candidates, constructing a relative difference set, and using a combination of different rewards to complete the quality assessment of the fusion results; in the inference stage, only the reference candidates are output as the final fusion result. The overall experiments are rich and well-designed, and model training is completed using GRPO ideas based on similar reinforcement learning.
Critical strengths
This paper proposes a method that drives optimization based on the relative quality difference between candidates and reward evaluation, which is different from the traditional optimization paradigm for a single output under fixed loss. This method has certain novelties in the design of optimization goals and training signals. It can use the relative merits and demerits information between candidates and guide model optimization through reward evaluation. Experiments on multiple data sets show that this method has good effectiveness and shows certain practicality in downstream tasks. Overall, the paper has a clear idea, the method design is obviously different from the existing paradigm, and it has certain innovation and application value.
Critical weaknesses
- The connection and difference between CGPO and GRPO can still be further explained in order to understand the specific role of adjustments made for image fusion.
- There are differences in the performance of different reward combinations. How gradient rewards complement pixel and structure rewards requires further explanation.
- The role of semantic feedback and shared decoder in the intermediate stage can still be presented more intuitively, and a small amount of intermediate fusion results help to understand the step-by-step refinement process.
- The overall model has a large number of parameters. What is the current main computational bottleneck?
- What is the main basis for determining the number of candidate fusion results? What impact does the number of candidates have on model training cost and fusion quality?
Questions for authors
- What key ideas does CGPO draw from GRPO, and what adjustments has been made to image fusion?
- In Table 7, after adding gradient reward, the performance of the combination of pixel and structure reward is significantly improved. Does this result indicate that gradient information is particularly important for balancing different quality goals? What are the differences between the different reward packages?
- Compared with only introducing semantic guidance on the final fusion result, what problems are mainly solved by embedding feedback in the intermediate stage?
- The same decoder is used for intermediate image reconstruction and final candidate generation. What is the purpose of this design?
- It is recommended to supplement the fusion results of intermediate reconstruction to visually illustrate the changes in image quality during the step-by-step refinement process.
4 comments
Response to Reviewer n2yV (Part 2/2)
Q5 / W3: Please provide intermediate fusion results to illustrate stage-wise changes in image quality.
To visualize changes in image quality during progressive refinement, we have added intermediate fusion visualizations to the revised manuscript (Appendix A.10.4, pp. 24–25). The newly added Figure 14 (p. 25) presents the intermediate images reconstructed by the shared decoder at Stages 3, 2, and 1, together with the final result at Stage 0.
Stage 3 reconstructs the initial fused image from features that have not undergone semantic modulation. Stage 2 uses feedback generated from the Stage 3 image and the prompt focusing on salient targets and global structure to modulate its features, increasing overall brightness and target responses. Stage 1 uses feedback from the Stage 2 image and the prompt focusing on boundaries and local contrast, reducing excessive brightness and producing clearer structural contours. Stage 0 uses feedback from the Stage 1 image and the prompt focusing on fine texture and artifact regulation, further reducing local brightness unevenness and producing a final image with more balanced brightness and more prominent pedestrians and vehicles.
W4: Why does the model have a large parameter count, and what is its main computational bottleneck?
As shown in Table 9, most parameters belong to the frozen DINOv3 and Qwen3-VL-Embedding backbones, while 17.2929 M parameters are trainable. For inputs, Qwen3-VL-Embedding accounts for approximately 85% of the total FLOPs. The main bottleneck is thus semantic embedding extraction across multiple stages of chained diagnosis.
Under the evaluation conditions used in the paper, the complete inference pipeline takes an average of 0.3421 s per image pair on the MSRS test set at its original resolution. Our design reflects a trade-off between fusion quality and inference efficiency.
W5: How is the number of candidates selected, and how does it affect training cost and fusion quality?
We conducted a hyperparameter analysis with to evaluate the effect of candidate count on fusion quality. As shown in Figure 10, when , the candidate’s reward equals the group-mean reward, making the relative advantage identically zero. The objective therefore provides no effective training gradients, preventing the model from being effectively optimized through this objective.
When the reference and perturbed candidates allow within-group quality comparisons, enabling optimization based on relative advantages. Increasing the candidate count provides more within-group quality references. The best performance among the tested settings is achieved at . However, increasing further to 5 reduces performance while increasing training cost, indicating that more candidates do not necessarily lead to continued improvements in fusion quality. An excessive number of candidates may make optimization more difficult without necessarily helping to improve the quality of the reference fused image. We therefore choose as the final setting to balance fusion quality and training cost.
Increasing the candidate count mainly increases the training cost of image decoding and reward computation. This additional overhead occurs only during training. At inference, no perturbed candidates are generated, and the final fused image is reconstructed solely from the reference candidate.
The authors' response has addressed most of my concerns. After considering the other reviewers' comments and the authors' replies, I have decided to keep my original positive rating.
Thank you very much for your reply. Your comments have been very helpful to us. Thank you again for taking the time to review our manuscript.
Response to Reviewer n2yV (Part 1/2)
Thank you for your review and valuable suggestions for improving our method. We respond to your comments and questions below.
Q1 / W1: What ideas does CGPO draw from GRPO, and what adaptations are made for image fusion?
CGPO draws on GRPO’s core idea of optimizing through reward comparisons among multiple candidate outputs for the same input. It aims to extend single-output optimization to optimization based on relative quality comparisons among candidates. Specifically, CGPO uses features refined through chained semantic feedback to construct one unperturbed reference candidate and several Gaussian-perturbed candidates in feature space. These candidates are decoded into fused images by the shared decoder, and the model is optimized using the computed rewards. During training, multiple candidate fused images provide local quality references. The model computes relative advantages by subtracting the group-mean reward from each candidate’s reward and uses these advantages to update its parameters, guiding improvements in the quality of the reference fused image.
Q2 / W2: How does the gradient reward complement the pixel and structural rewards, and how do different reward combinations compare?
The pixel reward uses the pixelwise maximum of the source images as a reference and preserves strong brightness responses. The SSIM reward focuses on the main structures, while the gradient reward emphasizes edges and textures, providing direct feedback for preserving local details. Combining these three rewards helps balance brightness, structure, and detail.
Furthermore, the quantitative results in Table 7 show varying degrees of performance degradation when reward components are omitted, compared with using the full reward combination. The qualitative results in Figure 11 further illustrate the differences among reward combinations. Combining the pixel and SSIM rewards accounts for both brightness and structure, but textural details may remain unclear. Combining the pixel and gradient rewards preserves both brightness and detail but provides relatively weaker overall structural preservation. The SSIM and gradient combination retains richer details but produces darker images. When all reward components are included, the fused images achieve a better balance among brightness responses, structural preservation, and detail preservation. These results illustrate the overall benefits of jointly optimizing all three rewards.
Q3: What is the role of intermediate semantic feedback compared with guiding only the final fusion result?
Intermediate semantic feedback enables the model to adjust subsequent fusion features based on the current fusion state before generating the final fused image. Compared with applying semantic guidance only to the final result, we use diagnostic feedback from the current intermediate fused image to modulate the next-stage features within the same fusion process, with subsequent diagnosis based on the updated fused image. Thus, semantic feedback is continually updated as the fusion state evolves and, together with stage-specific prompts, progressively guides feature adjustments according to the semantic requirements of each stage.
Q4: Why is the decoder shared between intermediate image reconstruction and final candidate generation?
Sharing the decoder ensures that intermediate fused images and final candidate fused images are reconstructed through the same decoding mapping, providing a consistent basis for semantic diagnosis and candidate evaluation. If decoders with separate parameters were used, even with the same architecture, image differences could arise from both the fusion features and differences in the decoding mappings, potentially creating a mismatch between intermediate diagnostic feedback and the reconstruction characteristics of the final candidates. Furthermore, decoder parameters updated using the final candidate rewards are reused for subsequent intermediate fusion reconstructions, linking intermediate semantic feedback with the optimization of final fusion quality.