- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents RefGC-SR², which aims to jointly perform super-resolution and generative artifact refinement on low-resolution generated images with the guidance of high-resolution reference images. To address the lack of training data, the authors propose DipRefGC, which decouples object appearance and pose to construct LRGI-HRRI-HRGT triplets. Furthermore, the authors introduce a frequency-adaptive mixture of LoRA experts and a frequency-adaptive loss to model global structure and fine-grained details. Experimental results demonstrate that the proposed method outperforms existing baselines in terms of multiple quantitative metrics and visual quality.
Critical strengths
1.The data construction strategy of DipRefGC is well motivated. By decoupling object appearance and pose, it generates training samples that contain generative artifacts while maintaining reasonable pose consistency with the target images, providing effective supervision for the proposed task.
2.The designs of FreqMoLE and the frequency-adaptive loss are reasonable and well motivated. They leverage the frequency characteristics of different network layers to encourage the modeling of global structure and the recovery of fine-grained textures.
3.The experimental evaluation is relatively comprehensive. The authors compare their method with various super-resolution and image refinement approaches and provide ablation studies, generalization experiments on outputs from real-world generative models, and a user study to demonstrate the effectiveness of the proposed method.
Critical weaknesses
1.Although the ablation studies demonstrate the effectiveness of FreqMoLE, there is still limited evidence that the LF and HF experts learn their intended frequency specializations. An analysis of expert responses and learned gating weights would help better validate the proposed frequency decomposition mechanism.
2.The high-frequency loss matches channel-wise statistics between the output and HRRI to avoid cross-view pixel alignment. However, statistical consistency does not necessarily ensure spatially accurate texture recovery, especially for structured details such as text, logos, and complex patterns.
3.While the authors evaluate common reference degradations such as blur, JPEG compression, and downsampling, the model’s robustness to more challenging cases remains unclear. Further analysis of occlusions, incorrect reference details, or irrelevant information would help assess whether the model can reliably control detail transfer.
Questions for authors
Please refer to the weaknesses above.
7 comments
[W2] The high-frequency loss matches channel-wise statistics
We thank the reviewer for raising this point. We agree that matching channel-wise statistics alone does not determine where details appear. Our HF loss serves as an auxiliary constraint to help exploit useful HRRI details that may not be fully utilized through flow-matching supervision alone. It is not designed to directly supervise the exact spatial arrangement of text, logos, or complex patterns, or to independently guarantee their accurate reconstruction.
Spatial conditioning is provided by the FLUX-Kontext backbone: HRRI and LRGI are encoded as context tokens alongside the target tokens, and every DiT block attends to these context tokens (Sec. 3, L289-299). The flow-matching loss supervises HRGT reconstruction conditioned on LRGI and HRRI, while the LF loss further anchors the output to the HRGT structure. However, these HRGT-based objectives do not explicitly encourage the output to incorporate HRRI's high-frequency characteristics. The HF loss complements this reconstruction and structural supervision by providing such guidance.
Because HRRI can differ from LRGI and the target output in viewpoint, pose, and object placement, direct pixel-wise supervision against HRRI can force noncorresponding spatial locations to match, potentially conflicting with reconstruction in the target viewpoint. We therefore use channel-wise statistical matching to incorporate HRRI's high-frequency characteristics without imposing incorrect spatial correspondences (Appendix G.1, L1303-1309).
Table 11 (Appendix G.1, L1320-1327) supports this design. Adding the frequency loss () to flow matching alone, (a)(d), improves all five metrics (CLIP-I 0.8673 → 0.8696, DINO 0.7317 → 0.7474, PSNR 17.3893 → 17.5148, SSIM 0.6221 → 0.6335, LPIPS 0.2896 → 0.2746). Under the same FreqMoLE backbone and training schedule, replacing pixel-wise HRRI supervision with channel-wise statistical supervision, (b)(d), improves PSNR from 16.9123 to 17.5148, SSIM from 0.6202 to 0.6335, and LPIPS from 0.2960 to 0.2746, while also improving CLIP-I and DINO. These results support using HRRI's high-frequency statistics as an auxiliary constraint.
[W3] Reference mismatch (1/2): Occlusion and incorrect details
We thank the reviewer for raising these challenging reference conditions. We evaluated occlusion and reference mismatch without retraining.
1. Occlusion
We occluded 25% of the foreground object region in the HRRI and evaluated the resulting outputs on 100 samples. RefGC-SR² achieved a DINO score of 0.734, PSNR of 16.97 dB, and LPIPS of 0.298, obtaining the best values among the three configurations reported in the table. These results indicate that the model can produce competitive outputs even when part of the reference object is unavailable. This evaluation was conducted using the same 100 samples as in Table 13. We have incorporated the corresponding qualitative and quantitative results into Fig. 10-(g), (h) and Table 13, respectively, and added the relevant discussion in L1384–L1401 of the revised manuscript.
| HRRI condition | Model | DINO ↑ | PSNR ↑ | LPIPS ↓ |
|---|---|---|---|---|
| Clean | AdaRefSR* | 0.692 | 17.34 | 0.289 |
| Clean | ImageCritic* | 0.742 | 17.62 | 0.285 |
| Clean | RefGC-SR² (Ours) | 0.751 | 18.54 | 0.256 |
| Occlusion | AdaRefSR* | 0.694 (+0.002) | 16.10 (−1.24) | 0.308 (+0.019) |
| Occlusion | ImageCritic* | 0.713 (−0.029) | 16.83 (−0.79) | 0.313 (+0.028) |
| Occlusion | RefGC-SR² (Ours) | 0.734 (−0.017) | 16.97 (−1.57) | 0.298 (+0.042) |
2. Incorrect details
In the RefGC-SR² setting, LRGI may contain generative artifacts introduced by the upstream reference-guided generation process, including substantial distortions of object identity. For example, Fig. 11 shows a DreamFuse compositing result in which the frog-shaped dispenser loses its protruding eyes and facial features and is rendered as a cylindrical bottle, although these characteristics are present in both HRRI and HRGT, as discussed in L1437-L1440. This example illustrates why preserving the appearance of LRGI alone may be insufficient for faithful refinement of outputs from reference-guided generation models. We therefore assume that a corresponding HRRI provides more reliable information about the target object’s identity and appearance than LRGI and use it to guide refinement. However, this reliance also makes the model susceptible to transferring incorrect instance-specific details when HRRI is unreliable.
To examine this limitation, we constructed 100 reference-substitution pairs by replacing the correct HRRI with an image of a different object from the same category. This setting evaluates references that are semantically similar but contain instance-specific details inconsistent with the target object. We measure DINO similarity against the substituted HRRI, whereas PSNR and LPIPS are measured against the original HRGT.
As shown in Table 14 and discussed in L1454-L1457, under same-category reference substitution, RefGC-SR² exhibits higher DINO similarity to the substituted HRRI than ImageCritic and AdaRefSR, while its fidelity to the original HRGT decreases, as reflected by lower PSNR and higher LPIPS relative to the clean-reference condition. However, DINO similarity alone does not identify which specific details are transferred.
The qualitative examples also illustrate how incorrect references can influence instance-specific appearance. In the bowl example in Fig. 12(a), RefGC-SR² produces a white interior and a denser exterior pattern resembling the substituted reference, rather than retaining the original green-toned appearance, as described in L1554-L1556. In the wine-glass example in Fig. 12(b), the originally yellow glass similarly changes to a turquoise appearance resembling the substituted reference, as described in L1556-L1559. These observations are consistent with the expected behavior of our model, which relies on HRRI as a reliable source of instance-specific appearance information to guide refinement.
A future research direction is to make this balance controllable, for example through a user-adjustable slider that regulates the influence of HRRI on the output. Such a mechanism could allow users to adjust reference guidance according to their preference in the reference.
| HRRI condition | Model | DINO ↑ | PSNR ↑ | LPIPS ↓ |
|---|---|---|---|---|
| Clean | AdaRefSR* | 0.6513 | 17.2706 | 0.2934 |
| Clean | ImageCritic* | 0.6730 | 17.0765 | 0.4148 |
| Clean | Ours | 0.7231 | 18.6731 | 0.2679 |
| Incorrect details | AdaRefSR* | 0.3818 | 17.2825 | 0.2990 |
| Incorrect details | ImageCritic* | 0.4792 | 14.5911 | 0.5015 |
| Incorrect details | Ours | 0.4868 | 13.4183 | 0.4174 |
To reviewer Yqy7
Thank you very much for your valuable comments that helped us significantly improve this submission. Below are detailed point-to-point responses about how we address your questions.
[W1] Frequency role of FreqMoLE
We thank the reviewer for suggesting a more direct analysis of the LF and HF experts. To investigate their utilization across network depth, we analyzed their gate-weighted Q/K/V LoRA responses across 57 layers using RefGC-SR² Benchmark and three denoising steps.
For each expert, we computed the squared magnitude of the 2D Fourier transform over spatial image tokens after applying its gate and excluded the DC component. Using a normalized frequency-radius cutoff of 0.25, we calculated each expert's share of the sum of the two experts' individual energies in each frequency band. We then divided the layers into four approximately equal groups and averaged the energy shares across layers, frequency bands, and denoising steps.
As shown in Table A, the LF expert dominates the early layers with an energy share of 84.18%, while the HF expert's share increases progressively with depth (15.82% → 60.74% → 88.56% → 98.25%). This pattern shows a shift in gate-weighted expert utilization from LF to HF across network depth, consistent with the intended depth-dependent design.
To examine whether the experts also have different frequency roles at the output level, we generate three outputs using the same input and seed: one with both experts enabled and two with either expert disabled. We apply a 2D Fourier transform to the difference between the full-model output and each ablated output within the object region, and measure the fraction of the change energy above a normalized frequency radius of 0.5. Across the same RefGC-SR² Benchmark, disabling the HF expert produces changes with a higher mean high-frequency share than disabling the LF expert (0.044 vs. 0.025), with a 95% confidence interval of [0.014, 0.023] for the difference, estimated using 10,000 image-level bootstrap resamples. The same direction holds in 150 of 200 images. These results support different relative frequency roles in the trained model, with changes induced by disabling the HF expert containing a larger proportion of high-frequency energy.
We further compare our proposed depth-dependent gate initialization with uniform and reversed initializations. As shown in Table B, our initialization achieves higher PSNR and SSIM and lower LPIPS than both alternatives. This ablation supports the effectiveness of the initialization strategy. The response analysis and output-level analysis provide complementary evidence: the former characterizes expert utilization across depth, while the latter reveals differences in the frequency composition of the output changes induced by disabling each expert.
Table A:
Table B: