acceptodds
Under review as a conference paper at ICLR 2027

HiCoSR: High-Fidelity Single-Step Real-World Super-Resolution via Hierarchical Latent Diffusion

Abstract

High-fidelity real-world image super-resolution (RealSR) remains challenging for Latent Diffusion Models (LDMs) due to the fixed spatial compression in pre-trained variational autoencoders (VAEs), which causes fine structural information loss during encoding and often introduces artifacts in generated images. To address this limitation, we propose HiCoSR, a hierarchical and distribution-consistent framework designed to preserve spatial details in the latent space. Specifically, we design a HiCo-Encoder that introduces a parallel latent branch to maintain richer spatial and structural information beyond the capacity of the representation. Based on this dual-scale representation, we adopt a coarse-to-fine cascaded generation strategy to overcome the artifacts and semantic inaccuracies inherent in single-scale denoisers. The coarse stage first establishes a robust global layout, while the fine stage refines local structural details with a single denoising step at each stage. Extensive experiments on three RealSR benchmarks show that HiCoSR achieves strong reconstruction fidelity and comparable perceptual quality to other diffusion-based models, using only a pretrained SD-Turbo backbone.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
real-world image super-resolution, diffusion model, hierarchical VAE

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xsFkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a real-image super-resolution framework, HiCoSR, which constructs a high-resolution latent branch by fusing intermediate features from the encoder with upsampled coarse-grained latent variables, thus mitigating the loss of detail caused by VAE space compression. The model employs a two-stage recovery process from coarse to fine, performing one step of denoising at each stage, and promotes compatibility between the new representation and the pre-trained latent space through shared decoder supervision. On three benchmarks, it achieves improvements in PSNR and SSIM, and its inference speed is competitive, but its perceptual quality metrics show varying performance.

Critical strengths

The high-resolution latent characterization design is effective. After removing the HiCo feature, the PSNR on RealSR decreased from 26.48 to 24.61 dB, while the DISTS increased from 0.1817 to 0.2142.

It achieves best PSNR and SSIM on all three benchmarks, and also best LPIPS and DISTS on RealSR; evaluation on both real and synthetic data enhances the reliability of the results.

Reusing SD-Turbo, employing LoRA, and omitting image description generation results in low inference costs. Processing a 512×512 image on an RTX 5090 takes 0.13 seconds, close to OSEDiff and faster than TVT.

Critical weaknesses

Reconstruction supervision under a shared frozen decoder is insufficient to prove the latent distribution is "strictly aligned." Direct distribution analysis and ablation experiments distinguishing the contributions of shared decoder supervision from the increase in spatial capacity are lacking.

The coarse stage improves PSNR by 0.41 dB, but LPIPS and FID decrease slightly. The complete model actually performs two sequential denoising steps, but the table labels it as one step; a single-scale baseline matching training conditions and computational cost is lacking, making it difficult to determine whether the gains come from hierarchical design or additional computation.

Questions for authors

What exactly does "strict alignment" mean, and what evidence supports this? Can you compare high-resolution encoders with and without shared decoder supervision, all other things being equal, to verify their contribution to the compatibility and recovery performance of the pre-trained denoiser?

How does the performance compare to a single-scale double-denoising baseline using the same decoder, discriminator, and loss settings? Please also report fidelity, perceptual quality, end-to-end time, and actual number of denoising iterations.

4 comments
AuthorsOpenReview ↗

Response to Reviewer xsFk Comments 1

We thank the reviewer for recognizing the contribution of our higher-resolution latent representation, the improvements in fidelity and reference-based perceptual metrics, and the practical inference efficiency. We also appreciate the detailed questions on latent compatibility and the hierarchical design, which we address point by point below with additional analyses and controlled comparisons.


Q1 & W1 — Latent Compatibility and Shared-Decoder Supervision

Here, alignment refers to compatibility with the frozen pretrained decoder: the f4 latent preserves finer detail while remaining decodable by the original Decoder×8. We will use "pretrained-decoder compatibility constraint" in the revision to state this property precisely. With the decoder unchanged, the f4 representation achieves 39.46 dB PSNR and 0.0197 LPIPS, versus 33.83 dB and 0.0601 for the original VAE (Table 1).

We isolate the effect of this compatibility constraint by comparing the same f4 HiCo-Encoder trained with either the frozen pretrained Decoder×8 or a jointly trained decoder under matched training settings. We denote the resulting latents as f4 locked and f4 unlocked, respectively. The latter removes the shared frozen-decoder supervision. We evaluate both variants in the same restoration framework on RealSR, with results below.

ConfigurationDecoder during HiCoVAE trainingPSNR↑SSIM↑LPIPS↓DISTS↓FID↓MANIQA↑MUSIQ↑
Without frozen-decoder constraint (f4 unlocked)trained jointly25.630.75020.26870.1963121.860.635264.71
HiCoSR (ours, f4 locked)pretrained, frozen26.480.76950.24760.1817115.370.653865.36

At identical f4 spatial capacity, the frozen-decoder constraint improves all seven reported metrics, with a 0.85 dB PSNR gain, lower LPIPS (0.2687→0.2476) and lower FID (121.86→115.37), demonstrating its benefit beyond increased latent resolution.

Latent visualization. Figure R1 compares the original VAE's f8 latent, HiCoVAE's f4 locked latent, and f4 unlocked using 3,000 crops of size 512×512 from the open-source Aesthetic-4K dataset. For each crop, we use a bicubically upsampled 1024×1024 input for the original f8 VAE and the original 512×512 input for both f4 encoders. This gives 4×128×128 latents in all three cases for the t-SNE visualization.

In the joint t-SNE plot, the original f8 and HiCoVAE f4 locked latents strongly overlap without visibly separate clusters, whereas f4 unlocked exhibits a clearly different distribution. The three individual distribution plots on the right show the same contrast. The remaining differences between the locked f8 and f4 views may partly reflect the bicubic upsampling applied to the f8 inputs. Together with the restoration ablation, these observations support the role of shared-decoder supervision in maintaining compatibility with the pretrained latent domain.

t-SNE visualization of original VAE f8, HiCoVAE f4 locked, and f4 unlocked latents.

Figure R1. Latent distributions with and without shared frozen-decoder supervision. Left: joint t-SNE visualization. Right: individual views of f8 locked, f4 locked, and f4 unlocked, respectively.

Shared-decoder supervision supports both cross-scale fusion and reuse of pretrained priors. Without this constraint, the f4 latent can drift from the original f8 latent domain. First, the upsampler and fusion module must then reconcile this distribution mismatch while transferring the denoised f8 structure to f4, making it harder to combine coarse guidance with fine source details. Second, the fine denoiser inherits SD-Turbo weights learned in the original VAE latent domain. A shifted f4 distribution weakens transfer of their semantic priors and increases the adaptation burden on LoRA. Keeping the two scales compatible therefore facilitates feature fusion and lets the fine stage use pretrained semantic knowledge more effectively.

AuthorsOpenReview ↗

Response to Reviewer xsFk Comments 2


Q2 & W2 — Single-Step Hierarchical Denoising and Ablation Results

HiCoSR is a single-step hierarchical diffusion framework, with one denoising step at each resolution. The coarse denoiser establishes global structure at f8, while the fine denoiser refines fused f4 features with additional LoRA modules and adapted effective weights. Each of these distinct, resolution-specific modules predicts a clean latent in one forward pass. "Single-step" describes denoising within each stage; NFE counts the total denoiser evaluations across the cascade. We will clarify this convention in Table 2 as one step per stage with a total NFE of 2. We compare against f8→f8 and f4→f4 two-pass baselines using the same discriminator and matched training settings. Final f8 latents are decoded by Decoder×8, and final f4 latents by Decoder×4. All timings are measured end-to-end at 512×512 on an RTX 5090.

ConfigurationNFETime (s)PSNR↑SSIM↑DISTS↓MANIQA↑
Single pass, 4× (w/o D_f8)10.1126.070.76640.18750.6407
Two passes, 8×20.1025.490.73860.19610.6437
Two passes, 4×20.2026.110.76650.18710.6413
HiCoSR (ours)20.1326.480.76950.18170.6538

With NFE fixed at 2, HiCoSR leads both same-scale baselines on all four quality metrics, gaining 0.99 dB PSNR over f8→f8 and 0.37 dB over f4→f4. Adding a second f4 pass yields only 0.04 dB over the fine-only variant while increasing latency by 0.09 s (0.11→0.20 s). The f8→f4 cascade gains 0.41 dB with only 0.02 s additional latency (0.11→0.13 s). Thus, HiCoSR achieves higher quality and 35% lower latency than f4→f4 at the same NFE.

Across all eight metrics in Table 3, adding the coarse stage improves PSNR, SSIM, DISTS, MANIQA, MUSIQ and Q-Align over the fine-only variant. LPIPS increases from 0.2445 to 0.2476 and FID from 112.98 to 115.37. This operating point prioritizes structural fidelity while improving reference-based perceptual similarity measured by DISTS and all three no-reference scores, consistent with our restoration objective.

Denoising entirely at f4 requires the model to recover global structure and fine detail jointly on a larger latent grid. Repeating this operation retains that coupled task. HiCoSR separates these responsibilities: the f8 stage establishes global structure on a compact 64×64 grid, providing guidance for the f4 stage to refine local detail on a 128×128 grid. This also reduces computation compared with two passes at f4. The controlled comparisons demonstrate that the gains come from the coarse-to-fine, multi-resolution design, beyond simply increasing NFE to 2.

Thank you for the clarifications and additional experiments; after considering the rebuttal, I maintain my original score of 3.

AuthorsOpenReview ↗

Thank You for Your Time and Positive Feedback

Thank you for carefully considering our clarifications and additional experiments. We sincerely appreciate your time and constructive feedback. If there are any remaining concerns you would like us to address, we would be happy to provide further clarification.

Reviewer_2NYJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes HiCoSR, aims to mitigate the loss of detail caused by the 8× spatial compression of the VAE in pre-trained latent diffusion models. It employs a "HiCo-Encoder" to construct dual-scale latent representations f8 and f4, supervised by a shared, frozen decoder, and conducts image restoration through a coarse-to-fine, two-stage denoising process. The coarse stage is to reconstructed the overall structure, while the fine stage utilizes high-resolution latent representations and LoRA adapters to further refine the results. Experiments conducted on the RealSR, DRealSR, and DIV2K-Val datasets demonstrate strong performance in terms of PSNR, SSIM, and various perceptual distance metrics.

Critical strengths
  1. The method achieves high PSNR and SSIM scores across all three datasets, while also offering a significant advantage in terms of inference speed.
  2. The research motivation is clear, the problem addressed is practically significant, and the design logic—incorporating a dual-scale latent space and cascaded generation—is well-structured.
Critical weaknesses
  1. The innovation is limited. Techniques such as 4× high-resolution latent spaces, feature fusion, cascaded denoising, and LoRA are already well-established; the paper fails to sufficiently demonstrate that the combination of these components constitutes a substantive innovation beyond standard architectural designs.
  2. In some of the provided visualizations, the handling of fine textures is inferior to that of existing methods, with issues such as the appearance of artifacts and over-smoothing.
  3. The results presented in the ablation studies show a clear degradation in perceptual quality, as the authors prioritize fidelity metrics (like PSNR) at the expense of the FID score.
  4. A comparison between a standard VAE and the 4× VAE is required to empirically demonstrate the actual contribution of the high-resolution latent space.
Questions for authors
  1. Resummarize the novelty of the proposed method. The current components—4× high-resolution latent space, feature fusion, cascaded denoising, and LoRA—are all based on well-established techniques for addressing perceptual quality and distortion.
  2. Conduct ablation studies to demonstrate that the proposed method offers advantages in terms of both perceptual quality and distortion quality.
  3. Provide a comparison between a standard VAE and the 4× VAE to substantiate the actual contribution of the high-resolution latent space.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 2NYJ Comments 1

We thank the reviewer for recognizing HiCoSR's strong PSNR/SSIM performance across all three benchmarks, fast inference, and clear motivation for combining dual-scale latents with cascaded restoration. We appreciate the constructive questions and suggestions and address each concern below.


Q1 & W1 — Novelty Beyond the Individual Components

Motivation. Standard f8 VAE compression removes fine image structures, leaving the denoiser to infer details that are no longer present in its input. Preserving more information in the latent can improve fidelity, but it also makes generation harder. LightningDiT / VA-VAE[1] shows that increasing latent channels can require larger diffusion models or longer training. Latent Forcing[2] discusses the slower convergence of less compressed representations and shows the benefit of generating latent structure before pixel detail. Our motivation is therefore to preserve more source details while making the richer representation easier to denoise.

Our design. HiCoSR increases spatial resolution from f8 to f4 while retaining four latent channels, providing a compromise between compact f8 latents and pixels. Skip connections bring in shallow VAE features to preserve details lost during f8 compression. We make this richer representation easier to use in two ways. Shared frozen-decoder supervision promotes compatibility with the pretrained latent domain so the model can reuse SD-Turbo priors. The f8 stage first recovers global structure, which guides the f4 stage to refine local detail. Our contribution combines detail preservation with pretrained compatibility and coarse-to-fine guidance, allowing the model to use a larger latent space without requiring the fine stage to recover global structure and local detail on its own. LoRA provides parameter-efficient adaptation within this framework.

Table 3 supports the contribution of each component. The w/o HiCo row keeps both denoisers and the full two-stage path, so its number of denoiser evaluations equals that of the full model, and only the authentic f4 source features are replaced by upsampled f8 features. PSNR then falls from 26.48 to 24.61 dB and DISTS rises from 0.1817 to 0.2142, the largest degradation among the three ablations on both metrics. Removing the fine denoiser, which confines denoising to the 8× latent, costs 1.12 dB, and removing the coarse denoiser, which forces the 4× stage to establish global semantics as well, lowers MANIQA and Q-Align. PSNR and SSIM are highest for the full model and decrease with every removal.

Controlled comparisons at the same NFE. We train f8→f8 and f4→f4 with the same discriminator and matched training settings as HiCoSR. Final f8 latents are decoded by Decoder×8, and final f4 latents by Decoder×4. All three two-pass configurations perform two denoiser evaluations, allowing us to compare same-scale refinement with the multi-resolution cascade.

ConfigurationPass 1Pass 2PSNR↑SSIM↑LPIPS↓DISTS↓MANIQA↑MUSIQ↑FID↓Q-Align↑
Single pass, 8× (w/o D_f4)f8—25.360.73310.26730.19780.639865.49107.313.527
Single pass, 4× (w/o D_f8)—f426.070.76640.24450.18750.640765.16112.983.356
Two passes, 8×f8f825.490.73860.26490.19610.643765.62115.623.519
Two passes, 4×f4f426.110.76650.24500.18710.641365.17115.483.358
HiCoSR (ours)f8f426.480.76950.24760.18170.653865.36115.373.443

HiCoSR outperforms both same-scale two-pass baselines on PSNR, SSIM, DISTS, MANIQA and FID, and against the stronger same-scale variant f4→f4 it also leads on MUSIQ and Q-Align. At an identical denoiser count, HiCoSR improves PSNR by 0.37 dB, reduces DISTS by 0.0054, and raises MANIQA and Q-Align by 0.0125 and 0.085, respectively, relative to f4→f4. LPIPS is 0.0026 higher. These comparisons demonstrate the benefit of the multi-resolution cascade beyond adding a second pass.

The two same-scale variants quantify what a second pass contributes on its own. f4→f4 reaches 26.11 dB against 26.07 dB for the single 4× pass, and f8→f8 reaches 25.49 dB against 25.36 dB for the single 8× pass, so repeating a pass at one scale adds 0.04 and 0.13 dB. The second pass revisits a latent in which the discarded structure is already absent, or in which global semantics must be established without the prior the 8× stage supplies. HiCoSR instead recovers 0.41 dB over the single 4× pass by placing the 8× stage first, where the pretrained SD-Turbo prior establishes global structure, and leaving the 4× stage to refine detail in the higher-density latent. f4→f4 retains a 0.0026 LPIPS advantage, while HiCoSR improves the other seven metrics. f8→f8 leads on MUSIQ and Q-Align while trailing by 0.99 dB PSNR.

AuthorsOpenReview ↗

Response to Reviewer 2NYJ Comments 2


W2 — Fine-Texture Artifacts and Over-Smoothing

HiCoSR prioritizes structural fidelity and natural texture recovery. The f4 branch and skip connections preserve source details, while the coarse-to-fine cascade establishes global structure before refining local textures. This grounds generation in the input and helps reduce distortions caused by synthesizing missing structures.

Figure R1 compares the restored details directly with the GT. In the upper example, HiCoSR preserves the diagonal ridges and fine irregular texture visible in the GT, while LucidFlux and TSD-SR produce sharper, more uniform bands. In the lower example, HiCoSR recovers distinct, connected chain links; OSEDiff blurs their boundaries, and LucidFlux introduces strong dark and bright edges. In these examples, HiCoSR produces fewer visible artifacts and more natural textures that closely follow the GT, preserving both local detail and structural continuity.

Visual comparisons with GT references for diagonal textures and chain links.

Figure R1. Visual comparisons with GT references. Upper: diagonal texture. Lower: chain links. Each example includes the LR input, GT and restoration results for the same region.

The quantitative results support this fidelity advantage. In Table 2, HiCoSR achieves the best PSNR, SSIM, LPIPS, DISTS and FSIM on RealSR, including LPIPS of 0.2476 and DISTS of 0.1817. It also achieves the lowest DISTS on DRealSR (0.1968), demonstrating strong agreement with the reference in both pixel fidelity and perceptual structure.


Q2 & W3 — Fidelity and Perceptual Quality in the Ablations

Table 3 of the paper reports all eight metrics for every ablation, covering both fidelity and perceptual quality. The variants without the coarse or fine denoiser assess each stage's contribution using the same discriminator and matched training settings. The same-scale two-pass comparisons above further assess the benefit of the hierarchical design at the same denoiser count.

ConfigurationPSNR↑SSIM↑LPIPS↓DISTS↓MANIQA↑MUSIQ↑FID↓Q-Align↑
w/o D_f8 (one stage)26.070.76640.24450.18750.640765.16112.983.356
HiCoSR (ours)26.480.76950.24760.18170.653865.36115.373.443
Change+0.41+0.0031+0.0031−0.0058+0.0131+0.20+2.39+0.087

Adding the coarse stage improves six of the eight metrics: PSNR rises by 0.41 dB, SSIM by 0.0031 and DISTS decreases by 0.0058, while MANIQA, MUSIQ and Q-Align all rise. The benefits therefore extend beyond pixel fidelity to reference-based perceptual similarity and no-reference quality scores. LPIPS and FID increase by 0.0031 and 2.39, respectively (1.3% and 2.1%), reflecting the trade-off between these evaluation criteria.

The trade-off is also a design decision, and Table 7 reports both configurations. The Decoder×8 variant achieves higher MANIQA and MUSIQ and lower FID. The Decoder×4 variant reported as ours gains 1.01 dB PSNR, with higher SSIM and Q-Align and lower LPIPS and DISTS, improving five of the eight metrics and all four fidelity metrics in the table. HiCoSR is a fidelity-oriented method for real-world SR, and the claim it makes is improved fidelity with competitive perceptual quality.

The GT measurements also help explain the no-reference scores. Evaluated with the same metrics as Table 2, the RealSR HR set scores MUSIQ 57.46, MANIQA 0.6017 and Q-Align 3.024, below every compared method on MUSIQ and Q-Align. These measurements suggest that closely reproducing the GT can also preserve appearance characteristics that receive lower no-reference scores. This may partly explain why a fidelity-oriented method such as HiCoSR does not lead every no-reference ranking. HiCoSR itself exceeds the GT on all three measures while prioritizing structural agreement with it.

AuthorsOpenReview ↗

Response to Reviewer 2NYJ Comments 3


Q3 & W4 — Standard VAE, 4× VAE, and HiCo-VAE

Reconstruction comparison. We compare the standard f8 VAE, a conventional f4 VAE, and HiCo-VAE below. For the conventional f4 VAE, we reproduce the reconstruction results using the publicly released pretrained weights from Latent Diffusion Models (LDM)[3]. The standard VAE and HiCo-VAE results are from Table 1; HiCo-VAE denotes HiCo-Encoder with Decoder×4.

ConfigurationLatentPSNR-Y↑SSIM-Y↑LPIPS↓DISTS↓FID↓
Standard VAEf833.830.91010.06010.066920.25
Conventional 4× VAE (LDM[3])f439.920.97490.01590.01954.51
HiCo-VAE (ours)f440.950.97960.01400.01714.23

The conventional f4 VAE gains 6.09 dB PSNR over the standard VAE and reduces LPIPS from 0.0601 to 0.0159, supporting higher spatial resolution as an effective way to alleviate compression loss. At the same spatial downsampling factor of four, HiCo-VAE adds 1.03 dB PSNR and improves all five metrics. The comparison supports our central motivation: retain more source information in a denser latent, with HiCo-VAE achieving further reconstruction improvements.

From richer latents to effective restoration. Denoising a denser f4 latent in a single stage requires global structure and fine detail to be modeled jointly. The single-pass results above illustrate the resulting trade-off: f4 improves PSNR over f8 (26.07 vs. 25.36 dB), while f8 retains better FID (107.31 vs. 112.98) and Q-Align (3.527 vs. 3.356). This motivates using the compact f8 space to establish global structure before refining the preserved details at f4, easing the fine stage's modeling burden.

The resulting cascade reaches 26.48 dB PSNR, 0.1817 DISTS and 0.6538 MANIQA, improving all three over either single-pass variant. It also exceeds f4→f4 by 0.37 dB at the same NFE, showing the value of coarse-to-fine guidance beyond an additional fine-scale pass. Our design thus combines the detail-preservation benefit of higher-resolution latents with a generation strategy suited to their increased spatial complexity.


References

[1] Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models. 2025. (LightningDiT / VA-VAE.)

[2] Alan Baade, Eric Ryan Chan, Kyle Sargent, Changan Chen, Justin Johnson, Ehsan Adeli, and Li Fei-Fei. Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation. 2026.

[3] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. CVPR, 2022.

Reviewer_wnvJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HiCoSR, a high-fidelity framework for real-world image super-resolution, aiming to mitigate the loss of fine-grained details caused by the high spatial compression ratio of pretrained VAEs. Specifically, the authors introduce HiCo-Encoder, which incorporates a higher-resolution 4× latent branch alongside the original 8× latent representation to preserve fine-grained structural information through feature fusion and shared-decoder supervision. Furthermore, a coarse-to-fine cascaded denoising framework is developed based on pretrained SD-Turbo, progressively restoring global structures and local textures in low- and high-resolution latent spaces, with a single denoising step at each stage. Experimental results demonstrate that the proposed method achieves competitive reconstruction fidelity in terms of PSNR and SSIM while maintaining high inference efficiency.

Critical strengths

1. Clear Motivation and Well-Justified Method Design.

This paper addresses the loss of fine-grained details caused by the 8× spatial compression of pretrained VAEs by introducing hierarchical latent representations and a coarse-to-fine cascaded denoising framework. The overall design is well motivated, with closely integrated components that effectively combine global structural restoration with local detail refinement.

2. Clear Presentation and Well-Organized Technical Framework.

The paper is well structured and logically organized, progressing from the limitations of VAE compression to the proposed HiCo-Encoder and dual-scale cascaded denoising strategy. The methodology is closely aligned with the research motivation, and the overall architecture and stage-wise training procedure are clearly presented, making the contributions of individual components easy to understand.

3. Efficient Inference and Comprehensive Experiments.

Despite employing a two-stage cascaded denoising framework, HiCoSR achieves an inference latency of only 0.13 s on an RTX 5090, comparable to OSEDiff (0.12 s) and notably faster than TVT (0.23 s). Moreover, the paper provides extensive evaluations, including comparisons across multiple benchmarks, component-wise ablation studies, inference efficiency analysis, and qualitative visual results.

Critical weaknesses

1. Unclear Evaluation Fairness and Performance Attribution of HiCo-Encoder

In Table 1, HiCo-Encoder + Decoder (8×) improves PSNR from 33.83 dB to 39.46 dB (+5.63 dB) compared with the original VAE, and further achieves 40.95 dB with Decoder (4×). However, the input/output resolutions, latent representations, and evaluation protocols for different configurations are not clearly specified, making it difficult to assess the fairness of these comparisons. Moreover, the architecture, implementation details, and parameter count of the Upsampler are insufficiently described. The authors are encouraged to provide controlled experiments under consistent settings and evaluate generalization on independent test sets to clarify the sources of performance gains and rule out potential overfitting.

2. Insufficient Validation of Texture Fidelity and Potential Hallucinations

The visual comparisons in Figure 5 and Appendix Figures 7–8 do not include corresponding HR reference images, making it difficult to assess whether the generated details faithfully reflect the original content. In particular, some horizontal patterns on wall tiles and fabrics appear unnatural, suggesting potential texture artifacts or hallucinations. Furthermore, Table 2 shows that HiCoSR underperforms TVT on RealSR in terms of MUSIQ (65.36 vs. 69.86) and FID (115.37 vs. 110.10). The authors are encouraged to provide visual comparisons with HR references and conduct a more systematic analysis of potential texture distortions and hallucinations.

3. Significant Impact of DINOv3 and Insufficient Validation of the Core Architecture's Independent Contribution

Table 6 shows that replacing DINOv1 with DINOv3 as the discriminator backbone improves PSNR from 25.34 dB to 26.48 dB (+1.14 dB), which is substantially larger than the improvement of HiCoSR over the best-performing baseline in Table 2 (+0.22 dB). Although these improvements cannot be directly compared for causal attribution, the results suggest that the stronger discriminator may play a significant role in the overall performance. The authors are encouraged to conduct more rigorous controlled experiments under identical discriminator and training settings to better distinguish the contributions of the DINOv3-based training strategy from those of the proposed HiCoSR architecture.

Questions for authors

1. Evaluation Settings and Performance Gains of HiCo-Encoder

In Table 1, HiCo-Encoder + Decoder (8×) achieves a 5.63 dB PSNR improvement over the original VAE. Could the authors clarify the latent branches, input/output resolutions, and evaluation protocols used in each configuration, and explain the source of this substantial improvement? Additionally, could the authors provide the architecture, parameter count, and training details of the Upsampler to better assess the contribution and generalization capability of HiCo-Encoder?

2. Texture Fidelity and Potential Hallucinations

In Figure 5 , some regions, particularly wall tiles and fabrics, exhibit seemingly unnatural repetitive textures. Could the authors clarify whether these generated patterns faithfully reflect the underlying image structures? Could they provide visual comparisons with HR references, where available, or other supporting evidence to assess reconstruction fidelity and potential hallucinations in complex texture regions?

3. Contributions of DINOv3 and the Core Architecture

Table 6 shows that replacing DINOv1 with DINOv3 improves PSNR by 1.14 dB, suggesting that the discriminator backbone substantially affects the final performance. Although Table 3 provides component-wise ablations, could the authors clarify whether these experiments use identical discriminator and training configurations? Furthermore, could they disentangle the contributions of the DINOv3-based training strategy and the proposed HiCoSR architecture to the overall performance gains?

5 comments
AuthorsOpenReview ↗

Response to Reviewer wnvJ Comments 1

We thank the reviewer for recognizing the clear motivation and coherent design of HiCoSR, its efficient inference, and the breadth of our experiments. We also appreciate the constructive questions and suggestions, which we address point by point below.


Q1 & W1 — Table 1: Configurations, Resolutions, and Source of the Gains

Table 1 is an autoencoding comparison. A 512×512 HR image from RealSR is encoded, decoded and compared against that same image at 512×512. No degraded LR input and no restoration step is involved.

ConfigurationEncoder inputLatentDecoderDecoder outputPost-processingScored at
Original VAE512×512f8, 64×64Decoder×8, frozen512×512none512×512
HiCo-Encoder + Decoder (8×)512×512f4, 128×128Decoder×8, frozen1024×1024bicubic ↓2512×512
HiCo-Encoder + Decoder (4×)512×512f4, 128×128Decoder×4, fine-tuned512×512none512×512

Rows 1 and 2 share the frozen VAE encoder and Decoder×8; row 2 introduces an f4 latent built from the f8 latent and shallow encoder features. Rows 2 and 3 use the same HiCo-Encoder and f4 latent with different decoding paths: Decoder×8 followed by downsampling, or the adapted Decoder×4 with direct output at the target resolution.

Evaluation protocol. All three rows use the HR images of RealSR, cropped to 512×512, with identical crops, implementation and preprocessing. The 1024×1024 output of row 2 is downsampled to 512×512 before scoring, so every configuration is compared at one resolution against one reference. PSNR and SSIM are computed on the Y channel.

Upsampler. The module inside HiCo-Encoder consists of a 3×3 convolution, two residual blocks, a 2× upsampling stage, a self-attention layer, two residual blocks and a 1×1 convolution. Each residual block contains two groups of 3×3 convolution, GroupNorm and Swish. After upsampling, the latent is fused through a skip connection with features from the original VAE encoder at the same spatial resolution. This connection supplies source-image details lost during f8 compression. Without it, the higher-resolution latent is derived solely from f8 and inherits its information bottleneck, yielding reconstruction quality similar to the original f8 representation. The skip connection is therefore essential for preserving additional structural information in the expanded latent. The module has 5.0M parameters and is trained independently on LSDIR and FFHQ with the HiCo-Encoder objective: L1, LPIPS and PatchGAN losses weighted 1.0:1.0:0.1 (Appendix A), while the VAE encoder stays frozen.

Source of the gains. The f4 branch reduces compression loss while preserving source details through skip connections. For a 512×512 RGB image, the f8 latent has shape 64×64×4, a 48× reduction in the number of scalar values, whereas the f4 latent has shape 128×128×4, a 12× reduction. The shallow encoder features supply boundary and texture information to this denser latent, and training through the frozen Decoder×8 against a 1024×1024 HR target encourages detail preservation. With Decoder×8 kept frozen, the HiCo configuration already improves PSNR by 5.63 dB, while LPIPS/DISTS/FID improve from 0.0601/0.0669/20.25 to 0.0197/0.0262/6.23. These gains support the effectiveness of the richer latent representation under a fixed pretrained decoder.

The Decoder×8 path requires bicubic downsampling after RGB decoding, which can smooth fine details and affect reconstruction quality. Adapting Decoder×4 to the f4 latents enables direct 512×512 output and avoids this extra resampling, yielding a further 1.49 dB PSNR and reaching 40.95 dB. This configuration combines the richer representation with a decoder suited to its spatial resolution and achieves the best reconstruction results across all five metrics in Table 1.

Generalization. HiCo-Encoder, the Upsampler and Decoder×4 are trained on LSDIR and FFHQ, whereas Table 1 is evaluated on the HR images of RealSR, which are not part of either training set. Evaluation on this independent dataset supports the generalization of the reconstruction gains.

AuthorsOpenReview ↗

Response to Reviewer wnvJ Comments 2


Q2 & W2 — Texture Fidelity in Figure 5 and the MUSIQ/FID Gap

HiCoSR balances reconstruction fidelity and generative texture synthesis, with priority given to preserving source structure. The f4 branch and skip connections retain details from the input, while the f8→f4 cascade uses the pretrained diffusion prior to establish global structure and refine local textures. This design grounds generation in the preserved source features, reducing reliance on synthesizing details lost during compression. Its fidelity advantage is reflected in Table 2: HiCoSR leads on RealSR in PSNR, SSIM, LPIPS, DISTS and FSIM.

Figure R1 adds GT references for the diagonal-texture and chain examples from Figure 5. In the upper example, the diagonal pattern is already visible in the LR, and HiCoSR recovers the fine irregular texture along the ridges seen in the GT. LucidFlux and TSD-SR produce smoother, more uniform bands. In the lower example, HiCoSR preserves distinct, connected chain links and wall boundaries consistent with the GT. These examples illustrate how the design supports texture recovery while maintaining structural fidelity.

Visual comparisons with GT references for diagonal textures and chain links.

Figure R1. Visual comparisons with GT references. Upper: diagonal texture. Lower: chain links. LR, GT and restoration crops show the same region; GT denotes the HR reference.

Table 7 demonstrates the fidelity–perception trade-off within our framework. Decoder×8 yields higher MUSIQ (67.84 vs. 65.36) and lower FID (108.69 vs. 115.37). Our selected Decoder×4 gains 1.01 dB PSNR (25.47→26.48), together with better SSIM, LPIPS and DISTS. We choose this configuration because it better serves our objective of faithful reconstruction, while retaining competitive perceptual quality.

The GT measurements help explain the no-reference scores. The RealSR HR set itself scores MUSIQ 57.46, MANIQA 0.6017 and Q-Align 3.024, below every compared method on MUSIQ and Q-Align. Closely reproducing the GT may therefore preserve appearance characteristics that receive lower no-reference scores. HiCoSR exceeds the GT on all three measures and achieves the highest MANIQA among the compared single-step methods on RealSR (0.6538), showing that its fidelity-oriented design also maintains strong perceptual quality.

AuthorsOpenReview ↗

Response to Reviewer wnvJ Comments 3


Q3 & W3 — DINOv3 versus the Architecture's Own Contribution

Choice of discriminator backbone. We select DINOv3 for its strong feature representation and effective adversarial supervision. With the full HiCoSR architecture fixed, the backbone comparison in Table 6 shows that DINOv3 achieves the best results on seven of eight metrics, including a 1.14 dB PSNR gain over DINOv1. In our experiments, the weaker discriminator backbones led to less stable training and more visible artifacts. DINOv3 provided the stable, informative supervision needed for our richer latent representation and cascade to realize their detail-recovery potential, motivating its use as the discriminator's frozen feature extractor.

Discriminator backbonePSNR↑SSIM↑LPIPS↓DISTS↓MANIQA↑MUSIQ↑FID↓Q-Align↑
DINOv125.340.75810.25450.20550.596462.53119.502.993
XCIT25.280.74870.25970.20510.635765.44128.893.315
DINOv3 (ours)26.480.76950.24760.18170.653865.36115.373.443

To examine how this benefit depends on the generator, we additionally compare DINOv1 and DINOv3 with only D_f8 (w/o D_f4), keeping the generator architecture fixed.

Discriminator backbone (only D_f8)PSNR↑SSIM↑LPIPS↓DISTS↓MANIQA↑MUSIQ↑FID↓
DINOv125.540.74010.23910.18880.624765.38108.75
DINOv325.360.73310.26730.19780.639865.49107.31

The stronger discriminator brings limited overall benefit to only D_f8. DINOv1 leads on the four fidelity metrics, while DINOv3 improves MANIQA, MUSIQ and FID. This suggests that the compressed f8 representation limits the generator's ability to exploit stronger supervision. HiCoSR's denser f4 representation and cascaded refinement provide greater detail-recovery capacity, for which stable DINOv3 supervision becomes more effective. Switching from DINOv1 to DINOv3 raises full-model PSNR by 1.14 dB, compared with −0.18 dB for only D_f8.

Architecture contributions under the same discriminator. Table 3 uses the same DINOv3 discriminator and matched training settings while varying the restoration components. The w/o HiCo variant retains the two-stage pipeline but replaces the authentic f4 source features with upsampled f8 features.

ConfigurationPSNR↑SSIM↑LPIPS↓DISTS↓MANIQA↑MUSIQ↑FID↓Q-Align↑
w/o D_f425.360.73310.26730.19780.639865.49107.313.527
w/o HiCo24.610.72230.29120.21420.631666.05125.873.512
w/o D_f826.070.76640.24450.18750.640765.16112.983.356
HiCoSR (ours)26.480.76950.24760.18170.653865.36115.373.443

Even with the same DINOv3 supervision, removing the HiCo source features costs 1.87 dB PSNR and increases DISTS from 0.1817 to 0.2142, demonstrating their importance for preserving fine structures. Removing the fine denoiser costs 1.12 dB, showing the value of refinement in the higher-resolution latent. Adding the coarse stage to the fine-only variant improves PSNR by 0.41 dB, reduces DISTS by 0.0058 and raises MANIQA by 0.0131, supporting its role in guiding structural recovery. These gains isolate the contribution of our representation and cascade under a common, strong discriminator.

Comments on the paper

1 comment
Area_Chair_nubGOpenReview ↗

Author-Reviewer Discussion

Dear Reviewers,

Thank you for your time and thoughtful reviews. With the author–reviewer discussion period ending on November 18 (AoE), please review the authors' rebuttal and share your updated assessment. Specifically, let us know whether your main concerns have been resolved, and if not, which points remain open.

Your input before the deadline is greatly appreciated!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.