acceptodds
Under review as a conference paper at ICLR 2027

TAPD-SR: Bridging the Detail-Conditioning Gap for Diffusion-Based Faithful Real-World Image Super-Resolution

Abstract

Real-world image super-resolution (Real-ISR) using pretrained text-to-image (T2I) diffusion models achieves strong perceptual quality but often suffers from unfaithful textures. We trace this failure to a detail-conditioning gap: loss of low-resolution (LR) features in latent space, under-specified by coarse textual prompts, and insufficiently enforced along the denoising trajectory. To address this issue, we present TAPD-SR, a unified framework that strengthens LR-derived conditioning to improve generated texture fidelity. At the representation level, a Time-Aware Detail Feature Extractor (TDFE) captures high-frequency information directly from the LR input and adaptively injects it into the diffusion process according to the denoising timestep. At the semantic level, a Reasoning-driven Fine-grained Prompt Generator (RFPG) produces fine-grained global and object-level prompts from the LR input using a fine-tuned multimodal large language model (MLLM). At the trajectory level, a Hierarchical Frequency Refinement Module (HFRM) extracts multi-scale frequency information from the LR input to predict a residual correction signal to the diffusion model's velocity output, providing additional structural and detail guidance for latent updates. Extensive experiments demonstrate that our TAPD-SR improves reconstruction fidelity while maintaining high perceptual quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
T2I diffusion model, super-resolution, image Fidelity, MLLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KVJROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets Real-World Image Super-Resolution (Real-ISR) with pretrained T2I diffusion models (Qwen-Image). The authors diagnose a “detail-conditioning gap” with three causes: (i) LR high-frequency loss in VAE latent space, (ii) coarse textual prompts, and (iii) weakening of conditioning along the denoising trajectory. They propose three components — TDFE , RFPG and HFRM. Experiments on DIV-2K, DrealSR, RealSR and UZ-ST report competitive fidelity/perceptual metrics and clear gains in text recognition

Critical strengths
  1. Clear three-level problem framing with a coherent, self-consistent ablation.
  2. Transparent reporting of cost and limitations.
Critical weaknesses
  1. In Figure 1, the VAE encoder and decoder icons are drawn with their trapezoid orientations swapped, "RPFG" for RFPG in the contribution list; Norm(·) in Eq. (3) never defined.
  2. Unfair comparison. Qwen-Image is a much larger model than the methods in the comparison list. The authors should either reproduce the other methods on Qwen-Image for comparison, or implement the proposed method on the models used by the other methods.
Questions for authors

See weaknesses

Reviewer_abXdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TAPD-SR, a diffusion-based framework for faithful real-world image super-resolution that addresses the loss of fine-grained conditioning information. Built on Qwen-Image, it introduces three components: a Time-Aware Detail Feature Extractor (TDFE) for preserving and injecting high-frequency details, a Reasoning-driven Fine-grained Prompt Generator (RFPG) for global and object-level semantic guidance, and a Hierarchical Frequency Refinement Module (HFRM) for correcting diffusion velocity predictions. Experiments on DIV2K, DRealSR, and RealSR demonstrate competitive reconstruction fidelity and perceptual quality, while evaluations on UZ-ST show improved text restoration accuracy. Ablation studies and qualitative comparisons further support the effectiveness of the proposed components.

Critical strengths
  1. TAPD-SR addresses detail preservation at three complementary levels: visual feature extraction, semantic prompting, and denoising trajectory refinement. In particular, TDFE preserves high-frequency information before VAE compression, while HFRM directly corrects diffusion velocity predictions using multi-scale frequency cues. This is a meaningful technical strength because it strengthens low-resolution conditioning at different stages of the generative process rather than relying solely on latent features or textual guidance.

  2. RFPG employs a degradation-aware, fine-tuned multimodal language model to generate global and object-level descriptions, which are injected through separate conditioning pathways. This design enables more detailed semantic guidance than conventional global captions or object tags. It is particularly valuable for restoring complex textures and textual content, where coarse semantic descriptions are often insufficient to guide accurate reconstruction.

Critical weaknesses
  1. RFPG is trained using HR-derived descriptions to supervise an MLLM that generates fine-grained prompts from degraded LR images. However, severely degraded inputs may not contain sufficient information to reliably recover specific textures or textual content. The model may therefore infer plausible but incorrect details from semantic priors. Although the paper reports improvements in OCR accuracy and reconstruction metrics, it does not sufficiently establish that the generated details are consistently supported by the LR observations.

  2. TAPD-SR combines TDFE, RFPG, and HFRM to strengthen detail conditioning at different stages of diffusion. While the paper provides ablation studies for individual components and their internal designs, it does not sufficiently investigate potential functional overlap or interactions among the three modules. In particular, both TDFE and HFRM exploit frequency information from the LR input, raising questions about whether their contributions are genuinely complementary. Without a more systematic comparison of module combinations, it is difficult to determine whether the additional architectural complexity is necessary.

Questions for authors
  1. Both TDFE and HFRM extract frequency-related information from the LR input, although they operate at different stages of the diffusion process. Could the authors provide a direct comparison of models using TDFE alone, HFRM alone, and both modules together under otherwise identical settings? This would help determine whether the two modules provide genuinely complementary benefits or introduce partially redundant conditioning.
  2. The paper reports substantial improvements in text reconstruction accuracy on UZ-ST. However, Qwen-Image already provides strong text-generation capabilities. How much of the improvement can be attributed specifically to RFPG rather than the pretrained backbone? Could the authors report OCR-A and NED results with and without RFPG under the same Qwen-Image backbone?
  3. RFPG is trained using HR-derived descriptions to supervise prompt generation from degraded LR images. However, severely degraded inputs may not contain sufficient information to recover specific textures or textual content. How does the method prevent the MLLM from introducing plausible but incorrect details?
Reviewer_8QmiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes TAPD‑SR, a real‑world image super‑resolution framework using a frozen Qwen‑Image diffusion backbone to address the “detail‑conditioning gap” causing texture unfaithfulness. It incorporates three collaborative modules: a timestep‑adaptive detail extractor to inject LR high‑frequency features via cross‑attention, a LoRA‑tuned Qwen3‑VL‑8B that yields global and object‑level prompts with separate injection paths, and a total‑variation‑based frequency refinement module for per‑step velocity residual correction. Trained on degraded Flickr2K, LSDIR and FFHQ data and evaluated on DIV2K, DRealSR, RealSR and text‑centric UZ‑ST, it achieves strong LPIPS, DISTS, MUSIQ and 48.8% OCR‑A / 0.6921 NED on UZ‑ST, while PSNR and MANIQA are not consistently best. With 20.51 B parameters and 20 sampling steps, it takes 8.512 s per image on A6000; RealSR‑based ablations verify its designs.

Critical strengths
  1. The system targets an important failure mode and evaluates text explicitly. Unfaithful small text and detailed structures are practical weaknesses of generative SR. The UZ-ST OCR/NED evaluation is more informative about text fidelity than generic perceptual metrics, and the reported improvement is substantial.

  2. The internal ablations are reasonably broad. The paper evaluates timestep modulation, interaction design, structural keys, global versus object prompts, prompt injection, frequency-level count, MLLM fine-tuning, decomposition alternatives, and module complementarity. These experiments support that all three components contribute inside the chosen backbone.

Critical weaknesses
  1. The methodological novelty is limited and fragmented. Timestep-aware LR feature injection is closely related to prior conditional diffusion SR; wavelet/high-frequency guidance is established; generating richer prompts with a fine-tuned MLLM extends existing prompt-based SR; and adding a frequency-conditioned residual to a frozen velocity prediction is a conventional adapter-style correction. The paper combines these at three locations but does not formulate a new deterministic-to-generative bridge, trajectory-alignment principle, objective, or theoretical account of the alleged conditioning gap.

  2. The main empirical comparison is not sufficiently fair to attribute gains to the proposed method. TAPD-SR uses a 20.51B Qwen-Image backbone plus an 8B Qwen3-VL teacher/student pipeline, whereas many baselines use substantially smaller Stable-Diffusion-family models or one-step sampling. The authors acknowledge that Chinese OCR gains partly arise from Qwen-Image itself. Baselines are not controlled for backbone, data, steps, parameters, or compute; TAPD-SR takes 8.512 seconds per image; and no same-Qwen-Image baseline isolates the three modules. Consequently, the reported SOTA metrics do not establish that the proposed components are superior to simpler conditioning on the same foundation model.

Questions for authors
  1. Please provide a same-backbone, same-training-data, same-20-step Qwen-Image baseline and compare TDFE, RFPG, and HFRM with the closest simpler alternatives under matched trainable parameters and compute. How much of the main-table and OCR improvement remains after controlling for Qwen-Image and Qwen3-VL?

  2. What happens when HR-derived MLLM pseudo-label supervision is removed, when the MLLM is replaced by a model comparable to those available to the baselines, or when prompts are generated by frozen Qwen3-VL without SFT? Please report OCR-A/NED and image metrics for these controls.

  3. Please conduct a blinded human study separating perceptual realism from faithfulness to LR/GT evidence and report hallucination-sensitive text/content error rates. How often does RFPG confidently complete unreadable text incorrectly under severe degradation?

  4. Please report the GPU count and wall-clock cost for TAPD-SR training, MLLM fine-tuning iterations, total-variation decomposition settings and solver, sampler/CFG parameters, random seeds and variance, exact baseline checkpoints, and whether all methods use identical resizing and degradation protocols.

  5. Is the proposed "detail-conditioning gap" empirically demonstrated inside Qwen-Image? Please measure how LR-detail information or correspondence decays across MMDiT depth and denoising time, and show that TDFE/HFRM specifically restores this signal rather than simply adding capacity and direct skip conditioning.

  6. What are the separate parameter counts, FLOPs, memory, and latency contributions of TDFE, RFPG/Qwen3-VL inference, TCA, and HFRM? Does the reported 8.512-second latency include prompt generation and all TV decomposition steps, and what is the end-to-end latency when no prompts are cached?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.