acceptodds
Withdrawn from ICLR 2027

Decouple then Revive: Towards Perceptual Super-Resolution via Reinforcement Learning with Anchored Generative Priors

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion models advance RealSR through progressive distribution matching, yet existing training paradigms remain challenged: supervised finetuning on paired data tends to produce over-smoothed textures or misaligned semantics, while RL-based approaches introduce structural and textural inconsistency. In this paper, we propose NA-DePO (Null-space Anchored Dense Preference Optimization), which decomposes the SR signal into observation-constrained and unconstrained components via null-space decomposition, and restricts RL exploration exclusively to the unconstrained generative subspace to leverage generative priors without compromising structural consistency. We decouple the optimization process: we maintain the LR fidelity via a range-space constraint, and conduct negative-aware contrastive policy learning within the null-space. Furthermore, to provide informative and stable guidance, we design a dense reward mechanism incorporating VLM-driven local reward maps and timestep-aware scheduling, effectively reducing multi-reward conflicts. Extensive experiments demonstrate that NA-DePO significantly improves the balance between realistic detail generation and faithful structural restoration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Real-world Super-Resolution, Reinforcement Learning, Generative Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K9t6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies RL post-training of diffusion-based real-world super-resolution models, seeking to improve perceptual detail without losing structural fidelity. It proposes NA-DePO, which combines an approximate range/null-space decomposition with local VLM-derived rewards, sigmoid reward reweighting, and timestep-dependent reward scheduling. The method is evaluated on DIV2K and RealLQ using M3ESR as backbone, with ablations and qualitative comparisons.

Critical strengths

1: A coherent fidelity-preserving RL formulation. Section 3.4 and Equations 8 to 12 provide a concrete way to combine preference optimization with a low-frequency observation constraint. Table 2 supports the basic motivation: unconstrained RL improves some no-reference scores but substantially harms PSNR and LPIPS relative to the proposed full method, while the complete method improves both perceptual metrics and LPIPS over M3ESR.

2: Consistent empirical improvements over the chosen backbone. Table 1 reports that M3ESR + Ours achieves the strongest listed no-reference perceptual scores on both DIV2K and RealLQ, while Figure 6 and Table 2 provide evidence of an improved fidelity-perceptuality trade-off relative to the pretrained backbone. The direct re-training of DPPO-SR and LucidNFT on the same M3ESR backbone is particularly valuable for making the RL comparison more meaningful.

Critical weaknesses
  1. The paper’s central “null-space anchored” interpretation is only heuristic in the actual implementation, but the main claims often describe it as a strict exploration restriction. Section 3.4 introduces Equation 7 using a pseudo-inverse and argues that RL is projected into the null-space while the range-space is preserved. Yet the implementation discussion on Pages 17 to 18 states that A†A^\dagger is interpolation rather than the Moore–Penrose pseudo-inverse, so P=A†AP = A^\dagger A is only an approximate projection. Consequently, (I−P)z(I - P)z is not generally a null-space component of the actual observation operator, and because the VAE decoder is nonlinear, the latent operation does not imply a corresponding observation-preserving decomposition in image space. The appendix appropriately acknowledges this limitation, but this substantially weakens language in the abstract, Section 3.4, and conclusion suggesting that RL exploration is restricted “exclusively” to an unconstrained null-space or that it ensures signal constancy. The method may still be a useful frequency-biased anchoring heuristic, but the claim should be narrowed accordingly. A resolution would be to clearly distinguish the practical approximate projection from a mathematical null-space constraint throughout the paper, and quantify LR-consistency directly, for example by reporting ∥AD(z^)−Axgt∥\left\| A D(\hat{z}) - A x_{\mathrm{gt}} \right\| or an equivalent observation-domain residual against relevant variants.

  2. The evidence does not isolate the two advertised dense-guidance contributions, namely spatial reward factorization and QwenVL-Local. Section 3.3 claims that QwenVL-Local yields precise local semantic supervision, and Sections 3.3 and 3.6 claim that spatialized rewards and scheduling reduce reward conflicts and stabilize training. However, Table 2 contains no “without QwenVL-Local,” global-Qwen, or scalar-reward counterpart under otherwise identical null-space anchoring, reward composition, and sigmoid mapping. Figure 12 is qualitative only, while the scalar DiffusionNFT row in Table 2 changes multiple factors at once. Likewise, the “w/o reward scheduling” ablation changes the schedule but does not establish that spatial factorization itself is beneficial, and no metric of multi-reward conflict or gradient disagreement is provided. Since dense VLM guidance is a central contribution rather than an implementation detail, controlled quantitative ablations are necessary to establish the claimed causal mechanism. At minimum, the authors should report a matched scalar version of the same rewards, a global-Qwen similarity version, and metrics for the Qwen-removal variant; otherwise the corresponding claims should be softened.

  3. The experimental positioning omits directly relevant recent RL-for-RealSR approaches, making the state-of-the-art and novelty claims insufficiently established. Table 1 compares DPPO-SR and LucidNFT, but the paper does not discuss or evaluate process-aware and VLM-guided RL alternatives that also explicitly address the realism-fidelity trade-off. This matters because the claimed contribution is not merely applying RL to SR, but specifically improving reward guidance and constraining exploration. A direct comparison or a careful discussion of the methodological distinctions from OARS and RealSR-R1 is needed; if comparable implementations are available, evaluation on the same MSE3R protocol would be the most informative resolution.

Potentially Missing Related Work

Zhao et al., OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution, 2026 — directly studies online RL alignment for generative RealSR with staged/process-aware reward design and fidelity-perception considerations. It should be discussed in Section 2.2 and compared conceptually with the proposed timestep-dependent reward scheduling and constrained exploration; it is also a strong relevant baseline for Table 1.

Qiao et al., RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought, 2025 — directly applies RL with multiple vision-language-based reward signals to RealSR. It should be discussed in Sections 1 and 2.2, especially relative to the paper’s claim that VLM-guided dense rewards provide a new route to semantic consistency.

Wu et al., One-Step Effective Diffusion Network for Real-World Image Super-Resolution, 2024 — introduces OSEDiff, a relevant generative RealSR baseline with a different diffusion parameterization and supervision strategy. It should be added to the diffusion-based RealSR discussion in Section 2.1 and considered in the quantitative comparison when compatible evaluation results are available.

Wei et al., Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models, 2025 — studies semantic understanding and multimodal generative priors for RealSR, directly relevant to the semantic-consistency framing of QwenVL-Local. It should be cited in Section 2.1’s discussion of high-level and multimodal guidance.

Hu et al., Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders, 2025 — addresses structure preservation under semantic constraints in RealSR. It should be discussed in Section 2.1 as related work on improving fidelity-perceptual quality under explicit semantic guidance.

Questions for authors
  1. In the actual latent-space implementation of Equations 8 to 12, what quantitative LR-observation consistency is achieved, for example after decoding, compared with M3ESR + DiffusionNFT and the “w/o Range Constraint” variant? Evidence that the proposed projection reduces observation residual would increase my confidence in the anchoring interpretation.

2.Can the authors provide quantitative results for a matched ablation that removes QwenVL-Local, and for a version replacing its patchwise map with a global Qwen embedding similarity? If the patchwise version consistently improves semantic and perceptual metrics under otherwise fixed training, it would resolve a central attribution concern.

3.Does the “w/o reward scheduling” result in Table 2 use exactly the same per-reward normalization, spatial maps, and aggregate reward weights as the full method, with only time assignment changed? Please clarify this, since the answer determines whether the ablation isolates scheduling.

  1. Could the method be extended to use an estimated or learned observation operator for real degradations, rather than fixed ×4\times 4 average pooling? This would test whether the anchoring mechanism remains effective when the LR formation process differs materially from the synthetic training degradation.
Reviewer_GmcjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NA-DePO, an RL post-training framework for diffusion-based real-world super-resolution, built on DiffusionNFT and evaluated using M3ESR. It recomposes policy predictions using the ground-truth range component and the policy's approximate null component, while separately supervising the low-frequency component of the current prediction. QwenVL-Local produces spatial semantic rewards through attention masking and local embedding comparisons. Sigmoid reward reweighting and timestep-dependent scheduling are used to stabilize and organize reward guidance. Experiments on DIV2K and RealLQ report improvements across six no-reference quality metrics, while comparisons with the pretrained backbone show an improved fidelity–perception trade-off. Ablations examine the range constraint, decomposition scale, reward reweighting, and reward scheduling.

Critical strengths
  1. The Motivation stems from problems that actually arise in RL for SR models and is relevant to the super-resolution field. The methods used also have some theoretical grounding.

  2. In Table 1, the proposed method achieves the best numerical results among the compared methods on all six no-reference metrics for both DIV2K and RealLQ. The authors also retrain DP2O-SR and LucidNFT using the same M3ESR backbone and training data, controlling two important sources of variation. The consistency of the results and these baseline controls support the practical value of the complete method.

Critical weaknesses
  1. Table 2 includes vanilla DiffusionNFT, a variant without the range-space constraint, and the complete method. However, it lacks a control that retains the Range Constraint but omits null-space recomposition, leaving the contribution of the Range Constraint relative to vanilla DiffusionNFT unexplained.

  2. Figure 6 demonstrates the effectiveness of the proposed method compared with the baseline, but does not demonstrate its superiority over other methods (e.g., DP2O-SR).

Questions for authors
  1. Do the vanilla DiffusionNFT, “w/o Range Constraint,” and full-method configurations in Table 2 use identical reward models, spatial weighting, scheduling, and sigmoid mapping? Does “w/o Range Constraint” retain Eqs. 8–10? With these settings fixed, how does NA-DePO compare with standard NFT that retains Eq. 11 but removes recomposition? This comparison would determine the incremental value of the central anchoring mechanism over low-frequency supervision alone.

  2. Table 1 compares methods only using no-reference metrics. Could the authors report comparisons between NA-DePO and other methods (DP2O-SR and LucidNFT) using PSNR/SSIM/LPIPS/MANIQA/MUSIQ/CLIP-IQA (commonly used metrics) under a common evaluation protocol? This would help demonstrate the core advantages of NA-DePO.

Reviewer_UhZuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces NA-DePO, which leverages reinforcement learning to improve the trade-off between fidelity and perceptual quality in real-world image super-resolution. The method employs an approximate null-space decomposition to decouple structural fidelity constraints from detail-generation optimization, thereby mitigating structural deviations that can arise during reinforcement learning exploration. Furthermore, the authors design a VLM-based local reward and incorporate a multi-reward scheduling strategy to enhance both semantic consistency and textural realism. Evaluations on DIV2K and RealLQ show that several no-reference quality metrics outperform the original baseline.

Critical strengths

The null-space formulation provides a clear way to separate fidelity preservation from perceptual optimization.

Applying different rewards at different denoising stages is also well motivated, allowing semantic and texture guidance to act when they are most relevant.

Critical weaknesses

The paper emphasizes improving the balance between fidelity and perceptual quality; however, the main table reports only NR (no-reference) metrics, omitting reference-based metrics such as PSNR, SSIM, and LPIPS. While Figure 6 and the ablation study table provide some results, these are primarily comparisons against the backbone or internal variants of the proposed method, making it difficult to determine whether it truly preserves fidelity better than other existing approaches.

Furthermore, the quantitative evaluation is limited to synthetic DIV2K and unpaired RealLQ datasets, lacking tests on real-world paired datasets such as RealSR or DRealSR.

The discriminative power of QwenVL-Local has not been sufficiently verified. The fact that a local embedding is similar to the ground truth does not guarantee its ability to detect erroneous textures or structural anomalies; qualitative comparisons showing the effect of removing this reward are not enough. Drawing inspiration from RefReward-SR, the authors should incorporate tests for alignment with human preferences to demonstrate that the reward can reliably differentiate between better and worse SR outputs.

RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution https://arxiv.org/pdf/2603.24198

Questions for authors

The reward schedule is not sufficiently justified. Early denoising mainly establishes global structure, so why apply QwenVL-Local, which focuses on local regions, at this stage? Following the authors’ motivation, it seems more suitable for later stages when the image becomes clearer. Moreover, the authors employ a large number of reward functions here, without conducting a thorough investigation into how different reward functions affect performance.

Why is the group size set to 12? To what extent do different group sizes affect reconstruction quality and training stability? Could a smaller group size reduce training costs while maintaining comparable performance?

This paper only validates NA-DePO on M3ESR; it does not apply the method to other models, such as FLUX or SD2, to demonstrate its generalizability. The authors should provide more comprehensive experimental results by following DP2OSR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.