acceptodds
Under review as a conference paper at ICLR 2027

Heterogeneous-Negative Preference Optimization for Real-World Super-Resolution

Abstract

We present a new approach to explore Direct preference optimization (DPO) technique for real-world image supre-resolution (SR). First, we propose a preference-pair construction strategy in which the ground-truth high-quality image serves as the preferred sample, while predictions from a heterogeneous pool of existing SR models serve as negative samples. This strategy requires neither human annotations nor external reward models, while capturing diverse failure modes and providing more informative supervision for preference learning. Second, based on the constructed preference pairs, we derive a general DPO formulation applicable to arbitrary representation spaces. Further, we instantiate this formulation in the feature space of a pretrained DINOv2 model and introduce an adaptive layer-weighting mechanism to integrate representations across different semantic levels. Third, we decouple the shared temperature coefficient in DPO into two independent coefficients that separately control attraction toward preferred reconstructions and repulsion from negative predictions, thereby accounting for the asymmetric optimization difficulty of these two opposite directions. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness and generality of the proposed framework.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Super-Resolution, Direct Preference Optimization, Feature-Space Preference Learning, Heterogeneous Negative Samples

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_mP7hOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HePO-SR: it uses paired GT as positive samples, randomly samples negative samples from the outputs of a heterogeneous pool of pretrained SR models, and constructs a DPO preference objective in a multi-layer DINOv2 feature space, without requiring human preference labels or an explicit reward model.

Critical strengths
  1. The method is clear, easy to understand, and well motivated.
  2. The ablation studies are relatively comprehensive. The paper ablates negative pool expansion, adaptive layer weights, decoupled temperature coefficients, etc., supporting the main design choices.
Critical weaknesses
  1. Insufficient discussion of the drop in full-reference metrics. In the experiments, full-reference metrics such as PSNR decrease while no-reference perceptual metrics increase. The paper provides little discussion on this and does not deeply analyze the phenomenon of “perceptual improvement but fidelity degradation,” nor does it explain its relationship with DINOv2 feature preferences, heterogeneous negative samples, or the decoupled coefficients.
  2. No separate ablation on the positive-sample strategy. The paper emphasizes using GT as positive samples, but does not compare GT positive samples with other positive-sample construction methods (e.g., the best candidate selected by IQA ranking, or positive samples selected by an external reward model). Therefore, the contribution of the GT positive-sample design cannot be independently verified.
  3. Lack of direct experimental comparison with preference optimization SR methods such as DP²O / GDPO. Although the authors explain architectural incompatibility and insufficient reproducibility conditions, readers still cannot judge the actual advantage of HePO relative to existing preference optimization SR methods.
Questions for authors
  1. The paper provides insufficient discussion on the drop in full-reference metrics. Why do full-reference metrics decrease while no-reference metrics increase compared with before fine-tuning? Intuitively, using DINOv2 features for sample weighting computation should optimize the corresponding LPIPS?
  2. Can you provide a specific explanation of the heterogeneous pool ablation, such as analyzing the contribution of each SR model?
6 comments
AuthorsOpenReview ↗

Response to Q1&&Weakness1:Why do full-reference metrics decrease while no-reference metrics increase compared with before fine-tuning? Intuitively, using DINOv2 features for sample weighting computation should optimize the corresponding LPIPS?

We appreciate the reviewer raising this question. This is a good question. We agree that the paper should discuss the full-reference (FR) drop more explicitly, and we will add such a discussion.

discussion on the drop in full-reference metrics. This is the well-known perception-distortion trade-off. HePO-SR is a preference objective that targets perceptual quality, so a moderate decrease in distortion-type metrics is expected rather than anomalous. The distortion-side change in our results is small: on DRealSR with OMGSR, PSNR moves from 27.56 to 27.50 dB and SSIM improves (0.7465 to 0.7518), while the main changes appear in LPIPS (0.3266 to 0.3378) and DISTS (0.2335 to 0.2512). The repulsion term is what moves the output away from the over-smoothed negatives, and the gains on most no-reference metrics come from that same term (see our response to Reviewer ucq3, Q1). The change in FR metrics is the cost of that term, and the ratio of β1\beta_1 to β2\beta_2 decides where the model sits on this trade-off. In addition, a blinded user study on DRealSR is reported in our response to Reviewer LHEm, Q4: raters prefer our output on fidelity in 63.4% of images and on naturalness in 73.1%, so the FR drop is not perceived as a loss of structural fidelity. We will add this discussion of the perception-distortion trade-off to the revised paper.

DINOv2-based sample weighting and LPIPS. We separate two questions: what the weighting does, and whether it should reduce LPIPS.

(1) What the weighting does. The adaptive weights enter Eq. (12): for each sample they combine the five layer-wise preference logits, and each logit is −log⁡σ(β/(2s2)(A+−A−))-\log \sigma(\beta/(2s^2)(A^+ - A^-)), a comparison between the model's and the reference's distance to the same target. The weights therefore decide how much each layer contributes to the preference comparison for that sample; they do not reweight channels of a feature distance to the GT. Because the layers encode different levels of content, we let the weights depend on the sample instead of fixing them, and Table 3 measures the effect: with the same layers and base weights, adaptive weighting raises CLIP-IQA (0.7408 to 0.7556), MUSIQ (70.13 to 70.55), TOPIQ (0.6868 to 0.7124) and AFINE-NR (-1.0095 to -1.0353), with MANIQA slightly lower (0.6376 to 0.6370).

(2) Whether this should reduce LPIPS. We have no evidence for such a relation, so we do not claim it. LPIPS is a fixed weighted distance in a pretrained VGG feature space, whereas our weights act on preference comparisons in DINOv2 space. We have not established a correspondence between the two, nor any result showing that a weighting in one space controls a distance in the other. What we can report is that the FR metrics move with the β1/β2\beta_1/\beta_2 ratio: on DRealSR, moving from (β1,β2)=(5,3)(\beta_1, \beta_2) = (5,3) to (4,4)(4,4) and (3,5)(3,5) changes PSNR from 27.50 to 27.16 and 27.03 dB.

AuthorsOpenReview ↗

Response to Weakness2: No separate ablation on the positive-sample strategy.

We appreciate the reviewer raising this question. This is a good suggestion. We agree that the contribution of the GT-positive design should be verified independently, and we have run exactly the comparison the reviewer suggests. In the alternative strategy, the GT positive is replaced as follows: for each training input, we rank the four pool outputs with a weighted ensemble of full-reference and no-reference metrics (PSNR, SSIM, LPIPS, DISTS, CLIP-IQA, MUSIQ, MANIQA) and take the best-ranked candidate as the positive sample; the negative is then sampled at random from the remaining three candidates. Everything else, including the backbone (OMGSR), the training configuration, and the evaluation, is identical to our main setting.

Results on RealSet65 (bold: best in each column; underline: second best):

VariantCLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
OMGSR baseline0.740070.29‾\underline{70.29}0.6360‾\underline{0.6360}0.6911‾\underline{0.6911}−1.0230‾\underline{-1.0230}
IQA-ranked positive0.7410‾\underline{0.7410}69.910.62930.6844-1.0187
HePO-SR (GT positive)0.755670.540.63700.7124-1.0353

The IQA-ranked-positive variant removes nearly all of the gain: CLIP-IQA rises by only 0.0010, and the other four metrics are worse than the baseline (MUSIQ 69.91 vs 70.29, MANIQA 0.6293 vs 0.6360, TOPIQ 0.6844 vs 0.6911, AFINE-NR -1.0187 vs -1.0230). GT-positive training, in contrast, improves all five metrics.

Two further analyses support this conclusion.

(1) Relative to the negatives, the GT acts mainly as an anchor rather than as a source of perceptual gain. As shown in our response to Reviewer 3, Q1, the GT attraction term alone already raises CLIP-IQA from 0.7400 to 0.7550, close to the full objective (0.7556), but MUSIQ and MANIQA fall below the baseline and TOPIQ gains only 0.0011. Replacing a fixed anchor with a candidate that changes with each input makes the anchor unstable, which in turn affects training.

(2) We also scored the GT and the existing negatives with IQA metrics: in most cases the GT's overall score is higher, and it strictly outscores the pool negatives in 84.88% of the sampled pairs (our response to Reviewer ucq3, Q3).

This experiment therefore independently verifies the contribution of the GT-positive design. It also shows why the role of the positive and the negative should not be conflated: the positive needs a single fixed, unbiased anchor, whereas the negative benefits from preference diversity. We will add it to the appendix of the revised paper.

AuthorsOpenReview ↗

Response to Q2: Can you provide a specific explanation of the heterogeneous pool ablation, such as analyzing the contribution of each SR model?

We appreciate this request. In Table 3, the heterogeneous pool ablation (Variant 1 vs. Full) compares the two-model pool {SwinIR, StableSR} with the four-model pool, which adds Real-ESRGAN and DiffBIR; all other settings are the same. Adding the two sources improves CLIP-IQA (0.7513 → 0.7556), TOPIQ (0.7040 → 0.7124) and AFINE-NR (−1.0323 → −1.0353), while MUSIQ (70.59 → 70.54) and MANIQA (0.6407 → 0.6370) are lower. Because this comparison adds two sources at once, it changes both the size and the composition of the pool, and it cannot show what each source contributes.

To analyze each source and the effect of pool composition, we therefore explored this systematically: on both OSEDiff and OMGSR, we trained every negative pool that can be built from the four sources (4 single-source, 6 two-model and 4 three-model pools, plus the full pool), 30 runs in total. Each backbone uses its protocol in Appendix B, and only the negative pool changes. The table averages RealSet65 results over all pools of the same size (bold: best pool size per backbone and column; underline: second best; baselines are listed for reference):

BackbonePool size (pools averaged)CLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
OSEDiffbaseline0.687168.910.60130.5957-0.9725
OSEDiff1 (4)0.699469.570.62040.6153-0.9638
OSEDiff2 (6)0.702669.730.62100.6200‾\underline{0.6200}-0.9662
OSEDiff3 (4)0.7047‾\underline{0.7047}69.82‾\underline{69.82}0.62290.6196−0.9688‾\underline{-0.9688}
OSEDiff4 (1)0.708269.880.6226‾\underline{0.6226}0.6248-0.9710
OMGSRbaseline0.740070.290.63600.6911-1.0230
OMGSR1 (4)0.747570.310.63700.6977-1.0220
OMGSR2 (6)0.7516‾\underline{0.7516}70.50‾\underline{70.50}0.63680.7056‾\underline{0.7056}-1.0254
OMGSR3 (4)0.750170.400.6369‾\underline{0.6369}0.7044−1.0289‾\underline{-1.0289}
OMGSR4 (1)0.755670.540.63700.7124-1.0353

On average, more sources generally work better. On both backbones, every multi-model average is above the single-source average on at least four of the five metrics, and the four-model pool has the best average on four of the five. The exception is MANIQA, where the differences are small: on OSEDiff the three-model average is higher by 0.0003, and on OMGSR the four-model pool ties with the single-source average for the highest value (0.6370).

Per-source contribution, measured by removing each source from the full pool. StableSR contributes most on both backbones: removing it causes the largest drops on CLIP-IQA, MUSIQ, TOPIQ and AFINE-NR (OMGSR: −0.0187, −0.51, −0.0269, and 0.0272 worse; OSEDiff: −0.0116, −0.31, −0.0158, and 0.0086 worse), and on OMGSR it brings four metrics below the baseline. On OMGSR, the other sources play smaller roles: SwinIR mainly supports MUSIQ, MANIQA and AFINE-NR, while Real-ESRGAN and DiffBIR mainly support CLIP-IQA and TOPIQ. The role of a source can also differ between backbones: the contribution of SwinIR to CLIP-IQA and TOPIQ is slightly negative on OMGSR (−0.0029 and −0.0006) but positive on OSEDiff (+0.0050 and +0.0079), and the effect of Real-ESRGAN on CLIP-IQA is negligible on OSEDiff (0.0001, versus 0.0041 on OMGSR).

The averaged table also shows two further advantages of the pool.

Robustness. The four-model pool is the only pool size that ranks first or second on every metric on both backbones. On OMGSR, every multi-model average improves all five metrics over the baseline, whereas the single-source average is slightly below the baseline on AFINE-NR (-1.0220 versus -1.0230). On OSEDiff, AFINE-NR improves steadily as sources are added (from -0.9638 with one source to -0.9710 with four).

Training cost. All pool sizes are trained with the same protocol (same number of steps, same batch size, one negative per training sample), so the gain of the four-model pool over the single-source average comes at no extra training cost (OSEDiff: CLIP-IQA +0.0088, MUSIQ +0.31, MANIQA +0.0022, TOPIQ +0.0095, AFINE-NR better by 0.0072; OMGSR: CLIP-IQA +0.0081, MUSIQ +0.23, MANIQA equal, TOPIQ +0.0147, AFINE-NR better by 0.0133). The only extra cost is producing the cached negatives of the additional SR models, which is inference only and done once (Appendix B). In addition, finding the best combination of k sources with cached outputs would require training all ∑i=1k(ki)=2k−1\sum_{i=1}^{k}\binom{k}{i}=2^k-1 combinations: 15 runs per backbone for our four sources (the search reported above), with the count roughly doubling for every added source and the search repeated for every new backbone. This cost is hard to justify in practice, while the fixed pool needs one run per backbone.

We will add the averaged table and the remaining results to the appendix of the revised paper.

Reviewer_ucq3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HePO-SR, which pairs ground-truth images with negative outputs from several SR models. It combines a DINOv2 feature-space preference loss, adaptive layer weighting, and separate attraction and repulsion coefficients. Experiments on OSEDiff and OMGSR show improvements in several no-reference quality metrics.

Critical strengths

The preference-pair construction is simple and practical. It uses existing SR models without requiring additional human preference annotations or a separate reward model. The improvements across two backbones and multiple datasets provide useful evidence that the approach works beyond one particular model.

Critical weaknesses
  • My main concern is that the contribution of the negative samples is not clearly isolated. HePO also introduces DINOv2 supervision toward the ground truth, which could account for part of the improvement. The reconstruction-only comparison in Appendix C.1 removes the entire HePO objective, so it does not separate these effects. Since heterogeneous negatives are a central contribution, I would like stronger evidence showing what they add beyond directly matching the ground-truth DINOv2 features.

  • Table 3 shows mixed results when expanding the pool from two models to four. How much does the choice of models matter? Would one well-chosen model work just as well?

  • What if an SR result keeps the content correct but looks better than the GT, for example by removing noise? Would treating it as a negative hurt training?

  • RealDPO already use demonstration or real data as positives and model-generated samples as negatives. How does your pair construction differ beyond using a heterogeneous pool?

Questions for authors

see weaknesses

7 comments
AuthorsOpenReview ↗

Response to Q1(Weakness1): How do you isolate the contribution of heterogeneous negatives from the DINOv2 ground-truth supervision?

We appreciate the reviewer raising this question. We agree that the reconstruction-only control removes the entire objective, so it does not isolate the negative term. The clean control for "what do the negatives add beyond directly matching the GT DINOv2 features" is a positive-only variant: the same reference-relative preference loss, with the same multi-level DINOv2 features, the same adaptive aggregation, the same reference model and the same compute, and only the negative term removed. Results on RealSet65 (OMGSR, protocol of Table 2; bold: best in each column, underline: second best):

VariantCLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
OMGSR baseline0.740070.29‾\underline{70.29}0.6360‾\underline{0.6360}0.6911-1.0230
Positive-only (no negative term)0.7550‾\underline{0.7550}70.010.61950.6922‾\underline{0.6922}−1.0277‾\underline{-1.0277}
HePO-SR (full)0.755670.540.63700.7124-1.0353

The full objective is better than positive-only matching of GT features on all five metrics, with the largest margins on TOPIQ (+0.020) and MUSIQ (+0.53). This comparison also shows what each term contributes. GT attraction acts mainly as an anchor: on its own it already raises CLIP-IQA from 0.7400 to 0.7550, close to the full objective (0.7556), while MUSIQ and MANIQA stay below the baseline and TOPIQ gains only 0.0011. The perceptual gains in these three metrics come from the negative term, which raises MUSIQ to 70.54, MANIQA to 0.6370 and TOPIQ to 0.7124.

One further observation supports this reading. For OMGSR on DRealSR, moving weight from attraction to repulsion degrades most FR metrics monotonically (bold: best in each column; underline: second best):

Variant(β1,β2)(\beta_1,\beta_2)PSNR ↑SSIM ↑LPIPS ↓DISTS ↓CLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
OMGSR baseline–27.560.7465‾\underline{0.7465}0.32660.23350.695766.670.63650.6558‾\underline{0.6558}−0.9808‾\underline{-0.9808}
Full (Ours)(5,3)(5,3)27.50‾\underline{27.50}0.75180.3378‾\underline{0.3378}0.25120.720567.35400.63750.6842-1.0058
Variant 3(4,4)(4,4)27.160.72280.33790.2346‾\underline{0.2346}0.6972‾\underline{0.6972}66.91040.6467‾\underline{0.6467}0.6519-0.9712
Variant 4(3,5)(3,5)27.030.72040.34190.23700.694666.9335‾\underline{66.9335}0.64820.6524-0.9753

PSNR falls from 27.50 to 27.03 dB, SSIM from 0.7518 to 0.7204, and LPIPS rises from 0.3378 to 0.3419. Four of the five no-reference metrics are also worse at (4,4)(4,4) and (3,5)(3,5) than at (5,3)(5,3). Reducing the attraction weight therefore moves the output away from the GT, which is consistent with attraction acting as the anchor rather than as the source of the perceptual gain. We will add the positive-only control to the appendix.

AuthorsOpenReview ↗

Response to Q3(Weakness3): What if an SR result keeps the content correct but looks better than the GT, for example by removing noise? Would treating it as a negative hurt training?

We appreciate the reviewer raising this question.This is a great insight, and it is worth examining:

1. Audit of the expected preference direction. Since the training loss is an expectation over sampled pairs, what matters is whether the expected ordering is correct on the training distribution. We scored 10,000 triplets (LQ input, GT, and an offline negative from the four-model pool) with four NR-IQA metrics (CLIP-IQA, MUSIQ, MANIQA, TOPIQ-NR). Scores were min-max normalized per image over the GT and all pool candidates, then equally weighted. The GT strictly wins 84.88% of pairs and the negatives win 15.12%, with ties at ~0; the mean weighted score is 0.854 for the GT versus 0.454 for the expected negative. The expected preference therefore points in the correct direction on the large majority of pairs.

2. Further discussion. However, the reviewer is right that the GT is not guaranteed to be the perceptually better reconstruction under every quality measure. HePO-SR takes the paired GT as the preferred sample, and a negative can outscore the GT under no-reference IQA metrics; such a pair is then mislabeled, and the negative is wrongly repelled. The GT nevertheless provides a fixed, unbiased anchor: in β1A+−β2A−\beta_1 A^+ - \beta_2 A^- (Eq. 13), the gradient pulls the output toward the GT with weight β1\beta_1 and pushes it away from the negative with weight β2\beta_2. Since β1>β2\beta_1 > \beta_2, the attraction is β1/β2\beta_1/\beta_2 times the repulsion for displacements of the same magnitude, so the repulsion from a mislabeled negative is partly counteracted. This point is not discussed in the present Limitations section, and we will add it.

AuthorsOpenReview ↗

Response to Q4(Weakness4): RealDPO already use demonstration or real data as positives and model-generated samples as negatives. How does your pair construction differ beyond using a heterogeneous pool?

We appreciate the reviewer pointing to RealDPO. The two constructions differ in where the negatives come from. RealDPO obtains its negatives from a single generator: it draws three random init noises, conditions them on the positive sample's caption, and samples that generator to obtain three negatives. Its diversity therefore comes from different noise initializations of one model. HePO-SR instead uses, for the same LQ input, the reconstructions of different pretrained SR models (SwinIR, Real-ESRGAN, StableSR, DiffBIR), produced once and cached offline (Appendix C.4). The diversity is across models: two negatives differ by which model produced them, not by which noise seed was drawn.

What the two share. From a higher level, both methods use the same labeling principle: real data as the preferred sample and model-generated outputs as the dispreferred one.

Constructing preference pairs alone, however, is not enough to apply DPO to an SR backbone. Our contribution therefore lies in three further method-level changes, not only in how the pairs are built.

(1) DPO in a general representation space. SR maps are deterministic and define no output likelihood, so we introduce auxiliary conditional distributions qθ(u∣x)=N(u;ψϕ(y^),s2I)q_\theta(u|x) = \mathcal{N}(u; \psi_\phi(\hat y), s^2 I) and qref(u∣x)=N(u;ψϕ(yr),s2I)q_{\mathrm{ref}}(u|x) = \mathcal{N}(u; \psi_\phi(y_r), s^2 I), with ψϕ(y)=vec(ϕ(y))/N\psi_\phi(y) = \mathrm{vec}(\phi(y))/\sqrt N (Eqs. 2-4). The preference log-ratio then has a closed form and equals the DPO reward (Eq. 5), which is what makes preference optimization possible without an explicit output likelihood.

(2) Layer-wise preference losses with adaptive aggregation. We instantiate ϕ\phi as DINOv2 layers {0,3,7,9,11}\{0,3,7,9,11\} and combine the five layer-wise preference logits with sample-dependent weights (Eqs. 9-12). With the same layers and base weights, adaptive aggregation raises CLIP-IQA from 0.7408 to 0.7556 (Table 3).

(3) Decoupled coefficients. We split the shared β\beta into β1\beta_1 for GT attraction and β2\beta_2 for negative repulsion (Eq. 13), which addresses the attraction-repulsion imbalance we observed in SR (Fig. 3); moving weight between the two significantly affects the results (Table 3, Fig. 6).

These three changes act on the objective, not on where the pairs come from. We will add the RealDPO citation and this comparison to the related work in the revision.

Reviewer_ect1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript proposes HePO-SR, a preference optimization method for real-world image super-resolution. It uses ground-truth images as positive samples and outputs from different SR models as heterogeneous negatives. The preference objective is formulated in a frozen DINOv2 feature space relative to a reference model, with adaptive layer weighting and separate coefficients for the positive and negative terms. Experiments on OSEDiff and OMGSR show improvements across several no-reference image quality metrics on real and synthetic datasets.

Critical strengths
  1. Practical, annotation-free supervision. Outputs from different SR models provide diverse negative examples, and the objective can supplement existing restoration training without human preference labels.

  2. Evaluation across backbones and datasets. Results on two SR backbones and multiple datasets, together with component ablations, support the method’s practical effectiveness.

Critical weaknesses
  1. The distinction from prior restoration and SR methods is unclear. C²PNet [1] uses other restoration models’ outputs as contrastive negatives, while HGGT [2] uses human-selected positive and negative model-generated examples for real-world SR. Despite differences in supervision and objectives, these relevant precedents should be discussed to clarify HePO-SR’s additional contribution.

  2. The ablations do not isolate the central contributions. There is no comparison against a standard contrastive loss using the same DINO features and negative pool. Comparing two-model and four-model pools without a single-source control also does not establish whether heterogeneity itself, rather than particular negative sources, drives the gains.

  3. The perceptual gains are not visually convincing. Improvements over the baselines in Figure 4 appear subtle. Without a user study or more extensive qualitative comparisons, it remains unclear whether the no-reference metric gains translate into consistent, perceptible improvements.

[1] Curricular Contrastive Regularization for Physics-Aware Single Image Dehazing. CVPR, 2023.

[2] Human Guided Ground-Truth Generation for Realistic Image Super-Resolution. CVPR, 2023.

Questions for authors
  1. How does HePO-SR differ from C²PNet and HGGT in using model-generated restoration results, and what additional contribution does its preference objective provide?

  2. With features, negatives, backbone, and training budget fixed, does the proposed objective outperform a standard contrastive loss or positive-only feature matching?

  3. Does a heterogeneous negative pool outperform a single-source pool under the same sampling budget?

  4. Can additional visual comparisons or user preference results substantiate the perceptual gains beyond Figure 4?

7 comments
AuthorsOpenReview ↗

Response to Q1:How does HePO-SR differ from C²PNet and HGGT in using model-generated restoration results, and what additional contribution does its preference objective provide?

We appreciate the reviewer raising this question. We discuss the two works in turn.

C2PNet. We agree that its negative construction is similar to ours, but the key is how the objective uses these negatives.

C2PNet trains from scratch with a pixel-L1 fidelity term and adds a contrastive regularizer, R∗=d(f(y^),f(GT))/(∑iwid(f(y^),negi)+zE)R^* = d(f(\hat y), f(\mathrm{GT})) / \big(\sum_i w_i d(f(\hat y), \mathrm{neg}_i) + zE\big), computed on fixed VGG-19 features, with wiw_i set by a PSNR curriculum. The negatives enter only through the denominator, where they reweight a distance to the GT; there is no preference pair and no reference model. The loss is therefore a contrastive penalty inside a distortion-oriented recipe, and its gains are reported on full-reference metrics (PSNR/SSIM). In HePO-SR, the same kind of sample fills the dispreferred slot of a preference pair that is scored against a frozen copy of the base model. The logit is β/(2s2)(A+−A−)\beta/(2s^2)(A^+ - A^-), with A+=d(y+,yr)−d(y+,y^)A^+ = d(y^+, y_r) - d(y^+, \hat y) and A−=d(y−,yr)−d(y−,y^)A^- = d(y^-, y_r) - d(y^-, \hat y), so the unit of supervision is the pair and the reference appears in both terms. The outcomes differ in the same direction: C2PNet improves full-reference metrics, while ours mainly improves no-reference perceptual metrics, and full-reference metrics are not guaranteed to improve (Table 1).

Our contribution is not the negative construction alone. Using DPO on a general SR model required three method-level changes.

(a) A tractable likelihood. SR maps are deterministic, so we define Gaussian policies qθ(u∣x)=N(u;ψϕ(y^),s2I)q_\theta(u|x) = \mathcal{N}(u; \psi_\phi(\hat y), s^2 I) and qref(u∣x)=N(u;ψϕ(yr),s2I)q_{\mathrm{ref}}(u|x) = \mathcal{N}(u; \psi_\phi(y_r), s^2 I) in an arbitrary representation space, with ψϕ(y)=vec(ϕ(y))/N\psi_\phi(y) = \mathrm{vec}(\phi(y))/\sqrt N (Eqs. 2-4). The preference log-ratio then has a closed form and equals the DPO reward (Eq. 5). This is what allows the objective to act on a deterministic SR model, and on features rather than pixels.

(b) Multi-level features with adaptive weights. We instantiate ϕ\phi as DINOv2 layers {0,3,7,9,11}\{0,3,7,9,11\} and aggregate the five layer-wise preference logits with learned weights (Eqs. 9-12), a degree of freedom that pixel-space preference learning does not have.

(c) Decoupled coefficients. We split DPO's β\beta into β1\beta_1 for attraction and β2\beta_2 for repulsion (Eq. 13). In SR the two terms behave asymmetrically, and the ablation shows that moving weight between them significantly affects the results (Table 3, Fig. 6).

The matched controls and ablations in Q2 show that the gain comes largely from these method choices rather than from the negatives alone: with the same negatives, features, and budget, a standard triplet loss only reproduces the baseline, and removing the reference drops four of the five metrics below it.

In the revision we will cite C2PNet in the related work.

HGGT. Both its negative construction and its objective differ from ours. Its negatives are not model restorations but human-labeled artifact regions (63,583 positive and 2,514 negative LR-GT pairs). The corresponding term is a pixel-space penalty on the labeled regions, weighted by a residual-variance map and applied inside the standard L1, perceptual, and GAN recipe, starting from a plain L1 initialization. We therefore treat HGGT as a dataset-level line of work rather than a different use of the same negatives, and we will cite it as such.

AuthorsOpenReview ↗

Response to Q2: With features, negatives, backbone, and training budget fixed, does the proposed objective outperform a standard contrastive loss or positive-only feature matching?

We appreciate the reviewer raising this question. With the training conditions held fixed, we compare three objectives: (a) our full preference objective, (b) a standard triplet contrastive loss, and (c) positive-only feature matching, that is, the GT-attraction term alone without negatives. All variants are evaluated on RealSet65 with the five no-reference metrics.

Results (testset: RealSet65, baseline: OMGSR; bold: best in each column, underline: second best):

VariantCLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
Baseline0.740070.29‾\underline{70.29}0.63600.6911-1.0230
Positive-only feature matching0.7550‾\underline{0.7550}70.010.61950.6922‾\underline{0.6922}−1.0277‾\underline{-1.0277}
Standard triplet contrastive0.739670.050.64040.6865-1.0063
HePO-SR (full objective)0.755670.540.6370‾\underline{0.6370}0.7124-1.0353

Two conclusions follow.

First, the standard contrastive loss with the same features, negatives, and budget essentially reproduces the baseline: it is worse than the baseline on TOPIQ (0.6865 vs 0.6911) and AFINE-NR (-1.0063 vs -1.0230). The same features and negatives alone are therefore not sufficient; the DPO preference formulation is what turns them into consistent gains.

Second, positive-only feature matching shows that attraction alone cannot substitute for the preference objective. It raises CLIP-IQA (0.7550), but MUSIQ and MANIQA fall below the baseline (70.01 vs 70.29 and 0.6195 vs 0.6360), and its gains on TOPIQ (0.6922 vs 0.6911) and AFINE-NR (-1.0277 vs -1.0230) are small. The full objective improves all five metrics, with TOPIQ gaining 0.0213 over the baseline. Negatives are necessary, but how the loss uses them also matters: the DPO log-ratio with decoupled coefficients and adaptive layer weighting outperforms both ablations under a strictly controlled budget.

AuthorsOpenReview ↗

Response to Q4: Can additional visual comparisons or user preference results substantiate the perceptual gains beyond Figure 4?

We appreciate the reviewer raising this question. This is a reasonable request. We agree that no-reference metric gains alone are not fully convincing, and we address this on two fronts:

Qualitative comparison. We have prepared an extended comparison beyond Figure 4, covering diverse scene categories for both backbones on DRealSR, RealSR, and RealSet65. It will be included in the revised paper.

User study. In response to a related question from Reviewer LHEm (Q4), we completed a blinded study on DRealSR. Each of the 93 test images was shown as one item containing the low-quality input, the ground truth, and two restorations: ours and the model before fine-tuning. The left-right order was randomized and the method names were hidden. Each image was rated by 30 annotators, giving 2,790 comparisons, on two independent questions: naturalness (richer texture and detail, and fewer structures or textures absent from the input) and fidelity (with the ground truth shown, closer to its structure and content). At the image level, ours wins on 59 of the 93 images (63.4%) for fidelity and on 68 (73.1%) for naturalness, with 12.9% and 14.0% ties. Excluding ties, the win rates are 76.3% and 81.6%.

Reviewer_LHEmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HePO-SR: the ground-truth image is the positive sample, and outputs of heterogeneous pretrained SR models are negatives. A preference objective is written with auxiliary Gaussians in DINOv2 feature space, with adaptive multi-layer weights, and the temperature split into β1 (toward GT) and β2 (away from the negative). OSEDiff and OMGSR are fine-tuned; no-reference metrics rise on paired and unpaired benchmarks.

Critical strengths

• Negatives are easy to get: failure modes of existing models, with no human preferences or a separate reward model. • Two backbones and several benchmarks; reconstruction-only, feature-space, β allocation, and offline/online cost are all compared. • Figures 3 and 6 make the training dynamic—“moving away from the negative without approaching GT”—clear; the motivation holds.

Critical weaknesses

• This is not DPO. It is a feature contrastive loss with a reference offset. SR models are deterministic; the auxiliary Gaussian is not a likelihood of the output policy. After expansion, the terms in ŷ are −β1·d(y+,ŷ)+β2·d(y−,ŷ). β1/β2 are attraction/repulsion weights and do not yield a policy ratio or a KL term. The title is written as DPO, which overstates the theoretical contribution. • The ablations do not reach the actual mechanism. “Reconstruction-only vs full HePO” changes the positive term, the negative term, the reference offset, and adaptive weighting at once. Two-model vs four-model pools change size and difficulty together. VGG/pixel controls only show that DINOv2 is better. OMGSR already has a DINOv3 perceptual loss and OSEDiff already has LPIPS. Missing: positive-only, standard triplet, and the reference-free equivalent. • The perceptual gains mainly come from learned no-reference metrics, fidelity is falling, and there is no human study. CLIP-IQA and related scores may share bias with DINO/CLIP; training in DINOv2 and evaluating with similar metrics risks circularity. PSNR drops on paired sets for both backbones; OSEDiff SSIM on DRealSR goes 0.7835→0.7440. Without a blinded study it is unclear whether outputs are more natural or more hallucinated. The main ablation is also run on the test set RealSet65, and results appear to be a single seed.

Questions for authors

• What policy model would make this objective DPO? If none, would you restate it as heterogeneous-negative contrastive learning? • Under matched compute, what do DINOv2 positive-only, triplet, and the reference-free form each give? • On which validation set were the layers, prior weights, and (5,3) chosen? Was RealSet65 inspected during selection? • When raters are asked about both naturalness and fidelity, do they prefer HePO? • Do the main results hold under a new seed and a new model pool?

6 comments
AuthorsOpenReview ↗

Response to Q1: What policy model would make this objective DPO?If none, would you restate it as heterogeneous-negative contrastive learning?

We appreciate the reviewer raising this question, and we are sorry for the confusion caused by our unclear presentation. The policy model is in fact defined in Eq. (4); we did not explain it clearly enough, and we will spell it out in Sec. 2.1.

1.Policy and reference. Eq. (4) defines qθϕ(u∣x)=N(u;ψϕ(y^),s2I)q^\phi_{\theta}(u|x) = \mathcal{N}(u; \psi_\phi(\hat y), s^2 I) and qrefϕ(u∣x)=N(u;ψϕ(yr),s2I)q^\phi_{\mathrm{ref}}(u|x) = \mathcal{N}(u; \psi_\phi(y_r), s^2 I), where xx is the low-quality input, fθf_\theta is the SR model being fine-tuned, freff_{\mathrm{ref}} is a frozen copy of the same model before fine-tuning, y^=fθ(x)\hat y = f_\theta(x), yr=fref(x)y_r = f_{\mathrm{ref}}(x), and ψϕ\psi_\phi is the normalized feature map of Eq. (2). The two means are therefore the representations of the trainable reconstruction and of the reference reconstruction. The Gaussian form is a modeling choice: it gives a deterministic SR map a density, which is what a likelihood ratio needs. Both Gaussians share the same variance s2s^2, and s2s^2 does not depend on the model parameters. It therefore cancels in the log-ratio and is absorbed into β\beta (Eq. 13), so no new hyperparameter is introduced. In what follows we write qθ(y∣x)q_\theta(y|x) for qθϕ(ψϕ(y)∣x)q^\phi_{\theta}(\psi_\phi(y)|x), as Eq. (5) does.

2. Pixel-space case. Let ϕ\phi be the identity, so that ψϕ(y)=y/N\psi_\phi(y) = y/\sqrt N. The 1/N1/\sqrt N in Eq. (2) makes dϕ(a,b)=∥a−b∥2/Nd_\phi(a,b) = \Vert a-b\Vert^2/N the mean squared error per dimension, so neither the distances nor the log-ratio grow with the feature dimension. Eq. (4) then reads

qθ(u∣x)=N(u;y^/N,s2I),qref(u∣x)=N(u;yr/N,s2I),q_\theta(u|x) = \mathcal{N}(u; \hat y/\sqrt N, s^2 I), \qquad q_{\mathrm{ref}}(u|x) = \mathcal{N}(u; y_r/\sqrt N, s^2 I),

which are two isotropic Gaussians with the same covariance s2Is^2 I. Evaluating them at u=y/Nu = y/\sqrt N gives the log-densities

log⁡qθ(y∣x)=−∥y−y^∥22Ns2+C,\log q_\theta(y|x) = -\frac{\Vert y-\hat y\Vert^2}{2Ns^2} + C,

log⁡qref(y∣x)=−∥y−yr∥22Ns2+C,\log q_{\mathrm{ref}}(y|x) = -\frac{\Vert y-y_r\Vert^2}{2Ns^2} + C,

where the constant CC is the same in both. The log-ratio of Eq. (5) is therefore

Δ(y;x)=∥y−yr∥2−∥y−y^∥22Ns2.\Delta(y;x) = \frac{\Vert y-y_r\Vert^2 - \Vert y-\hat y\Vert^2}{2Ns^2}.

This is exactly DPO's log⁡πθ(y∣x)−log⁡πref(y∣x)\log \pi_\theta(y|x) - \log \pi_{\mathrm{ref}}(y|x), with the token log-likelihood replaced by the Gaussian log-density at yy. The remaining terms also correspond one to one: (yw,yl)(y_w, y_l) is (y+,y−)(y^+, y^-), and Eq. (6) becomes the DPO objective

−Elog⁡σ(β(Δ(y+;x)−Δ(y−;x))),-\mathbb{E}\log \sigma\big(\beta(\Delta(y^+;x) - \Delta(y^-;x))\big),

with implicit reward r(x,y)=βΔ(y;x)r(x,y) = \beta\Delta(y;x). For a general ϕ\phi the same expressions hold with the distances taken in feature space, which is the form Eq. (5) states.

3. DPO derivation. Because both densities are Gaussian and share a covariance that does not depend on the parameters, their ratio has a closed form, and Eq. (5) is the DPO log-ratio itself rather than an analogy. DPO's derivation (Rafailov et al., Eqs. 4-7) only requires a policy and a reference whose log-ratio can be computed; it does not require discrete outputs or a language model. Substituting Δ\Delta into Eq. (6) therefore gives the standard Bradley-Terry DPO loss, whose KL-regularized optimum is π∗∝πrefexp⁡(r/β)\pi^* \propto \pi_{\mathrm{ref}} \exp(r/\beta). Diffusion-DPO performs the same substitution with an ELBO surrogate.

4. Why the loss is not merely contrastive. Expanding the loss does give it an attraction/repulsion form, but DPO takes the same form once its implicit reward is substituted, so the algebraic shape of a loss does not tell us where it comes from. What distinguishes the two is that the reference appears in the reward, which makes the optimum a reweighted reference policy rather than a point estimate. Empirically, a standard triplet loss with the same features, the same negatives, and the same budget does not reproduce the gains (Q2).

AuthorsOpenReview ↗

Response to Q2: Under matched compute, what do DINOv2 positive-only, triplet, and the reference-free form each give?

We appreciate the reviewer raising this question. We ran the three controls on OMGSR under the same protocol as Table 2, changing only the preference term. Results on RealSet65 (bold: best in each column; underline: second best):

VariantCLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
OMGSR baseline0.740070.29‾\underline{70.29}0.63600.6911-1.0230
Positive-only (negative term removed)0.7550‾\underline{0.7550}70.010.61950.6922‾\underline{0.6922}−1.0277‾\underline{-1.0277}
Reference-free (reference removed)0.724469.710.63550.6730-0.9951
Triplet (standard margin loss)0.739670.050.64040.6865-1.0063
HePO-SR (full)0.755670.540.6370‾\underline{0.6370}0.7124-1.0353

Three findings.

(1) Without the reference, four of the five metrics fall below the baseline, and MANIQA equals it. In this case the loss can decrease simply by moving the output away from the samples, and nothing keeps it close to a reasonable solution. The reference is therefore necessary, as the DPO argument in Q1 predicts.

(2) A standard triplet loss is at or below the baseline on four of the five metrics, with MANIQA slightly higher. Being close to the GT and far from the negatives is therefore not enough: the gains come from the reference-relative form, not from contrast in DINOv2 space.

(3) Positive-only training helps on three metrics, but on two of them the improvement is small (TOPIQ 0.6922 vs 0.6911, AFINE-NR -1.0277 vs -1.0230); the only clear gain is CLIP-IQA (0.7550 vs 0.7400). The full objective is better than positive-only on all five metrics, for example TOPIQ 0.7124 vs 0.6922 and MUSIQ 70.54 vs 70.01. It also restores MANIQA, which positive-only lowers from 0.6360 at the baseline to 0.6195. The negative term therefore provides information that positive-only training does not.

All variants use the same compute as the full method. We will add this table to the appendix.

AuthorsOpenReview ↗

Response to Q3:On which validation set were the layers, prior weights, and (5,3) chosen? Was RealSet65 inspected during selection?

We appreciate the reviewer raising this question. Yes: the layer set, the prior weights, and (β1,β2)(\beta_1, \beta_2) were chosen on RealSet65 using its no-reference metrics together with visual inspection. We acknowledge our oversight, we share the reviewer's concern about this evaluation protocol, and we will state it clearly in the revision. Three points reduce this concern.

(1) The full ablation replicates on DRealSR, a benchmark that played no role in selection. All Table 3 variants were trained under one fixed protocol and evaluated with both FR and NR metrics (bold: best in each column; underline: second best):

VariantExpanded PoolAdaptive Weights(β1,β2)(\beta_1,\beta_2)PSNR ↑SSIM ↑LPIPS ↓DISTS ↓CLIP-IQA ↑MUSIQ ↑MANIQA ↑TOPIQ ↑AFINE-NR ↓
OMGSR baseline–––27.560.7465‾\underline{0.7465}0.32660.2335‾\underline{0.2335}0.695766.670.63650.6558-0.9808
Variant 1×✓(5,3)(5,3)27.50‾\underline{27.50}0.73780.33120.23520.7031‾\underline{0.7031}66.890.64300.6586‾\underline{0.6586}−0.9822‾\underline{-0.9822}
Variant 2✓×(5,3)(5,3)27.470.73570.3309‾\underline{0.3309}0.23230.702266.710.64100.6515-0.9690
Variant 3✓✓(4,4)(4,4)27.160.72280.33790.23460.697266.910.6467‾\underline{0.6467}0.6519-0.9712
Variant 4✓✓(3,5)(3,5)27.030.72040.34190.23700.694666.93‾\underline{66.93}0.64820.6524-0.9753
Full (Ours)✓✓(5,3)(5,3)27.50‾\underline{27.50}0.75180.33780.25120.720567.350.63750.6842-1.0058

Three observations can be made from this table:

(a) Full (Ours) is best on five of the nine metrics and ties Variant 1 for the best PSNR among the trained variants. It is also the only variant whose SSIM is above the baseline (0.7518 vs 0.7465).

(b) The three design conclusions of Table 3 hold again: (5,3)(5,3) is best on the same NR metrics; adaptive weighting improves the NR metrics, and by more than on RealSet65 (TOPIQ 0.6842 vs 0.6515); and the larger pool improves CLIP-IQA, TOPIQ and AFINE-NR. MANIQA is again the exception: on both datasets it is higher when repulsion is stronger.

(c) As the emphasis moves from attraction to repulsion, the FR metrics decrease: from 27.50 dB / 0.7518 for Full (Ours) to 27.16 / 0.7228 for Variant 3 and 27.03 / 0.7204 for Variant 4, as shown in Fig. 6. On DRealSR, Full (Ours) is also below some ablated variants on LPIPS and DISTS, and below the baseline on both. This is the same dataset-specific FR pattern as in Table 1, and Q4 examines it from the human-perception side.

(2) The configuration chosen on RealSet65 was then evaluated on DRealSR, RealSR and DIV2K, none of which was inspected during selection. In Table 1, CLIP-IQA, MUSIQ and TOPIQ improve on all three benchmarks for both backbones.

(3) The allocation (5,3)(5,3) was additionally chosen from the training dynamics of the GT-attraction component A+A^+. Among the three allocations with β1+β2=8\beta_1 + \beta_2 = 8, only (5,3)(5,3) lets A+A^+ recover and turn positive during training (Fig. 6); under (4,4)(4,4) and (3,5)(3,5) it stays negative.

We will add this table to the appendix and state in the revision that RealSet65 was used for selection.

Comments on the paper

3 comments
Area_Chair_2BW7OpenReview ↗

Reminder for Discussion

Dear Reviewers,

We’re about halfway through the discussion period. This is a friendly reminder to check all of your assigned papers. If the authors have answered questions or responded to critiques in reviews, please leave a comment for the authors!

Thank you for your contribution to the community!

Your AC

Dear Reviewers,

We sincerely appreciate the time and effort you have devoted to reviewing our paper. Your comments have helped us improve both the presentation of the method and the experimental analysis.

During the rebuttal period, we conducted a series of additional experiments to answer the reviewers' questions. These include a full ablation on DRealSR, comparisons with alternative preference objectives and positive-sample constructions, 30 training runs covering every negative-pool combination on both backbones, and a blinded user study on DRealSR with 30 annotators per image. We hope these results address your concerns, and we look forward to your feedback.

As promised in our responses, we have revised the paper accordingly. In the revised PDF, added text is shown in blue with a wavy underline, and removed text is shown in red. The main changes are as follows.

  1. Policy definition (Sec. 2.1; Reviewer 1). We state more explicitly how the Gaussian distributions in Eq. (4) correspond to the policy and the reference policy in DPO, and how the trainable SR model and its frozen copy enter them. We also explain that the shared variance cancels in the log-ratio and introduces no additional hyperparameter, give the identity-mapping case of Eq. (5) as an example, and identify the implicit reward in Eq. (6).

  2. Relation to DP²O-SR and GDPO-SR (Sec. 2.5 and Appendix A.2; Reviewer 4). We describe more clearly how HePO-SR relates to these methods: they study how preference pairs or groups are constructed, while HePO-SR studies how the DPO objective is adapted to a deterministic SR backbone, so the two lines of work are complementary. We also explain in more detail why a controlled direct comparison is not feasible, and make the difference in the source of supervision explicit in the related work.

  3. Full-reference metrics (Sec. 3.2; Reviewer 4). We add a discussion of the full-reference results in terms of the perception–distortion trade-off, with reference to the DRealSR ablation and the user study.

  4. Selection protocol (Sec. 3.3; Reviewer 1). We state that the DINOv2 layers, the prior layer weights, and (β1,β2)(\beta_1, \beta_2) were selected on RealSet65, and refer to the full ablation on DRealSR, which was not used for selection (Appendix C.5).

  5. Qualitative comparison (Fig. 4 and Sec. 3.2; Reviewer 2). We updated Fig. 4 and describe the differences between the compared methods in more detail.

  6. Related work (Appendix A.2). We add RealDPO (Reviewer 3), as well as C2PNet and HGGT (Reviewer 2), and describe how their constructions and objectives differ from ours.

  7. Additional experiments (Appendix C).

    • C.5: Ablation on DRealSR with both full-reference and no-reference metrics (Reviewer 1).
    • C.6: Comparison with positive-only, reference-free, and triplet objectives (Reviewers 1 and 3).
    • C.7: Comparison of the GT positive with an IQA-ranked positive (Reviewer 4).
    • C.8: Results for every negative pool built from the four sources on OSEDiff and OMGSR, together with the averages over pools of the same size (Reviewers 2, 3, and 4). Sec. 3.3 now refers to this appendix.
    • C.9: Blinded user study on DRealSR (Reviewers 1 and 2).
  8. Limitations (Appendix D; Reviewer 3). We discuss the case in which the GT is not the perceptually better image, report an audit of 10,000 training triplets, and explain how the coefficient allocation partly counteracts such pairs.

  9. Pipeline figure (Fig. 2). We updated Fig. 2 so that the sign of the negative component A−A^{-} shown in the figure is consistent with its definition in Eq. (8).

Best regards, The Authors

Area_Chair_2BW7OpenReview ↗

Author-Reviewer Discussion Period is About to Finish

Dear reviewers,

Thank you for your reviews and constructive feedback.

The author-reviewer discussion period will be finishing soon. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:

  • Whether the rebuttal addresses your main concerns;

  • Whether any significant issues remain unresolved;

  • Whether your overall recommendation has changed.

If appropriate, please also update your review and recommendation in the system.

Thank you for your time and valuable contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.