acceptodds
Under review as a conference paper at ICLR 2027

Localizing Failures: Hallucination-Grounded Preference Learning for Faithful Generative Super-Resolution

10 views

Abstract

Generative Super-Resolution (SR) can improve perceptual quality but often introduces SR hallucinations—semantically unsupported objects, textures, structures, or text that are inconsistent with the paired Ground Truth (GT). Existing preference optimization methods rely on either image-level signals without spatial information or predefined spatial units that do not directly localize actual SR hallucinations. To address this limitation, we propose a framework comprising Hallucination-Grounded-Marks (HGM) and Hallucination-Selective Preference Optimization (HSPO). HGM uses a VLM to localize candidate regions on a grid and verifies them against the GT to produce a cell-level binary hallucination mask, without requiring precomputed object regions or boxes. Since existing perceptual metrics struggle to quantify hallucination severity, we further introduce the Area-weighted Hallucination Score (AHS) as a composite evaluation metric. HSPO exchanges hallucinated and faithful regions among SR candidates generated from the same Low-Resolution (LR) input to construct preference pairs for region-level optimization. This alleviates global preference ambiguity caused by inconsistent hallucination locations while preventing faithful regions from dominating the optimization signal. Experiments on Syn-TestSet and RealSR-4x show lower hallucination scores than C-FLUX and DPO-SR while maintaining competitive perceptual quality.

Primary area:
generative models
Keywords:
Generative Super-Resolution, RLHF

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LaREOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses semantic hallucinations in generative image super-resolution. The authors argue that full-image preference optimization cannot accurately localize failure regions, and that object-level semantic regions may not align with hallucination boundaries. They propose HGM, which uses an indexed grid and a VLM for candidate localization, followed by GT/SR local crops for verification, and constructs AHS from region severity and area. Building on HGM, HSPO exchanges local content across multiple random candidates, with GT fallback when necessary, to construct chosen/rejected preference pairs and restricts the preference loss to the relevant regions. The authors claim that this reduces hallucinations while maintaining competitive perceptual quality, supported by two initializations (C-FLUX and DP²O), two VLM evaluators, component ablations, and human evaluation. The core contribution is organizing preference supervision around actual failure locations, rather than being the first to propose SR hallucination or DPO.

Critical strengths
  1. Clear method and strong motivation. The paper is well organized and easy to follow. The argument for localizing hallucinations rather than optimizing globally is convincing, and the preliminary taxonomy of SR hallucinations offers a useful reference for future work.
  2. Region-level preference construction is sound. Building local chosen/rejected pairs at the same spatial location avoids local strengths and weaknesses canceling out in image-level comparisons. Treating failure regions as supervision units for post-training has strong potential for transfer to other generative restoration tasks.
Critical weaknesses
  1. Validity and reproducibility of the severity metric remain insufficient. Figure 3 mainly validates area correlation and localization F1, but does not directly verify the consistency between the continuous severity score and human judgment. The threshold used to convert Stage 2 scores into the retained binary mask is unclear. In addition, the matching and counting rule when multiple predicted cells correspond to the same human-annotated region needs clarification.
  2. The meaning of "fewer hallucinations" should be further qualified. A lower average AHS does not always coincide with a lower number of images containing hallucinations, and LPIPS degrades in some settings. The current evidence is therefore better suited to supporting improvements in average area/severity than synchronized fidelity gains across all dimensions.
  3. Training cost is not disclosed. Without GPU resources, training steps, batch size, learning rate, and total time, the practical advantage for online training remains unclear.
Questions for authors
  1. Can the authors provide a curve with the score threshold (0 to 1) on the x-axis and F1@score on the y-axis, to show how finely the current VLM localizes hallucinations beyond a single operating point?
  2. In online training, what is the additional time cost of introducing an extra VLM for Set-of-Marks localization compared with directly performing flow matching? How many training GPU hours are typically needed for convergence, and can the corresponding training cost be disclosed?
  3. Since training is performed only on a small masked region, does this affect the generation result of the full image? How well are non-masked regions preserved before and after optimization? Is there any risk of degrading the original generation quality outside the masked regions?
  4. What is the additional contribution of local preference optimization relative to direct GT supervision? The authors are encouraged to prioritize controlled experiments under the same mask and the same training budget: (i) a local GT reconstruction baseline, and (ii) an HSPO variant with GT fallback removed. These comparisons would address the attribution question more directly than adding several external SR baselines.
3 comments
AuthorsOpenReview ↗

Response to LaRE (Part1/2) : Evaluation Protocol and Computational Costs

We thank the reviewer for their careful review. We provide clarifications below.

Q1. Severity Threshold and F1 Matching Protocol

The second stage outputs a region score in [0,1]; by default, we retain regions with score>0, while score=0 classifies a candidate as not containing a semantic hallucination. For localization evaluation, we process annotated regions in annotation order and select the unmatched prediction with the highest coverage, defined as intersection area divided by annotated-region area. A match is counted as TP when coverage is at least 0.3. Each annotated region contributes at most one TP; the remaining predictions are FP and undetected annotated regions are FN.

The reviewer asks us to clarify how the second-stage threshold affects the results. Our prompt specifies that a score greater than 0 indicates a hallucination. In practice, the VLM's scores for local hallucinations are not uniformly distributed. The distribution computed from the test-set scoring results is shown below.

Table 1. Distribution of second-stage hallucination scores.

Score range0[0.5, 0.6)[0.6, 0.7)[0.7, 0.8)[0.8, 0.9)[0.9, 1]
Proportion33.64%4.05%25.23%11.84%14.02%11.21%

Thus, the VLM effectively produces a discretized assessment: score=0 indicates no hallucination, while score>0 measures hallucination severity.

Q2. Offline Data Construction and Training Costs

Our method uses offline preference optimization. Before optimization begins, we generate SR candidates in advance and use HGM for Set-of-Marks localization and verification. The dataset is saved for subsequent optimization.

During dataset construction, the VLM performs HGM evaluation on the candidates, taking approximately 2 seconds per image on two A800 GPUs. For 38K images, this requires 53 GPU-hours.

During training, the cost of preference optimization is equivalent to that of DPO training. With a batch size of 128 and 500 steps, HSPO requires 25 GPU-hours, while direct flow-matching SFT requires 10 GPU-hours for the same number of iterations. Although HSPO has a higher training cost, SFT alone cannot provide semantic preference information and mitigate SR hallucinations (see Q4).

AuthorsOpenReview ↗

Response to LaRE (Part2/2) : Effects of Masked Optimization and Contribution Beyond GT SFT

Q3. Impact of Masked Optimization on Unmasked Regions

HSPO computes the preference loss within the detected hallucination masks and introduces a global GT-MSE loss with a weight of 0.1 to prevent degradation of overall reconstruction capability. Local preferences are designed to avoid imposing unnecessary preference constraints on regions that are already correct, thereby reducing the risk of disturbing normal regions.

As shown in Table 2 of the paper, HSPO achieves lower hallucination levels without degrading perceptual metrics. In the ablation study, Ours achieves a lower hallucination level than Global, which computes the loss globally without a mask. Overall, because model parameters are shared spatially, local optimization affects unmasked regions. However, local masks prevent local strengths and weaknesses from potentially canceling each other out and avoid incorrectly computing preferences for normal regions. Therefore, they preserve the original generation quality while reducing hallucinations.

Q4. HSPO versus Masked GT SFT

We have added two ablations: (1) removing GT fallback and the GT-MSE loss, with the mask defined as the union of the masks for A and B minus their intersection; and (2) SFT training, computing the GT loss only within the masked regions to measure the difference between HSPO and SFT while keeping the training budget the same. The results are shown below:

Table 3. Ablation study.

MethodPSNR ↑MUSIQ ↑CLIPIQA ↑AHS-Q ↓Hal.Img-Q ↓
Ours22.0965.910.69500.028681/300
w/o GT22.2364.890.68200.030282/300
Random Mask22.0164.830.63600.037285/300
GT SFT21.2669.160.68500.038189/300
Global21.8965.270.63400.038989/300
Global w/o exchange21.6566.070.64000.045592/300

The experiments show that GT fallback has some impact on performance, but preference learning makes the main contribution. GT SFT also reduces hallucinations, potentially benefiting from both adaptation to the training degradation distribution and hallucination-aware supervision within the detected regions, while HSPO achieves further improvements by leveraging region-level preference learning.

Reviewer_LaREOpenReview ↗

Reviewer Comments

Thank you for the additional experiments and clarifications. My concerns have been largely addressed.

Reviewer_1xdROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes a new method, HGM, for locally identifying hallucinations in generative super-resolution using a VLM. It also introduces a loss function, HSPO, designed to suppress hallucinations in generative super-resolution models by exploiting the stochastic variation in the locations and characteristics of hallucinations across different seeds. The effectiveness of the proposed method is validated through a comprehensive set of experiments.

Critical strengths
  1. The manuscript is well written. The proposed method is well motivated by the observation that hallucinations in super-resolution are local and stochastic. The positioning of the work with respect to prior studies is also clearly organized.

  2. The effectiveness of the proposed method is demonstrated through comparisons with multiple methods, ablation studies, and human evaluation.

Critical weaknesses
  1. The training pipeline, including data construction, is somewhat complex and computationally heavy.

  2. The proposed method is applied to only two super-resolution models in the paper.

Questions for authors
  1. What criteria were used to select C-FLUX and DP^2O?

  2. What results would be expected if models specifically designed to suppress hallucinations, such as those discussed in Section 2.3, were evaluated using the same metrics as in Table 2?

3 comments
AuthorsOpenReview ↗

Response to 1xdR (Part1/2) : Generalization Across SR Architectures

We thank the reviewer for their helpful suggestions. We respond below.

Q1. Generalization Across SR Architectures

In our previous observations, generative SR models exhibit a trade-off between perceptual quality and hallucination metrics. C-FLUX and DP²O, as relatively recent SR baselines, provide models before and after post-training, with different perceptual scores and initial hallucination levels. This allows us to examine the effectiveness of HSPO from different optimization starting points.

We agree that experiments should not be limited to two SR models with the same architecture. We perform HSPO training on FiDeSR using a mixture of 25% FiDeSR-constructed preference pairs and 75% preference pairs from the original training dataset (due to time constraints, we could not construct the full FiDeSR preference dataset). We replace the flow-matching preference loss with a loss adapted to single-step diffusion, retain the relative weights of the loss terms, and train rank-64 LoRA in the UNet. The results are shown below:

Table 1. HSPO on FiDeSR.

Syn-TestSet

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR23.700.62270.253765.640.68153.71173.62750.630.019758/300—
FiDeSR + Ours23.770.62480.253665.380.67563.68873.56750.920.013945/300+29.42%

RealSR-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR23.910.70510.272369.910.69203.667112.97862.260.062933/100—
FiDeSR + Ours24.040.70970.269469.480.68243.642113.41859.920.060029/100+4.67%

DRealSR-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR26.640.76200.286465.340.69643.608129.53884.100.02346/93—
FiDeSR + Ours26.740.76580.284464.990.69263.584129.74882.080.02126/93+9.30%

DIV2K-difficult-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR16.620.40720.399566.300.64873.600125.02929.190.074427/100—
FiDeSR + Ours16.640.40810.400465.940.64083.579125.31927.220.060028/100+19.31%

FiDeSR improves fidelity through modules such as Detail-aware Weighting and LRRB, indirectly reducing its hallucinations and giving it a relatively low hallucination level to begin with. HSPO can further reduce its hallucination level, indicating that architectural optimization and preference optimization are compatible. This further demonstrates the effectiveness of our method.

AuthorsOpenReview ↗

Response to 1xdR (Part2/2) : Other Hallucination Mitigation Baselines

Q2. Availability of Hallucination Mitigation Baselines

We agree that comparisons with hallucination mitigation methods would strengthen the evaluation. At the time of our experiments, we could not obtain the code and/or pretrained weights needed for a controlled comparison with methods (such as FinPercep-RM, DSPO, SRSR, and Hallucination Score) under the same evaluation protocol. Meanwhile, the performance is strongly correlated with the choice of backbone, and the discrepancy introduced by different base models can be substantially larger than the hallucination difference brought by the method itself.

Meanwhile, HSPO operates during post-training and is potentially complementary to inference-time methods such as SRSR.

Thank you for conducting the additional experiments and providing the results. These results adequately address my concern regarding this aspect of the paper.

Reviewer_uzqzOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a hallucination-grounded preference learning framework for generative super-resolution. Specifically, Hallucination-Grounded-Marks (HGM) uses VLMs to localize and assess hallucinated regions, while Hallucination-Selective Preference Optimization (HSPO) constructs region-level preference pairs and selectively optimizes hallucinated regions. The authors also introduce an Area-Weighted Hallucination Score (AHS) for evaluation. Experiments on Syn-TestSet and RealSR-4x demonstrate reduced hallucination scores compared with the corresponding baseline models.

Critical strengths
  1. Well-motivated region-level preference optimization. The paper identifies the limitations of image-level preferences when hallucinations occur in different spatial regions. The proposed region-exchange strategy provides a reasonable way to construct more targeted preference pairs.
  2. Encouraging experimental results. The proposed method consistently reduces AHS compared with its corresponding backbones, and the ablation studies and human evaluation provide additional evidence of its effectiveness.
Critical weaknesses
  1. Limited evaluation of generalizability. The proposed method is evaluated only on C-FLUX and DP²O, which are closely related models, and on two relatively limited test sets. Since the framework is presented as a general approach to hallucination mitigation, experiments on more diverse generative SR architectures and additional benchmarks, such as DRealSR and standard DIV2K-Val, are necessary to establish its generalizability and effectiveness.
  2. Insufficient validation of the proposed hallucination metric and unclear advantages over existing methods. Although the proposed method improves AHS over its own baselines, existing methods such as FiDeSR, which do not explicitly optimize hallucinations, achieve lower or comparable AHS while outperforming the proposed method on several fidelity and perceptual metrics. This raises questions about both the practical benefits of HSPO and the reliability of AHS. Moreover, the area-weighted formulation lacks sufficient justification, and its agreement with human judgments across different SR methods has not been adequately established.
Questions for authors
  1. Generalizability: Can the authors evaluate HSPO on additional generative SR architectures and datasets, particularly DRealSR and standard DIV2K-Val, to demonstrate that its effectiveness is not limited to the two closely related models currently considered?
  2. Validity of AHS: Why does FiDeSR achieve lower or comparable hallucination scores without explicitly optimizing hallucinations? Can the authors provide cross-method human evaluations and correlation analyses to validate whether AHS reliably reflects semantic hallucination severity?
  3. Contribution of preference optimization: Can the authors provide ablations isolating the effects of GT fallback and the auxiliary GT-MSE loss to demonstrate the independent contribution of HSPO?
4 comments
AuthorsOpenReview ↗

Response to uzqz (Part1/3) : Generalization Across Datasets

We thank the reviewer for their constructive feedback. We address the concerns below.

Q1. Generalization Across Datasets

Our HSPO is positioned as a general method for mitigating hallucinations in super-resolution. We have demonstrated its effectiveness on C-FLUX and DP²O, two models based on FLUX.1-dev, but have not yet tested it on more datasets and model architectures.

We agree that evaluation on more datasets and generative SR architectures is needed. We have conducted additional experiments on DRealSR and standard DIV2K-Val, as mentioned by the reviewer.

The results on DRealSR and the difficult subset of DIV2K-Val are shown below. For DIV2K-Val, we use the difficult subset. For both datasets, we use 512 × 512 center crops for inference and scoring. The values in each table row are the average of eight seeds.

Table 1. Results on DRealSR and the difficult subset of DIV2K-Val.

DRealSR-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
C-FLUX25.550.70550.354461.910.57813.226151.92985.180.037110/93—
C-FLUX + Ours25.770.70560.359162.310.63463.334143.28948.630.02658/93+28.60%
DP²O25.390.68610.361869.380.72513.815148.931018.560.042712/93—
DP²O + Ours25.740.69160.356768.680.74023.771146.551004.940.035510/93+16.78%

DIV2K-difficult-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
C-FLUX16.340.37920.476864.120.60463.549152.701177.660.099135/100—
C-FLUX + Ours16.440.37710.483863.220.64263.451153.651156.830.063225/100+36.18%
DP²O16.300.37150.447571.170.69653.911150.471149.410.125239/100—
DP²O + Ours16.460.37480.454869.670.69973.711154.491143.830.094230/100+24.76%

In addition, we have performed post-training on other SR models; please see Q2 for details.

AuthorsOpenReview ↗

Response to uzqz (Part2/3) : FiDeSR's Lower AHS and Generalization Across Models

Q2. FiDeSR's Lower AHS and Further Improvement with HSPO

The reviewer notes that models such as FiDeSR, which do not explicitly optimize for hallucinations, achieve lower AHS than our optimized models. This is expected, because we deliberately selected C-FLUX and DP²O backbones characterized by high perceptual metrics and high hallucination risks to demonstrate the effectiveness of our method. In addition, single-step diffusion models such as FiDeSR are inherently more stable than multi-step models, and FiDeSR improves fidelity through modules such as Detail-aware Weighting and LRRB, indirectly reducing its hallucinations.

Fortunately, our preference learning method is compatible with improvements to model architecture, and the two can jointly reduce hallucinations. We perform HSPO training on FiDeSR using a mixture of 25% FiDeSR-constructed preference pairs and 75% preference pairs from the original training dataset (due to time constraints, we could not construct the full FiDeSR preference dataset). We replace the flow-matching preference loss with a loss adapted to single-step diffusion, retain the relative weights of the loss terms, and train rank-64 LoRA in the UNet. The results are shown below:

Table 2. HSPO on FiDeSR.

Syn-TestSet

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR23.700.62270.253765.640.68153.71173.62750.630.019758/300—
FiDeSR + Ours23.770.62480.253665.380.67563.68873.56750.920.013945/300+29.42%

RealSR-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR23.910.70510.272369.910.69203.667112.97862.260.062933/100—
FiDeSR + Ours24.040.70970.269469.480.68243.642113.41859.920.060029/100+4.67%

DRealSR-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR26.640.76200.286465.340.69643.608129.53884.100.02346/93—
FiDeSR + Ours26.740.76580.284464.990.69263.584129.74882.080.02126/93+9.30%

DIV2K-difficult-4x

MethodPSNR ↑SSIM ↑LPIPS ↓MUSIQ ↑CLIPIQA ↑Q-Align ↑FID ↓FD-DINOv2 ↓AHS-Q ↓Hal.Img-Q ↓AHS-Q reduction (%) ↑
FiDeSR16.620.40720.399566.300.64873.600125.02929.190.074427/100—
FiDeSR + Ours16.640.40810.400465.940.64083.579125.31927.220.060028/100+19.31%

Although the improvement is smaller than for the FLUX-based multi-step models, there is still room for optimization.

AuthorsOpenReview ↗

Response to uzqz (Part3/3) : The Impacts of the GT Rollback Mechanism, Etc.

Q3. The Impacts of the GT Rollback Mechanism, Etc.

We have added two ablations: (1) removing GT fallback and the GT-MSE loss, with the mask defined as the union of the masks for A and B minus their intersection; and (2) SFT training, computing the GT loss only within the masked regions to measure the difference between HSPO and SFT while keeping the training budget the same. The results are shown below:

Table 3. Ablation study.

MethodPSNR ↑MUSIQ ↑CLIPIQA ↑AHS-Q ↓Hal.Img-Q ↓
Ours22.0965.910.69500.028681/300
w/o GT22.2364.890.68200.030282/300
Random Mask22.0164.830.63600.037285/300
GT SFT21.2669.160.68500.038189/300
Global21.8965.270.63400.038989/300
Global w/o exchange21.6566.070.64000.045592/300

The experiments show that GT fallback has some impact on performance, but preference learning makes the main contribution.

Reviewer_uzqzOpenReview ↗

Reply to the rebuttal

I have carefully read the other reviewers' comments and the authors' responses. The authors have demonstrated the generalizability of their method across additional datasets and model architectures, which thoroughly addresses my concerns. Taking the other reviewers' comments into account, I have raised my score to 3, and I hope these generalizability experiments will be included in the final revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.