acceptodds
Withdrawn from ICLR 2027

FinerSR: Restoring Small but Semantically Critical Regions in Generative Video Super-Resolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative video super-resolution has made remarkable progress in restoring realistic visual content from severely degraded videos. However, high global perceptual quality does not imply semantic fidelity in small but semantically critical regions, where visually plausible reconstructions may contain incorrect text or altered facial identities. To address this challenge, we propose FinerSR, a unified framework that strengthens small but semantically critical regions in both representation and supervision levels while dynamically coordinating local semantic fidelity with global perceptual quality. We first establish a strong four-step general restorer through supervised fine-tuning, on-policy distillation, and perceptual and reconstruction refinement. Building on this foundation, we introduce object-aware conditioning and region-normalized supervision to enhance the representation and optimization of small but semantically critical regions. We further develop discriminative objectives for two distinct semantic properties: alignment-aware identity and appearance matching for faces, and complete-string likelihood with edit-graph probability contrasts for text. Finally, we formulate semantic refinement as a quality-constrained joint optimization problem, using adaptive primal-dual updates to coordinate local semantic objectives with full-frame fidelity and perceptual constraints relative to the general restorer. Experiments on faces and text demonstrate that FinerSR enhances semantic fidelity in small but semantically critical regions while preserving overall restoration quality, bridging the gap between global visual quality and local semantic correctness.

Primary area:
generative models
Keywords:
Generative Video Super-Resolution, Face Restoration, Text Restoration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zY5LOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

FinerSR addresses semantic errors in generative video super-resolution, where visually plausible outputs may alter facial identities or produce incorrect text. It combines object-aware conditioning and region-normalized supervision with specialized face identity–appearance matching and text recognition objectives. An adaptive quality-constrained optimization scheme balances local semantic fidelity with overall restoration quality. Experiments show improved facial identity preservation, small-face quality, and text recovery while maintaining competitive full-frame quality.

Critical strengths
  1. It addresses a meaningful limitation of generative video super-resolution. Visually realistic results restored by state-of-the-art models can still contain incorrect identities or text.
  2. Experiments support improved local semantic fidelity while retaining competitive overall visual quality.
Critical weaknesses
  1. Unclear advantage over separate local restoration. Since the method already relies on frame-wise face and text detection, a natural alternative is to restore the detected regions separately and blend them into the general restorer’s output. This could offer a more direct way to improve local content while preserving the rest of the frame. Table 3 makes this comparison particularly relevant: the full method improves local semantic metrics but slightly worsens full-frame PSNR and LPIPS. Moreover, frame-wise detection fluctuations or missed detections could destabilize the injected features, so joint generation does not by itself establish better temporal consistency. Without a comparison against a well-designed local-restoration-and-blending baseline, including regional quality, boundary artifacts, temporal consistency, and computational cost, the benefit of the proposed integration remains unclear.
  2. Reliability of the semantic evaluation. The evidence specifically supporting small-region restoration is limited. Small-face LPIPS is evaluated on only eight clips, while the text subset includes instances of unrestricted size. Consequently, the aggregate improvements do not establish whether the gains are consistent across videos or concentrated on small targets. Furthermore, the reference text labels are generated by PP-OCR without manual verification, and PP-OCR also serves as a primary evaluator. This creates a risk of evaluator-specific bias: agreement with PP-OCR’s own predictions may be rewarded even when those predictions are incorrect. Evaluating with EasyOCR does not fully resolve this concern because it uses the same PP-OCR-derived reference labels. Human-verified annotations and a larger evaluation stratified by target size, with video-level confidence intervals, are needed to substantiate the central semantic-fidelity claim.
Questions for authors
  1. Comparison evidence: Are there existing results comparing FinerSR with separate local restoration and blending under the same detections? If so, what advantages or trade-offs do they show?
  2. Detection robustness: Is any temporal stabilization applied to the frame-wise detections? What evidence is available regarding the effect of detection jitter or missed detections on output consistency?
Reviewer_DfqUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes FinerSR, a generative video super-resolution framework for improving the identity and text fidelity of small regions. It combines object-aware conditioning, region-normalized supervision, face and text recognition losses, and adaptive quality-constrained optimization. Experiments report improved local face and text metrics with comparable global restoration quality.

Critical strengths

The paper targets an important failure mode of generative video super-resolution and provides extensive experiments on local semantic metrics.

Critical weaknesses
  1. The proposed framework is excessively complicated, combining a four-step restorer, object conditioning, region-normalized losses, multiple face/text objectives, and adaptive primal-dual updates. The ablations do not identify a simple or generalizable principle that could guide future work; the contribution largely appears to be an engineering combination of many components.

  2. The method and evaluation rely heavily on external pretrained face recognition and OCR systems. These recognizers are not necessarily accurate on severely degraded or restored small regions, while OCR-derived pseudo-labels are also used for text supervision. Errors from detection, recognition, pseudo-labeling, and metric computation can accumulate through the pipeline.

Questions for authors

Please see the weaknesses above.

Reviewer_prRLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FinerSR targets identity and transcription errors in small regions during generative video super-resolution. It builds a four-step restorer, adds features from LR object detections, and averages semantic supervision over observations, tracks, and videos rather than image area. Faces use reference-conditioned identity–appearance matching; text uses complete-string likelihood and edit-graph probability contrasts. Primal–dual updates coordinate these losses with seven full-frame quality measures relative to a frozen restorer. Evaluation covers 80 synthetically degraded paired clips, 50 unpaired clips, selected face/text subsets, and component ablations, reporting improved local metrics with competitive full-frame quality.

Critical strengths

Regional conditioning and loss aggregation have practical support. Removing object conditioning reduces IDS from 0.5619 to 0.5481 and PP-OCR token coverage from 0.3420 to 0.3068. Area weighting worsens small-face LPIPS from 0.2955 to 0.3081 (Table 3, p. 9). Appendix B.4 matches source exposure and update budgets, with calibration performed by the same rule. These comparisons support the regional design on the evaluated subsets.

The loss constructions have interpretable, appropriately limited properties. The face metric satisfies P ⪯ G ⪯ 2P (Equation 25, p. 21), and the text loss’s excess risk is KL divergence plus a nonnegative quadratic for a fixed supervision context (Equation 35, p. 23). The appendices explicitly distinguish these properties from guarantees of correct identity or readable text. This supports the technical rationale for the losses, without treating it as a semantic-recovery theorem.

Critical weaknesses

The main semantic results lack independent and sufficiently direct validation. Appendix B.1 states that the face/text subsets “also informed model selection during development” (p. 27). They therefore are not untouched semantic test sets; the extent of selection remains unclear. This matters with only 24 face clips, including eight small-face clips, and 46 text clips. Text targets are OCR-derived pseudo-labels without manual filtering, and Equation 44 uses neither spatial matching nor penalties for extra strings. Coverage can improve while incorrect text remains or a token appears at the wrong location. EasyOCR supplies another recognizer, but shares the same pseudo-targets. There is also no text-size breakdown. These results support improved token coverage, but do not yet establish reliable correctness in small text regions.

The distinctive face geometry and adaptive controller need clearer attribution. The face control replaces the entire reference-conditioned MMD objective with direct cosine and feature errors (Appendix B.4, p. 29), changing metric geometry, kernel matching, and view aggregation together. It supports the combined objective, not specifically the alignment-aware metric. For quality coordination, the fixed-weight control uses calibrated scales with all multipliers set to one, and no final source-wise constraint residuals are reported (Appendix A.8, pp. 25–26; Table 3). The comparison supports a quality benefit over that setting, but leaves the advantage over tuned fixed weighting unresolved. These are evidence gaps, not demonstrated failures; the small test-set deterioration relative to F₀ does not itself prove a training-constraint violation.

Questions for authors

Clarification and targeted validation: Which choices were selected using the reported semantic subsets? Is an untouched evaluation available? A modest independent set with manually checked, spatially matched word labels and a small-text breakdown would most directly strengthen the central conclusion.

Controller validation: Can you report final source-wise mean constraint residuals and one tuned fixed-weight control at comparable semantic performance? This would clarify whether adaptation delivers effective constraint control beyond the chosen fixed-weight baseline.

Face-metric attribution: Keeping MMD and the training protocol unchanged, how do results change with G = I? A comparison with G = P, if available, would further separate whitening from competitor emphasis. A consistent benefit would strengthen the case for the added geometry.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.