acceptodds
Under review as a conference paper at ICLR 2027

Learning What Structure to Preserve for Source-Faithful Generative Image Super-Resolution

Abstract

Generative super-resolution can synthesize plausible details that do not match the source. We introduce MetaSR, a source-assisted framework that learns how to use structural metadata and which structure to transmit under a budget. The receiver encodes the LR image and edge map with a shared VAE and fuses aligned tokens through native transformer attention. We define structural utility as the change in whole-image reconstruction quality produced by an assignment, relative to MetaSR w/o edge with the same trained receiver. A lightweight sender-side predictor learns reusable region-action scores from these counterfactual outcomes. Combining the scores with measured coding costs enables budgeted selection without online receiver queries, and changing the budget requires no retraining or recalibration. Across three benchmarks, MetaSR improves PSNR by 0.47–1.60 dB over MetaSR w/o edge at matched aggregate transmission rates. On a held-out evaluation set with the receiver fixed, reusing the learned scores at five unseen budgets outperforms random allocation by 0.50 dB on average under shared metadata caps. These results show that receiver-dependent utility can guide which source structure to preserve for source-faithful reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image super-resolution, generative image restoration, diffusion transformers

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MqxUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MetaSR, a source-assisted generative super-resolution framework. The sender extracts structural metadata from the HR source, while a learned utility predictor selects which regional edge information to transmit under a bitrate budget. Experiments evaluate reconstruction quality, structural fusion, allocation policies, and generalization to unseen metadata budgets.

Critical strengths

The paper introduces MetaSR, a source-assisted framework that learns how to use structural metadata and which structure to transmit under a budget.

Critical weaknesses
  1. Since the sender has access to the complete HR image, the task is closer to source-assisted compression than conventional super-resolution.

  2. Direct HR coding is compared only with H.264 in an exploratory experiment. Modern learned codecs and more advanced conventional codecs are missing.

  3. The allocation methods transmit different actual numbers of bits. Therefore, the benefits of the selection strategy are not fully isolated from bitrate differences.

  4. The qualitative images exhibit strong generative reconstruction effects, but the paper provides insufficient evidence that they are direct and unselectively presented outputs of the reported pipeline.

  5. The qualitative evaluation includes only a few selected regions, without random examples, full resolution results, error maps, original output files, or complete reproduction scripts.

Questions for authors
  1. Can the authors provide unedited full-resolution outputs and their corresponding fixed random seeds?

  2. Can the authors provide randomly selected qualitative examples rather than only manually selected local crops?

  3. Are all displayed results direct outputs of the corresponding models, without manual modification, regional replacement, or additional generative processing?

  4. How does MetaSR compare with modern learned image codecs at exactly matched total bitrates?

5.How can the authors demonstrate that the recovered details originate from the transmitted structural metadata rather than the generative prior?

6 comments

[This is the first section of our rebuttal (1/5)]

We thank the reviewer for asking what transmitting source structure adds when the sender already has the high-resolution (HR) image. We could have made the delivery requirements clearer. When HR is the only required output, encoding the source at its native resolution is a strong choice, as the submitted Appendix I and the new comparisons in W2/Q4 illustrate. A service may also need to deliver an independently decodable low-resolution (LR) rendition. Receivers with limited processing capacity or memory can decode and display it directly, while receivers with super-resolution (SR) support can use that same decoded image to reconstruct at HR. On RealSR100, allocating part of the same total transmission budget to selected source structure improves PSNR-Y by 0.86 dB over spending the entire budget on HEVC LR coding (W1). Both pipelines deliver independently decodable HEVC LR, use the same SR receiver and per-image latent seeds, and match total transmitted bytes exactly across the dataset, counting metadata and signaling where applicable (0.1496 bpp).

Downsampling before encoding lets a service offer a playable rendition under a tighter bandwidth budget. YouTube creates multiple playback resolutions from a single upload, and Apple's HLS authoring guide gives example HEVC renditions at 720p (2.4 Mbps) and 540p (0.6 Mbps). A viewer with a 4K screen may receive the 540p rendition of a 4K source when bandwidth is limited. The LR rendition is usable on its own, while reconstructing 3840×2160 from 960×540 requires 4× upscaling along each axis. Downsampling can merge strokes in small text or obscure thin contours, leaving their original shapes ambiguous in the decoded image. The sender can provide cues about those shapes directly from the HR source.

Still-image services also deliver multiple renditions. The HTML srcset attribute lets browsers choose an image version suited to its display size and the device's pixel density. HTTP Archive found it on 42% of the mobile pages in its 2024 sample.

MetaSR uses a standard JPEG or HEVC bitstream for the LR rendition and sends selected source cues in a separate packet for receivers that perform SR. In a separate experiment with exactly matched metadata and signaling bits, learned selection improves PSNR-Y by 0.46 dB over random selection (W3). The revised manuscript makes the LR delivery requirement explicit and distinguishes the results for this setting from the HR-only coding comparisons.

All page numbers refer to the revised paper. Figs. 15–19 are in its Appendix U (pp. 51–55); the review package calls them Figs. M1–M5. Submitted figures use the original manuscript numbering. Fig. 9 refers to the revised manuscript.

W1: Source-assisted gains at a fixed total rate

The total transmission budget includes the coded LR image, structural metadata, and signaling (revised Introduction and Section 4.3, Table 3). In the matched-byte RealSR100 test, both arms transmit exactly 490,368 bytes (0.1496 bpp), and the LR-only arm selects one QP per image by measured code length. Both arms use the same receiver and inference seeds. Selected structure raises PSNR-Y from 26.21 to 27.07 dB, a paired gain of 0.86 dB (95% CI [0.73, 0.99] dB). SSIM is 0.020 higher and DISTS 0.007 lower; LPIPS changes by −0.002, with an interval that includes zero, and MUSIQ is 2.1 points lower (Table M9).

Table M9. RealSR100 at exactly 490,368 transmitted bytes

ConditionTotal bytesbppPSNR-Y ↑SSIM ↑LPIPS ↓DISTS ↓MUSIQ ↑
LR-only HEVC490,3680.149626.210.7270.1700.18962.8
MetaSR + predictor490,3680.149627.070.7470.1680.18260.7
Paired difference+0.86+0.020−0.002−0.007−2.1
95% CI[0.73, 0.99][0.012, 0.029][−0.009, 0.005][−0.013, −0.002][−3.3, −0.9]

Total bytes include the LR stream, structural metadata, and signaling. Differences are MetaSR + predictor minus LR-only, paired by image.

Over the HEVC sweep with the LR base at QP 30, the gains of the eight predictor caps against the nearest measured LR-only points at equal or higher total rates are 0.31–1.30 dB on RealSR and 0.51–1.57 dB on DRealSR.

[To be continued in the next section]

[This is the second section of our rebuttal (2/5)]

Under the original JPEG protocol, MetaSR improves mean PSNR-Y from 26.29 to 27.34 dB while using 1.36% fewer total bits than LR-only (Table M1 and revised Appendix N, Table 17).

Table M1. RealSR100 at the original Table 1 JPEG operating point

ConditionTotal bytesbppPSNR-Y ↑SSIM ↑LPIPS ↓DISTS ↓MUSIQ ↑
LR-only1,736,7040.530026.290.7310.1670.18862.6
Full edges1,736,7040.530027.640.7880.1340.16564.3
MetaSR + predictor1,713,0090.522827.340.7660.1480.17262.2

Displayed PSNR-Y means are rounded.

W3 + Q5: Benefits of learned selection and aligned structure

W3. Learning which structure to send improves fidelity at equal realized rates. Across 73 exact bit-matched pairs from 43 sources, learned allocation gains 0.46 dB PSNR-Y over random allocation (95% CI [0.27, 0.67]). We compare the assignments on 80 held-out sources (40 LSDIR and 40 DIV8K), using the same decoded LR, receiver, latent seed, candidate bank, and coded-cost optimizer. Exact matches in metadata and signaling bits give equal total rates.

Allowing random to use up to 1% fewer bits yields 0.45 dB over 268 pairs from 72 sources (95% CI [0.33, 0.57]). A one-sided 3% check gives the same mean gain. We bootstrap sources after averaging their paired gains. Revised Appendix J (Table 14) reports coverage and bit gaps by cap.

Table M3. Learned minus random across five unseen budgets

Actual-bit pairingPairsSourcesΔPSNR-Y (dB)95% CI (dB)
Exact metadata bit match7343+0.46[+0.27, +0.67]
Random ≤ learned, within 1%26872+0.45[+0.33, +0.57]

Q5. The received structure guides geometry when it is aligned with the source. With decoded LR, receiver, and latent seed fixed on the same 80 sources, aligned learned structure adds 0.915 dB PSNR-Y over no edge (95% CI [0.760, 1.077]). We intervene after metadata decoding and hold the transmitted byte count fixed within each metadata arm. A (64, 64)-pixel shift, a tile shuffle, and an edge map from another source each bring mean PSNR-Y below no edge in both the learned and full-edge arms (Table M6).

Table M6. Structure interventions on held-out sources

Decoded conditionPSNR-Y (dB)Δ vs. aligned (95% CI)Δ vs. no edge (95% CI)
No edge24.070—Reference
Learned 50%, aligned24.985Reference+0.915 [0.760, 1.077]
Learned 50%, shifted23.548−1.437 [−1.618, −1.263]−0.522 [−0.656, −0.373]
Learned 50%, shuffled23.555−1.431 [−1.614, −1.250]−0.515 [−0.657, −0.361]
Learned 50%, other source23.168−1.817 [−2.231, −1.459]−0.902 [−1.317, −0.565]
Full edge, aligned25.186Reference+1.115 [0.887, 1.358]
Full edge, shifted23.269−1.917 [−2.205, −1.646]−0.801 [−0.932, −0.667]
Full edge, shuffled23.253−1.932 [−2.230, −1.656]−0.817 [−0.950, −0.686]
Full edge, other source22.776−2.409 [−2.903, −1.965]−1.294 [−1.771, −0.890]

Fig. 19 (p. 55) shows the case with median aligned gain among examples with a nonempty donor raster. Shift and shuffle preserve the aligned raster's edge-pixel count. With LR fixed, learned metadata improves reference line-segment recall by 1.7 percentage points on Urban100 and 1.9 points on DIV2K. The receiver uses the spatial structure for geometry and synthesizes texture from the LR image and its learned prior.

[To be continued in the next section]

[This is the third section of our rebuttal (3/5)]

W2 + Q4: Direct HR coding and measured quality trade-offs

W2. Perceptual trade-off at the measured predictor points. Table M10 compares the eight measured MetaSR + predictor points (LR QP 30, caps of 12.5% to 100%) with direct HEVC and VVC at the same total rate. Direct coding has higher PSNR-Y and SSIM at all eight points on both datasets (Table M2), and lower LPIPS and DISTS at most of them. On RealSR100, the predictor has lower LPIPS than direct HEVC only at the 12.5% cap (0.184 vs. 0.187 at 0.0837 bpp); at the 100% cap the values are 0.153 vs. 0.083 at 0.2270 bpp, and its DISTS is higher at all eight caps. On DRealSR93, the predictor has lower LPIPS than direct HEVC at the two lowest caps (0.188 vs. 0.218 at 0.0536 bpp and 0.185 vs. 0.192 at 0.0655 bpp) and lower DISTS at the lowest cap. Against direct VVC, its LPIPS is higher at all eight points on both datasets.

Along the interpolated MetaSR curve, which starts at the LR-only point, LPIPS is lower than direct HEVC on RealSR at approximately 0.062–0.087 bpp and DISTS at 0.062–0.077 bpp; on DRealSR, LPIPS is lower than direct VVC at approximately 0.043–0.053 bpp and DISTS at 0.043–0.056 bpp. These intervals use interpolation within measured rate support and end at or before the lowest predictor caps.

MetaSR + predictor is higher than both direct codecs on the no-reference metrics MUSIQ, MANIQA, and CLIPIQA at all eight points on both datasets (Table M10). On RealSR100, learned structure allocation gives 60.7 on MUSIQ at 0.1496 bpp, compared with 52.2 for direct HEVC at 0.1500 bpp. The margins are smallest at the highest caps; at the 100% cap on DRealSR93, MANIQA is 0.485 vs. 0.481 for direct VVC.

Table M10. MetaSR + predictor against direct HR coding at the eight measured predictor points (LR QP 30)

MetricRealSR100 vs. HEVCRealSR100 vs. VVCDRealSR93 vs. HEVCDRealSR93 vs. VVC
PSNR-Y (dB)0/8; −10.2 to −5.90/8; −11.6 to −7.20/8; −9.1 to −6.10/8; −10.3 to −7.4
SSIM0/8; −0.162 to −0.1300/8; −0.182 to −0.1620/8; −0.126 to −0.1030/8; −0.144 to −0.130
LPIPS1/8; −0.071 to +0.0040/8; −0.080 to −0.0332/8; −0.048 to +0.0300/8; −0.055 to −0.002
DISTS0/8; −0.068 to −0.0050/8; −0.070 to −0.0191/8; −0.050 to +0.0101/8; −0.048 to +0.002
MUSIQ8/8; +7.8 to +12.38/8; +5.9 to +9.48/8; +4.4 to +8.78/8; +3.1 to +6.6
MANIQA8/8; +0.026 to +0.1148/8; +0.021 to +0.0858/8; +0.010 to +0.0998/8; +0.004 to +0.076
CLIPIQA8/8; +0.20 to +0.238/8; +0.13 to +0.188/8; +0.18 to +0.198/8; +0.09 to +0.15

Each cell gives the number of the eight predictor points at which MetaSR + predictor is better, followed by the range of its margin over the direct codec; a positive margin means that MetaSR + predictor is better. The codec value is interpolated linearly in bpp between its two neighboring measured points at the predictor's rate. The predictor rates are 0.084–0.227 bpp on RealSR100 and 0.054–0.126 bpp on DRealSR93.

Table M2. Direct HR coding PSNR-Y advantage over MetaSR

DatasetEvaluated overlap (bpp)Direct HEVC (dB)Direct VVC (dB)
RealSR1000.062–0.227+4.9 to +10.2+6.1 to +11.6
DRealSR930.043–0.126+5.8 to +9.1+6.9 to +10.3

Revised Appendix I gives the direct HR comparisons in Fig. 9 and Tables 11 and 13. The reported StableCodec results on RealSR cover 0.0256–0.0347 bpp and 26.62–27.30 dB PSNR-Y. Those rates lie below the evaluated HEVC-base MetaSR range.

[To be continued in the next section]

Reviewer_8kofOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MetaSR, a source-assisted generative super-resolution method, where the encoder looks at the HR image, encodes and sends an LR image and the compressed edges. The decoder uses a common VAE encoder and transformer attention to fuse spatially aligned tokens. The core of the paper introduces a training method for the encoder-side predictor that learns additive region–action scores for fixed-edge representations based on the global reconstruction improvement achieved by a fixed decoder, and, using a multiple-choice knapsack solver with measured coding costs, selects the structural information without interacting with the decoder online. The evaluation includes three x4 reconstruction benchmark results, comparison of conditioning methods, ablations of the predictor, and evaluation of allocation strategies using 80 images that were held out from training. The authors achieve 0.47–1.60 dB improvements compared to a version without edges at the same total rates and approximately 0.50 dB better than random allocation for five unseen metadata budgets.

Critical strengths
  1. Meaningful evidence for learning transmission utility from receiver outcomes. With given candidates, coding costs, decoder, and optimizer, the learned predictor achieves 0.870 dB improvement over the no-edge decoder as the primary seed, while random allocation only achieves 0.355 dB, and the two saliency heuristics have negative values (Table 3), and the three-seed mean is 0.834 ± 0.032 dB. Other regression-only control and unseen-budget experiments further support the learned objective. This shows that the method can go beyond just showing that adding source information can improve reconstruction.

2. Careful accounting and useful experimental controls. The authors report actual payloads and signaling, control for random decoder seeds in the paired comparisons, and group statistical uncertainty by source to account for repeated observations. In addition, Appendix G.1 provides additional approximate matched-rate controls to supplement comparisons across shared metadata caps, and for the 1% matching subset, learned allocation retains a 0.432 dB advantage over the rate-selected random comparator. The authors acknowledge that the controls are not comprehensive and clarify that the comparison to a finite decoder search is not an optimization over all possibilities, further supporting the allocation results.

Critical weaknesses
  1. The system-level benefit of learned structural transmission remains insufficiently established. Since the encoder has access to the HR source, the relevant comparison could include direct HR coding and adding more bits to the base stream. Table 1 compares the approach to image restoration methods and transmits full-edge maps without the learned selector. The learned selector experiments only optimize metadata allocation given a fixed base stream. Appendix I acknowledges the direct coding option but uses H.264 in an exploratory low-rate regime with another lossy structural representation without the predictor, at rates roughly ten times lower than Table 1 (~0.03–0.06 vs ~0.53 bpp). At some point, direct HR coding performs better. This does not establish whether the proposed learned allocation method is useful in the overall transmission system. A comparison using MetaSR + predictor, total transmitted rates, and representative direct and generative codecs would substantially improve the claim's significance. For example, StableCodec (ICCV 2025) uses one-step generative compression to address the source fidelity problem. The question is whether the proposed transmission method offers useful advantages over relevant alternatives, rather than whether it achieves state-of-the-art performance.

  2. The contribution needs clearer positioning against prior structural and semantic transmission methods. Text + Sketch presented at the ICML 2023 Neural Compression Workshop, already transmits compressed edge maps to condition a diffusion decoder, preserving the structure of the source. GESCO, IEEE TCCN 2026 builds a sender–receiver system that transmits compressed semantic layouts to guide generative reconstruction. Neither is in the bibliography. The language-oriented image-transmission paper at IEEE MLSP 2024 provides additional context by also transmitting captions and image latents. They constrain the amount of novelty that can be attributed to the system as a whole. The paper should more directly address what additional benefits arise from the aligned receiver interface and learned region allocation. Using the same scores for different budgets is a useful technique, but this follows by construction, since the predictor does not take the budget as input; the substantive question is the quality of the resulting decisions. It would be useful to evaluate the efficacy of such decisions. Fixing these problems would make the contribution easier to contextualize.

Questions for authors
  1. Can you compare the MetaSR + predictor system end-to-end with a direct HR coder and with a no-edge system that spends the same total bits on its base stream? Table 1 includes the latter comparison for complete edge maps, but not for MetaSR + predictor. PSNR and perceptual metrics within the targeted operating range would help demonstrate the conditions under which transmitting structural information is beneficial.

  2. What specifically is the contribution of MetaSR over other systems (see Weakness 2)? Please disentangle the contributions of the receiver interface, utility-learning objective, and region allocation procedure.

  3. How do you make sure Table 1 corresponds to equivalent total byte budgets? Which base codec do you use? What quality levels are available for it? How do you distribute them among the images? Do you select the configuration based on reconstruction quality?

9 comments

[This is the first section of our rebuttal (1/5)]

We thank the reviewer for asking when learned structural transmission offers a useful system-level advantage. We could have explained the delivery requirements more clearly. A service may need an independently decodable low-resolution (LR) rendition so that clients with limited processing capacity or memory can decode and display it directly. Clients with super-resolution (SR) support can use that same decoded image for high-resolution (HR) reconstruction.

Under this delivery requirement, MetaSR + predictor improves PSNR-Y by 0.86 dB on RealSR100 over spending the entire budget on HEVC LR coding (paired 95% CI [0.73, 0.99] dB). Both pipelines deliver independently decodable HEVC LR and use the same SR receiver and per-image latent seeds. Total transmitted bytes match exactly across the dataset, including structural payload and signaling where applicable (0.1496 bpp, Table R2).

Under the original Table 1 JPEG protocol, MetaSR + predictor improves PSNR-Y over LR-only by 1.04 dB on RealSR100, 1.24 dB on DRealSR93, and 0.34 dB on DIV2K100, while using 1.36%, 0.32%, and 1.72% fewer total bytes, respectively. LPIPS and DISTS also decrease on all three datasets. The revised Table 1 includes the complete predictor results at their measured total rates.

A service can encode a 4K source at its native resolution and deliver a stream that decodes at 4K. It can also downsample the source before encoding to deliver a playable LR rendition under a tighter bandwidth budget. YouTube creates multiple playback resolutions from a single upload, and Apple's HLS authoring guide gives example HEVC tiers at 720p (2.4 Mbps) and 540p (0.6 Mbps). A lower delivered bitrate can also reduce volume-based CDN charges by reducing the data sent for the same viewing duration.

Consider a viewer with a 4K display who receives the 540p rendition of a 4K source. Reconstructing 3840×2160 from 960×540 requires 4× upscaling along each axis. Downsampling can merge strokes in small text or obscure thin contours. Spending more bits on LR coding can preserve the downsampled image more accurately. Spending some of those bits on source cues can tell the receiver where the strokes or contours lay in the original image.

Still-image services also deliver multiple renditions. The HTML srcset attribute lets browsers choose an image version suited to its display size and the device's pixel density. HTTP Archive found it on 42% of the mobile pages in its 2024 sample. MetaSR delivers a standard JPEG or HEVC bitstream for direct LR viewing and a separate packet of source cues for clients that perform SR. The predictor selects cues based on their estimated reconstruction benefit and measured coding costs.

When HR is the only required deliverable, direct HR coding is a strong choice. The submitted Appendix I already showed a PSNR-Y advantage for direct H.264 toward the upper end of its exploratory sweep. W1/Q1 extends that comparison to the complete MetaSR system and modern direct codecs. The revised introduction makes the delivery requirements explicit so that readers can interpret each comparison in its intended setting.

W1 and Q1: System Benefit at Actual Transmission Rates

Exact Total Byte Comparison

On RealSR100, both SR arms transmit exactly 490,368 bytes (0.1496 bpp) and use the same HEVC LR codec, receiver, and per-image latent seeds. Allocating part of this budget to selected structure improves PSNR-Y by 0.86 dB (95% CI [0.73, 0.99]) over spending the entire budget on the LR stream (Table R2). SSIM increases by 0.020, and DISTS decreases by 0.007 (95% CI [0.002, 0.013] for the decrease), a small reduction whose interval excludes zero. LPIPS changes by −0.002 (95% CI [−0.009, 0.005]), and MUSIQ decreases by 2.1 points (95% CI [0.9, 3.3] for the decrease).

We use libx265 intra coding with 4:4:4 sampling and one IDR picture. The LR-only arm assigns quality by measured file length, selecting QP 19 for 89 images, QP 20 for eight, and QP 21 for three to meet the dataset budget exactly.

Table R2. RealSR100 at a target of 490,368 transmitted bytes

MethodTotal bytesbppPSNR-Y (dB)SSIMLPIPSDISTSMUSIQ
LR-only HEVC490,3680.149626.210.7270.1700.18962.8
MetaSR + predictor490,3680.149627.070.7470.1680.18260.7
Direct DCAE490,3650.149637.730.9410.0970.13758.0

The two SR arms match exactly. DCAE uses 490,340 bitstream bytes and 25 bytes of checkpoint signaling (2 bits per image), for a total 3 bytes below the target. Displayed bpp values are rounded.

[To be continued in the next section]

[This is the second section of our rebuttal (2/5)]

Original JPEG Benchmark

The benefit also appears under the original Table 1 JPEG protocol. MetaSR + predictor improves PSNR-Y over LR-only by 1.04 dB on RealSR100 (95% CI [0.92, 1.17]) and 1.24 dB on DRealSR93 (95% CI [1.06, 1.43]), while using 1.36% and 0.32% fewer total bytes, respectively. Its measured rates are 0.5228 and 0.5283 bpp. Both LPIPS and DISTS decrease on each dataset. The totals include the LR stream, structural payload, and signaling.

Table R1. RealSR100 under the original JPEG protocol

MethodTotal bytesbppPSNR-Y (dB)SSIMLPIPSDISTSMUSIQ
LR-only1,736,7040.530026.290.7310.1670.18862.6
MetaSR with full edges1,736,7040.530027.640.7880.1340.16564.3
MetaSR + predictor1,713,0090.522827.340.7660.1480.17262.2

Table R1b. DRealSR93 under the original JPEG protocol

MethodTotal bytesbppPSNR-Y (dB)SSIMLPIPSDISTSMUSIQ
LR-only1,615,1350.530028.750.7610.1730.19253.9
MetaSR with full edges1,615,1350.530030.350.8060.1490.17653.5
MetaSR + predictor1,609,9220.528329.990.7930.1610.18252.1

Displayed PSNR-Y means are rounded. Paired PSNR-Y gains use unrounded per-image scores.

On RealSR100, the predictor is 1.84 dB ahead of AdcSR, the highest-PSNR-Y added SR baseline, and gives lower LPIPS and DISTS than ODTSR. Both baselines score higher on MUSIQ. On DRealSR93, the predictor uses the full-edge arm's JPEG level on each image and sends zero structure for 12 of 93 images. Relative to LR-only, MUSIQ decreases by 1.8 points (95% CI [0.4, 3.2] for the decrease). The full-edge arm achieves 0.36 dB higher PSNR-Y than the predictor (95% CI [0.24, 0.47]) at the rates in Table R1b.

The revised Table 1 has a MetaSR + predictor row with the RealSR100 and DRealSR93 results at their measured total rates. The row also covers DIV2K100, where the predictor uses 1.72% fewer total bytes than LR-only and improves PSNR-Y by 0.34 dB (95% CI [0.27, 0.40]), and the full-edge arm is 0.13 dB higher (95% CI [0.07, 0.20]).

When Selective Transmission Helps

Across the eight predictor operating points at LR QP 30 in Table 20 of the revised paper (Appendix S; caps of 12.5% to 100%), the predictor gains 0.31–1.30 dB on RealSR100 and 0.51–1.57 dB on DRealSR93 over the nearest measured LR-only points using at least as many total bits. With the same QP 30 LR base, selective transmission at the 25% cap uses 0.106 bpp on RealSR100 and 0.065 bpp on DRealSR93, below the full-edge rates of 0.1836 and 0.0906 bpp. At those selective points, PSNR-Y exceeds that of the slightly higher-rate LR-only controls by 0.56 and 0.86 dB, respectively. LPIPS and DISTS changes are mixed.

Table R3. Measured operating points at the 25% HEVC metadata cap

DatasetPredictor bppLR-only bppFull edges same base bppPSNR-Y gain (dB)LPIPS and DISTS changes
RealSR1000.1060.1110.1836+0.56+0.005 / −0.003
DRealSR930.0650.0710.0906+0.86+0.005 / +0.002

Changes compare the predictor with the nearest tested LR-only point at an equal or higher total rate. The cap uses the per-image cost of full-resolution sparse tile coding, including 40 signaling bits per tile. The full-edge arm uses a whole-image MEG1 packet.

At the four measured full-edge rates, full edges give 0.10–0.26 dB higher PSNR-Y and lower LPIPS and DISTS than the predictor curve interpolated between bracketing measurements with the same decoded LR base. On the tested bases, selection provides structural guidance at tighter rates, and the complete map gives higher fidelity at these four matched operating points.

Table R3b. Full edges and the predictor at matched dataset rates

DatasetLR QPTotal bppFull-edge PSNR-Y (dB)Predictor PSNR-Y interpolated (dB)Full-edge gain (dB) [95% CI]
RealSR300.183627.4327.33+0.10 [0.01, 0.18]
RealSR360.158227.2027.05+0.15 [0.06, 0.23]
DRealSR300.090630.1529.90+0.25 [0.14, 0.36]
DRealSR360.073329.8629.60+0.26 [0.15, 0.37]

[To be continued in the next section]

[This is the third section of our rebuttal (3/5)]

Scope Relative to Direct HR Coding

The submitted Appendix I (Fig. 9) already shows direct H.264 overtaking the earlier lossy coarse-to-fine MetaSR configuration at higher rates in the exploratory sweep. The new comparisons extend that analysis to MetaSR with the learned selector and to modern direct HR codecs. Over the overlapping measured rate ranges, direct HEVC and VVC achieve 4.9–10.2 dB and 6.1–11.6 dB higher PSNR-Y than MetaSR on RealSR100 (0.062–0.227 bpp), and 5.8–9.1 dB and 6.9–10.3 dB higher on DRealSR93 (0.043–0.126 bpp).

The perceptual metrics show a trade-off at a measured predictor point. On DRealSR93, MetaSR + predictor at the 12.5% cap (0.0536 bpp) has LPIPS 0.188 and DISTS 0.202. Direct HEVC, interpolated at the same rate between its measured QP 48 and QP 45 points, gives about 0.218 and 0.212, while its PSNR-Y is higher (about 35.3 vs. 29.13 dB). Direct VVC at that rate has about the same LPIPS (0.186) and a DISTS of 0.205.

Table 20 also contains a direct HEVC point on RealSR100 at 0.5300 bpp, which uses exactly the Table 1 byte total (1,736,704 bytes). It reaches 42.88 dB PSNR-Y, LPIPS 0.036, and DISTS 0.056. At 0.5228 bpp, the JPEG-based MetaSR + predictor of Table R1 reaches 27.34 dB, 0.148, and 0.172; only MUSIQ favors MetaSR (62.2 vs. 57.4). This near-rate pair lies at the Table 1 operating point, not in the low-rate window of the submitted Appendix I (about 0.05 bpp).

All rates are measured total transmitted rates. Figure 14 and Tables 20 and 22 of the revised paper (Appendix S, pp. 41–44) give the curves, every measured point, these ranges, and the LPIPS and DISTS advantage intervals, where present. The revised manuscript reports them in Appendix I (Figure 9, p. 27; Table 11, p. 28).

At the near-matched point in Table R2, direct DCAE achieves 37.73 dB versus 27.07 dB for MetaSR + predictor, a difference of 10.66 dB (95% CI [10.23, 11.11]). DCAE has lower LPIPS and DISTS, while MetaSR has higher MUSIQ (60.7 vs. 58.0).

StableCodec was run with its ft2, ft3, and ft4 checkpoints on all 100 RealSR images and all 93 DRealSR images (Table 23 of the revised paper). On RealSR100, the highest of these full-set rates is 0.0347 bpp, where it reaches 27.30 dB with LPIPS 0.088 and DISTS 0.111. On DRealSR93, the three checkpoints lie at 0.0237–0.0326 bpp and reach 30.89–31.77 dB, with LPIPS 0.076–0.086, DISTS 0.099–0.110, and MUSIQ 52.3–53.0. Each of them is ahead of the DRealSR93 MetaSR + predictor points in the original HEVC sweep (Table 20) in all four metrics; the best of those points reaches 30.28 dB at 0.1257 bpp. These full-set points lie below the predictor sweeps and serve as low-rate references. For the remaining released weights we measured only the rate, on a fixed sample of the first eight RealSR100 images. On that sample, the stage-1 base weight gives the highest rate, 0.0508 bpp, and the other inference checkpoints (ft6 to ft32) give lower rates than ft2. The lowest measured MetaSR + predictor point is 0.0809 bpp on RealSR100 and 0.0482 bpp on DRealSR93, so neither the full-set points nor the sampled rates reach a rate in common with MetaSR + predictor.

W2 and Q2: Component Contributions and Related Work

We agree with the reviewer that score reuse across budgets follows from the predictor's design: the region–action scores do not depend on the cap, so one predictor pass serves every budget, and reuse alone does not show that the resulting decisions are good. We therefore test decision quality directly. With the same candidate pool, receiver, and optimizer, learned selection improves PSNR-Y over random selection by 0.46 dB (95% CI [0.27, 0.67]) in 73 pairs from 43 sources whose realized metadata bits are exactly identical. The pairs come from five caps outside the training caps of 25%, 50%, and 75%, and each pair also shares its LR input and latent seed. Table R4 lists the control for each component of MetaSR; each row changes one component and holds the others fixed.

[To be continued in the next section]

Reviewer_8Es1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MetaSR, a source-assisted image super-resolution framework. The sender provides structural information extracted from the high-resolution source to help the receiver reconstruct details more faithfully. The authors also introduce a lightweight predictor to select which structural information to transmit under different budgets. Experiments examine the benefits of structural guidance, the effectiveness of learned selection, and its use at different transmission budgets.

Critical strengths
  1. The paper addresses a meaningful problem in generative super-resolution: visually plausible details may not match the original image. Using source structure to improve fidelity is a reasonable direction.
  2. The structural fusion comparison in Table 2 is useful. The proposed fusion design performs well across the three evaluated datasets.
Critical weaknesses

The main comparisons report full-reference metrics, including PSNR, SSIM, LPIPS, and DISTS. While these assess reconstruction fidelity and perceptual similarity to the reference, they do not fully establish the perceived quality or naturalness of the generated images. It would be helpful to complement them with no-reference quality metrics, such as MUSIQ, MANIQA, or CLIP-IQA, and representative visual comparisons.

The comparison could also include recent generative SR methods, such as OMGSR, ODTSR, DiT4SR. One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution [ODTSR] OMGSR: You Only Need One Mid-timestep Guidance for Real-World Image Super-Resolution [OMGSR] DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution

The Main Experiments Do Not Evaluate the Complete Proposed Method. My main concern is the connection between the paper's central contribution and its main results. Learning which structure to transmit is presented as an important contribution. However, Table 1 uses complete edge maps without the predictor. The predictor is mainly evaluated on a separate set of 80 images. Therefore, the main benchmark demonstrates that additional structural information helps reconstruction, but does not show whether the proposed selection method improves the complete system on those datasets. It would be helpful to include MetaSR + predictor in Table 1 and compare it with complete-edge transmission at several transmission budgets. This would make the practical benefit of learning what to preserve much clearer.

The Contribution of Utility Learning Needs Clearer Explanation. The predictor learns from the reconstruction quality difference between an edge-guided result and a result without edges. This is an understandable training target. However, I find it difficult to identify what is methodologically new about this formulation beyond learning to predict which structural choices improve reconstruction. The regression-only ablation is helpful and shows that the combined training losses improve the results. Nevertheless, this supports the training design more directly than the broader claim about counterfactual utility. Could the authors explain more clearly which part constitutes the main methodological contribution and why it is necessary? The current presentation makes this contribution appear stronger than the evidence establishes.

Questions for authors

The Evidence for Faithfully Preserving Source Details Is Still Limited. The allocation experiments mainly assess improvements in whole-image PSNR. However, the motivation emphasizes preserving thin structures, repeated contours, and local geometry. An improvement in average PSNR does not necessarily show that these specific details are reconstructed more faithfully. Figure 4 provides useful examples, but only covers two sources under additional edge-density constraints. More representative comparisons, including unsuccessful cases, would help establish whether learned selection consistently preserves the structures highlighted in the motivation. A structural evaluation or a more systematic visual analysis would strengthen this part of the paper.

The efficiency discussion should distinguish the predictor's running time from the complete sender processing time, since the supplementary results show a substantial difference between them.

12 comments

This is the first section of our rebuttal (1/5)

We thank the reviewer for pointing out that the main benchmark should evaluate MetaSR with its learned selector and compare it with full-edge transmission across budgets. Under the original Table 1 JPEG protocol, MetaSR + predictor improves PSNR-Y over LR-only by 1.04 dB on RealSR100, 1.24 dB on DRealSR93, and 0.34 dB on DIV2K100, while using 1.36%, 0.32%, and 1.72% fewer total bytes, respectively (Table R2). The revised Table 1 includes the complete system and its measured rates.

The HEVC sweep shows where selection is useful. With the QP 30 LR base fixed, selective transmission operates at 0.1063 bpp on RealSR100 and 0.0655 bpp on DRealSR93, compared with 0.1836 and 0.0906 bpp for full-edge transmission. At these tighter rates, selection improves PSNR-Y by 0.56 and 0.86 dB over the nearest uniform-QP LR-only points in the sweep, which use slightly higher rates (Table R3(a)). At the four measured full-edge rates, the full map gives 0.10–0.26 dB higher PSNR-Y than the predictor curve interpolated from measurements with the same decoded LR base (Table R3(b)).

1. Naturalness and Representative Visual Comparisons

With the same receiver at a dataset-level total rate of 0.5300 bpp, full-edge guidance raises PSNR-Y on all three datasets and improves MUSIQ, MANIQA, and CLIP-IQA on DIV2K and RealSR (Table R1). DRealSR shows the trade-off between fidelity and the no-reference scores: full edges raise PSNR-Y from 28.75 to 30.35 dB, while MUSIQ decreases from 53.87 to 53.55 and CLIP-IQA from 0.49 to 0.48, and MANIQA is 0.32 in both arms at the reported precision. MetaSR transmits source structure so that the receiver reconstructs the strokes and contours of the HR image rather than plausible substitutes. We therefore report naturalness as a separate dimension next to fidelity.

Table R1 also reports the complete system at its measured total rate, scored with the same no-reference models. On RealSR, the paired 95% confidence interval of the difference from LR-only includes zero for each metric: MUSIQ −0.39 [−1.35, 0.63], MANIQA −0.001 [−0.009, 0.007], and CLIP-IQA +0.003 [−0.013, 0.019]. On DIV2K, it raises MANIQA from 0.36 to 0.38 and CLIP-IQA from 0.58 to 0.60, with MUSIQ unchanged within its interval. On DRealSR, where its PSNR-Y gain is largest (+1.24 dB), MUSIQ decreases by 1.78 points (95% CI [0.39, 3.15]), MANIQA by 0.012, and CLIP-IQA by 0.026. Its no-reference scores are below those of full edges on all three datasets. The revised appendix includes all three no-reference measures for the original benchmark (Table A1; Table 25 of the revised paper).

Table R1. Fidelity and no-reference quality under the original JPEG protocol

DatasetConditionTotal bppPSNR-Y (dB)MUSIQMANIQACLIP-IQA
DIV2K100LR-only0.530027.4666.220.360.58
Full edges0.530027.9368.020.390.62
MetaSR + predictor0.520927.7966.150.380.60
RealSR100LR-only0.530026.2962.630.370.48
Full edges0.530027.6464.260.390.50
MetaSR + predictor0.522827.3462.240.370.48
DRealSR93LR-only0.530028.7553.870.320.49
Full edges0.530030.3553.550.320.48
MetaSR + predictor0.528329.9952.080.310.46

Higher is better for the three no-reference measures. All rows use the same PyIQA models (MUSIQ and MANIQA with KonIQ-10k weights, CLIP-IQA with RN50) on the full output images. The predictor rows score the outputs of Table R2.

Figures 10 and 11 of the revised paper (Appendix R, pp. 36–37) show whole images and aligned crops for the first random draw and one low-gain case from each of RealSR and DRealSR. In the RealSR random draw (Figure 10, top), the LR-only reconstruction renders the small emblem in the crop as a triangle, whereas with full-edge guidance the emblem's loop and horizontal base bar reappear as in the HR image. Whole-image PSNR-Y increases from 25.01 to 27.49 dB and MUSIQ from 66.7 to 72.5, at 0.5235 versus 0.7055 bpp for LR-only. Figure 12 (p. 38) links this difference to the transmitted structure: with the same decoded LR input, receiver, and latent seed, MetaSR + predictor raises the PSNR-Y of the emblem crop from 23.66 to 26.18 dB. Each panel reports its measured per-image rate. Table 19 (p. 35) places these cases among all test images: with full edges, MetaSR has higher PSNR-Y than ODTSR on 98 of 100 RealSR100 images and 86 of 93 DRealSR93 images, and lower LPIPS on 85 and 65 of them, while ODTSR has higher MUSIQ on 74 and 82 of them.

[To be continued in the next section]

[This is the second section of our rebuttal (2/5)]

2. Recent Generative SR Baselines

Table A2 adds OMGSR, ODTSR, DiT4SR-f/q, and AdcSR on the decoded LR-only JPEGs used in the original benchmark. The captioned DiT4SR runs use the authors' LLaVA pipeline on those inputs. Compared with the highest-PSNR-Y added baseline on each dataset, MetaSR + predictor gains 1.84 dB over AdcSR on RealSR100 and 1.69 dB over ODTSR on DRealSR93, at slightly lower total rates. It also gives lower LPIPS and DISTS than ODTSR on both datasets. The added baselines score higher on MUSIQ, while the reference-based results favor MetaSR at these operating points.

3. Complete System in the Main Benchmark and Across Budgets

Under the original JPEG protocol, the paired PSNR-Y gains of MetaSR + predictor over LR-only are 1.04 dB on RealSR100 (95% CI [0.92, 1.17]), 1.24 dB on DRealSR93 (95% CI [1.06, 1.43]), and 0.34 dB on DIV2K100 (95% CI [0.27, 0.40]). The complete system uses 1.36%, 0.32%, and 1.72% fewer total bits, respectively, and lowers LPIPS and DISTS on all three datasets. Total rate includes the LR stream, metadata, and signaling. The revised Table 1 has a MetaSR + predictor row with the results and measured rates for all three datasets.

Table R2. Complete system under the original JPEG protocol

Dataset and conditionTotal bytesbppPSNR-Y (dB)SSIMLPIPSDISTSMUSIQ
RealSR100 LR-only1,736,7040.530026.290.7310.1670.18862.6
RealSR100 full edges1,736,7040.530027.640.7880.1340.16564.3
RealSR100 + predictor1,713,0090.522827.340.7660.1480.17262.2
DRealSR93 LR-only1,615,1350.530028.750.7610.1730.19253.9
DRealSR93 full edges1,615,1350.530030.350.8060.1490.17653.5
DRealSR93 + predictor1,609,9220.528329.990.7930.1610.18252.1
DIV2K100 LR-only18,742,6020.530027.460.7310.1350.09466.2
DIV2K100 full edges18,742,6020.530027.930.7540.1120.07668.0
DIV2K100 + predictor18,420,4890.520927.790.7400.1260.08266.2

Displayed PSNR-Y means are rounded. Paired PSNR-Y gains use unrounded per-image scores.

With the same QP 30 HEVC LR base, Table R3(a) lists every measured predictor point whose total rate is below that of full-edge transmission, each paired with the nearest uniform-QP LR-only point in the sweep (one HEVC QP for every image) at an equal or higher total rate. At the 25% cap, selective transmission uses 0.1063 bpp versus 0.1836 bpp for full edges on RealSR100, and 0.0655 versus 0.0906 bpp on DRealSR93, and improves PSNR-Y by 0.56 and 0.86 dB over LR-only points at 0.1112 and 0.0714 bpp. Across the listed caps, the gain rises from 0.31 to 0.99 dB on RealSR100 and from 0.51 to 1.18 dB on DRealSR93, and every 95% interval lies above zero. At these caps, LPIPS and DISTS change by at most 0.008 in either direction, while the no-reference scores are lower on average, by 1.6–4.3 points in MUSIQ, 0.011–0.036 in MANIQA, and 0.011–0.067 in CLIP-IQA, the trade-off discussed in Section 1. Each metadata cap is a fraction of the summed per-image cost of coding all tiles in full-resolution sparse mode, including 40 signaling bits per tile. The full-edge arm uses a whole-image MEG1 packet.

At the four measured full-edge rates, full edges give 0.10–0.26 dB higher PSNR-Y, 0.012–0.015 lower LPIPS, and 0.006–0.008 lower DISTS than the predictor curve interpolated from bracketing measurements with the same LR base, with every interval excluding zero (Table R3(b)). The sweep identifies rates where selective transmission makes structural guidance available under a tighter budget and rates where sending the complete map gives higher fidelity.

[To be continued in the next section]

[This is the third section of our rebuttal (3/5)]

Table R3. Selective and full-edge transmission on the same HEVC LR base

(a) Measured operating points below the full-edge rate (LR base QP 30)

Structure sentTotal bppPSNR-Y (dB)Uniform-QP LR-only pair QP (bpp)ΔPSNR-Y [95% CI]ΔLPIPSΔDISTSΔMUSIQΔMANIQAΔCLIP-IQA
RealSR100
Predictor 12.5% cap0.083726.4226 (0.0872)+0.31 [0.21, 0.40]+0.007+0.001−2.5−0.021−0.033
Predictor 25% cap0.106326.7223 (0.1112)+0.56 [0.44, 0.68]+0.005−0.003−2.8−0.024−0.029
Predictor 37.5% cap0.128326.9321 (0.1310)+0.73 [0.61, 0.86]0.000−0.006−2.7−0.024−0.024
Predictor 50% cap0.149627.0719 (0.1519)+0.86 [0.73, 0.99]−0.002−0.007−2.1−0.015−0.017
Predictor 62.5% cap0.170627.2315 (0.2049)+0.99 [0.85, 1.13]−0.005−0.008−1.6−0.011−0.011
Full edges0.183627.4315 (0.2049)+1.19 [1.02, 1.36]−0.019−0.015+0.3+0.005+0.013
DRealSR93
Predictor 12.5% cap0.053629.1327 (0.0558)+0.51 [0.38, 0.65]+0.004+0.002−3.2−0.026−0.048
Predictor 25% cap0.065529.5024 (0.0714)+0.86 [0.69, 1.05]+0.005+0.002−4.3−0.035−0.067
Predictor 37.5% cap0.076729.6723 (0.0774)+1.05 [0.88, 1.24]+0.004+0.002−4.2−0.036−0.065
Predictor 50% cap0.087829.8621 (0.0910)+1.18 [0.99, 1.38]+0.004+0.003−3.7−0.030−0.056
Full edges0.090630.1521 (0.0910)+1.47 [1.23, 1.72]−0.009−0.006−1.8−0.014−0.033

All points are measured. Δ = point minus its LR-only pair, the nearest uniform-QP HEVC LR-only point in the sweep at an equal or higher total rate; brackets are 95% paired source-bootstrap intervals over these fixed-QP pairs (10,000 replicates). MUSIQ, MANIQA, and CLIP-IQA use the same models as Table R1. The exactly-equal-byte LR-only control with per-image QP in our response to Reviewer 8kof (Table R2 there) is a separate comparison: at the RealSR100 50% cap it gives +0.859 dB [0.732, 0.992], against +0.858 dB [0.732, 0.993] for the fixed QP 19 pair here (0.1519 bpp), because 89 of its 100 images also use QP 19.

(b) Full edges and the predictor at the full-edge rates (predictor interpolated)

The three Δ columns give full edges minus interpolated predictor [95% CI].

DatasetLR QPTotal bppFull-edge PSNR-Y (dB)Predictor PSNR-Y (dB)ΔPSNR-Y (dB)ΔLPIPSΔDISTS
RealSR300.183627.4327.33+0.10 [0.01, 0.18]−0.012 [−0.017, −0.008]−0.006 [−0.009, −0.004]
RealSR360.158227.2027.05+0.15 [0.06, 0.23]−0.015 [−0.020, −0.011]−0.007 [−0.009, −0.004]
DRealSR300.090630.1529.90+0.25 [0.14, 0.36]−0.012 [−0.017, −0.008]−0.008 [−0.012, −0.005]
DRealSR360.073329.8629.60+0.26 [0.15, 0.37]−0.015 [−0.020, −0.011]−0.008 [−0.012, −0.005]

Full-edge points are measured. Predictor values are interpolated, per metric and per image, between the two measured predictor points with the same decoded LR base that bracket the full-edge rate. Negative ΔLPIPS and ΔDISTS favour full edges. Brackets are 95% paired source-bootstrap intervals (10,000 replicates).

[To be continued in the next section]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.