- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes MetaSR, a source-assisted generative super-resolution framework. The sender extracts structural metadata from the HR source, while a learned utility predictor selects which regional edge information to transmit under a bitrate budget. Experiments evaluate reconstruction quality, structural fusion, allocation policies, and generalization to unseen metadata budgets.
Critical strengths
The paper introduces MetaSR, a source-assisted framework that learns how to use structural metadata and which structure to transmit under a budget.
Critical weaknesses
-
Since the sender has access to the complete HR image, the task is closer to source-assisted compression than conventional super-resolution.
-
Direct HR coding is compared only with H.264 in an exploratory experiment. Modern learned codecs and more advanced conventional codecs are missing.
-
The allocation methods transmit different actual numbers of bits. Therefore, the benefits of the selection strategy are not fully isolated from bitrate differences.
-
The qualitative images exhibit strong generative reconstruction effects, but the paper provides insufficient evidence that they are direct and unselectively presented outputs of the reported pipeline.
-
The qualitative evaluation includes only a few selected regions, without random examples, full resolution results, error maps, original output files, or complete reproduction scripts.
Questions for authors
-
Can the authors provide unedited full-resolution outputs and their corresponding fixed random seeds?
-
Can the authors provide randomly selected qualitative examples rather than only manually selected local crops?
-
Are all displayed results direct outputs of the corresponding models, without manual modification, regional replacement, or additional generative processing?
-
How does MetaSR compare with modern learned image codecs at exactly matched total bitrates?
5.How can the authors demonstrate that the recovered details originate from the transmitted structural metadata rather than the generative prior?
6 comments
[This is the second section of our rebuttal (2/5)]
Under the original JPEG protocol, MetaSR improves mean PSNR-Y from 26.29 to 27.34 dB while using 1.36% fewer total bits than LR-only (Table M1 and revised Appendix N, Table 17).
Table M1. RealSR100 at the original Table 1 JPEG operating point
| Condition | Total bytes | bpp | PSNR-Y ↑ | SSIM ↑ | LPIPS ↓ | DISTS ↓ | MUSIQ ↑ |
|---|---|---|---|---|---|---|---|
| LR-only | 1,736,704 | 0.5300 | 26.29 | 0.731 | 0.167 | 0.188 | 62.6 |
| Full edges | 1,736,704 | 0.5300 | 27.64 | 0.788 | 0.134 | 0.165 | 64.3 |
| MetaSR + predictor | 1,713,009 | 0.5228 | 27.34 | 0.766 | 0.148 | 0.172 | 62.2 |
Displayed PSNR-Y means are rounded.
W3 + Q5: Benefits of learned selection and aligned structure
W3. Learning which structure to send improves fidelity at equal realized rates. Across 73 exact bit-matched pairs from 43 sources, learned allocation gains 0.46 dB PSNR-Y over random allocation (95% CI [0.27, 0.67]). We compare the assignments on 80 held-out sources (40 LSDIR and 40 DIV8K), using the same decoded LR, receiver, latent seed, candidate bank, and coded-cost optimizer. Exact matches in metadata and signaling bits give equal total rates.
Allowing random to use up to 1% fewer bits yields 0.45 dB over 268 pairs from 72 sources (95% CI [0.33, 0.57]). A one-sided 3% check gives the same mean gain. We bootstrap sources after averaging their paired gains. Revised Appendix J (Table 14) reports coverage and bit gaps by cap.
Table M3. Learned minus random across five unseen budgets
| Actual-bit pairing | Pairs | Sources | ΔPSNR-Y (dB) | 95% CI (dB) |
|---|---|---|---|---|
| Exact metadata bit match | 73 | 43 | +0.46 | [+0.27, +0.67] |
| Random ≤ learned, within 1% | 268 | 72 | +0.45 | [+0.33, +0.57] |
Q5. The received structure guides geometry when it is aligned with the source. With decoded LR, receiver, and latent seed fixed on the same 80 sources, aligned learned structure adds 0.915 dB PSNR-Y over no edge (95% CI [0.760, 1.077]). We intervene after metadata decoding and hold the transmitted byte count fixed within each metadata arm. A (64, 64)-pixel shift, a tile shuffle, and an edge map from another source each bring mean PSNR-Y below no edge in both the learned and full-edge arms (Table M6).
Table M6. Structure interventions on held-out sources
| Decoded condition | PSNR-Y (dB) | Δ vs. aligned (95% CI) | Δ vs. no edge (95% CI) |
|---|---|---|---|
| No edge | 24.070 | — | Reference |
| Learned 50%, aligned | 24.985 | Reference | +0.915 [0.760, 1.077] |
| Learned 50%, shifted | 23.548 | −1.437 [−1.618, −1.263] | −0.522 [−0.656, −0.373] |
| Learned 50%, shuffled | 23.555 | −1.431 [−1.614, −1.250] | −0.515 [−0.657, −0.361] |
| Learned 50%, other source | 23.168 | −1.817 [−2.231, −1.459] | −0.902 [−1.317, −0.565] |
| Full edge, aligned | 25.186 | Reference | +1.115 [0.887, 1.358] |
| Full edge, shifted | 23.269 | −1.917 [−2.205, −1.646] | −0.801 [−0.932, −0.667] |
| Full edge, shuffled | 23.253 | −1.932 [−2.230, −1.656] | −0.817 [−0.950, −0.686] |
| Full edge, other source | 22.776 | −2.409 [−2.903, −1.965] | −1.294 [−1.771, −0.890] |
Fig. 19 (p. 55) shows the case with median aligned gain among examples with a nonempty donor raster. Shift and shuffle preserve the aligned raster's edge-pixel count. With LR fixed, learned metadata improves reference line-segment recall by 1.7 percentage points on Urban100 and 1.9 points on DIV2K. The receiver uses the spatial structure for geometry and synthesizes texture from the LR image and its learned prior.
[To be continued in the next section]
[This is the third section of our rebuttal (3/5)]
W2 + Q4: Direct HR coding and measured quality trade-offs
W2. Perceptual trade-off at the measured predictor points. Table M10 compares the eight measured MetaSR + predictor points (LR QP 30, caps of 12.5% to 100%) with direct HEVC and VVC at the same total rate. Direct coding has higher PSNR-Y and SSIM at all eight points on both datasets (Table M2), and lower LPIPS and DISTS at most of them. On RealSR100, the predictor has lower LPIPS than direct HEVC only at the 12.5% cap (0.184 vs. 0.187 at 0.0837 bpp); at the 100% cap the values are 0.153 vs. 0.083 at 0.2270 bpp, and its DISTS is higher at all eight caps. On DRealSR93, the predictor has lower LPIPS than direct HEVC at the two lowest caps (0.188 vs. 0.218 at 0.0536 bpp and 0.185 vs. 0.192 at 0.0655 bpp) and lower DISTS at the lowest cap. Against direct VVC, its LPIPS is higher at all eight points on both datasets.
Along the interpolated MetaSR curve, which starts at the LR-only point, LPIPS is lower than direct HEVC on RealSR at approximately 0.062–0.087 bpp and DISTS at 0.062–0.077 bpp; on DRealSR, LPIPS is lower than direct VVC at approximately 0.043–0.053 bpp and DISTS at 0.043–0.056 bpp. These intervals use interpolation within measured rate support and end at or before the lowest predictor caps.
MetaSR + predictor is higher than both direct codecs on the no-reference metrics MUSIQ, MANIQA, and CLIPIQA at all eight points on both datasets (Table M10). On RealSR100, learned structure allocation gives 60.7 on MUSIQ at 0.1496 bpp, compared with 52.2 for direct HEVC at 0.1500 bpp. The margins are smallest at the highest caps; at the 100% cap on DRealSR93, MANIQA is 0.485 vs. 0.481 for direct VVC.
Table M10. MetaSR + predictor against direct HR coding at the eight measured predictor points (LR QP 30)
| Metric | RealSR100 vs. HEVC | RealSR100 vs. VVC | DRealSR93 vs. HEVC | DRealSR93 vs. VVC |
|---|---|---|---|---|
| PSNR-Y (dB) | 0/8; −10.2 to −5.9 | 0/8; −11.6 to −7.2 | 0/8; −9.1 to −6.1 | 0/8; −10.3 to −7.4 |
| SSIM | 0/8; −0.162 to −0.130 | 0/8; −0.182 to −0.162 | 0/8; −0.126 to −0.103 | 0/8; −0.144 to −0.130 |
| LPIPS | 1/8; −0.071 to +0.004 | 0/8; −0.080 to −0.033 | 2/8; −0.048 to +0.030 | 0/8; −0.055 to −0.002 |
| DISTS | 0/8; −0.068 to −0.005 | 0/8; −0.070 to −0.019 | 1/8; −0.050 to +0.010 | 1/8; −0.048 to +0.002 |
| MUSIQ | 8/8; +7.8 to +12.3 | 8/8; +5.9 to +9.4 | 8/8; +4.4 to +8.7 | 8/8; +3.1 to +6.6 |
| MANIQA | 8/8; +0.026 to +0.114 | 8/8; +0.021 to +0.085 | 8/8; +0.010 to +0.099 | 8/8; +0.004 to +0.076 |
| CLIPIQA | 8/8; +0.20 to +0.23 | 8/8; +0.13 to +0.18 | 8/8; +0.18 to +0.19 | 8/8; +0.09 to +0.15 |
Each cell gives the number of the eight predictor points at which MetaSR + predictor is better, followed by the range of its margin over the direct codec; a positive margin means that MetaSR + predictor is better. The codec value is interpolated linearly in bpp between its two neighboring measured points at the predictor's rate. The predictor rates are 0.084–0.227 bpp on RealSR100 and 0.054–0.126 bpp on DRealSR93.
Table M2. Direct HR coding PSNR-Y advantage over MetaSR
| Dataset | Evaluated overlap (bpp) | Direct HEVC (dB) | Direct VVC (dB) |
|---|---|---|---|
| RealSR100 | 0.062–0.227 | +4.9 to +10.2 | +6.1 to +11.6 |
| DRealSR93 | 0.043–0.126 | +5.8 to +9.1 | +6.9 to +10.3 |
Revised Appendix I gives the direct HR comparisons in Fig. 9 and Tables 11 and 13. The reported StableCodec results on RealSR cover 0.0256–0.0347 bpp and 26.62–27.30 dB PSNR-Y. Those rates lie below the evaluated HEVC-base MetaSR range.
[To be continued in the next section]
[This is the first section of our rebuttal (1/5)]
We thank the reviewer for asking what transmitting source structure adds when the sender already has the high-resolution (HR) image. We could have made the delivery requirements clearer. When HR is the only required output, encoding the source at its native resolution is a strong choice, as the submitted Appendix I and the new comparisons in W2/Q4 illustrate. A service may also need to deliver an independently decodable low-resolution (LR) rendition. Receivers with limited processing capacity or memory can decode and display it directly, while receivers with super-resolution (SR) support can use that same decoded image to reconstruct at HR. On RealSR100, allocating part of the same total transmission budget to selected source structure improves PSNR-Y by 0.86 dB over spending the entire budget on HEVC LR coding (W1). Both pipelines deliver independently decodable HEVC LR, use the same SR receiver and per-image latent seeds, and match total transmitted bytes exactly across the dataset, counting metadata and signaling where applicable (0.1496 bpp).
Downsampling before encoding lets a service offer a playable rendition under a tighter bandwidth budget. YouTube creates multiple playback resolutions from a single upload, and Apple's HLS authoring guide gives example HEVC renditions at 720p (2.4 Mbps) and 540p (0.6 Mbps). A viewer with a 4K screen may receive the 540p rendition of a 4K source when bandwidth is limited. The LR rendition is usable on its own, while reconstructing 3840×2160 from 960×540 requires 4× upscaling along each axis. Downsampling can merge strokes in small text or obscure thin contours, leaving their original shapes ambiguous in the decoded image. The sender can provide cues about those shapes directly from the HR source.
Still-image services also deliver multiple renditions. The HTML
srcsetattribute lets browsers choose an image version suited to its display size and the device's pixel density. HTTP Archive found it on 42% of the mobile pages in its 2024 sample.MetaSR uses a standard JPEG or HEVC bitstream for the LR rendition and sends selected source cues in a separate packet for receivers that perform SR. In a separate experiment with exactly matched metadata and signaling bits, learned selection improves PSNR-Y by 0.46 dB over random selection (W3). The revised manuscript makes the LR delivery requirement explicit and distinguishes the results for this setting from the HR-only coding comparisons.
All page numbers refer to the revised paper. Figs. 15–19 are in its Appendix U (pp. 51–55); the review package calls them Figs. M1–M5. Submitted figures use the original manuscript numbering. Fig. 9 refers to the revised manuscript.
W1: Source-assisted gains at a fixed total rate
The total transmission budget includes the coded LR image, structural metadata, and signaling (revised Introduction and Section 4.3, Table 3). In the matched-byte RealSR100 test, both arms transmit exactly 490,368 bytes (0.1496 bpp), and the LR-only arm selects one QP per image by measured code length. Both arms use the same receiver and inference seeds. Selected structure raises PSNR-Y from 26.21 to 27.07 dB, a paired gain of 0.86 dB (95% CI [0.73, 0.99] dB). SSIM is 0.020 higher and DISTS 0.007 lower; LPIPS changes by −0.002, with an interval that includes zero, and MUSIQ is 2.1 points lower (Table M9).
Table M9. RealSR100 at exactly 490,368 transmitted bytes
Total bytes include the LR stream, structural metadata, and signaling. Differences are MetaSR + predictor minus LR-only, paired by image.
Over the HEVC sweep with the LR base at QP 30, the gains of the eight predictor caps against the nearest measured LR-only points at equal or higher total rates are 0.31–1.30 dB on RealSR and 0.51–1.57 dB on DRealSR.
[To be continued in the next section]