acceptodds
Under review as a conference paper at ICLR 2027

AMID: Attribution Matching and Inverse-Network Distillation for Heterogeneous Super-Resolution

Abstract

Distilling image super-resolution (SR) models across heterogeneous architectures is challenging because their hidden representations are not directly comparable, while output matching alone does not transfer how the teacher exploits low-resolution evidence. We propose Attribution Matching and Inverse-Network Distillation (AMID), which uses teacher outputs and output gradients without accessing teacher intermediate features. Global Attribution Map Distillation (GAMD) matches integrated-gradient attribution maps of guidance-weighted structural responses, transferring the teacher's low-resolution pixel utilization and stabilizing reconstruction fidelity. Inverse Self-Consistency Distillation (ISCD) passes teacher and student outputs through a frozen inverse network architecturally isomorphic to the student, enabling multi-layer contrastive alignment in a shared, degradation-aware proxy space. An inverse reconstruction constraint further enforces degradation consistency. Experiments with five heterogeneous teachers across five benchmarks demonstrate improvements over representative distillation methods at , , and . At with a SwinIR teacher, AMID outperforms all compared baselines across all eight metrics and improves MUSIQ by over the strongest competing method. All auxiliary components are used only during training. AMID preserves the student's architecture and inference cost, making it practical for deployment-oriented heterogeneous SR distillation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Super-resolution, Heterogeneous distillation, Attribution Matching

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_9UNYOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This study proposes a novel method for distilling a super-resolution teacher model into a student model without accessing the teacher's hidden states. The proposed method, AMID, consists of two novel components, GAMD and ISCD. GAMD conveys to the student how the teacher model utilizes the input image, while ISCD uses an invertible model that is isomorphic to the student in the reverse direction to mitigate representation-level mismatch. The effectiveness of the proposed method is demonstrated through comparisons with multiple prior methods across several models and datasets. Sufficient ablation studies also demonstrate the importance of each component.

Critical strengths
  1. The method consistently outperforms multiple prior methods across several models and datasets. Its computational cost is also lower than that of competing methods.

  2. The individual components of the method are novel, and the methodology is convincing. The analysis showing that GAMD acts as a fidelity regularizer while ISCD contributes to perceptual improvement is also interesting.

  3. ISCD, in particular, is a generalizable approach and may have some impact on other domains as well.

Critical weaknesses
  1. The method involves a somewhat large number of hyperparameters. However, the chosen hyperparameter values appear sufficiently robust across different teacher models, so this is not a major weakness.
Questions for authors
  1. Tables 16 and 17 suggest that the inverse reconstruction loss plays a meaningful role. How do the authors justify not using the inverse reconstruction loss in the VGG control?
3 comments

We thank the reviewer for the time and patience devoted to helping us improve this work.

W1. Number of hyperparameters

We agree that AMID uses a somewhat large number of hyperparameters, and we thank the reviewer for finding the chosen values robust across teachers.

Each teacher produces ISCD terms on its own numerical scale, so Appendix A.4 chooses λout\lambda_{\mathrm{out}}, λirc\lambda_{\mathrm{irc}}, and λisc\lambda_{\mathrm{isc}} once per teacher and then keeps them at 2×2\times, 3×3\times, and 4×4\times. The change stays inside a narrow set. λout\lambda_{\mathrm{out}} remains 0.10.1 and λirc\lambda_{\mathrm{irc}} remains 1.01.0 for all five teachers. λisc\lambda_{\mathrm{isc}} takes 0.010.01 or 0.050.05, and NGAMDN_{\mathrm{GAMD}} takes 5050 or 100100. We thank the reviewer for treating this limited range as sufficient and not as a major weakness.

Q1. Inverse reconstruction in the VGG control

We thank the reviewer for leading us to clarify this point further. We include the VGG control because this is the feature space used by CSD-style distillation. Section 3.2 states that these methods compare reconstructed images through a VGG extractor. Appendix A.16 follows that setting. The network is a frozen ImageNet-pretrained VGG, and its intermediate features form a fixed perceptual space for aligning the teacher and student reconstructions.

This VGG is trained for image classification and does not implement an HR-to-LR mapping. During distillation, the SR output is passed through it and the alignment uses its multi-layer features. Section 3.2 defines Lirc=∥I(S(x))−x∥1\mathcal{L}_{\mathrm{irc}}=\|\mathcal{I}(\mathcal{S}(x))-x\|_1, which requires the proxy to turn the student prediction into an image that can be compared with the LR input. VGG does not produce that degraded image, so the loss has no target. Appendix A.16 therefore disables Lirc\mathcal{L}_{\mathrm{irc}} for the VGG proxy.

The rest of the objective is shared. Both runs use the reconstruction loss, the output distillation loss, and the same batch-wise contrastive form. The inverse network provides a degradation-aware feature space that matches the student architecture and can also carry Lirc\mathcal{L}_{\mathrm{irc}}. VGG provides a generic classification feature space, and the inverse reconstruction term is not available there. Appendix A.16 compares these as the complete design each proxy can support. Tables 16 and 17 in Appendix A.13 show why the inverse term is kept where it is defined. With HYPIR at 2×2\times, adding Lirc\mathcal{L}_{\mathrm{irc}} to Lout\mathcal{L}_{\mathrm{out}} reduces LPIPS from 0.12800.1280 to 0.10110.1011.

Thank you for your clarifications. After considering your responses and the discussions with other reviewers, I will maintain my original rating.

We sincerely thank the reviewer for the careful reading and for keeping the original rating.

The question raised in the review is now stated more clearly in Appendix A.18, with the supporting loss ablation in Appendix A.15.

Reviewer_1PxWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AMID is a heterogeneous image super-resolution distillation framework with two components. Global Attribution Map Distillation (GAMD) aligns teacher and student integrated-gradient attribution maps, aiming to transfer which low-resolution regions the teacher uses. Inverse Self-Consistency Distillation (ISCD) sends teacher and student outputs through a frozen inverse network built to match the student architecture and aligns representations in that shared space. The method is evaluated with five teachers, five benchmarks, and scales from 2x to 4x, mainly using an RRDB student. Auxiliary networks and losses are used only during training, so student inference cost is unchanged.

Critical strengths
  1. The method is decomposed into meaningful components, and the ablation study examines the main design choices rather than treating the full system as a black box. This makes it easier to understand where the reported improvements come from.
  2. The evaluation spans several teacher families, datasets, scaling factors, and more than one student architecture. The fact that all auxiliary networks and losses are removed at inference time is also a clear practical advantage for deployment.
Critical weaknesses
  1. The empirical scope is concentrated around the RRDB student and DF2K paired-data regime. The EDSR study covers one teacher and one scale. Appendix A.18 notes that the inverse network depends on the same HR-to-LR pair distribution, leaving real-camera degradations untested.
  2. AMID adds substantial training overhead. The paper estimates 5.5x to 9.0x the baseline training compute, depending on teacher, including repeated gradient-based attribution computation. This cost should be weighed against gains that are small or metric-dependent in some configurations. Without seed-level uncertainty in the main tables, it is difficult to assess the robustness of reported improvements, particularly where absolute gains are modest.
Questions for authors
  1. How sensitive are the gains to student training seeds and changes in the paired degradation distribution? Can uncertainty intervals be reported for the main PSNR and perceptual comparisons?
  2. Does the method retain its advantage on real-camera or otherwise mismatched degradations, where the learned inverse network may not match test-time image formation?
  3. Can the authors provide a cost-benefit analysis relating the additional training compute to the magnitude and consistency of student-quality gains?
10 comments

We thank the reviewer for the time and patience devoted to helping us improve this work.

W1. Student architecture scope

The main experiments fix an RRDB student so teacher effects and distillation losses can be compared under one backbone. Appendix EDSR uses a SwinIR teacher at 4×4\times and is limited to that pair. We additionally distill CAT-R-2[1] from a frozen HAT teacher at 2×2\times under the same DF2K/RGB protocol with 100K iterations.

AMID attains the best PSNR/SSIM and the strongest results on most perceptual metrics. Pixel-wise KD improves PSNR by only 0.04 dB. DCKD raises PSNR further but is worse than KD on LPIPS, MUSIQ, and MANIQA and worse than the baseline on BRISQUE. AMID improves PSNR by 0.37 dB over the CAT-R-2 baseline and by 0.19 dB over DCKD and reduces LPIPS from 0.0946 to 0.0852. The deployed architecture and inference cost are unchanged.

Conclusion. AMID is therefore not restricted to CNN students. Real-camera degradations are discussed with W1 & Q2.

ConfigurationPSNR↑SSIM↑LPIPS↓BRISQUE↓NIQE↓CLIPIQA↑MUSIQ↑MANIQA↑
Baseline(no distillation)31.65590.91750.094629.87655.19990.645866.74070.4214
KD31.69870.92160.090329.16705.16110.662167.14310.4343
DCKD31.83280.92050.092630.69765.15140.657466.99120.4304
AMID(ours)32.02160.92240.085229.37775.04830.654767.35640.4378

W1 & Q2. Real-camera degradations

Appendix A.18 states that the inverse is pretrained on the student HR-to-LR pairs. Q2 asks whether AMID retains its advantage on real-camera or mismatched degradations when this inverse may not match test-time image formation.

We train the undistilled baseline and the distilled students on DF2K HR images degraded by the standard Real-ESRGAN[2] pipeline used in many blind SR papers. The inverse is pretrained on the same pairs with HR and LR reversed. The student is EDSR at 2×2\times, and the inverse network stays isomorphic to it. The teacher is the official SwinIR real-world GAN trained with BSRGAN degradations. All students use the same iteration budget and are tested on the camera-captured DRealSR[3] dataset at 2×2\times under the main-paper RGB protocol.

AMID is the strongest distilled student on PSNR, SSIM, and LPIPS. Relative to the undistilled EDSR baseline, PSNR rises by 0.40 dB and LPIPS drops from 0.1896 to 0.1661. Pixel-wise KD improves PSNR but lowers CLIPIQA, MUSIQ, and MANIQA. DCKD is close and leads on BRISQUE, NIQE, and MUSIQ while AMID leads on the remaining metrics.

The official GAN teacher still has the highest PSNR and leads on most no-reference scores. AMID is slightly better on SSIM and LPIPS because GAN textures may lead to deviations from the camera GT even when no-reference models assign high scores.

Conclusion. This evaluation follows the established blind-SR protocol of high-order synthetic degradation plus camera-captured testing. The undistilled baseline, the distilled students, and the inverse share the same Real-ESRGAN DF2K pairs, and the inverse uses those pairs with HR and LR reversed as in the main paper. AMID still improves over this baseline and KD on DRealSR and is stronger than DCKD on most metrics.

ConfigurationPSNR↑SSIM↑LPIPS↓BRISQUE↓NIQE↓CLIPIQA↑MUSIQ↑MANIQA↑
Baseline(no distillation)27.94030.85320.189647.35636.04730.517535.55160.3067
SwinIR-GAN teacher28.41840.86000.171228.90544.33520.568036.94230.3579
KD28.26480.86090.181146.21846.03310.499534.60260.3017
DCKD28.28900.86280.167745.61995.65360.535038.86390.3393
AMID(ours)28.34460.86400.166145.74955.80340.546238.48100.3463

Q1. Seed-level uncertainty

Q1 asks how sensitive the gains are to training seeds and whether uncertainty can be reported for the main PSNR and perceptual numbers. Changes in the paired degradation are discussed with W1 & Q2.

We independently retrain full AMID three times at 2×2\times with a SwinIR teacher and an RRDB student on default DF2K pairs, using the main paper run as s1s_1. Across seeds the five-benchmark RGB-mean PSNR under the main-paper protocol is 32.2074±0.000932.2074±0.0009 dB, and SSIM and LPIPS move only in the fourth decimal. The AMID–DCKD PSNR gap in the main table is 0.16 dB and far exceeds this scatter, with DCKD (the closest competing method) shown as a single paper run for reference.

MethodSeedPSNR↑SSIM↑LPIPS↓
AMIDs1s_1(paper)32.20720.92400.0860
AMIDs2s_232.20660.92420.0863
AMIDs3s_332.20830.92410.0863
AMIDmean±std32.2074±0.00090.9241±0.00010.0862±0.0002
DCKDpaper32.04910.92250.0873

W2 & Q3. Training cost versus gains

AMID costs 5.5×5.5\times to 9.0×9.0\times the undistilled RRDB training budget at 2×2\times depending on the teacher, including the frozen inverse and GAMD path-integrated gradients applied every 50 to 100 iterations. This overhead is confined to training and leaves the deployed student unchanged.

The table relates that budget to student quality using five-benchmark RGB means from the main tables. DCKD is a strong competing method in our comparisons, and the return is larger against the undistilled student than against DCKD. At 3×3\times with SwinIR the perceptual gap over DCKD is clear, and at 4×4\times PSNR is slightly below DCKD while perceptual scores remain higher. Under HAT at 2×2\times, AMID is 8.7×8.7\times and DCKD is 23.2×23.2\times.

AMID is therefore a practical SR distillation method, since its additional training cost is a reasonable trade-off for the quality gains and its inference cost remains unchanged.

Teacher/scaleCost vs undistilled RRDBΔPSNR vs baselineΔMUSIQ vs baselineΔPSNR vs DCKDΔMUSIQ vs DCKD
SwinIR 2×2\times5.5×5.5\times+0.84+0.75+0.16+0.005
HAT 2×2\times8.7×8.7\times+0.80+0.80+0.09+0.08
SwinIR 3×3\times3.29×3.29\times+0.55+9.66+0.05+1.22
SwinIR 4×4\times3.20×3.20\times+0.75+15.30-0.02+0.48

Reference

[1] Chen, Zheng, et al. "Cross aggregation transformer for image restoration." NeurIPS (2022): 25478-25490.

[2] Wang, Xintao, et al. "Real-esrgan: Training real-world blind super-resolution with pure synthetic data." ICCVW, 2021.

[3] Wei, Pengxu, et al. "Component divide-and-conquer for real-world image super-resolution." ECCV, 2020.

Thank you for the additional experiments. The CAT-R-2 and DRealSR results substantially address my concerns about student architecture coverage and real-camera evaluation. Two questions remain:

  1. Uncertainty of the comparative gains. The three AMID runs provide useful evidence of stability under the SwinIR/RRDB ×2 setting. However, DCKD is represented by a single run, so the variability of the improvement over this baseline remains unclear. Could you provide matched multi-seed results for DCKD under the same setting and report the mean and variability of the AMID–DCKD differences in PSNR and LPIPS?

  2. Accounting for inverse-network pretraining. Do the reported 5.5–9.0× training-compute estimates include the cost of pretraining the inverse network? Appendix A.1.6 states that this pretraining uses the same number of iterations as student training, whereas Tables 7–8 appear to report per-iteration costs during distillation. Please clarify the accounting and, if pretraining is excluded, report the total compute including this stage, with any reuse or amortization assumptions stated explicitly.

Reviewer_KD1mOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper addresses the challenge of knowledge distillation across heterogeneous architectures for image super-resolution (SR). It proposes Attribution Matching and Inverse-Network Distillation, which transfers knowledge without accessing the intermediate features. Global Attribution Map Distillation matches integrated-gradient attribution maps of guidance-weighted structural responses. Inverse Self-Consistency Distillation uses a frozen inverse network for multi-layer contrastive alignment in a degradation-aware proxy space. Extensive experiments demonstrate that AMID improves the student’s performance and preserves inference cost.

Critical strengths
  1. The paper addresses a practical challenge in heterogeneous SR distillation without requiring access to intermediate features.
  2. GAMD and ISCD provide complementary and well-motivated distillation mechanisms. In particular, combining integrated-gradient attribution matching with a residual-based guidance map is a meaningful design.
  3. The proposed method improves the baseline student's performance without introducing additional parameters or inference-time computational overhead.
Critical weaknesses

This work addresses the challenge of heterogeneous knowledge distillation for SR demonstrating potential real-world value. However, several concerns remain:

  1. The experiments primarily focus on CNN-based students. Broader validation across diverse lightweight student architectures would better demonstrate the generalizability and practical applicability of AMID.

  2. The writing could be further refined and condensed (including the abstract):

    a.Method: The section should add a clearer high-level overview. And the relationships between the proposed components and the overall framework are not sufficiently clear, making the technical flow difficult to follow.

    b.Experiments: The experimental section needs better organization, as the dense and fragmented descriptions disrupt readability. Important experiments and key findings should be incorporated into the main text rather than being scattered throughout the supplementary material.

  3. Although AMID introduces no additional inference cost, its training cost is approximately 5.5–9.0× that of standard KD. In comparison, AugKD official reports approximately 1.4× the training cost of KD. This substantial overhead raises concerns about the practical efficiency of AMID.

  4. Section 3.1, Eqs. (1)-(5), forms GAMD guidance from a residual involving the ground-truth target and applies min-max normalization over spatial positions and the batch. Appendix A.10 compares mapping functions, but not sensitivity to batch size or composition, nor batch-level with per-sample normalization. Because examples are coupled by this normalization, please report its effect on the guidance maps and student results. The paper states that the target is used to compute the map during training and that the map is unused at inference; the remaining question is how robust training is to batch composition.

  5. This inverse network is pretrained on paired data with ideal bicubic downsampling. However, in real-world blind super-resolution scenarios, images often suffer from unknown blur kernels, compound noise, and nonlinear degradations. Under such conditions, it is difficult to pretrain an accurate and stable single-step inverse mapping network. This dependency substantially limits the generalizability of the proposed framework to real-world super-resolution scenarios.

Questions for authors

Please refer to Weaknesses.

3 comments

We thank the reviewer for the time and patience devoted to helping us improve this work.

W1. Generalization beyond CNN students

The main experiments use CNN students. To test whether AMID depends on this choice, we distill a light Transformer student CAT-R-2[1] from a frozen HAT teacher at $2\times$. Training follows the same DF2K/RGB protocol with 100K iterations.

AMID attains the best PSNR/SSIM and the strongest results on most perceptual metrics. Pixel-wise KD improves PSNR by only 0.04 dB. DCKD raises PSNR further but is worse than KD on LPIPS, MUSIQ, and MANIQA and worse than the baseline on BRISQUE. AMID improves PSNR by 0.37 dB over the CAT-R-2 baseline and by 0.19 dB over DCKD and reduces LPIPS from 0.0946 to 0.0852. The deployed CAT-R-2 architecture and inference cost are unchanged.

Conclusion. AMID is therefore not specific to CNN students. The main paper uses a smaller-capacity CNN to test a limited hypothesis space, and the same recipe can be applied to students from other SR architectures.

ConfigurationPSNR↑SSIM↑LPIPS↓BRISQUE↓NIQE↓CLIPIQA↑MUSIQ↑MANIQA↑
Baseline(no distillation)31.65590.91750.094629.87655.19990.645866.74070.4214
KD31.69870.92160.090329.16705.16110.662167.14310.4343
DCKD31.83280.92050.092630.69765.15140.657466.99120.4304
AMID(ours)32.02160.92240.085229.37775.04830.654767.35640.4378

W2. Writing and organization

We will revise and improve the manuscript for clarity as suggested.

The method already has a pipeline figure. We will add a short textual roadmap that maps each block to GAMD or ISCD and links both modules to the total loss.

We will rearrange the experiment section around the main questions. Key appendix results will be moved into the main text, including the GAMD correspondence controls and the ISCD proxy comparison.

W3. Cost-quality trade-off

The 5.5−9.0×5.5-9.0\times training overhead is incurred only during training, and inference uses the original student with no extra compute.

We agree that AMID costs more to train than KD and CSD. Practicality also depends on the quality obtained for that budget. In our setting, cheaper methods transfer less, and the closest quality competitor is not cheaper to train.

KD is cheapest because it matches HR pixels, but the quality gap is clear. With SwinIR at 3×3\times, KD reaches 61.05 MUSIQ and AMID reaches 64.54. The PSNR difference is small while the perceptual difference is not.

The official AugKD report cites about 1.4× KD. Under our protocol, AugKD stays close to the undistilled student in PSNR. With SwinIR at 3×3\times, AugKD is 0.16 dB below the baseline and AMID is 0.55 dB above.

DCKD is the closest method in overall quality. Under HAT at 2×2\times, it costs 23.2×23.2\times the baseline and AMID costs 8.7×8.7\times. GAMD is amortized and is applied only every 50-100 iterations.

Among the compared methods, AMID is a stronger cost-quality trade-off than KD and AugKD. It is also less expensive than DCKD in the profiled HAT 2×2\times setting. The deployed runtime does not change, which keeps AMID a practical recipe for deployment-oriented SR distillation.

W4. GAMD guidance-map normalization

GAMD min–max is taken jointly over the batch and space. The concern is that this couples examples and may make training depend on batch composition.

Batch+spatial min-max is our default. It ranks each residual against all pixels in the current mini-batch and targets batch-relative extremes (inverted-Gaussian U\mathbf{U}). Per-sample spatial min-max stretches each sample to [0,1] on its own and forces every image to have both tails. These two settings define different guidance maps U\mathbf{U}. We measure their effect on student metrics below.

Under HAT at 2×2\times, we use the GAMD-only protocol, which matches Appendix GAMD+Ours and HAT-w/G.

  • For per-sample normalization with batchsize 8, PSNR drops by 0.09 dB relative to batch+spatial, and LPIPS and MUSIQ worsen slightly. Easy images with a narrow residual range receive inflated tail weights. The drop stays small because GAMD acts as a regularizer on top of LREC\mathcal{L}_{\mathrm{REC}}. Every GAMD setting remains above the undistilled baseline of 31.37 dB PSNR.

  • For batch+spatial normalization with batchsize 4, we train for 500K iterations with a matched sample count relative to BS 8. PSNR changes by 0.006 dB, and the other metrics move only marginally.

Conclusion. GAMD training remains robust when the batch size used for joint min-max is changed in this range. Switching to per-sample guidance maps weakens GAMD slightly, but does not reverse the gain over the undistilled student.

R^\hat{R} normalizationbatchsizeItersPSNR↑SSIM↑LPIPS↓BRISQUE↓NIQE↓CLIPIQA↑MUSIQ↑MANIQA↑
undistilled baseline (main paper)8250K31.37000.91420.099631.00045.22420.620366.88140.4225
batch+spatial(ours, main paper)8250K31.63680.91750.094430.23265.13660.629566.98660.4277
per-sample spatial8250K31.54430.91640.096130.92785.10270.620066.88910.4268
batch+spatial(ours)4500K31.64250.91750.094330.30725.13420.629466.99070.4280

W5. Inverse network beyond bicubic pairs

We test whether AMID still works after the inverse is retrained on a practical synthetic degradation.

We follow the main-paper recipe on DF2K HR images degraded by the standard Real-ESRGAN[2] pipeline. The undistilled EDSR baseline, KD, DCKD, and AMID are all trained on these pairs. The inverse is isomorphic to EDSR and is pretrained on the same pairs with HR and LR reversed. The teacher is the official SwinIR real-world GAN trained with BSRGAN degradations. All students use the same iteration budget and are tested on the camera-captured DRealSR[3] dataset at 2×2\times under the main-paper RGB protocol.

AMID is the strongest distilled student on PSNR, SSIM, and LPIPS. Relative to the undistilled EDSR baseline, PSNR rises by 0.40 dB and LPIPS drops from 0.1896 to 0.1661. Pixel-wise KD improves PSNR but lowers CLIPIQA, MUSIQ, and MANIQA. DCKD is close and leads on BRISQUE, NIQE, and MUSIQ while AMID leads on the remaining metrics.

The official GAN teacher still has the highest PSNR and leads on most no-reference scores. AMID is slightly better on SSIM and LPIPS because GAN textures may deviate from the camera GT even when no-reference models assign high scores.

Conclusion. The inverse is not restricted to bicubic pairs. After Real-ESRGAN-style pretraining, AMID still improves an EDSR student on DRealSR (camera-captured pairs).

ConfigurationPSNR↑SSIM↑LPIPS↓BRISQUE↓NIQE↓CLIPIQA↑MUSIQ↑MANIQA↑
Baseline(no distillation)27.94030.85320.189647.35636.04730.517535.55160.3067
SwinIR-GAN teacher28.41840.86000.171228.90544.33520.568036.94230.3579
KD28.26480.86090.181146.21846.03310.499534.60260.3017
DCKD28.28900.86280.167745.61995.65360.535038.86390.3393
AMID(ours)28.34460.86400.166145.74955.80340.546238.48100.3463

Reference

[1] Chen, Zheng, et al. "Cross aggregation transformer for image restoration." NeurIPS (2022): 25478-25490.

[2] Wang, Xintao, et al. "Real-esrgan: Training real-world blind super-resolution with pure synthetic data." ICCVW, 2021.

[3] Wei, Pengxu, et al. "Component divide-and-conquer for real-world image super-resolution." ECCV, 2020.

I read through the rebuttal, and I still keep my original score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of our work and the rebuttal discussion

Dear Area Chair, Senior Area Chair, and Program Chairs,

We thank Reviewers 9UNY, KD1m, and 1PxW. The questions raised before the rebuttal have all been explained or resolved. We kindly ask that the main text and the rebuttals be weighed together.

GAMD matches how the teacher uses LR pixels. ISCD aligns reconstructions in a frozen student isomorphic HR to LR network. Neither module is used at inference. The main tables cover five teachers, three scales, five benchmarks, and eight metrics. At 3x with SwinIR, AMID leads all eight metrics and raises MUSIQ by 1.22 over the strongest competing method.

Reviewer 9UNY

After carefully reading our rebuttal and the discussion with the other reviewers, Reviewer 9UNY continues to support the paper with a strong accept. The reviewer asked about the number of hyperparameters and about inverse reconstruction in the VGG control. Both points have been explained. ISCD weights are chosen once per teacher and then shared across 2x, 3x, and 4x. λout\lambda_\text{out} stays at 0.1 and λirc\lambda_\text{irc} stays at 1.0 for all five teachers. λisc\lambda_\text{isc} is only 0.01 or 0.05. VGG is the CSD style perceptual proxy used in Appendix A.16. It does not map HR to LR, so Lirc\mathcal{L}_\text{irc} has no target there and is disabled. The inverse network can carry this term. VGG cannot. The reviewer treated the limited hyperparameter range as sufficient.

Reviewer KD1m

The reviewer asked about CNN students, writing, cost, GAMD normalization, and inverses beyond bicubic pairs. These issues have been explained or resolved. CAT-R-2 distilled from HAT at 2x shows that AMID is not specific to CNN students. PSNR rises by 0.37 dB over the undistilled CAT-R-2 baseline and by 0.19 dB over DCKD. Writing will be revised as suggested, with a textual roadmap and key appendix controls moved into the main text. Training overhead is extra only during training. Inference is unchanged. Per-sample GAMD normalization lowers PSNR by 0.09 dB and does not reverse the gain over the undistilled student. Batch size 4 versus 8 changes PSNR by 0.006 dB. Real-ESRGAN training plus camera captured DRealSR shows that the inverse is not restricted to bicubic pairs. PSNR rises by 0.40 dB over the undistilled EDSR baseline.

Reviewer 1PxW

The reviewer asked about CNN only students, bicubic only inverses, seeds, training cost, and novelty. The empirical issues have been explained or resolved. The reviewer has requested no further experiments. CAT-R-2 and DRealSR give the same conclusions as above. Three seeds for both AMID and DCKD keep the SwinIR to RRDB 2x PSNR gap at about 0.15 dB. Inverse pretraining is counted in full as an extra 1.0x. Under HAT at 2x the end to end budget is 9.7x, still below DCKD at 23.2x. The remaining novelty point is already stated in the paper and in the rebuttal. If closer papers are named we will cite them and add the comparison.

Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.