acceptodds
Desk rejected at ICLR 2027

SPARK: Spatial Policy-driven Adaptive Reinforcement learning for Knowledge distillation

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Low-bit quantization enables deployment of image restoration (IR) networks on resource-constrained devices, but introduces rounding noise that disproportionately degrades high-frequency regions such as edges and fine textures. Existing knowledge distillation (KD) methods apply distillation signals uniformly across all spatial locations, overlooking the varying reconstruction difficulty across image regions. To address this, we propose SPARK (Spatial Policy-driven Adaptive Reinforcement learning for Knowledge distillation), a framework that adaptively allocates distillation effort using a lightweight reinforcement learning (RL) policy network. At each training step, a difficulty feature extractor computes four signals, namely Laplacian variance, pixel variance, student reconstruction error, and teacher-student knowledge gap, which are fed into a compact policy CNN that produces a stochastic spatial weight map to modulate the KD loss during quantization-aware training (QAT). SPARK is IR task-agnostic, adds no inference cost, and integrates into any existing QAT pipeline without architectural changes. Experiments on benchmark datasets demonstrate that SPARK consistently outperforms PTQ, QAT, and state-of-the-art (SOTA) KD approaches across multiple student architectures, achieving reconstruction quality closest to the full-precision teacher under significant computational constraints.

Primary area:
reinforcement learning
Keywords:
Image restoration, Knowledge distillation, Reinforcement Learning, Image enhancement

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ujJdOpenReview ↗
Rating
1
Confidence
1(−1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SPARK, a spatially adaptive distillation scheme for quantization-aware training (QAT) of image restoration networks. The starting observation is that standard KD averages the teacher–student loss uniformly over pixels, while quantization damage concentrates on edges and textures.

At each training step, SPARK proceeds as follows:

  • It computes four difficulty maps without gradients: Laplacian variance and pixel variance of the student output, student error ∣I^s−IGT∣|\hat I_s - I_{GT}|, and teacher–student gap ∣I^t−I^s∣|\hat I_t - \hat I_s|.
  • It average-pools these maps to an 8×88\times 8 grid.
  • A three-layer CNN maps the pooled state to the mean of an isotropic Gaussian (σ=0.1\sigma = 0.1). A sample is clipped to [0,1][0,1], detached, and upsampled into a weight map W~\tilde W for an MSE distillation term.
  • The student minimizes Lrec+LKD\mathcal{L}_{rec} + \mathcal{L}_{KD}, with an L1 loss to the ground truth.
  • The policy is trained as a contextual bandit with REINFORCE and a running-mean baseline. The reward is α ΔPSNR+β ΔSSIM\alpha\,\Delta\mathrm{PSNR} + \beta\,\Delta\mathrm{SSIM} relative to a student output "obtained without spatial weighting."

The policy is discarded after training, so there is no inference cost.

Experiments use INT8 students in four teacher–student pairs:

  • MIRNet → self-quantized MIRNet and MIRNet → DnCNN, on LOLv1 and SIDD;
  • RFDN → self-quantized RFDN and RFDN → SRCNN, on Urban100 at ×2 and ×4.

SPARK is compared with PTQ, QAT, and KD/FAKD/DCKD/SLKD combined with QAT, and is best on every metric. Ablations remove the policy (uniform L1 or MSE KD) and vary the grid size (4/8/16). An appendix shows that the learned weights correlate weakly with per-patch student error (Pearson r=0.158r = 0.158), while a teacher-error heuristic correlates negatively (r=−0.121r = -0.121).

Critical strengths

1. The method is simple and costs nothing at deployment. The policy is tiny, runs only during training, and needs no change to the student architecture. If the gains were real, it could be dropped into any existing QAT pipeline, which is the right kind of practical target for compressed restoration models. The prose is easy to follow, and Appendix E specifies the quantization graph in useful detail: observers, freezing schedule, treatment of the first and last layers, and handling of sigmoid gates and softmax. It also states that all compared methods share the same quantized graph. That level of protocol detail is better than in many papers in this area.

2. The evaluation is broad in coverage. It spans three restoration tasks, two kinds of student (self-quantized and a smaller architecture), and two SR scales, so the claim is at least tested beyond a single setting. This breadth is the main reason I would not dismiss the idea outright. Unfortunately, the issues below make me unable to trust what the experiments show.

Critical weaknesses

1. It is unclear that the RL component does anything, and the controls needed to tell are missing.

The reward is not well defined. I^s\hat I_s is computed before the weight map is sampled, and W~\tilde W only enters the loss, so the action can affect reconstruction only through the subsequent parameter update. A meaningful ΔPSNR\Delta\mathrm{PSNR} "relative to uniform weighting" would then require a second student update (or a lookahead copy) with uniform weights on the same batch. Algorithm 1 does not show this. If the reward is computed from the same pre-update output, it does not depend on aa, and REINFORCE receives only noise. Even with a proper lookahead, the effect of one 8×88\times 8 map on one step with batch size 4 is tiny relative to the reward variance. The reward weights α\alpha, β\beta and the decay η\eta are not reported, and the training overhead is explicitly left unquantified.

A simpler explanation for the gains is not ruled out. Since W∈[0,1]W\in[0,1], SPARK's KD term is always no larger than uniform MSE KD. The appendix reports weights in [0.16,0.69][0.16, 0.69] with standard deviation 0.097, so in effect the KD loss is scaled down by roughly half with mild spatial variation. Its correlation with student error is only r=0.158r = 0.158. The paper never compares against:

  • uniform KD with a constant weight equal to the mean of WW;
  • a random map drawn from the same distribution;
  • the non-RL baseline that uses the paper's own features directly (e.g., W=normalize(D3)W = \mathrm{normalize}(D_3) or D4D_4).

Without these, I cannot attribute anything to learned spatial allocation, let alone to RL. The only heuristic in the paper (one minus teacher error) is compared by correlation rather than accuracy. It is also expected to correlate negatively with student error, because regions that are hard for the teacher tend to be hard for the student as well.

Related work does not position the contribution. Learned "what and where to transfer" in distillation (e.g., L2T-ww, Jang et al., ICML 2019) and learned loss or sample reweighting are not discussed. The claim that this is the first such formulation for low-bit IR is therefore a narrow one.

2. The baseline numbers look implausible, which makes the reported margins hard to interpret.

The baselines collapse far more than INT8 normally causes. With per-channel INT8 weights and a histogram observer, PTQ of MIRNet on SIDD drops from 39.72 to 28.42 dB (SSIM 0.959 → 0.510). In my experience, INT8 PTQ of restoration networks typically costs around a dB, not eleven. QAT recovers only 0.7 dB on top of that. The implementation details say the student is trained from scratch, including the self-quantized MIRNet and RFDN, whereas QAT is normally initialized from the full-precision weights. This probably explains why every method sits 6 dB or more below the teacher on SIDD. Against such weak baselines, a +4.3 dB, +0.24 SSIM improvement from reweighting a KD term does not tell us much about SPARK in a realistic QAT pipeline.

Several metrics are internally inconsistent:

  • On Urban100 ×2, SPARK is within 0.16 dB PSNR of the RFDN teacher but 0.10 lower in SSIM (0.825 vs. 0.928).
  • The quantized SRCNN has higher SSIM (0.884) than the quantized RFDN (0.825) despite 3.4 dB lower PSNR.
  • On SIDD, SPARK's LPIPS nearly matches the teacher (0.325 vs. 0.320) while PSNR is 5.8 dB lower.

Basic reporting is missing. There are no seeds, no variances, and no significance tests, and LOLv1 has only 15 test images.

The setting avoids the regime where this problem matters most. Only INT8 is evaluated, and the baselines include no quantization-specific IR methods (e.g., PAMS, DAQ, CADyQ, QuantSR). The literature that motivates this work is mostly about 4 bits and below.

Questions for authors
  1. How exactly is the reward computed? Is a second, uniformly weighted student update performed on the same batch? Is PSNR measured on that batch before or after the update, and on which output? Does rr depend on the sampled action aa in your implementation? Please also give α\alpha, β\beta, η\eta, and the extra training cost per iteration.

  2. Do the gains survive simple controls? Please report SPARK against:

    • (a) uniform MSE KD with a constant weight equal to the mean of the learned WW;
    • (b) a random map with the same mean and variance;
    • (c) a deterministic map built directly from D3D_3 or D4D_4, without RL.

    If (a) or (c) recovers most of the gain, the RL contribution disappears.

  3. Why are students trained from scratch? In particular, why is the self-quantized student not initialized from the full-precision teacher? What do the full Tables 1–2 look like under the standard protocol, i.e., QAT fine-tuned from FP weights? Can you also explain the 11 dB INT8 PTQ drop on SIDD?

  4. Are the teacher numbers measured under the same evaluation pipeline as the students (Appendix E), or taken from the original papers? Please also explain the PSNR/SSIM/LPIPS inconsistencies listed above.

Smaller clarifications

  1. Results over multiple seeds, with standard deviations, at least for LOLv1 and SIDD.
1 comment

Thank you for the rebuttal and the additional results. I have carefully read your response and will take these clarifications into account during the final discussion.

Reviewer_Tk4YOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SPARK, a quantization-aware training (QAT) distillation method for low-bit image restoration networks. The authors believe that uniform pixel-by-pixel distillation ignores the difficulty differences in different regions: quantization noise mainly damages edges and textures, while smooth areas account for the majority of the loss. Therefore, SPARK extracts four difficulty features (Laplacian variance, pixel variance, student error, and teacher-student gap) in each training step, and outputs a spatial weight map with a Gaussian distribution by a 3-layer CNN strategy on an 8×8 grid, to weight the MSE distillation loss. The strategy is trained through REINFORCE, with the reward being relatively uniformly weighted ΔPSNR/ΔSSIM. After training, the strategy network is discarded, and inference is no cost.

Critical strengths
  1. The motivation is clear and the method is practical. The starting point of "quantitative errors are not uniform spatially, therefore distillation should not be uniform" is intuitive and reasonable. The method only acts during the training stage, does not change the student structure, nor increases the inference cost, and can be directly embedded into the existing QAT process. Moreover, it is universal across three tasks and does not require modification based on the task.

  2. The experiments are enough. It includes three tasks, two types of teacher-student combinations (isomorphic and heterogeneous), and multiple scales.

Critical weaknesses
  1. The spatial weighting must be achieved through RL learning and has not been proven. The learned weights mostly fall within the range of 0.2 to 0.6, which is equivalent to reducing the overall KD loss by about half, while there is no balance coefficient in the total loss. As a result, the performance improvement might only be due to the reduction in the KD intensity, and has nothing to do with the spatial distribution.

  2. The credibility of the experimental results is insufficient. Firstly, there are multiple abnormal values. INT8 PTQ causes MIRNet to decrease by 11 dB on SIDD, which is far beyond common sense and the baseline may have been weakened. On SIDD, simply weighting the KD term brings +4.3 dB and an increase of +0.24 in SSIM, while the same mechanism on LOLv1 only results in +0.56 dB. In the SR results, there are also cases where PSNR is high by 5 dB while SSIM is actually lower. Secondly, the title and introduction emphasize low bit rates, but the experiment only conducted INT8 with the minimum quantization error, without results for 4-bit or lower bit rates.

Questions for authors

See Critical Weaknesses

Reviewer_TpuFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SPARK, a reinforcement learning (RL) framework for spatially adaptive knowledge distillation (KD) in low-bit image restoration. A lightweight policy CNN maps four difficulty signals—Laplacian variance, pixel variance, student reconstruction error, and the teacher–student gap—to a stochastic spatial weight map that modulates the distillation loss during quantization-aware training. The policy is optimized with REINFORCE using a reconstruction-quality reward defined relative to uniform weighting, and is removed at inference. Experiments cover low-light enhancement, denoising, and super-resolution. The motivation is clear, but the action–reward computation remains insufficiently specified, the necessity of RL is not established through matched comparisons with simpler weighting schemes, and the baseline, cross-architecture, and edge-deployment evidence remains limited.

Critical strengths

The paper has a clear motivation. In low-bit image restoration, different spatial regions may differ in their sensitivity to quantization error and their need for distillation supervision, making adaptive allocation of distillation weights practically relevant. The policy network is used only during training and removed at inference, which also gives the method potential for integration into existing quantization training pipelines.

Critical weaknesses
  1. The coverage of baselines, backbones, and qualitative comparisons could be strengthened. Comparisons with relevant distillation methods and quantization methods designed specifically for image restoration are insufficient. For example, MiPKD (ICLR 2025) [1] and PREFILT (ICCV Workshops 2025) [2], both already cited in the paper, are absent from the main experimental comparisons. Additional comparisons should be conducted on their respective applicable tasks using comparable backbones, bit widths, and training budgets, with any necessary architectural differences disclosed. The current student backbones are mainly CNNs. Representative Transformer and state-space configurations, such as Restormer [3] and MambaIRv2 [4], should also be included to test applicability across architectures. More detailed qualitative comparisons with these methods should use the same images, crop regions, and display ranges.

  2. The action–reward connection remains insufficiently explained and concerns the core learning mechanism. Section 3.3 and Eq. (2) define the reward as the improvement in reconstruction quality relative to uniform weighting. It is currently unclear whether the updated student is evaluated through a new forward pass and how the uniformly weighted reference is constructed. If the reward still uses cached student outputs obtained before the update, then, conditioned on the current state, it is independent of the action sampled in the current iteration. REINFORCE would therefore lack an effective immediate learning signal for distinguishing better actions from worse ones. The complete computation should explicitly specify the action-dependent update, post-update evaluation, reference branch, and policy update.

  3. The existing ablations are insufficient to establish the necessity of reinforcement learning. The experiments do not report the restoration performance of a fixed teacher-error heuristic after training under the same budget. They also lack sufficient comparisons with normalized student-error weighting, edge weighting, teacher–student discrepancy weighting, and a learnable deterministic policy with the same inputs and network capacity. In particular, Eq. (4) restricts weights to [0,1] without constraining their sum; compared with uniform KD using all-one weights, the method may also reduce the overall distillation strength. The mean distillation weight, number of student updates, and computational budget should be matched. Controls such as uniform weights matched to the mean of the policy map or spatially shuffled weight maps should be included to distinguish the contributions of spatial allocation, overall loss scaling, and stochastic exploration. The deterministic policy should also use an appropriate outer reconstruction objective or constraints to avoid degeneration to zero weights when minimizing only the weighted KD loss. Non-RL meta-reweighting research can inform the design of this control [5]. Please report the mean and standard deviation over multiple runs for RL and the strongest non-RL control. Some differences between grid sizes in Tables 3 and 4 are small, so performance gains need to be distinguished from training variability.

  4. Evidence for edge deployment is insufficient, and the execution path underlying the reported speed results also requires clarification. The paper lacks measured latency, GPU memory usage, model storage, and power consumption/energy per image on target edge devices for SPARK. The existing measurements are insufficient to support benefits in specific edge-deployment scenarios. More importantly, Appendix E states that PyTorch FX quantization uses the FBGEMM backend, whereas this native quantization backend targets x86 CPU execution [6]. The deployment path connecting this configuration to the INT8 throughput reportedly measured on an RTX 3080 in Table 5 is unclear. Please specify whether a separate GPU deployment backend is used, whether the timed model has been converted to actual INT8 operators, and the batch size, warm-up procedure, device synchronization, and input/output resolutions. Please also describe the CUDA kernel design used for actual deployment. Provide the quantization-conversion and deployment workflow, identifying the actual execution device, operator precisions, and any operations that may fall back to floating point, together with reproducible timing code or configurations.

[1] S. Li, Y. Zhang, W. Li, H. Chen, W. Wang, B. Jing, S. Lin, and J. Hu, “Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution,” in Proceedings of the International Conference on Learning Representations (ICLR), 2025.

[2] D. Makhov, R. Ostapets, I. Zhelavskaya, and D. Song, “PREFILT: Prefiltering for Fully Quantized Image Restoration Neural Networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 4101–4110.

[3] S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, and M.-H. Yang, “Restormer: Efficient Transformer for High-Resolution Image Restoration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5728–5739.

[4] H. Guo, Y. Guo, Y. Zha, Y. Zhang, W. Li, T. Dai, S.-T. Xia, and Y. Li, “MambaIRv2: Attentive State Space Restoration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 28124–28133.

[5] M. Ren, W. Zeng, B. Yang, and R. Urtasun, “Learning to Reweight Examples for Robust Deep Learning,” in Proceedings of the 35th International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, vol. 80, 2018, pp. 4334–4343.

[6] PyTorch, “INT8 Quantization for x86 CPU in PyTorch,” 2023.

Questions for authors
  1. Please clarify: Is the reward evaluated after a student-parameter update determined by the sampled action, and compared against a matched update branch using uniform weighting? Please describe the exact computation.

  2. Key comparison: How does reinforcement learning compare with student-error weighting and a deterministic policy when the mean distillation weight and training budget are matched?

1 comment

Thank you for the careful and constructive review. All new experiments use the same INT8 setting, teacher, training budget, and evaluation protocol as the main paper.

1. Baselines and Transformer backbones MiPKD [1] and PREFILT [2] were reimplemented under the SPARK protocol (identical teachers, INT8 students, epochs, optimizer, and data). MiPKD's multi-granularity prior mixing, originally designed for super-resolution, was applied to the intermediate feature stages of each student. PREFILT was applied as specified in [2] with QAT. All adaptations will be documented in the revised appendix.

Table R1. INT8 CNN students, PSNR / SSIM

MethodSIDD (quantized DnCNN)Urban100 ×2 (quantized RFDN)
Uniform KD + QAT28.87 / 0.54231.33 / 0.772
MiPKD + QAT29.45 / 0.57031.60 / 0.811
PREFILT + QAT28.90 / 0.54431.38 / 0.775
SPARK31.71 / 0.67732.01 / 0.825

SPARK outperforms MiPKD by 2.26 dB on SIDD and 0.41 dB on Urban100 ×2. PREFILT gives only a marginal gain over uniform KD, since its prefiltering targets activation range rather than the distillation signal.

To test generality beyond CNNs, SPARK was also applied to an INT8 Restormer [3] student.

Table R2. Transformer student (INT8 Restormer), SIDD, PSNR / SSIM

MethodSIDD
Restormer FP32 (teacher)40.02 / 0.960
PTQ36.67 / 0.920
Uniform KD + QAT37.56 / 0.931
MiPKD + QAT37.60 / 0.933
PREFILT + QAT37.56 / 0.930
SPARK38.53 / 0.946

SPARK recovers 1.86 dB of the 3.35 dB PTQ gap, versus 0.93 dB for the best baseline. MambaIRv2 [4] is not included, since its selective-scan operators are not supported by the standard PyTorch INT8 flow and a fair comparison requires custom handling.

2. Action dependence of the reward The reviewer is correct that a reward computed from the pre-update student output would not depend on the action. Algorithm 1 omitted the step that prevents this. The full procedure at each iteration is as follows.

  1. The policy samples a spatial weight map Wt∼πϕ(⋅∣st)W_t \sim \pi_\phi(\cdot \mid s_t) from the batch difficulty features sts_t.
  2. Two virtual one-step updates are run from identical parameters θt\theta_t and optimizer state, one with the WtW_t-weighted KD loss (θ~tW\tilde{\theta}_t^{W}) and one with uniform weights (θ~tU\tilde{\theta}_t^{U}).
  3. Both are evaluated against IGTI_{GT}, giving rt=α ΔPSNR+β ΔSSIMr_t = \alpha\,\Delta\mathrm{PSNR} + \beta\,\Delta\mathrm{SSIM}, where Δ\Delta is the score of θ~tW\tilde{\theta}_t^{W} minus that of θ~tU\tilde{\theta}_t^{U}. Hence rtr_t depends directly on at=Wta_t = W_t.
  4. REINFORCE updates the policy with advantage rt−btr_t - b_t, where btb_t is a running-mean baseline. The virtual students are discarded and the real student step proceeds unchanged.

Settings are α=1\alpha = 1, β=10\beta = 10, η=0.9\eta = 0.9, and σ=0.1\sigma = 0.1. The revised Algorithm 1, Section 3.3, and Eq. (2) state this step explicitly. Training takes 0.43 s per iteration versus 0.21 s for KD + QAT (about 2×), with no inference overhead.

3. Necessity of reinforcement learning (Q2) The requested controls were run with the same training budget and number of student updates, over three seeds.

Table R3. KD weighting controls, [student] INT8, mean ± SD over three seeds

Weighting of KDSIDD PSNRSIDD SSIMUrban100 ×4 PSNRUrban100 ×4 SSIM
Uniform, λ=1\lambda = 129.63 ± 0.110.57224.58 ± 0.060.701
Uniform, tuned λ\lambda29.91 ± 0.090.57924.69 ± 0.050.703
(a) Constant, mean(W)29.88 ± 0.100.57824.67 ± 0.070.702
(b) Random, matched mean and SD29.71 ± 0.140.57524.61 ± 0.080.701
(c) Normalized student error (D3)31.24 ± 0.120.64825.08 ± 0.060.708
(d) Normalized teacher–student discrepancy (D4)31.62 ± 0.100.66325.21 ± 0.060.710
(e) Policy frozen at initialization30.37 ± 0.130.60124.83 ± 0.070.705
SPARK33.89 ± 0.120.80725.89 ± 0.050.719

SPARK exceeds the strongest heuristic (D4) by 2.27 dB on SIDD and 0.68 dB on Urban100 ×4, and the frozen policy recovers little of this gain. The improvement therefore comes from the learned policy rather than from reweighting alone.

4. Deployment efficiency SPARK is a KD + QAT framework that reduces the accuracy loss of INT8 conversion. It does not aim to accelerate INT8 inference. Latency, memory, and energy are set by the architecture, the INT8 backend (such as FBGEMM, QNNPACK, or TensorRT), and the hardware. The policy network and difficulty features are discarded after training, so a SPARK student has exactly the same INT8 graph, operators, and inference cost as a uniformly distilled student of the same architecture. Device-level measurements would therefore characterize the architecture and backend rather than SPARK.

Comments on the paper

2 comments

Dear Reviewer,

I hope this message finds you well. I am writing to kindly remind you that the authors’ rebuttal has been posted and is now available for your review in the system.

We would greatly appreciate it if you could review the rebuttal and provide your assessment at your earliest convenience, as your input is important for us to move forward with a timely decision.

Thank you very much for your time and continued support.

ACs

As only 24 hours remain in the author rebuttal period, I sincerely invite the reviewers to review the author's response and promptly raise any remaining questions or indicate whether you believe the author has addressed the issues you have raised.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.