- Rating
- 1
- Confidence
- 1(−1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SPARK, a spatially adaptive distillation scheme for quantization-aware training (QAT) of image restoration networks. The starting observation is that standard KD averages the teacher–student loss uniformly over pixels, while quantization damage concentrates on edges and textures.
At each training step, SPARK proceeds as follows:
- It computes four difficulty maps without gradients: Laplacian variance and pixel variance of the student output, student error , and teacher–student gap .
- It average-pools these maps to an grid.
- A three-layer CNN maps the pooled state to the mean of an isotropic Gaussian (). A sample is clipped to , detached, and upsampled into a weight map for an MSE distillation term.
- The student minimizes , with an L1 loss to the ground truth.
- The policy is trained as a contextual bandit with REINFORCE and a running-mean baseline. The reward is relative to a student output "obtained without spatial weighting."
The policy is discarded after training, so there is no inference cost.
Experiments use INT8 students in four teacher–student pairs:
- MIRNet → self-quantized MIRNet and MIRNet → DnCNN, on LOLv1 and SIDD;
- RFDN → self-quantized RFDN and RFDN → SRCNN, on Urban100 at ×2 and ×4.
SPARK is compared with PTQ, QAT, and KD/FAKD/DCKD/SLKD combined with QAT, and is best on every metric. Ablations remove the policy (uniform L1 or MSE KD) and vary the grid size (4/8/16). An appendix shows that the learned weights correlate weakly with per-patch student error (Pearson ), while a teacher-error heuristic correlates negatively ().
Critical strengths
1. The method is simple and costs nothing at deployment. The policy is tiny, runs only during training, and needs no change to the student architecture. If the gains were real, it could be dropped into any existing QAT pipeline, which is the right kind of practical target for compressed restoration models. The prose is easy to follow, and Appendix E specifies the quantization graph in useful detail: observers, freezing schedule, treatment of the first and last layers, and handling of sigmoid gates and softmax. It also states that all compared methods share the same quantized graph. That level of protocol detail is better than in many papers in this area.
2. The evaluation is broad in coverage. It spans three restoration tasks, two kinds of student (self-quantized and a smaller architecture), and two SR scales, so the claim is at least tested beyond a single setting. This breadth is the main reason I would not dismiss the idea outright. Unfortunately, the issues below make me unable to trust what the experiments show.
Critical weaknesses
1. It is unclear that the RL component does anything, and the controls needed to tell are missing.
The reward is not well defined. is computed before the weight map is sampled, and only enters the loss, so the action can affect reconstruction only through the subsequent parameter update. A meaningful "relative to uniform weighting" would then require a second student update (or a lookahead copy) with uniform weights on the same batch. Algorithm 1 does not show this. If the reward is computed from the same pre-update output, it does not depend on , and REINFORCE receives only noise. Even with a proper lookahead, the effect of one map on one step with batch size 4 is tiny relative to the reward variance. The reward weights , and the decay are not reported, and the training overhead is explicitly left unquantified.
A simpler explanation for the gains is not ruled out. Since , SPARK's KD term is always no larger than uniform MSE KD. The appendix reports weights in with standard deviation 0.097, so in effect the KD loss is scaled down by roughly half with mild spatial variation. Its correlation with student error is only . The paper never compares against:
- uniform KD with a constant weight equal to the mean of ;
- a random map drawn from the same distribution;
- the non-RL baseline that uses the paper's own features directly (e.g., or ).
Without these, I cannot attribute anything to learned spatial allocation, let alone to RL. The only heuristic in the paper (one minus teacher error) is compared by correlation rather than accuracy. It is also expected to correlate negatively with student error, because regions that are hard for the teacher tend to be hard for the student as well.
Related work does not position the contribution. Learned "what and where to transfer" in distillation (e.g., L2T-ww, Jang et al., ICML 2019) and learned loss or sample reweighting are not discussed. The claim that this is the first such formulation for low-bit IR is therefore a narrow one.
2. The baseline numbers look implausible, which makes the reported margins hard to interpret.
The baselines collapse far more than INT8 normally causes. With per-channel INT8 weights and a histogram observer, PTQ of MIRNet on SIDD drops from 39.72 to 28.42 dB (SSIM 0.959 → 0.510). In my experience, INT8 PTQ of restoration networks typically costs around a dB, not eleven. QAT recovers only 0.7 dB on top of that. The implementation details say the student is trained from scratch, including the self-quantized MIRNet and RFDN, whereas QAT is normally initialized from the full-precision weights. This probably explains why every method sits 6 dB or more below the teacher on SIDD. Against such weak baselines, a +4.3 dB, +0.24 SSIM improvement from reweighting a KD term does not tell us much about SPARK in a realistic QAT pipeline.
Several metrics are internally inconsistent:
- On Urban100 ×2, SPARK is within 0.16 dB PSNR of the RFDN teacher but 0.10 lower in SSIM (0.825 vs. 0.928).
- The quantized SRCNN has higher SSIM (0.884) than the quantized RFDN (0.825) despite 3.4 dB lower PSNR.
- On SIDD, SPARK's LPIPS nearly matches the teacher (0.325 vs. 0.320) while PSNR is 5.8 dB lower.
Basic reporting is missing. There are no seeds, no variances, and no significance tests, and LOLv1 has only 15 test images.
The setting avoids the regime where this problem matters most. Only INT8 is evaluated, and the baselines include no quantization-specific IR methods (e.g., PAMS, DAQ, CADyQ, QuantSR). The literature that motivates this work is mostly about 4 bits and below.
Questions for authors
-
How exactly is the reward computed? Is a second, uniformly weighted student update performed on the same batch? Is PSNR measured on that batch before or after the update, and on which output? Does depend on the sampled action in your implementation? Please also give , , , and the extra training cost per iteration.
-
Do the gains survive simple controls? Please report SPARK against:
- (a) uniform MSE KD with a constant weight equal to the mean of the learned ;
- (b) a random map with the same mean and variance;
- (c) a deterministic map built directly from or , without RL.
If (a) or (c) recovers most of the gain, the RL contribution disappears.
-
Why are students trained from scratch? In particular, why is the self-quantized student not initialized from the full-precision teacher? What do the full Tables 1–2 look like under the standard protocol, i.e., QAT fine-tuned from FP weights? Can you also explain the 11 dB INT8 PTQ drop on SIDD?
-
Are the teacher numbers measured under the same evaluation pipeline as the students (Appendix E), or taken from the original papers? Please also explain the PSNR/SSIM/LPIPS inconsistencies listed above.
Smaller clarifications
- Results over multiple seeds, with standard deviations, at least for LOLv1 and SIDD.
Thank you for the rebuttal and the additional results. I have carefully read your response and will take these clarifications into account during the final discussion.