- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the full-reference degradation score SymKL: Firstly, it combines and freezes the INR on the reference signal, calculates the average of the squared gradient per coordinate caused by the reference and degraded signals, and takes the resulting diagonal vector as the precision of the two Gaussian distributions. Then, it measures the difference using the symmetric KL. The authors also provide the calculation formula for not explicitly storing the gradient of the parameters per coordinate, the hypernetwork for image reference fitting, and conduct experiments on images, audio, 3D occupancy fields, and medical images. The work in the article is meaningful, but I still have concerns about some of the issues present in the article.
Critical strengths
-
Using INRs to study signal degradation is an intuitive and well-motivated direction. Their shared coordinate-based formulation provides a natural starting point for analyzing different signal modalities.
-
Evaluating the reference and degraded signals with the same frozen INR avoids the parameter-alignment problem that arises when separate networks are fitted to each signal.
-
The audio and occupancy-field experiments explore the method’s applicability beyond natural images.
Critical weaknesses
1.Unclear novelty relative to prior work. INRIQ [1] already fits and freezes an INR on the reference signal, constructs diagonal profiles from squared per-coordinate gradients, and compares those profiles using a symmetrized KL divergence. SymKL adds a more efficient computation, amortized image fitting, and experiments in additional modalities, but does not cite INRIQ or clearly distinguish these additions from the previously proposed core measure.
2.The main results show that SymKL changes more rapidly with distortion severity. A monotonic power transformation can increase this sensitivity without improving a metric’s ranking or accuracy, while the absolute value in the sensitivity formula can reward changes in the wrong direction. An independent measure of degradation accuracy is needed.
3.The paper lacks some robustness tests for imperfect references and small misalignments, paired uncertainty estimates, and a complete accounting of reference-fitting cost. These gaps limit the strength of its generality and efficiency claims.
[1] D. Jayasundara, S. Rajagopalan, and V. M. Patel. “INRIQ: Implicit Neural Representation for Image Quality Assessment.” ICLR 2026 submission, 2026.
Questions for authors
-
A direct comparison with INRIQ would clarify the paper’s contribution. A side-by-side account of the methods, together with controlled ablations, would help distinguish the effects of the proposed computation, hypernetwork, and cross-modal experiments.
-
Evaluation beyond relative sensitivity would make the empirical conclusions more convincing. Calibration to measured degradation, monotonicity or inversion rates, cross-distortion ranking, and downstream task performance would provide complementary evidence; paired uncertainty estimates would clarify the reported gains.
7 comments
1. Calibration to measured degradation
For each reference signal and degradation type, we measured how closely each score follows the physical severity of the degradation (for example, the noise level or the blur width), using Spearman's rank correlation, 1 means that the score rises exactly with the severity. The values are averaged over all references and degradation types.
| Data | SymKL | Other measures |
|---|---|---|
| 3D occupancy fields | 1.000 | Chamfer 1.000 · IoU 1.000 · F-score 0.934 |
| Audio | 0.991 | SNR 0.992 · SI-SDR 0.992 · PESQ 0.990 |
| MRI | 0.980 | PSNR 0.980 · SSIM 1.000 · DreamSim 0.940 · CLIP 0.800 |
| Ultrasound | 1.000 | PSNR 1.000 · SSIM 1.000 · DreamSim 1.000 · CLIP 0.970 |
| X-ray | 1.000 | PSNR 1.000 · SSIM 1.000 · DreamSim 1.000 · CLIP 0.950 |
| Natural images (KADID-10k) | 0.960 | PSNR 0.977 · SSIM 0.976 · DreamSim 0.982 · CLIP 0.901 |
Result. SymKL follows the measured severity closely in every modality (0.96 to 1.00), close to the measures designed for each modality and above CLIP wherever CLIP applies. It does so as a single measure, applied in exactly the same form to all of these signals.
2. Inversion rates
A violation is a step to a higher severity at which the score does not increase. Unlike sensitivity, the violation rate records the direction of change. The following table presents the percentage of steps that are violations for each measure, with all degradation types included.
| Data | SymKL | Other measures |
|---|---|---|
| 3D occupancy fields | 0% | Chamfer 0% · IoU 0% · F-score 16.3% |
| Audio | 1.44% | SNR 1.29% · SI-SDR 1.23% · PESQ 1.49% |
| MRI | 3.3% | PSNR 3.3% · SSIM 0% · DreamSim 3.3% · CLIP 20.0% |
| Ultrasound | 0% | PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5% |
| X-ray | 0% | PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5% |
| Natural images, KADID-10k | 5.22% | PSNR 4.11% · SSIM 3.06% · DreamSim 2.91% · CLIP 10.63% |
SymKL has no inversions on 3D shapes, ultrasound and X-ray, is on par with the specialized measures on audio and MRI, and has fewer inversions than F-score on 3D shapes and than CLIP on every image dataset. On natural images, 76% of its inversions come from five KADID-10k types (pixelate, contrast, colour blocks, non-eccentricity and colour shift), which also hold 85% to 96% of the inversions of PSNR, SSIM and DreamSim.
3. Cross-distortion ranking
As a simple experiment, we used the measured signal error, the mean squared difference from the reference, as a common scale for all degradation types. For each reference image, we took all 125 of its distorted versions (25 KADID-10k types at 5 levels), ranked them once by each measure and once by the measured error, and compared the two rankings using Spearman's rank correlation, where 1 means the same order. The table gives the mean over the 81 reference images.
| Measure | Agreement with the measured error (Spearman) |
|---|---|
| SymKL | 0.965 |
| DreamSim | 0.746 |
| SSIM | 0.665 |
| CLIP | 0.574 |
Among the measures, SymKL ranks different kinds of damage most consistently with the measured error. The same holds on audio, pooled over all 9 degradation types: SymKL agrees at 0.994, on par with SI-SDR (0.997) and well above PESQ (0.619).
4. Downstream task: benchmarking restoration methods
A common use of a full-reference measure is to rank the outputs of several restoration methods applied to the same degraded signal. We applied standard denoisers to noisy signals and let each measure rank their outputs:
For this we used, 30 KADID-10k reference images, Gaussian noise at four strengths, and 9 methods, from simple filters to total-variation, wavelet and non-local-means denoising (120 ranking problems). As no single measure is the ground truth, we compared each measure's ranking with the consensus of all the other measures, that is, their average ranking with the measure being evaluated left out, using Kendall's rank correlation (1 means the same ranking). The following table gives the mean over all ranking problems.
| Measure | Agreement |
|---|---|
| SymKL | 0.737 |
| PSNR | 0.688 |
| SSIM | 0.564 |
| DreamSim | 0.531 |
| CLIP | 0.319 |
As can be seen, SymKL agrees with the consensus of the other measures more closely than any of them does.
5. Paired uncertainty estimates
We compared SymKL with a strong perceptual measure on the same degraded signals, DreamSim on images and PESQ on speech, in three tests:
-
Test 1, a 1% increase in noise (images). For each of 20 KADID-10k reference images, we generated 20 independently drawn noisy versions with Gaussian noise of standard deviation 0.05 (on intensities in [0, 1]) and 20 with standard deviation 0.0505. Discrimination is the AUC: the share of the 400 pairs (one version from each level) in which the version from the higher noise level receives the higher score, averaged over images (1 means always correct, 0.5 means chance).
-
Test 2, graded image degradations. On the same 20 images, we applied JPEG compression and multilevel quantization, each at ten increasing severity levels, more finely spaced than the five levels of KADID-10k. Discrimination is the share of the 180 steps of each degradation (9 per image) at which the score increases.
-
Test 3, graded speech degradations. On the 54 speech clips used in the paper, we applied high-pass and notch filtering, each at five increasing severity levels. Discrimination is the share of the 216 steps of each degradation (4 per clip) at which the score increases.
For each image (or clip), we first compute the gain of SymKL over the comparison measure: the difference between the shares of steps that the two measures order correctly. The gain reported below is the average of these per-image gains. To check that it does not depend on the particular images used, we drew 10,000 new sets of the same size from our images, at random and with repeats allowed, and recomputed the average gain for each set. Because both measures are always evaluated on the same images, this is a paired bootstrap. The range that contains 95% of these averages is the 95% interval, and a gain is significant when the whole interval lies above zero. Sensitivity is shown for comparison.
| Image test | Sensitivity (SymKL vs DreamSim) | Discrimination (SymKL vs DreamSim) | Uncertainty |
|---|---|---|---|
| 1% noise increase | 2.33 vs 1.75 | AUC 1.000 vs 0.535 | SymKL is correct for every pair on every image; DreamSim's 95% interval (0.50 to 0.57) contains chance |
| JPEG compression | 1.42 vs 0.82 | 100.0% vs 91.7% ordered | gain of 8.3 points, 95% interval 4.4 to 12.2 |
| Multilevel quantization | 2.48 vs 1.72 | 98.9% vs 87.2% ordered | gain of 11.7 points, 95% interval 6.7 to 16.7 |
| Speech test | Sensitivity (SymKL vs PESQ) | Discrimination (SymKL vs PESQ) | Uncertainty |
|---|---|---|---|
| High-pass filtering | 1.31 vs 0.81 | 100.0% vs 95.8% ordered | gain of 4.2 points, 95% interval 1.9 to 6.9 |
| Notch filtering | 3.92 vs 2.18 | 100.0% vs 96.3% ordered | gain of 3.7 points, 95% interval 1.4 to 6.0 |
As can be seen, under a 1% increase in noise, SymKL orders every pair correctly on every image, and every gain on the graded degradations is significant. In each of these tests, SymKL also has the higher sensitivity, so its stronger response comes together with a measurably better ability to tell degradation levels apart.
Weakness 3. Imperfect references, small misalignments, uncertainty and fitting cost
We thank the reviewer for pointing out these gaps; each is addressed below.
1. Imperfect references
We fitted reference networks to natural images from three random initializations each, and saved each network when it reached 30, 33, 35 and 38 dB. Each network then scored all 125 KADID-10k distortions of its image (25 types at 5 levels). We compared the order of these distortions with the order at 35 dB, using Spearman's rank correlation (1 means the same order). The results are averaged over the five images that reached all four accuracies and over the three initializations.
| Fitting accuracy | Agreement with the 35 dB order |
|---|---|
| 30 dB | 0.936 |
| 33 dB | 0.990 |
| 35 dB | 1 (reference) |
| 38 dB | 0.995 |
Between 33 and 38 dB, the order of the distortions closely matches that at 35 dB (0.990 to 0.995); only the considerably coarser fit of 30 dB departs from it more noticeably. A less accurate reference therefore changes SymKL's ordering only gradually, and all 81 natural-image references used in the experiments are fitted at or above 35.05 dB, within the range where the ordering is stable.
We thank the reviewer for the careful review and the constructive suggestions, which helped us strengthen the evaluation. Below, we compare SymKL side by side with INRIQ and separate the effects of the computation, the hypernetwork and the cross-modal experiments. We also report the five forms of evidence the reviewer suggests, all independent of the form of the score, and address imperfect references, small misalignments and the reference-fitting cost.
Q1. Comparison with INRIQ
We thank the reviewer for this suggestion. INRIQ (Jayasundara et al., a withdrawn ICLR 2026 submission) is an unpublished manuscript that has not appeared in any peer-reviewed or archival venue, and therefore we did not treat it as a baseline. Nevertheless, to address the reviewer's question: both works compare the Fisher profiles of a frozen INR fitted to the reference, the approach INRIQ applies to natural images. Below, we compare the two methods side by side in the respects in which they differ, and then use controlled ablations to separate the effects of the proposed computation, the hypernetwork and the cross-modal experiments.
1. Side-by-side account
Fisher computation. INRIQ computes a parameter gradient for every pixel. This paper derives an exact factorization that never forms these per-pixel gradients.
Reference fitting. INRIQ fits each reference network by optimization from a meta-learned initialization. This paper predicts the network's weights with a hypernetwork and then refines them briefly.
Signals and evaluation. INRIQ evaluates natural images, with two MRI examples, using relative sensitivity. This paper evaluates natural images, audio, 3D occupancy fields, and MRI, ultrasound and X-ray, each against the established measures of that modality, using sensitivity together with order-based evidence (Q2).
2. Controlled ablations
Proposed computation. Storing the per-pixel gradients of a 512 × 384 image for the 198,915-parameter network used here would take 146 GB, so the per-pixel computation must be split into chunks. On the same network, image and GPU, the factorization is 17 times faster than the fastest chunked version, and both give the same Fisher profiles up to floating-point rounding. The factorization therefore reduces the cost of computing the score without changing it.
Hypernetwork. We compared the hypernetwork with the standard random initialization of SIREN on KADID-10k references, using the same refinement settings. With the hypernetwork, a reference reaches the target fitting accuracy in about 5 times fewer refinement steps, and does so more reliably. The networks fitted from either starting point rank the distortions of each reference almost identically. The hypernetwork therefore makes reference fitting fast and reliable without changing what SymKL measures.
Cross-modal experiments. This paper formulates the measure for any signal on a coordinate domain, from one-dimensional waveforms to three-dimensional volumes. Applied in exactly the same form to natural images, audio, 3D occupancy fields and medical images, and compared in each case with the established measures of that modality, it follows the physical severity of the degradation closely and orders it perfectly on 3D shapes, ultrasound and X-ray (Q2, Parts 1 and 2). These experiments show that the measure is not specific to natural images.
In summary, the factorization and the hypernetwork make the measure fast to compute and to fit without changing it, and the cross-modal experiments establish its generality. The revised introduction and related-work section will state these contributions relative to INRIQ explicitly.
Q2. Evidence beyond relative sensitivity
We thank the reviewer for this helpful suggestion. The reviewer is correct in this regard: relative sensitivity (hereafter, sensitivity) shows how strongly a score responds to degradation, but a power transformation can raise it, and its absolute value ignores the direction of change. The revised paper will therefore present sensitivity only as response strength. To assess how well SymKL measures degradation, we report below the five forms of evidence the reviewer suggests. All of them depend only on how the scores are ordered, so no power or other increasing transformation can change them, and SymKL is always used in its original form.