acceptodds
Under review as a conference paper at ICLR 2027

SymKL: A Unified Representation-Space Measure of Signal Degradation

Abstract

Quantifying degradation between a pristine signal and its degraded observation is fundamental to signal processing, yet existing full-reference measures are inherently modality-specific. We introduce SymKL, a unified full-reference degradation measure based on implicit neural representations (INRs). An INR is fitted once to the pristine reference and characterized by its diagonal Fisher information, which captures the local information geometry of the learned representation. Evaluating the Fisher information with the pristine reference and a degraded observation yields two parameter-sensitivity profiles whose symmetric KL divergence defines the degradation score. The proposed formulation requires only the reference and degraded observation, without knowledge of the corruption process or degradation severity, making it applicable across images, audio, and 3D data. We further derive an exact closed-form expression for the diagonal Fisher of SIREN, enabling efficient computation. Experiments on image, speech, and volumetric 3D benchmarks demonstrate that SymKL accurately tracks degradation severity, remains monotonic where conventional metrics saturate or fail, and provides a unified modality-agnostic measure of signal degradation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
INRs, Weight Space Representations, Weight Space Measure

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UnyvOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the full-reference degradation score SymKL: Firstly, it combines and freezes the INR on the reference signal, calculates the average of the squared gradient per coordinate caused by the reference and degraded signals, and takes the resulting diagonal vector as the precision of the two Gaussian distributions. Then, it measures the difference using the symmetric KL. The authors also provide the calculation formula for not explicitly storing the gradient of the parameters per coordinate, the hypernetwork for image reference fitting, and conduct experiments on images, audio, 3D occupancy fields, and medical images. The work in the article is meaningful, but I still have concerns about some of the issues present in the article.

Critical strengths
  1. Using INRs to study signal degradation is an intuitive and well-motivated direction. Their shared coordinate-based formulation provides a natural starting point for analyzing different signal modalities.

  2. Evaluating the reference and degraded signals with the same frozen INR avoids the parameter-alignment problem that arises when separate networks are fitted to each signal.

  3. The audio and occupancy-field experiments explore the method’s applicability beyond natural images.

Critical weaknesses

1.Unclear novelty relative to prior work. INRIQ [1] already fits and freezes an INR on the reference signal, constructs diagonal profiles from squared per-coordinate gradients, and compares those profiles using a symmetrized KL divergence. SymKL adds a more efficient computation, amortized image fitting, and experiments in additional modalities, but does not cite INRIQ or clearly distinguish these additions from the previously proposed core measure.

2.The main results show that SymKL changes more rapidly with distortion severity. A monotonic power transformation can increase this sensitivity without improving a metric’s ranking or accuracy, while the absolute value in the sensitivity formula can reward changes in the wrong direction. An independent measure of degradation accuracy is needed.

3.The paper lacks some robustness tests for imperfect references and small misalignments, paired uncertainty estimates, and a complete accounting of reference-fitting cost. These gaps limit the strength of its generality and efficiency claims.

[1] D. Jayasundara, S. Rajagopalan, and V. M. Patel. “INRIQ: Implicit Neural Representation for Image Quality Assessment.” ICLR 2026 submission, 2026.

Questions for authors
  1. A direct comparison with INRIQ would clarify the paper’s contribution. A side-by-side account of the methods, together with controlled ablations, would help distinguish the effects of the proposed computation, hypernetwork, and cross-modal experiments.

  2. Evaluation beyond relative sensitivity would make the empirical conclusions more convincing. Calibration to measured degradation, monotonicity or inversion rates, cross-distortion ranking, and downstream task performance would provide complementary evidence; paired uncertainty estimates would clarify the reported gains.

7 comments

We thank the reviewer for the careful review and the constructive suggestions, which helped us strengthen the evaluation. Below, we compare SymKL side by side with INRIQ and separate the effects of the computation, the hypernetwork and the cross-modal experiments. We also report the five forms of evidence the reviewer suggests, all independent of the form of the score, and address imperfect references, small misalignments and the reference-fitting cost.

Q1. Comparison with INRIQ

We thank the reviewer for this suggestion. INRIQ (Jayasundara et al., a withdrawn ICLR 2026 submission) is an unpublished manuscript that has not appeared in any peer-reviewed or archival venue, and therefore we did not treat it as a baseline. Nevertheless, to address the reviewer's question: both works compare the Fisher profiles of a frozen INR fitted to the reference, the approach INRIQ applies to natural images. Below, we compare the two methods side by side in the respects in which they differ, and then use controlled ablations to separate the effects of the proposed computation, the hypernetwork and the cross-modal experiments.

1. Side-by-side account

  • Fisher computation. INRIQ computes a parameter gradient for every pixel. This paper derives an exact factorization that never forms these per-pixel gradients.

  • Reference fitting. INRIQ fits each reference network by optimization from a meta-learned initialization. This paper predicts the network's weights with a hypernetwork and then refines them briefly.

  • Signals and evaluation. INRIQ evaluates natural images, with two MRI examples, using relative sensitivity. This paper evaluates natural images, audio, 3D occupancy fields, and MRI, ultrasound and X-ray, each against the established measures of that modality, using sensitivity together with order-based evidence (Q2).

2. Controlled ablations

Proposed computation. Storing the per-pixel gradients of a 512 ×\times 384 image for the 198,915-parameter network used here would take 146 GB, so the per-pixel computation must be split into chunks. On the same network, image and GPU, the factorization is 17 times faster than the fastest chunked version, and both give the same Fisher profiles up to floating-point rounding. The factorization therefore reduces the cost of computing the score without changing it.

Hypernetwork. We compared the hypernetwork with the standard random initialization of SIREN on KADID-10k references, using the same refinement settings. With the hypernetwork, a reference reaches the target fitting accuracy in about 5 times fewer refinement steps, and does so more reliably. The networks fitted from either starting point rank the distortions of each reference almost identically. The hypernetwork therefore makes reference fitting fast and reliable without changing what SymKL measures.

Cross-modal experiments. This paper formulates the measure for any signal on a coordinate domain, from one-dimensional waveforms to three-dimensional volumes. Applied in exactly the same form to natural images, audio, 3D occupancy fields and medical images, and compared in each case with the established measures of that modality, it follows the physical severity of the degradation closely and orders it perfectly on 3D shapes, ultrasound and X-ray (Q2, Parts 1 and 2). These experiments show that the measure is not specific to natural images.

In summary, the factorization and the hypernetwork make the measure fast to compute and to fit without changing it, and the cross-modal experiments establish its generality. The revised introduction and related-work section will state these contributions relative to INRIQ explicitly.

Q2. Evidence beyond relative sensitivity

We thank the reviewer for this helpful suggestion. The reviewer is correct in this regard: relative sensitivity (hereafter, sensitivity) shows how strongly a score responds to degradation, but a power transformation can raise it, and its absolute value ignores the direction of change. The revised paper will therefore present sensitivity only as response strength. To assess how well SymKL measures degradation, we report below the five forms of evidence the reviewer suggests. All of them depend only on how the scores are ordered, so no power or other increasing transformation can change them, and SymKL is always used in its original form.

1. Calibration to measured degradation

For each reference signal and degradation type, we measured how closely each score follows the physical severity of the degradation (for example, the noise level or the blur width), using Spearman's rank correlation, 1 means that the score rises exactly with the severity. The values are averaged over all references and degradation types.

DataSymKLOther measures
3D occupancy fields1.000Chamfer 1.000 · IoU 1.000 · F-score 0.934
Audio0.991SNR 0.992 · SI-SDR 0.992 · PESQ 0.990
MRI0.980PSNR 0.980 · SSIM 1.000 · DreamSim 0.940 · CLIP 0.800
Ultrasound1.000PSNR 1.000 · SSIM 1.000 · DreamSim 1.000 · CLIP 0.970
X-ray1.000PSNR 1.000 · SSIM 1.000 · DreamSim 1.000 · CLIP 0.950
Natural images (KADID-10k)0.960PSNR 0.977 · SSIM 0.976 · DreamSim 0.982 · CLIP 0.901

Result. SymKL follows the measured severity closely in every modality (0.96 to 1.00), close to the measures designed for each modality and above CLIP wherever CLIP applies. It does so as a single measure, applied in exactly the same form to all of these signals.

2. Inversion rates

A violation is a step to a higher severity at which the score does not increase. Unlike sensitivity, the violation rate records the direction of change. The following table presents the percentage of steps that are violations for each measure, with all degradation types included.

DataSymKLOther measures
3D occupancy fields0%Chamfer 0% · IoU 0% · F-score 16.3%
Audio1.44%SNR 1.29% · SI-SDR 1.23% · PESQ 1.49%
MRI3.3%PSNR 3.3% · SSIM 0% · DreamSim 3.3% · CLIP 20.0%
Ultrasound0%PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5%
X-ray0%PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5%
Natural images, KADID-10k5.22%PSNR 4.11% · SSIM 3.06% · DreamSim 2.91% · CLIP 10.63%

SymKL has no inversions on 3D shapes, ultrasound and X-ray, is on par with the specialized measures on audio and MRI, and has fewer inversions than F-score on 3D shapes and than CLIP on every image dataset. On natural images, 76% of its inversions come from five KADID-10k types (pixelate, contrast, colour blocks, non-eccentricity and colour shift), which also hold 85% to 96% of the inversions of PSNR, SSIM and DreamSim.

3. Cross-distortion ranking

As a simple experiment, we used the measured signal error, the mean squared difference from the reference, as a common scale for all degradation types. For each reference image, we took all 125 of its distorted versions (25 KADID-10k types at 5 levels), ranked them once by each measure and once by the measured error, and compared the two rankings using Spearman's rank correlation, where 1 means the same order. The table gives the mean over the 81 reference images.

MeasureAgreement with the measured error (Spearman)
SymKL0.965
DreamSim0.746
SSIM0.665
CLIP0.574

Among the measures, SymKL ranks different kinds of damage most consistently with the measured error. The same holds on audio, pooled over all 9 degradation types: SymKL agrees at 0.994, on par with SI-SDR (0.997) and well above PESQ (0.619).

4. Downstream task: benchmarking restoration methods

A common use of a full-reference measure is to rank the outputs of several restoration methods applied to the same degraded signal. We applied standard denoisers to noisy signals and let each measure rank their outputs:

For this we used, 30 KADID-10k reference images, Gaussian noise at four strengths, and 9 methods, from simple filters to total-variation, wavelet and non-local-means denoising (120 ranking problems). As no single measure is the ground truth, we compared each measure's ranking with the consensus of all the other measures, that is, their average ranking with the measure being evaluated left out, using Kendall's rank correlation (1 means the same ranking). The following table gives the mean over all ranking problems.

MeasureAgreement
SymKL0.737
PSNR0.688
SSIM0.564
DreamSim0.531
CLIP0.319

As can be seen, SymKL agrees with the consensus of the other measures more closely than any of them does.

5. Paired uncertainty estimates

We compared SymKL with a strong perceptual measure on the same degraded signals, DreamSim on images and PESQ on speech, in three tests:

  • Test 1, a 1% increase in noise (images). For each of 20 KADID-10k reference images, we generated 20 independently drawn noisy versions with Gaussian noise of standard deviation 0.05 (on intensities in [0, 1]) and 20 with standard deviation 0.0505. Discrimination is the AUC: the share of the 400 pairs (one version from each level) in which the version from the higher noise level receives the higher score, averaged over images (1 means always correct, 0.5 means chance).

  • Test 2, graded image degradations. On the same 20 images, we applied JPEG compression and multilevel quantization, each at ten increasing severity levels, more finely spaced than the five levels of KADID-10k. Discrimination is the share of the 180 steps of each degradation (9 per image) at which the score increases.

  • Test 3, graded speech degradations. On the 54 speech clips used in the paper, we applied high-pass and notch filtering, each at five increasing severity levels. Discrimination is the share of the 216 steps of each degradation (4 per clip) at which the score increases.

For each image (or clip), we first compute the gain of SymKL over the comparison measure: the difference between the shares of steps that the two measures order correctly. The gain reported below is the average of these per-image gains. To check that it does not depend on the particular images used, we drew 10,000 new sets of the same size from our images, at random and with repeats allowed, and recomputed the average gain for each set. Because both measures are always evaluated on the same images, this is a paired bootstrap. The range that contains 95% of these averages is the 95% interval, and a gain is significant when the whole interval lies above zero. Sensitivity is shown for comparison.

Image testSensitivity (SymKL vs DreamSim)Discrimination (SymKL vs DreamSim)Uncertainty
1% noise increase2.33 vs 1.75AUC 1.000 vs 0.535SymKL is correct for every pair on every image; DreamSim's 95% interval (0.50 to 0.57) contains chance
JPEG compression1.42 vs 0.82100.0% vs 91.7% orderedgain of 8.3 points, 95% interval 4.4 to 12.2
Multilevel quantization2.48 vs 1.7298.9% vs 87.2% orderedgain of 11.7 points, 95% interval 6.7 to 16.7
Speech testSensitivity (SymKL vs PESQ)Discrimination (SymKL vs PESQ)Uncertainty
High-pass filtering1.31 vs 0.81100.0% vs 95.8% orderedgain of 4.2 points, 95% interval 1.9 to 6.9
Notch filtering3.92 vs 2.18100.0% vs 96.3% orderedgain of 3.7 points, 95% interval 1.4 to 6.0

As can be seen, under a 1% increase in noise, SymKL orders every pair correctly on every image, and every gain on the graded degradations is significant. In each of these tests, SymKL also has the higher sensitivity, so its stronger response comes together with a measurably better ability to tell degradation levels apart.

Weakness 3. Imperfect references, small misalignments, uncertainty and fitting cost

We thank the reviewer for pointing out these gaps; each is addressed below.

1. Imperfect references

We fitted reference networks to natural images from three random initializations each, and saved each network when it reached 30, 33, 35 and 38 dB. Each network then scored all 125 KADID-10k distortions of its image (25 types at 5 levels). We compared the order of these distortions with the order at 35 dB, using Spearman's rank correlation (1 means the same order). The results are averaged over the five images that reached all four accuracies and over the three initializations.

Fitting accuracyAgreement with the 35 dB order
30 dB0.936
33 dB0.990
35 dB1 (reference)
38 dB0.995

Between 33 and 38 dB, the order of the distortions closely matches that at 35 dB (0.990 to 0.995); only the considerably coarser fit of 30 dB departs from it more noticeably. A less accurate reference therefore changes SymKL's ordering only gradually, and all 81 natural-image references used in the experiments are fitted at or above 35.05 dB, within the range where the ordering is stable.

Reviewer_Q129OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a full-reference degradation measure built on implicit neural representations. A SIREN is fitted to a pristine reference signal and subsequently frozen. The method evaluates the diagonal second moments of per-coordinate reconstruction gradients using either the reference or a degraded observation as the target. These two empirical Fisher profiles are interpreted as the precisions of diagonal Gaussian distributions sharing the same parameter mean, and their symmetrized KL divergence defines the degradation score. The paper also presents an efficient matrix-based computation of the gradient second moments. To accelerate reference fitting for images, it introduces a DINOv2-conditioned hypernetwork trained through reconstruction pretraining and first-order meta-learning. Other modalities use direct INR optimization.

Critical strengths
  1. A coherent construction that can be reused across signal modalities. The choice to fit one reference INR and freeze it for both evaluations is reasonable. It establishes an exact correspondence between the parameters being compared and avoids introducing differences caused by independently optimizing a network for each degraded observation. Reference preparation can also be reused when evaluating multiple degraded versions of the same signal. The system seems to be pretty efficient in the experimental evidence.

  2. The weight-space baselines in Tables 1 and 9 are evaluated using the same frozen reference INR. This controls an important source of variation and helps isolate the effect of the chosen parameter-response statistic. The additional experiments in the appendix also broaden the evidence beyond the examples in the main text.

Critical weaknesses
  1. It's pretty hard to read the paper. The main part lacks the overview of the literature which makes it hard to understand the true improvement comparing to the existing works. The method part is written with mostly inline equations which is hard to distinguish and recall the use of them.

  2. My main concern is the evaluation criterion in Section 2.6. Most of the quantitative comparisons treat higher relative sensitivity as evidence of a better degradation measure. However, this sensitivity can be increased simply by applying a nonlinear transformation to an existing metric, without improving its ability to assess degradation.

The proposed sensitivity is defined as

\frac{ \left|(M_{k+1}-M_k)/M_k\right| }{ (d_{k+1}-d_k)/d_k }.$$ Consider two metrics that depend on degradation severity as follows. $$M_1(d) = d, \qquad M_2(d) = d^2.$$ When severity doubles, the first metric has a sensitivity of 1, while the second has a sensitivity of 3. Both produce exactly the same ordering of samples, and either score can be recovered from the other. Squaring the output therefore adds no information about degradation, yet it improves the result under the proposed evaluation. Taking higher powers would increase the reported sensitivity further. This makes the main comparisons difficult to interpret. SymKL may capture useful information about degradation, but higher sensitivity alone does not establish this. Its advantage could also arise from the nonlinear dependence of the score on the Fisher ratios. The current experiments do not sufficiently separate these explanations. The absolute value in the sensitivity definition also removes the direction of the response. A metric whose score decreases as degradation becomes more severe still contributes positively to the average sensitivity. Consequently, a high sensitivity does not establish that a metric preserves the correct ordering. The illustrated monotonic sequences are useful, but they do not show how frequently ordering errors occur across the full test set. I would find the claims more convincing with rank correlations or pairwise ordering accuracy within each corruption type, together with the frequency of monotonicity violations across all references. For stochastic corruptions, repeated realizations would help establish whether differences between severity levels remain distinguishable despite random variation. Simple power transformations of the existing error baselines would also be a useful control. These comparisons would help determine whether SymKL provides a more reliable assessment of degradation beyond producing a larger relative change in its output.
Questions for authors
  1. How do you justify treating higher relative sensitivity as evidence of a better degradation measure? A simple power transformation can increase this sensitivity without changing the ordering of any samples. What evidence shows that the reported gains reflect more reliable degradation assessment?

  2. Can you report rank correlations and monotonicity violation rates across the full test set, separately for each corruption type? Since the sensitivity calculation takes the absolute value of each change, it would be useful to know how often SymKL assigns a lower score to an observation with greater degradation severity.

  3. For stochastic corruptions such as noise and color blocks, how are the realizations coupled across severity levels? Do the differences between successive levels remain consistent across repeated realizations and reference signals? This would help establish whether the stronger response also provides reliable separation between degradation levels.

I am not fully into this field, but the problem and the paper itself are interesting. Please feel free to correct me if I didn't understand some parts correctly. However, it my main concern can be addressed, I am willing to increate the score. Thanks!

6 comments

We thank the reviewer for the thoughtful review and important questions. In the revised manuscript, we will present sensitivity only as response strength, move the related work into the main paper, and use numbered display equations. Below, we show that SymKL, in its original form, orders degradation reliably, with per-type results and repeated realizations.

Q1. Does higher relative sensitivity reflect a better degradation measure?

We thank the reviewer for raising this central question. The reviewer is correct in this regard: relative sensitivity (hereafter, sensitivity) measures how strongly a score responds to degradation, and a power transformation can raise it without changing any ordering, so sensitivity alone does not establish a better measure. We therefore use sensitivity only to describe response strength, and base the evidence of reliable assessment on statistics that depend only on how the scores are ordered. These statistics are unaffected by squaring or by any other increasing transformation of a score. Our answer has four parts.

1. Sensitivity and SymKL's original form

Sensitivity is the percentage change in a measure's score per percentage change in the physical severity of the degradation. For consecutive severity levels dk<dk+1d_k < d_{k+1} with scores MkM_k and Mk+1M_{k+1},

Sk = ∣(Mk+1−Mk)/Mk(dk+1−dk)/dk∣,S_k \ = \ \left|\frac{(M_{k+1} - M_k)/M_k}{(d_{k+1} - d_k)/d_k}\right| ,

and the reported sensitivity is the mean of SkS_k over all steps. Because it uses relative changes, sensitivity does not depend on the units or the range of a measure, so measures with very different scales can be placed side by side. Sensitivity does, however, depend on the form in which a score is used: squaring a score, for example, changes its sensitivity. SymKL is therefore used in its original form throughout: the symmetrized Kullback--Leibler divergence between the reference and degraded Fisher profiles, exactly as defined, with no power or other transformation. In the revised version of he paper will present sensitivity as response strength only: table captions will read "higher means a stronger response" instead of "higher is better", and the abstract, introduction and conclusion will be reworded in the same way.

2. Higher sensitivity and better discrimination: a simple ablation

If SymKL's higher sensitivity came only from the form of its score, it would not help to tell degradation levels apart. We therefore measured, on the same degraded signals, both how strongly each measure responds (sensitivity) and how reliably it orders the degradation levels (discrimination). We compared SymKL with a strong perceptual measure in each modality: DreamSim, a recent learned measure, for images, and PESQ, the standard speech-quality measure, for speech. Discrimination depends only on the order of the scores, so it cannot be changed by squaring or any other increasing transformation.

  • Test 1: a 1% increase in noise (images). For each of 20 KADID-10k reference images, we generated 20 independently drawn noisy versions with Gaussian noise of standard deviation 0.05, and 20 with standard deviation 0.0505, which is 1% higher. Sensitivity is the relative change in the average score between the two levels, divided by the 1% change in noise. Discrimination is the AUC: for each image, we form all 400 pairs made of one version from each level and count the share in which the version from the higher noise level receives the higher score (1 means always correct, 0.5 means chance), then average over images.

  • Test 2: graded image degradations. On the same 20 images, we attempted another degradations, which are, JPEG compression and Multilevel quantization each at ten increasing severity levels, more finely spaced than the five levels of KADID-10k. Sensitivity is computed as in the paper over the steps between consecutive levels. Discrimination is the share of the 180 steps of each degradation (9 per image) at which the score increases with severity.

  • Test 3: graded speech degradations. On the 54 speech clips used in the paper, we applied high-pass and notch filtering, each at five increasing severity levels. Sensitivity is computed as in the paper, and discrimination is the share of the 216 steps of each degradation (4 per clip) at which the score increases with severity.

Image testSensitivity (SymKL vs DreamSim)Discrimination (SymKL vs DreamSim)
1% noise increase2.33 vs 1.75AUC 1.000 vs 0.535
JPEG compression1.42 vs 0.82100.0% vs 91.7% ordered
Multilevel quantization2.48 vs 1.7298.9% vs 87.2% ordered
Speech testSensitivity (SymKL vs PESQ)Discrimination (SymKL vs PESQ)
High-pass filtering1.31 vs 0.81100.0% vs 95.8% ordered
Notch filtering3.92 vs 2.18100.0% vs 96.3% ordered

As can be seen, in every test, SymKL has both the higher sensitivity and the better discrimination.

3. One measure across all modalities

A violation is a step to a higher severity at which the score does not increase. Unlike sensitivity, the violation rate records the direction of change. The following table presents the percentage of steps that are violations for each measure, with all degradation types included.

DataSymKLOther measures
3D occupancy fields0%Chamfer 0% · IoU 0% · F-score 16.3%
Audio1.44%SNR 1.29% · SI-SDR 1.23% · PESQ 1.49%
MRI3.3%PSNR 3.3% · SSIM 0% · DreamSim 3.3% · CLIP 20.0%
Ultrasound0%PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5%
X-ray0%PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5%
Natural images5.22%PSNR 4.11% · SSIM 3.06% · DreamSim 2.91% · CLIP 10.63%

SymKL is a single measure with one definition, applied in exactly the same form to all of these data, whereas most of the other measures are designed for a single type of signal. With this one formula, SymKL orders degradation perfectly on 3D shapes, ultrasound and X-ray, is on par with the specialized measures on audio and MRI, and has fewer violations than F-score on 3D shapes and than CLIP on every image dataset. On natural images, 76% of SymKL's violations come from five KADID-10k types (pixelate, contrast, colour blocks, non-eccentricity and colour shift), and the same five types also hold 85% to 96% of the violations of PSNR, SSIM and DreamSim.

4. The nonlinear dependence on the Fisher ratios

We also thank the reviewer for the closely related question of whether SymKL's stronger response could come from the nonlinear dependence of the score on the Fisher ratios. This is a fair and insightful point, and it helped us sharpen how the evaluation is presented. This nonlinearity is an inherent part of the measure as defined, not a transformation applied afterwards to raise the score.

To separate a stronger response from more reliable assessment, the evidence in Parts 2 and 3 relies only on the order of the scores: whether the score increases from one severity level to the next (the share of correctly ordered steps and the violation rate), and whether the higher noise level receives the higher score (the AUC). No power or other increasing transformation of a score can change these statistics, so a stronger response alone cannot account for SymKL's results there. Instead, these results show that the measure, with its nonlinearity included, orders degradation correctly.

Q2. Per-type rank correlations and violation rates

We thank the reviewer for this question. Over the full KADID-10k test set, SymKL assigns a lower score to the more degraded observation in 5.22% of adjacent severity steps. In the tables below, Spearman is the rank correlation between score and severity level, averaged over references.

Across modalities. The violation rates for every modality, with all degradation types included, are given in Q1 (Part 3): SymKL has no violations on 3D shapes, ultrasound and X-ray, and is on par with the specialized measures on audio and MRI.

Natural images, full test set. Over all 81 references and 25 types of KADID-10k:

MeasureViolation rateMean Spearman
SymKL5.22%0.960
PSNR4.11%0.977
SSIM3.06%0.976
DreamSim2.91%0.982
CLIP10.63%0.901

Where the violations occur. Most of SymKL's violations (76%) come from five types: pixelate, contrast, colour blocks, non-eccentricity (displaced patches) and colour shift. These five types are equally challenging for the other image measures, accounting for 85% to 96% of the violations of PSNR, SSIM and DreamSim. On four of them, another measure has more violations than SymKL:

  • PSNR on pixelate (29.9%, against 29.6% for SymKL);
  • DreamSim on colour blocks (21.9%, against 19.4% for SymKL) and on non-eccentricity (21.3%, against 14.5% for SymKL);
  • SSIM on colour shift (22.2%, against 10.2% for SymKL).

Colour blocks, non-eccentricity and colour shift are random degradations, and each KADID-10k level contains only a single draw of them, so a comparison between two levels also includes chance variation between draws; the effect of this variation is examined in Q3.

Per-type results for the 12 types reported in the paper. Each cell gives the violation rate and the mean per-reference Spearman correlation with severity.

TypeSymKLPSNRSSIMDreamSimCLIP
Mean shift0.0% / 1.0000.0% / 1.0000.0% / 1.0000.0% / 1.0000.0% / 1.000
JPEG20000.0% / 1.0000.0% / 1.0000.0% / 1.0000.0% / 1.0002.5% / 0.990
White noise0.0% / 1.0000.0% / 1.0000.0% / 1.0000.0% / 1.00016.4% / 0.857
Colour-component noise0.0% / 1.0000.0% / 1.0000.0% / 1.0000.0% / 1.00014.2% / 0.886
Impulse noise0.0% / 1.0000.0% / 1.0000.0% / 1.0000.0% / 1.00019.4% / 0.809
Multiplicative noise0.0% / 1.0000.0% / 1.0000.0% / 1.0000.6% / 0.99812.7% / 0.875
Multilevel quantization0.0% / 1.0000.0% / 1.0000.9% / 0.9961.9% / 0.99320.7% / 0.719
JPEG0.3% / 0.9990.3% / 0.9990.0% / 1.0000.6% / 0.99819.1% / 0.847
Denoising1.2% / 0.9930.6% / 0.9950.0% / 1.0000.0% / 1.0006.2% / 0.964
Colour quantization2.2% / 0.9890.9% / 0.9941.5% / 0.9941.2% / 0.99318.2% / 0.801
Motion blur2.8% / 0.9890.6% / 0.9980.3% / 0.9990.3% / 0.9990.3% / 0.999
Contrast25.0% / 0.75022.2% / 0.93121.0% / 0.7902.2% / 0.9787.4% / 0.926

Q3. Coupling of stochastic corruptions and repeated realizations

We thank the reviewer for this question, which identifies an important source of variation in the benchmark.

Coupling. The realizations are not coupled across severity levels: each level of a random degradation in KADID-10k is a separate, independent random draw, rather than the same random pattern applied with increasing strength. The KADID-10k distortion code sets no random seed and generates each distorted image with a separate call that draws new random numbers. A comparison between two adjacent levels therefore mixes the change in severity with chance variation between draws, and this affects every measure.

Repeated realizations. To separate the two, we re-implemented a few random KADID-10k degradation types in Python, with the same severity levels and the same random elements as the KADID-10k code, and generated 40 independent versions of the benchmark. In each version, each of 20 KADID-10k reference images receives a new random draw at each of the five severity levels, exactly as in KADID-10k, and all degraded images were scored with the paper's reference networks. For each reference image and each pair of adjacent severity levels, we compared every draw at the lower level with every draw at the higher level (40×40=1,60040 \times 40 = 1{,}600 pairs) and recorded the share of pairs in which the higher level receives the higher score. This share, an AUC, is 1 when a measure always separates the two levels, whatever the draws, and 0.5 when it separates them only by chance. The following table reports it for the stochastic corruptions the reviewer mentions.

Consistency across repeated realizations and reference signals: The share of the 1,600 pairs of draws (40 at each of two adjacent levels) in which the higher severity level receives the higher score, averaged over 20 reference images and the 4 steps between adjacent levels (1 means always correct, 0.5 means chance).

DegradationSymKLPSNRSSIMDreamSimCLIP
White noise1.0001.0001.0000.9980.883
Colour-component noise1.0001.0001.0001.0000.844
Impulse noise1.0001.0001.0001.0000.814
Multiplicative noise1.0001.0001.0000.9990.869
Colour blocks0.7950.8340.9940.7620.657

On all four noise types, SymKL separates adjacent severity levels in every comparison (AUC 1.000), as PSNR and SSIM do: it orders every step correctly in every one of the 40 versions, for all 20 reference images, whereas DreamSim makes occasional errors and CLIP frequent ones. For colour blocks, whose block positions and colours are redrawn at every level, the variation between draws is large for every measure, and SymKL separates adjacent levels more reliably than DreamSim and CLIP.

Q4. Presentation: readability

We thank the reviewer for this constructive feedback and agree that the presentation can indeed be made considerably clearer. To show the improvement over existing work, the revised paper will move the related work from the appendix to a dedicated section in the main paper, covering full-reference measures for each modality, implicit neural representations and related representation-based approaches. This section will also state the key distinction of SymKL: a single measure applied in exactly the same form across images, audio, 3D shapes and medical images. To make the method easier to follow, the key definitions will be given as numbered display equations rather than inline expressions.

Reviewer_gt9kOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SymKL, a full-reference degradation measure built on a frozen implicit neural representation that is fitted to the reference signal. It compares the diagonal empirical Fisher profiles induced by the reference and the degraded targets via a symmetrized KL score. The authors also provide an efficient and exact way to compute these profiles, and use a meta-learned hypernetwork to speed up reference fitting for images. Experiments on natural images, audio, occupancy fields, and medical images then examine how the measure responds as distortion severity increases.

Critical strengths

The shared reference representation is a useful design choice. Both targets are evaluated using the same frozen INR, avoiding the additional variation introduced by fitting a separate network to each degraded observation. Applying this procedure across images, audio, and occupancy fields gives the work value beyond a single application.

The paper provides a clear computational route for making the descriptor practical. The derivation in Appendix A.3 computes the diagonal empirical Fisher without explicitly materializing per-coordinate parameter gradients. Together with the cross-modal experiments and illustrative degradation sequences, this provides a useful proof of concept. These methodological and implementation contributions are the main reasons for my positive recommendation.

Critical weaknesses

The experiments provide useful evidence of responsiveness, and the illustrated sequences show encouraging monotonic behavior. My reservation concerns the interpretation of the aggregate sensitivity rankings: higher absolute elasticity can reflect differences in the nonlinear parametrization of metric outputs, and the statistic itself does not record response direction. These rankings therefore do not, on their own, establish better discrimination between degradation levels. Aggregate ordering statistics and a clearer justification for comparing the response scales would strengthen the comparative conclusions.

Some numerical and robustness details remain unclear. The reference empirical Fisher depends on the residual error of the fitted INR, and its entries can become very small as reconstruction improves. Since the score uses ratios of Fisher entries, the treatment of zero or near-zero values should be specified. It would also be useful to establish whether the reported conclusions remain stable across reasonable changes in reference-fitting accuracy and initialization.

Questions for authors

Could you clarify why higher elasticity across metrics with different response scales should be interpreted as better discrimination between degradation levels? Aggregate monotonicity-violation rates would help substantiate the ordering claims. A simple nonlinear-transform control, such as squared Chamfer, could also help assess how much the sensitivity ranking depends on score parametrization.

What numerical convention is used for zero or very small Fisher entries? Could you clarify the reference-fitting accuracy used in the experiments and provide a small robustness check across fitting accuracies or initializations?

3 comments

We thank the reviewer for the careful and positive review, and for recognizing the value of a shared frozen reference representation and of the exact Fisher computation. The questions helped us make the evaluation clearer. We now present sensitivity only as response strength, and show that SymKL, used in its original form, also orders degradation reliably across all modalities. We also state the convention for small Fisher entries and show that SymKL is stable across fitting accuracies.

Q1. Sensitivity, discrimination and score parametrization

We thank the reviewer for this question. The reviewer is correct in this regard: sensitivity (the elasticity) measures how strongly a score responds to degradation, and a stronger response alone does not establish better discrimination between degradation levels. We therefore use sensitivity only to describe response strength, and assess discrimination with statistics that depend only on how the scores are ordered. These statistics are unaffected by squaring or by any other increasing transformation of a score.

1. Sensitivity and SymKL's original form

Sensitivity is the percentage change in a measure's score per percentage change in the physical severity of the degradation. For consecutive severity levels dk<dk+1d_k < d_{k+1} with scores MkM_k and Mk+1M_{k+1},

S_k \= \left|\frac{(M_{k+1} - M_k)/M_k}{(d_{k+1} - d_k)/d_k}\right| ,

and the reported sensitivity is the mean of SkS_k over all steps. Because it uses relative changes, sensitivity does not depend on the units or the range of a measure, so measures with very different scales can be placed side by side. Sensitivity does, however, depend on the form in which a score is used: squaring a score, for example, changes its sensitivity. SymKL is therefore used in its original form throughout: the symmetrized Kullback--Leibler divergence between the reference and degraded Fisher profiles, exactly as defined, with no power or other transformation. The revised version of the paper will present sensitivity as response strength only, table captions will read "higher means a stronger response" instead of "higher is better", and the abstract, introduction and conclusion will be reworded in the same way.

2. Higher sensitivity and better discrimination: a simple ablation

If SymKL's higher sensitivity came only from the form of its score, it would not help to tell degradation levels apart. We therefore measured, on the same degraded signals, both how strongly each measure responds (sensitivity) and how reliably it orders the degradation levels (discrimination). We compared SymKL with a strong perceptual measure in each modality: DreamSim, a recent learned measure, for images, and PESQ, the standard speech-quality measure, for speech. Discrimination depends only on the order of the scores, so it cannot be changed by squaring or any other increasing transformation.

  • Test 1: a 1% increase in noise (images). For each of 20 KADID-10k reference images, we generated 20 independently drawn noisy versions with Gaussian noise of standard deviation 0.05, and 20 with standard deviation 0.0505, which is 1% higher. Sensitivity is the relative change in the average score between the two levels, divided by the 1% change in noise. Discrimination is the AUC: for each image, we form all 400 pairs made of one version from each level and count the share in which the version from the higher noise level receives the higher score (1 means always correct, 0.5 means chance), then average over images.

  • Test 2: graded image degradations. On the same 20 images, we attempted another degradations, which are JPEG compression and multilevel quantization, each at ten increasing severity levels. Sensitivity is computed as in the paper over the steps between consecutive levels. Discrimination is the share of the 180 steps (9 per image) at which the score increases with severity.

  • Test 3: graded speech degradations. On the 54 speech clips used in the paper, we applied high-pass and notch filtering, each at five increasing severity levels. Sensitivity is computed as in the paper, and discrimination is the share of the 216 steps (4 per clip) at which the score increases with severity.

Image testSensitivity (SymKL vs DreamSim)Discrimination (SymKL vs DreamSim)
1% noise increase2.33 vs 1.75AUC 1.000 vs 0.535
JPEG compression1.42 vs 0.82100.0% vs 91.7% ordered
Multilevel quantization2.48 vs 1.7298.9% vs 87.2% ordered
Speech testSensitivity (SymKL vs PESQ)Discrimination (SymKL vs PESQ)
High-pass filtering1.31 vs 0.81100.0% vs 95.8% ordered
Notch filtering3.92 vs 2.18100.0% vs 96.3% ordered

As can be seen, in every test, SymKL has both the higher sensitivity and the better discrimination.

3. Aggregate monotonicity-violation rates

A violation is a step to a higher severity at which the score does not increase. Unlike sensitivity, the violation rate records the direction of change. The following table presents the percentage of steps that are violations for each measure, with all degradation types included.

DataSymKLOther measures
3D occupancy fields0%Chamfer 0% · IoU 0% · F-score 16.3%
Audio1.44%SNR 1.29% · SI-SDR 1.23% · PESQ 1.49%
MRI3.3%PSNR 3.3% · SSIM 0% · DreamSim 3.3% · CLIP 20.0%
Ultrasound0%PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5%
X-ray0%PSNR 0% · SSIM 0% · DreamSim 0% · CLIP 7.5%
Natural images, KADID-10k5.22%PSNR 4.11% · SSIM 3.06% · DreamSim 2.91% · CLIP 10.63%

SymKL is a single measure with one definition, applied in exactly the same form to all of these data, whereas most of the other measures are designed for a single type of signal. With this one formula, SymKL orders degradation perfectly on 3D shapes, ultrasound and X-ray, is on par with the specialized measures on audio and MRI, and has fewer violations than F-score on 3D shapes and than CLIP on every image dataset. On natural images, 76% of SymKL's violations come from five KADID-10k types (pixelate, contrast, colour blocks, non-eccentricity and colour shift), and the same five types also hold 85% to 96% of the violations of PSNR, SSIM and DreamSim.

4. The squared-Chamfer control

We thank the reviewer for suggesting this control. Squaring a score raises its sensitivity without changing how it orders any degradation, so the sensitivity ranking of different measures can change under such a transformation. This is precisely why, throughout the paper, every measure is used in its standard form and SymKL is used in its original form, exactly as defined, and why discrimination is judged by statistics that depend only on ordering (Parts 2 and 3).

On the 3D experiments (4 shapes, 5 degradations, 80 steps between consecutive levels), squared Chamfer orders every step correctly, exactly as Chamfer does, and SymKL in its original form also orders every step correctly. SymKL achieves this as a single measure applied in exactly the same form to 3D shapes, audio, medical and natural images.

Q2(a). Zero or very small Fisher entries

We thank the reviewer for raising this point; the revised paper will state the convention explicitly. Every ratio of Fisher entries in SymKL is regularized with a small constant ε=10−12\varepsilon = 10^{-12} in both numerator and denominator,

rj  =  Fjdeg+εFjref+ε,r_j \; = \; \frac{F^{\text{deg}}_j + \varepsilon}{F^{\text{ref}}_j + \varepsilon},

where jj indexes the network parameters. The logarithm in the KL divergence is taken of this regularized ratio, so no entry is ever divided by zero. Further, we measured the reference Fisher profile for to see whether any near-zero entries occur. The following tables summerizes the results for each modality.

DataZero entriesMedian entry
Imagesnone5.3×10−45.3 \times 10^{-4}
3D occupancy fieldsnone1.8×10−41.8 \times 10^{-4}
Audionone4.0×10−94.0 \times 10^{-9}

Q2(b). Reference-fitting accuracy and initialization

We thank the reviewer for this suggestion. The reference networks of the 81 KADID-10k natural images reach an average PSNR of 35.53±0.5435.53 \pm 0.54 dB, with every reference at or above 35.05 dB.

Robustness to fitting accuracy. We fitted reference networks to natural images from three random initializations each, and saved each network when it reached 30, 33, 35 and 38 dB. Each network then scored all 125 KADID-10k distortions of its image (25 types at 5 levels). We compare the order of these distortions with the order at 35 dB using Spearman's rank correlation (1 means the same order). Results are averaged over the five images that reached all four accuracies and over the three initializations.

Fitting accuracyAgreement with the 35 dB order
30 dB0.936
33 dB0.990
35 dB1 (reference)
38 dB0.995

Between 33 and 38 dB, the order of the distortions closely matches that at 35 dB (0.990 to 0.995); only the considerably coarser fit of 30 dB departs from it more noticeably. All 81 natural-image references used in the experiments are fitted at or above 35.05 dB, within this stable range.

We sincerely thank the reviewer again for the positive assessment and constructive feedback. We have provided additional experiments addressing degradation ordering, numerical stability, and robustness across reference-fitting accuracies, as suggested. As the discussion deadline is approaching, we would be grateful if the reviewer could take a look at our responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.