acceptodds
Under review as a conference paper at ICLR 2027

RaFL: An Empirical Study of Random Filter Losses for Image Reconstruction

Abstract

Pixel losses train image reconstruction networks toward a point statistic of the target distribution, often averaging away fine textures under uncertainty. Perceptual losses alleviate this issue using pretrained features, but those features are inherently domain-specific. We study a random filter loss (RaFL) that compares prediction and target through filters with random weights, avoiding both pretrained features and task-specific loss engineering. RaFL combines random convolutions, random Fourier features, and random wavelet filters, automatically balancing their contributions through gradient-norm matching, eliminating manual loss-weight tuning and grid search. We systematically investigate three design choices: filter family, weight resampling schedule, and weight initialization. Across all settings, the best configuration consistently uses all three filter families, redraws filters every optimization step, and employs Gaussian initialization. Using this configuration, we evaluate RaFL on three 2D and two 3D image reconstruction tasks. RaFL outperforms a pixel loss on three tasks and matches it on two, while outperforming prior random-filter losses on all 2D tasks. RaFL adds less than 2% to the backbone's forward FLOPs and no inference cost, remaining 182x cheaper than a VGG19 perceptual loss. These findings suggest that RaFL is a simple, tuning-free, and domain-agnostic loss objective for 2D and 3D image reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Reconstruction, Loss Functions, Random Filter Loss, Perceptual Loss, Deep Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vsvmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RaFL, a random filter loss for image reconstruction. It combines convolutional, Fourier, and wavelet features with adaptive loss weighting, and studies filter selection, resampling frequency, and initialization.

Critical strengths

The loss requires no pretrained features or additional inference computation. Evaluation across natural, remote-sensing, and medical images, with eight random seeds per configuration, provides useful evidence of transferability and training stability.

Critical weaknesses
  1. The comparison with pretrained perceptual losses is missing. Although avoiding domain-specific pretrained features is a central motivation, the VGG19 baseline uses random weights.
  2. The design choices are mainly evaluated on SIDD using DnCNN. It remains unclear whether the same filter combination and initialization are preferable for other tasks and network architectures.
  3. The improvement in perceptual quality is relatively limited. On SIDD, RaFL performs slightly worse than the pixel loss on both LPIPS and DISTS, and using only convolution and wavelet filters gives better results than the full combination. These results do not sufficiently support the motivation of improving texture preservation.
  4. The actual training cost is substantially higher despite the low FLOPs. On SIDD, RaFL takes 17.8 ms per iteration, compared with 3.3 ms for the pixel loss and 5.4 ms for untrained VGG19. Even with Adam alone, it takes 12.1 ms.
Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer vsvm (1/4): CW1 pretrained perceptual loss

Response to Reviewer vsvm (R3)

We thank Reviewer vsvm for the careful review and for recognizing that RaFL is simple, transferable, and stable. Please find the following answers to your concerns:

CW1. Comparison with a pretrained perceptual loss.

We have added a pretrained perceptual loss baseline. We trained pixel ℓ1\ell_1 + pretrained VGG19 (Lℓ1+LVGG19\mathcal{L} _{\ell_1} + \mathcal{L} _{\mathrm{VGG19}} with ImageNet-pretrained VGG19 features) on SIDD and LOL, the only tasks with 3-channel RGB images where such features are directly applicable, using the same eight seeds (42–49).

Task & MethodPSNR ↑SSIM ↑LPIPS ↓DISTS ↓Loss GFLOPs
SIDD (2D Denoising, DnCNN)
RaFL (Ours)40.28 ± 0.130.9557 ± 0.00120.2488 ± 0.00360.1847 ± 0.00341.12
pixel ℓ1\ell_139.64 ± 1.000.9551 ± 0.00110.2478 ± 0.00560.1830 ± 0.00570.00
uPL, VGG19 (untrained)39.18 ± 0.160.9410 ± 0.00200.2631 ± 0.00580.1903 ± 0.0032203.8
pixel ℓ1\ell_1 + pretrained VGG1939.28 ± 0.260.9443 ± 0.00140.2328 ± 0.00270.1807 ± 0.0033203.8
LOL (2D Enhancement, DRBN)
RaFL (Ours)21.17 ± 0.330.8035 ± 0.00320.2568 ± 0.00430.1995 ± 0.00321.12
pixel ℓ1\ell_121.16 ± 0.460.7799 ± 0.00310.3055 ± 0.00750.2273 ± 0.00490.00
pixel ℓ1\ell_1 + pretrained VGG1920.74 ± 0.590.8292 ± 0.00510.1946 ± 0.00380.1655 ± 0.0020203.8

Compared with pixel ℓ1\ell_1 + pretrained VGG19, RaFL achieves higher PSNR on both tasks (+1.00 dB on SIDD, +0.43 dB on LOL). Adding the pretrained VGG19 term lowers PSNR below pixel ℓ1\ell_1 alone (-0.36 dB on SIDD, -0.42 dB on LOL). The pretrained term does achieve lower LPIPS and DISTS on both tasks and higher SSIM on LOL, which is expected since LPIPS and DISTS are themselves computed from ImageNet-pretrained features. In contrast, RaFL requires no pretrained weights, and uses 182× fewer FLOPs in the loss and applies directly to multispectral and 3D data, whereas pretrained VGG19 features are limited to 2D RGB images. We have added this baseline to Table 2 and discussed these results in Sec. 4.3.

AuthorsOpenReview ↗

Response to Reviewer vsvm (2/4): CW2 transfer to other tasks and architecture

CW2. Do the filter-family and initialization choices transfer to other tasks and architectures?

Yes, they transfer as a PSNR- and stability-oriented default. We have repeated the filter-family and initialization ablations on another task and architecture, and tested the default on two more architectures.

(1) WorldView-2 with SFINet (App. A.7, Table S10, which also lists SSIM, ERGAS and SAM):

ConfigurationSIDD, DnCNN: PSNR ↑ (Table 1)WorldView-2, SFINet: PSNR ↑
Default: all three families, Gaussian (RaFL)40.28 ± 0.1339.62 ± 0.08
Filter families (Gaussian initialization)
Fourier + Wavelet39.28 ± 1.3639.64 ± 0.15
Fourier only39.92 ± 0.3239.47 ± 0.13
Convolution + Fourier39.98 ± 0.3339.41 ± 0.13
Convolution + Wavelet39.88 ± 0.7539.28 ± 0.13
Convolution only39.75 ± 0.1738.20 ± 0.20
Wavelet only32.26 ± 3.1334.74 ± 4.15
Initialization (all three families)
Kaiming normal39.92 ± 0.6039.57 ± 0.12
Xavier uniform39.98 ± 0.4139.60 ± 0.18
Sparse39.91 ± 0.3139.59 ± 0.12

On WorldView-2 the default ties with Fourier + Wavelet for the highest PSNR, outperforms the other five subsets, and has the lowest standard deviation across seed (0.08 dB), as on SIDD. The four initializations lie within 0.05 dB of each other, and Gaussian initialization again has the highest mean PSNR and the lowest standard deviation across seed.

(2) Other backbones on SIDD (App. A.7, Table S11):

Backbone (SIDD)LossPSNR ↑SSIM ↑LPIPS ↓DISTS ↓
DnCNN (Table 2)pixel ℓ1\ell_139.64 ± 1.000.9551 ± 0.00110.2478 ± 0.00560.1830 ± 0.0057
RaFL40.28 ± 0.130.9557 ± 0.00120.2488 ± 0.00360.1847 ± 0.0034
U-Net (new)pixel ℓ1\ell_135.21 ± 0.650.9079 ± 0.00220.3406 ± 0.00750.2414 ± 0.0031
RaFL35.45 ± 0.890.9208 ± 0.00740.3090 ± 0.00710.2253 ± 0.0025
DRUNet (new)pixel ℓ1\ell_142.02 ± 0.030.9669 ± 0.00010.2215 ± 0.00090.1873 ± 0.0007
RaFL42.06 ± 0.050.9669 ± 0.00020.2197 ± 0.00070.1863 ± 0.0007

On both backbones the default matches pixel ℓ1\ell_1 in PSNR and achieves lower LPIPS and DISTS, clearly with U-Net and slightly with DRUNet.

Accordingly, we have revised the manuscript to present the SIDD-selected configuration as a practical PSNR- and stability-oriented default, added both tables to App. A.7, and updated Sec. 5 to state that the design study now covers two datasets (SIDD and WorldView-2).

AuthorsOpenReview ↗

Response to Reviewer vsvm (3/4): CW3 perceptual gains

CW3. Perceptual gains and the texture-preservation motivation.

We thank the reviewer for this careful observation. We agree that SIDD with DnCNN does not show a perceptual advantage for RaFL. There, RaFL and the pixel loss do not differ significantly on LPIPS or DISTS. On this setting, RaFL's benefit is instead in PSNR and stability: +0.64 dB on average, with a much smaller standard deviation (0.13 vs. 1.00 dB). Similarly, adding the Fourier family to Convolution + Wavelet changes LPIPS and DISTS little but adds 0.40 dB PSNR and cuts the standard deviation from 0.75 to 0.13 dB (Table 1).

The perceptual gains are clear on other backbones and tasks:

  • U-Net and DRUNet on SIDD (CW2): lower LPIPS and DISTS than the pixel loss, at matched PSNR.
  • LOL (Table 2): the best LPIPS (0.2568 vs. 0.3055 for pixel ℓ1\ell_1) and DISTS (0.1995 vs. 0.2273) among methods without pretrained features, and SSIM 0.8035 vs. 0.7799.
  • WorldView-2 (Table 2): better than all baselines on all four metrics.
  • OASIS3 (Table 3): better than the pixel loss and uPL in PSNR, SSIM and LPIPS.

We agree that the original framing overstated texture preservation as the main benefit on SIDD. In the revised Abstract and Introduction, we now frame the selected configuration around PSNR and cross-seed stability, and present the perceptual gains as task- and backbone-dependent.

AuthorsOpenReview ↗

Response to Reviewer vsvm (4/4): CW4 training cost

CW4. Training cost.

We thank the reviewer for highlighting the gap between theoretical FLOPs and wall-clock training time. We have revised the Abstract to state "requiring 182× fewer loss FLOPs than a VGG19 perceptual loss", so that it refers to loss FLOPs rather than training cost.

Profiling our submitted code showed that the 12.1–17.8 ms step time on SIDD was caused by CUDA kernel-launch overhead from sequential small-filter evaluations rather than arithmetic volume. Keeping the backbone and eager execution unchanged, we refactored the loss by batching the filter evaluations and reusing intermediate output gradients. This mathematically equivalent implementation cuts the step time by more than half:

Loss on SIDD (DnCNN)Loss GFLOPsSubmitted ms/it (former Table 4)Refactored ms/it (Table S12)Ratio vs. pixel ℓ1\ell_1
RaFL (MAdam)1.1217.8 ms8.3 ms (2.1× faster)2.52× (was 5.4×)
RaFL (Adam only)1.1212.1 ms4.6 ms (2.6× faster)1.39× (was 3.7×)
pixel ℓ1\ell_10.003.3 ms3.3 ms1.00×
uPL, SimpleNet10.123.6 ms3.6 ms1.09×
CDC5.004.2 ms4.2 ms1.27×
uPL, VGG19203.805.4 ms5.4 ms1.64×

After refactoring, RaFL with Adam only (4.6 ms) is faster than the untrained VGG19 loss (5.4 ms), and RaFL with MAdam (8.3 ms) takes 1.5× as long, down from 3.3×. RaFL also reaches 1.10 dB higher PSNR than the untrained VGG19 loss on SIDD (40.28 vs. 39.18 dB, Table 2) with 182× fewer loss FLOPs.

On OASIS3 (SwinUNETR), RaFL with Adam only is within 6% of the pixel loss, and most of the remaining overhead comes from the MAdam preconditioner:

Loss on OASIS3 (SwinUNETR)Loss GFLOPsSubmitted ms/it (former Table 4)Refactored ms/it (Table S12)Ratio vs. pixel ℓ1\ell_1
RaFL (MAdam)11.8296.0 ms265.2 ms2.07× (was 2.3×)
RaFL (Adam only)11.8166.6 ms135.6 ms1.06× (was 1.3×)
pixel ℓ1\ell_10.00128.4 ms128.1 ms1.00×
uPL, VGG191821.0138.4 ms138.4 ms1.08×

We have updated Sec. 4.4, Sec. 5, App. C.3, and Table S12 (formerly Table 4), and we will include the refactored loss in the code release.

We hope our responses have addressed your concerns, and we would appreciate it if you would take them into account in your final evaluation. We are happy to clarify any remaining points.

Reviewer_Z12FOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies random filter losses for image reconstruction. RaFL combines random convolutions, random Fourier features, and random wavelet filters, automatically balancing their contributions through gradient-norm matching, eliminating manual loss-weight tuning and grid search. The reconstruction backbone and inference procedure remain unchanged. The paper studies filter families, redraw schedules and initialization on SIDD, then evaluates the selected configuration on five 2D and 3D tasks. The reported gains depend on the task and evaluation metric.

Critical strengths
  1. The paper provides a simple but clear loss interface without pretrained features or additional inference computation. Experiments demonstrating that a single random-filter objective can transfer across reconstruction problems spanning natural and medical images in both 2D and 3D.
  2. The paper provides detailed ablations with eight seeds per configuration. It separately examines loss weighting and preconditioning, and reports cross-task redraw results, including failed runs and metric trade-offs.
Critical weaknesses
  1. Firstly, I still have concerns about the claim of “a consistently best configuration”. Filter-family and initialization ablations are mainly conducted on SIDD. Applying the selected configuration to other tasks provides evidence of transfer. Whether the same design ranking still needs further verifications. In Table 1, Convolution + Wavelet has better mean LPIPS and DISTS than the full combination. In Table S9, despite lower PSNR, per-epoch redraw on LOL has better SSIM, LPIPS and DISTS than per-step redraw. My concern is that the current evidence supports a useful PSNR-oriented default instead of a general rule for random-filter loss design.
  2. Additionally, you still need to double check your figures. There is a contradiction between Fig1 and it's introduction: the figure has mixed up Kaiming normal (He et al., 2015) and Xavier uniform (Glorot & Bengio, 2010). Moreover I think it would be better to put the REFERENCES in the main paper rather than supplementary materials.
Questions for authors

The formulation is useful, and it would be better to retain reported task-specific improvements. My main concerns lie in the scope of the design conclusions and the practical value of the full training recipe. These limitations may affect my decision.

  1. Could you clarify the metric used to select the default configuration and provide existing filter-family or initialization comparisons beyond SIDD?
  2. Could you provide the run configurations and paired per-seed results for Tables 2–3 and S5, specifying how MAdam and loss weighting were applied to each baseline?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Z12F (1/3): CW1 scope of the default configuration, CW2 figure and references

Response to Reviewer Z12F (R2)

We thank Reviewer Z12F for helpful comments and for highlighting generality and rigorous evaluation of our method.

CW1. Clarifying the scope of the “consistently best configuration” claim.

We agree with this assessment. The current evidence supports a PSNR- and stability-oriented default configuration that transfers effectively across tasks, rather than a universal design ranking across all metrics.

Specifically, adding the Fourier family slightly worsens LPIPS and DISTS on SIDD (Table 1) but raises mean PSNR by 0.40 dB and lowers the cross-seed standard deviation from 0.75 to 0.13 dB. Likewise, the per-step redraw on LOL trades SSIM, LPIPS, and DISTS for PSNR (Table S9) while still outperforming all non-pretrained baselines on LPIPS and DISTS. The new WorldView-2 ablation (see Q1 below) supports the same conclusion: the default ties for the highest PSNR there, but Fourier + Wavelet matches it, so the ordering of the subsets is task-dependent.

Accordingly, we have revised the Abstract, Introduction and Sec. 5 to describe the configuration as a default selected on SIDD for PSNR and seed stability, replacing “the best configuration consistently uses...”.

CW2. Correction of figure and reference.

Thank you for catching both issues. We have corrected Fig. 1 in the revised manuscript so that the initialization labels match Sec. 3.2 and Table 1. The References are in the main paper after Sec. 5, before the Appendix.

AuthorsOpenReview ↗

Response to Reviewer Z12F (2/3): Q1 scope of the design conclusions

Q1. Metric used to select the default configuration and comparisons beyond SIDD.

(1) Selection criterion. The default configuration in Sec. 4.2 was selected on the SIDD validation split using mean validation PSNR as the primary criterion and cross-seed PSNR standard deviation as the secondary stability criterion (now stated in Sec. 4.1).

(2) Filter-family and initialization ablations on WorldView-2 (SFINet). To evaluate these design choices beyond SIDD, we conducted ablations on WorldView-2 with SFINet under the exact settings of Table 2 (added to App. A.7, Table S10):

ConfigurationSIDD, DnCNN: PSNR ↑ (Table 1)WorldView-2, SFINet: PSNR ↑SSIM ↑ERGAS ↓SAM ↓
Default: all three families, Gaussian (RaFL)40.28 ± 0.1339.62 ± 0.080.9631 ± 0.00051.318 ± 0.0110.0297 ± 0.0002
Filter families (Gaussian initialization)
Fourier + Wavelet39.28 ± 1.3639.64 ± 0.150.9634 ± 0.00121.304 ± 0.0240.0295 ± 0.0005
Fourier only39.92 ± 0.3239.47 ± 0.130.9629 ± 0.00051.338 ± 0.0140.0300 ± 0.0004
Convolution + Fourier39.98 ± 0.3339.41 ± 0.130.9626 ± 0.00081.350 ± 0.0190.0303 ± 0.0005
Convolution + Wavelet39.88 ± 0.7539.28 ± 0.130.9608 ± 0.00081.365 ± 0.0180.0309 ± 0.0004
Convolution only39.75 ± 0.1738.20 ± 0.200.9544 ± 0.00191.582 ± 0.0540.0351 ± 0.0009
Wavelet only32.26 ± 3.1334.74 ± 4.150.9563 ± 0.00642.229 ± 1.3720.0685 ± 0.0474
Initialization (all three families)
Kaiming normal39.92 ± 0.6039.57 ± 0.120.9631 ± 0.00091.321 ± 0.0190.0298 ± 0.0004
Xavier uniform39.98 ± 0.4139.60 ± 0.180.9631 ± 0.00121.318 ± 0.0260.0297 ± 0.0006
Sparse39.91 ± 0.3139.59 ± 0.120.9632 ± 0.00081.318 ± 0.0170.0297 ± 0.0004
  • Filter families: On WorldView-2, the default configuration ties with Fourier + Wavelet for the highest mean PSNR while achieving the lowest standard deviation across seed (0.08 dB), and outperforms the remaining five subsets. Unlike on SIDD (where Fourier + Wavelet has higher variance, 39.28 ± 1.36 dB), Fourier + Wavelet matches the default on WorldView-2, indicating that the relative ordering of proper subsets depends on the task while the full three-family default remains strong and stable.
  • Initialization: Across the four initializations, mean PSNR on WorldView-2 varies by at most 0.05 dB, with Gaussian again achieving the highest mean PSNR and smallest cross-seed variance.
AuthorsOpenReview ↗

Response to Reviewer Z12F (3/3): Q2 practical value of the full training recipe

Q2. The practical value of the full training recipe.

(1) Run configurations and optimizer/weighting protocol (Sec. 4.1 & App. C.1). Within each task, all methods share the exact same backbone, scene/subject-disjoint data splits, patch/volume size, epoch budget, learning rate, and random seeds (42–49), and are evaluated at the final epoch:

  • Single-term baselines (pixel ℓ1\ell_1, uPL SimpleNet, uPL VGG19): Trained with standard Adam and a unit loss weight (λ=1\lambda = 1). Because their objective has a single fixed weight, the second-moment mismatch addressed by MAdam does not arise.
  • Two-term random prototype baselines (CDC, reverse, Taylor, INN): Following Zhou et al. (2023), each prototype is added to pixel ℓ1\ell_1 with fixed equal weights (Lℓ1+Lproto\mathcal{L} _{\ell_1} + \mathcal{L} _{\mathrm{proto}}), redrawn once per epoch, and trained with Adam.
  • RaFL (Ours): Combines ϕ1,ϕ2,ϕ3\phi_1, \phi_2, \phi_3 without a pixel ℓ1\ell_1 term, updates λ1,λ2,λ3\lambda_1, \lambda_2, \lambda_3 every 50 steps via output gradient-norm matching, and preconditions gradients with MAdam followed by Adam. We have updated Sec. 4.1 to state this protocol explicitly.

(2) Contribution of the filter loss vs. MAdam (Table S5 vs. Table 2). Comparing RaFL trained with Adam (Table S5) against the baselines in Table 2 isolates the effect of the loss from MAdam:

  • SIDD: RaFL with Adam alone reaches 40.19 ± 0.37 dB, outperforming pixel ℓ1\ell_1 (39.64 ± 1.00 dB) and uPL SimpleNet (40.09 ± 0.07 dB). Adding MAdam further reduces cross-seed variance to 40.28 ± 0.13 dB.
  • LOL: RaFL with Adam alone (20.91 ± 0.34 dB) outperforms all six non-pretrained baselines in Table 2 on LPIPS (0.2588) and DISTS (0.2018). Adding MAdam improves PSNR by +0.26 dB to 21.17 ± 0.33 dB, matching pixel ℓ1\ell_1 (21.16 dB) while retaining the perceptual gains.
  • WorldView-2: RaFL with Adam alone (38.74 ± 0.13 dB) outperforms pixel ℓ1\ell_1 (38.10 ± 0.13 dB) and matches CDC (38.75 ± 0.14 dB), while adding MAdam yields an additional +0.87 dB gain (to 39.62 ± 0.08 dB), outperforming all baselines across all four metrics.

(3) Paired per-seed results. As requested, we have added the complete paired per-seed results (seeds 42–49) for Table S5, Table 2, and Table 3 to Appendix D (Tables S13–S17) of the revised manuscript. RaFL has higher PSNR than pixel ℓ1\ell_1 on 8/8 seeds on SIDD, WorldView-2, and OASIS3, and on 4/8 seeds on LOL and IXI, where its mean PSNR is still higher. On LOL, RaFL has better SSIM, LPIPS, and DISTS than pixel ℓ1\ell_1 on 8/8 seeds, and on WorldView-2 it is ahead of every baseline on all four metrics on 8/8 seeds. Adding MAdam raises the PSNR of RaFL on 4/8, 7/8, and 8/8 seeds on SIDD, LOL, and WorldView-2, respectively (Table S13).

We hope these responses clarify the points you raised, and we welcome any further questions.

Reviewer_paEXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

RaFL combines random convolutions, random Fourier features, and wavelet details into an image reconstruction loss. Filter weights are resampled every optimization step, while gradient-norm matching balances the loss terms. The paper studies filter selection, resampling schedules, and initialization, then transfers the selected configuration across five 2D and 3D reconstruction tasks.

Critical strengths
  1. Five tasks cover natural images, remote sensing, and medical volumes. Transferring the same loss configuration across these settings is a useful contribution.
  2. Reporting seeds, standard deviations, and paired comparisons provides stronger evidence than single-run evaluations.
  3. WorldView-2 shows consistent improvements across all four metrics, while OASIS3 also improves over pixel loss. The method requires no pretrained loss network or additional inference computation.
Critical weaknesses
  1. The cited Gaussian-kernel approximation concerns feature inner products, whereas RaFL uses an L1 feature discrepancy. Consequently, the claimed mechanism of emphasizing high-frequency image content needs a clearer derivation or direct empirical support. See the original random Fourier feature formulation.
  2. On SIDD, RaFL takes 17.8 ms per step versus 3.3 ms for pixel loss and 5.4 ms for untrained VGG19: approximately 5.4× and 3.3× slower, respectively. Even RaFL with Adam alone takes 12.1 ms. On OASIS3, it takes 296.0 ms versus 128.4 ms for pixel loss. The paper acknowledges this limitation, but the abstract’s “182× cheaper” wording should explicitly refer to loss FLOPs rather than practical training cost.
  3. The main comparisons are against pixel loss and prior random-network losses. Although pretrained perceptual losses motivate the paper, no pretrained perceptual-loss baseline is reported on the natural-image tasks, where such features are applicable. The reported Sobel/Laplacian additions also do not by themselves establish superiority over a competitive standalone deterministic composite loss.
Questions for authors

see weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer paEX (1/3): CW1 Fourier feature discrepancy and empirical support

Response to Reviewer paEX (R1)

We thank Reviewer paEX for the valuable comments and for recognizing the strength for broad applicability and rigorous evaluation of our experiments. Please find the following answers to your concerns:

CW1. Clarification on Fourier feature discrepancy and empirical support.

We agree with your point that the original paper (Rahimi & Recht, 2007) concerns inner products of features, whereas RaFL uses the ℓ1\ell_1 distance between them. The Fourier family (ϕ2\phi_2) draws its construction from that work. Since networks fit low-frequency content first (Rahaman et al., 2019), fine detail is the part of the residual that is fitted last, when most patches are already nearly correct. Empirically, removing Fourier feature (Convolution + Wavelet) lowers PSNR by 0.40 dB and increases the cross-seed standard deviation from 0.13 to 0.75 dB on SIDD (Table 1), and lowers PSNR by 0.34 dB on WorldView-2 with SFINet (Table S10). Accordingly, we have revised Sec. 3.1 and Table S2.

As suggested, we conducted an empirical frequency analysis on the SIDD test set by partitioning the reconstruction error in the DFT domain into low (∣f∣<1/8|f| < 1/8), mid (1/8≤∣f∣<1/41/8 \le |f| < 1/4), and high (∣f∣≥1/4|f| \ge 1/4 cycles per pixel) frequency bands. Comparing RaFL against its variant without the Fourier family (Convolution + Wavelet) yields the following mean band PSNR (in dB):

ConfigurationLowMidHighFull image
RaFL (With Fourier)42.3046.9449.1540.28
Without Fourier (Conv + Wavelet)42.1146.4848.3139.88
Difference (Without − With)−0.19−0.46−0.84−0.40

Removing the Fourier family lowers PSNR across all three bands, with the largest drop occurring in the high-frequency band (−0.84 dB vs. −0.19 dB in the low band), confirming that random Fourier features primarily aid high-frequency recovery. We have added this analysis and a qualitative example (Fig. S6) to Appendix A.5.

AuthorsOpenReview ↗

Response to Reviewer paEX (2/3): CW2 Efficiency

CW2. Clarification of Efficiency

Thank you for pointing this out. We have revised the Abstract to state "requiring 182× fewer loss FLOPs than a VGG19 perceptual loss", as in Sec. 1, while keeping the wall-clock runtime comparison in Sec. 4.4 and App. C.3.

Profiling our submitted code showed that the 12.1–17.8 ms step time on SIDD was caused by CUDA kernel-launch overhead from sequential small-filter evaluations rather than arithmetic volume. Keeping the backbone and eager execution unchanged, we refactored the loss by batching the filter evaluations and reusing intermediate output gradients. This mathematically equivalent implementation cuts the step time by more than half:

Loss on SIDD (DnCNN)Loss GFLOPsSubmitted ms/it (former Table 4)Refactored ms/it (Table S12)Ratio vs. pixel ℓ1\ell_1
RaFL (MAdam)1.1217.8 ms8.3 ms (2.1× faster)2.52× (was 5.4×)
RaFL (Adam only)1.1212.1 ms4.6 ms (2.6× faster)1.39× (was 3.7×)
pixel ℓ1\ell_10.003.3 ms3.3 ms1.00×
uPL, SimpleNet10.123.6 ms3.6 ms1.09×
CDC5.004.2 ms4.2 ms1.27×
uPL, VGG19203.805.4 ms5.4 ms1.64×

After refactoring, RaFL with Adam (4.6 ms) is faster than the untrained VGG19 loss (5.4 ms), and RaFL with MAdam (8.3 ms) takes 1.5× as long, down from 3.3×. RaFL also reaches 1.10 dB higher PSNR than the untrained VGG19 loss on SIDD (40.28 vs. 39.18 dB, Table 2) with 182× fewer loss FLOPs.

On OASIS3 (SwinUNETR), RaFL with Adam only is within 6% of the pixel loss, and most of the remaining overhead comes from the MAdam preconditioner:

Loss on OASIS3 (SwinUNETR)Loss GFLOPsSubmitted ms/it (former Table 4)Refactored ms/it (Table S12)Ratio vs. pixel ℓ1\ell_1
RaFL (MAdam)11.8296.0 ms265.2 ms2.07× (was 2.3×)
RaFL (Adam only)11.8166.6 ms135.6 ms1.06× (was 1.3×)
pixel ℓ1\ell_10.00128.4 ms128.1 ms1.00×
uPL, VGG191821.0138.4 ms138.4 ms1.08×

We have updated Table S12 (formerly Table 4), Sec. 4.4, Sec. 5, and App. C.3, and we will include the refactored loss in the code release.

AuthorsOpenReview ↗

Response to Reviewer paEX (3/3): CW3 Pretrained perceptual loss and deterministic composite baselines

CW3. Pretrained perceptual loss and deterministic composite baselines

(1) Pretrained perceptual loss. We added a baseline trained with pixel ℓ1\ell_1 + ImageNet-pretrained VGG19 (Lℓ1+LVGG19\mathcal{L} _{\ell_1} + \mathcal{L} _{\mathrm{VGG19}}) on SIDD and LOL, the two tasks involving 3-channel RGB images where pretrained VGG19 features are directly applicable:

Task & MethodPSNR ↑SSIM ↑LPIPS ↓DISTS ↓Loss GFLOPs
SIDD (2D Denoising, DnCNN)
RaFL (Ours)40.28 ± 0.130.9557 ± 0.00120.2488 ± 0.00360.1847 ± 0.00341.12
pixel ℓ1\ell_139.64 ± 1.000.9551 ± 0.00110.2478 ± 0.00560.1830 ± 0.00570.00
uPL, VGG19 (untrained)39.18 ± 0.160.9410 ± 0.00200.2631 ± 0.00580.1903 ± 0.0032203.8
pixel ℓ1\ell_1 + pretrained VGG1939.28 ± 0.260.9443 ± 0.00140.2328 ± 0.00270.1807 ± 0.0033203.8
LOL (2D Enhancement, DRBN)
RaFL (Ours)21.17 ± 0.330.8035 ± 0.00320.2568 ± 0.00430.1995 ± 0.00321.12
pixel ℓ1\ell_121.16 ± 0.460.7799 ± 0.00310.3055 ± 0.00750.2273 ± 0.00490.00
pixel ℓ1\ell_1 + pretrained VGG1920.74 ± 0.590.8292 ± 0.00510.1946 ± 0.00380.1655 ± 0.0020203.8

Compared with pixel ℓ1\ell_1 + pretrained VGG19, RaFL achieves higher PSNR on both tasks (+1.00 dB on SIDD, +0.43 dB on LOL). Adding the pretrained VGG19 term lowers PSNR below pixel ℓ1\ell_1 alone (-0.36 dB on SIDD, -0.42 dB on LOL). The pretrained term does achieve lower LPIPS and DISTS on both tasks and higher SSIM on LOL, which is expected since LPIPS and DISTS are themselves computed from ImageNet-pretrained features. In contrast, RaFL requires no pretrained weights, and uses 182× fewer FLOPs in the loss and applies directly to multispectral and 3D data, whereas pretrained VGG19 features are limited to 2D RGB images. We have added this baseline to Table 2 and discussed these results in Sec. 4.3.

(2) Standalone deterministic composite loss. We also trained a standalone deterministic composite loss (pixel ℓ1\ell_1 + fixed Sobel + fixed Laplacian) on SIDD with DnCNN:

Method (SIDD, DnCNN)PSNR ↑SSIM ↑LPIPS ↓DISTS ↓
RaFL (Ours)40.28 ± 0.130.9557 ± 0.00120.2488 ± 0.00360.1847 ± 0.0034
pixel ℓ1\ell_1 + fixed Sobel + fixed Laplacian39.33 ± 0.730.9544 ± 0.00090.2547 ± 0.00550.1875 ± 0.0057

RaFL outperforms this deterministic composite across all four metrics, improving mean PSNR by 0.95 dB with much lower cross-seed variance. We have added this baseline to Table S1 and Sec. 4.2.

We hope these responses address your concerns, and we would be grateful if you would consider them in your final assessment. We are happy to answer any further questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

General response to all reviewers

We thank all reviewers for their thoughtful and constructive feedback. We are encouraged that the reviewers found RaFL simple to use, noting that it requires no pretrained features and adds no inference-time computation (paEX, Z12F, vsvm). Reviewers also highlighted that our evaluation across natural, remote-sensing, and medical images provides useful evidence of transferability (paEX, Z12F, vsvm), and that our eight-seed reporting with paired comparisons is more rigorous than single-run evaluation (paEX, Z12F, vsvm). Their comments and suggestions have helped us substantially improve the quality and clarity of the paper.

Below, we first briefly address the shared concerns and then list the changes in the revised PDF. Each point is discussed in full in the individual responses.

Shared concerns

  • Scope of the design conclusions (Z12F, vsvm). We now present the configuration as a default selected on SIDD for PSNR and seed stability, not as a setting that is best on every metric. We repeated the filter-family and initialization ablations on WorldView-2 with SFINet, where the default ties for the highest PSNR and has the lowest standard deviation across seed. With U-Net and DRUNet on SIDD, it matches pixel ℓ1\ell_1 in PSNR with lower LPIPS and DISTS (App. A.7, Table S11).

  • Pretrained perceptual loss (paEX, vsvm). Pixel ℓ1\ell_1 + pretrained VGG19 on SIDD and LOL gives lower LPIPS and DISTS than RaFL, but 1.00 dB and 0.43 dB lower PSNR (Table 2, Sec. 4.3), respectively. However, VGG19 requires a pretrained network, which is only available for 2D RGB images. RaFL needs no pretraining and works on any image type.

  • Training cost (paEX, vsvm). We refactored the loss implementation, reducing the RaFL training step on SIDD from 17.8 ms in the submission to 8.3 ms (App. C.3, Table S12). We also revised the Abstract to refer to loss FLOPs for clarity.

Main revisions in the uploaded PDF (changes in blue)

  • Abstract, Sec. 1 and Sec. 5: the configuration is described as a default selected on SIDD; "182× fewer loss FLOPs" (paEX, Z12F, vsvm).
  • Sec. 3.1 and Table S2: the Fourier term is described by the ℓ1\ell_1 distance between feature vectors; App. A.5 and Fig. S6: band-wise analysis with and without the Fourier family (paEX).
  • Table 2 and Sec. 4.3: pixel ℓ1\ell_1 + pretrained VGG19 on SIDD and LOL (paEX, vsvm).
  • Table S1 and Sec. 4.2: standalone pixel ℓ1\ell_1 + fixed Sobel + fixed Laplacian baseline (paEX).
  • App. A.7, Tables S10 and S11: filter-family and initialization ablations on WorldView-2, and the default on two more backbones (Z12F, vsvm).
  • Sec. 4.1: optimizer and loss weighting of each baseline (Z12F).
  • Fig. 1: Kaiming normal and Xavier uniform labels corrected (Z12F).
  • App. D, Tables S13–S17: per-seed results for Tables 2, 3 and S5 (Z12F).
  • App. C.3 and Table S12 (former Table 4): refactored timings and the completed OASIS3 block (paEX, vsvm).

Our detailed response to each reviewer is posted under each review.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Before the Discussion Ends

Dear Reviewers,

Thank you again for your constructive reviews. As the discussion period is nearing its end, we wanted to politely follow up to see if our responses and the revised manuscript (with updates highlighted in blue) have satisfactorily addressed your concerns, and if you would consider updating your score accordingly.

Please let us know if you have any remaining questions or if there is anything else we can clarify.

Sincerely,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.