acceptodds
Under review as a conference paper at ICLR 2027

Learning Image-Adaptive Gabor Frames for Self-supervised Image Restoration

Abstract

Learning image representations that adapt to local structures while retaining explicit reconstruction guarantees offers a principled foundation for image restoration. We propose GaborCanvas, a framework for learning image-adaptive Gabor frames. A neural network predicts the scales, orientations, and frequencies of Gabor atoms, aligning them with edges and textures so that relatively few significant coefficients can capture most image structures. We construct corresponding dual frames that ensure perfect reconstruction under suitable conditions. Together, structural adaptation and an explicit reconstruction formula provide an efficient and interpretable image representation that supports coefficient-based processing. We exploit the resulting coefficient sparsity through self-supervised formulations of three tasks: image denoising, inpainting, and super-resolution. The restoration models are trained without accessing truth images. GaborCanvas achieves competitive performance across all three tasks.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Gabor frame, Image restoration, Representation learning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_JedJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GaborCanvas, an image-adaptive Gabor representation for self-supervised image restoration. A shared network takes local image content and predicts spatially varying Gabor parameters, including window orientation, scales, frequencies, and phases. When the local atom matrix has full row rank, the corresponding dual synthesis operator is constructed explicitly. The local reconstructions are then combined by overlap averaging. Under the stated conditions, this reconstructs the unmodified coefficients exactly.

For denoising, the method learns coefficient thresholds using the R2R loss. For SR and inpainting, it uses internal self-supervision and differentiable reconstruction with a Gabor analysis-sparsity penalty. In the implementation, SR updates the frame over several reconstruction rounds, while inpainting uses a quadratic term around a biharmonic initialization. The paper includes continuous and discrete frame results, restoration benchmarks, and additional experiments on spatial adaptation, sparse approximation, atom types, the inpainting initialization, and numerical reconstruction accuracy.

Critical strengths
  • The main methodological value is the combination of input-conditioned local Gabor parameters, spatially varying analysis systems, and corresponding local duals. The network predicts interpretable spatial-frequency structure, rather than directly producing restored pixels. In denoising, it is possible to see separately what is done by analysis, coefficient modification, and synthesis.

  • Appendix B.2 compares with learned Gaussian atoms and reports gains of 2.77 and 2.99 dB on Kodak24. This is useful evidence for the Gabor family against that alternative. It is not exactly a "remove only cosine modulation" experiment, since the Gaussian counterpart also learns its centers and scales.

  • Appendix B.4 includes the biharmonic initialization baseline. Adding the Gabor regularizer improves inpainting PSNR by 0.84 and 0.49 dB at the two missing-pixel ratios. So there is already evidence that the method adds something beyond the classical initialization. These experiments are a main reason for my positive assessment.

Critical weaknesses
  • The novelty needs to be discussed against closer prior work. An important missing work is GDLNet (Janjušević et al., 2022). It learns real 2D Gabor analysis/synthesis dictionaries in a sparse-coding-inspired denoising network. The broad combination of learned Gabor representations, sparsity, and self-supervised denoising already has precedent. This method is not the same as GaborCanvas. In particular, the input-conditioned per-location predictor and explicit local dual construction seem to be important differences. At present, the reader has to work out this positioning.

  • The AWGN R2R construction is explicit, but the noise generation and parameter estimation for SIDD and FMD are not described at the same level. These details matter for the validity and interpretation of the self-supervised training.

  • The baseline training conditions also need some clarification. For example, standard TTT-MIM uses supervised pretraining and then self-supervised test-time adaptation. It should not be grouped as entirely clean-target-free unless the authors used a different training protocol. This is a question about the comparison conditions, not evidence that GaborCanvas itself used clean targets.

  • ZSSR is a relevant missing SR baseline because it also learns from pairs constructed inside the input image. This is especially important for the broad claim about GT-free SR: at x4, the PSNR improvement over DIP is only 0.010 dB, although SSIM and LPIPS improve. A matched comparison would help. One established learned-frame baseline would also be useful in addition to the Gaussian-atom comparison. The supervision conditions should be kept clear when making these comparisons.

  • Finally, a small parameter count does not tell the actual cost here. There is image-specific fitting, local matrix computation, and iterative image reconstruction. End-to-end runtime and peak memory are needed. I am not claiming the method is necessarily slow, but the current tables do not allow a practical efficiency assessment.

Questions for authors
  • Could the authors explain the precise differences from GDLNet's self-supervised Gabor denoising, nonstationary Gabor-frame constructions, and region-adaptive learned frames? In particular, which part is new in the input-conditioned local parameterization, and which part follows from existing sparse representation and dual-frame results? This could change my novelty assessment.

  • For SIDD and FMD, what noise distribution, parameter estimates, and image domain are used for R2R? What data are used to choose hyperparameters and stopping rules, including any clean validation data? Please also clarify whether TTT-MIM uses its standard supervised pretraining, and which method was actually implemented for the PIP/Fourier INR row.

  • How does the method compare with ZSSR under the same downsampling and evaluation protocol? A comparison with one established learned-frame method under comparable information conditions would also help. I am asking for these because they are close to the current claims, not asking the authors to run every related method.

  • Please report runtime and peak memory at representative image resolutions, with the hardware specified. It would be useful to separate image-specific fitting from final reconstruction and include local matrix computations and iterative updates. Parameter counts alone are not enough for this part.

6 comments
AuthorsOpenReview ↗

Response to Reviewer JedJ (1/4)

We thank the reviewer for careful reading and instructive feedback.


W1 / Q1. Positioning against related work; new versus established components.

Response. We thank the reviewer for identifying these related works. Our contribution has two complementary aspects.

(1) Learning image-conditioned, spatially varying Gabor frames. GDLNet learns Gabor filters shared across spatial locations within each layer. DDTF learns one filter bank shared across an image, while region-adaptive extensions, such as piecewise DDTF (Du et al., 2026), learn separate frames for segmented regions. Our shared network predicts atom-specific orientations, frequencies, and phases from local image content at every location, without requiring a regional partition. Thus, the representation adapts both to the input image and to different locations within it, while retaining an explicit Gabor parameterization.

Table 5 (main paper) isolates the benefit of this spatial adaptation: under the same fixed-threshold protocol, the spatially varying frame improves PSNR by 0.50–0.55 dB over a learned stationary Gabor frame across all four settings. Additional comparisons with GDLNet under our self-supervised protocol and with DDTF are provided in our response to W4. GDLNet itself is trained supervised with MSE on BSD432 clean/noisy pairs; our per-image R2R experiment below adapts its architecture to our noisy-only training setting.

(2) Frame analysis with nonuniform local frequencies. In the time-adaptive nonstationary Gabor construction of Balazs et al. (2011), windows and frequency spacings may vary across locations, but frequencies remain uniformly spaced within each local atom set. Their painless frame proof uses this structure and suitable support conditions to obtain a diagonal frame operator. Our atom-dependent nonuniform frequencies break the Fourier orthogonality used in that argument, so their proof cannot be applied directly. We establish explicit, verifiable sufficient conditions for frame preservation; our continuum proof combines Gaussian fiber-matrix estimates, nonharmonic Fourier bounds, and phase-error control. Please also see our response to Reviewer Yg8B, W1, for further discussion.

What follows established methods. The local pseudo-inverse, coefficient shrinkage, and R2R loss follow established methods. Our contribution lies in the image-conditioned, spatially varying Gabor frame and the accompanying analysis of its frame property.

References.

  • Du, W., Li, J., & Zhou, Q. (2026). Piecewise data-driven tight frame and ℓ0\ell_0-balanced image denoising method. Inverse Problems and Imaging.

W2 / W3 / Q2. SIDD/FMD training protocols; TTT-MIM pretraining; the PIP/Fourier INR row.

Response. We thank the reviewer for these questions. We clarify the training protocols and baseline conditions below and will include these details in the revised manuscript, submitted before the discussion deadline.

(a) SIDD and FMD: noise models and image domains. For SIDD, we use the RAW-domain re-corruption procedure of R2R (Pang et al., 2021). For each noisy RAW image yy in SIDD-Medium, we generate the input–target pair

(y+2z,  y−z/2),z∼N(0,β1y+β2),(y+2z,\;y-z/2),\qquad z\sim\mathcal N(0,\beta_1y+\beta_2),

using the supplied camera noise-level parameters (β1,β2)(\beta_1,\beta_2) for the Bayer channels. The re-corrupted images are rendered to sRGB through the camera ISP, and training uses 128×128128\times128 crops from the resulting pairs. Separately, a per-channel median absolute deviation (MAD) estimate from the sRGB network input, σ^=MAD(d)/0.6745\hat\sigma=\mathrm{MAD}(d)/0.6745 with dd a 2×2 diagonal-difference high-pass response, sets the scale of the normalized thresholds.

For FMD, we use Gaussian re-corruption with its noise level estimated from the noisy image by MAD. The image intensities are the 8-bit grayscale values divided by 255, with no variance-stabilizing transform, and the noise is treated as homoscedastic Gaussian in this domain with level σ=c σ^\sigma=c\,\hat\sigma, where σ^=MAD(d)/0.6745\hat\sigma=\mathrm{MAD}(d)/0.6745 uses the same diagonal difference dd as for SIDD and cc is a per-modality multiplier (4 for Confocal, 2 for Two-Photon, 8 for Wide-Field). The R2R input–target pair is built at each update from a random 48×48 crop yy of the noisy image,

(y+ασε,  y−σε/α),ε∼N(0,I), α=0.5,(y+\alpha\sigma\varepsilon,\;y-\sigma\varepsilon/\alpha),\qquad \varepsilon\sim\mathcal N(0,I),\ \alpha=0.5,

so the input and target perturbations have variances σ2/4\sigma^2/4 and 4σ24\sigma^2, and the noise level given to the model is σ1+α2\sigma\sqrt{1+\alpha^2}.

AuthorsOpenReview ↗

Response to Reviewer JedJ (2/4)

(b) Hyperparameters and checkpoint selection. For SIDD, we train for four epochs (16,531 steps). The re-corruption procedure and the four-epoch budget are inherited from the official R2R code, the atom count and patch size (96 atoms, 8×8) and the learning rates were set empirically without validation data. As in the official R2R code, the model is evaluated on the SIDD validation set during training, here on a fixed subset of 32 of the 1,280 validation blocks every 250 steps, and we report the checkpoint with the highest PSNR on this subset, at step 15,500, near the end of training. Clean references enter only this checkpoint selection; they are not used in the training loss or for the hyperparameters. The same checkpoint and settings are used for SIDD Benchmark without further selection; its score is obtained through the benchmark's online evaluation server. For FMD, we optimize a separate model for each noisy image using a fixed number of 500 updates, without early stopping. The multiplier cc in its noise level, the atom count, and the patch size are chosen by mean PSNR against the clean references of validation images from different modality (5 Confocal, 4 Two-Photon, and 3 Wide-Field), which are among all 48 evaluation images. In both cases, the training loss uses no clean targets; clean references enter only model selection. We acknowledge that GT-free is accurate only for the training stage, and we will use the term in this narrower sense. To our knowledge, such validation-based model selection is the default protocol on these benchmarks, so the compared self-supervised methods are under the same condition.

We also evaluated sensitivity to the main representation hyperparameters by varying them individually in the Gaussian-denoising model, using all 24 Kodak24 images at σ=25\sigma=25 with the noise realizations of Table 1. With 8×88\times8 patches, PSNR varies by only 0.16 dB across 88–144 atoms; performance decreases at 72 and 80 atoms.

Table R4-1. Number of atoms, with 8×88\times8 patches; PSNR / SSIM:

72808896104112128144
29.41 / .815  29.80 / .829  29.98 / .835  30.03 / .837  29.91 / .831  30.07 / .836  29.98 / .836  29.96 / .835

When varying the patch size while maintaining approximately 1.5 atoms per pixel, PSNR varies by 0.08 dB across 7×77\times7–9×99\times9 patches.

Table R4-2. Patch size / number of atoms; PSNR / SSIM:

6×6 / 547×7 / 748×8 / 969×9 / 12210×10 / 150
29.80 / .827  29.95 / .834  30.03 / .837  30.01 / .835  29.86 / .830

These results support robustness around the adopted configuration.

(c) TTT-MIM. Yes, TTT-MIM is first trained on 38,400 ImageNet images corrupted with synthetic Gaussian noise (variance 0.005), with a supervised denoising loss against the clean images and a masked-image-modeling loss; it is then adapted to each noisy test image with 8 self-supervised gradient steps (Mansour et al., 2024). It therefore uses external clean training data and should not be grouped as entirely clean-target-free. We did not re-run it; its SIDD Validation, SIDD Benchmark, and FMD results are quoted from Table 1 of Guo et al. (NeurIPS 2025). In the revision, Table 2 will list it separately under “supervised pretraining + self-supervised test-time adaptation.”

(d) The PIP/Fourier INR row. It is our implementation of a Fourier-feature MLP (Tancik et al., 2020): 64 Gaussian random features (σ=10) and a 4×256 ReLU MLP predicting a bounded residual over bicubic upsampling. It is fitted for 1,800 Adam steps on low-resolution (LR) consistency through the bicubic downsampler (MSE plus total variation). The label "PIP" was inaccurate; in the revision, Table 3 will rename the row "Fourier-feature INR (residual)".

AuthorsOpenReview ↗

Response to Reviewer JedJ (3/4)

W4 / Q3. Comparisons with ZSSR and a learned-frame method; 0.010 dB gain over DIP at ×4 SR.

Response. We thank the reviewer for suggesting these comparisons.

(a) ZSSR. We re-implemented ZSSR in PyTorch following Shocher et al. (2018): an 8-layer, 64-channel residual network trained on internal low-resolution pairs, followed by back-projection. We use the same LR inputs, bicubic downsampler, and evaluation protocol as Table 3.

Table R4-3. Set14; PSNR-Y / SSIM-Y / LPIPS, with the same evaluation border crop:

Method×2×4×8
ZSSR32.05 / .911 / .151  26.69 / .753 / .339  23.32 / .559 / .510
GaborCanvas  31.37 / .906 / .16726.65 / .751 / .34123.63 / .578 / .505

We agree that the 0.01 dB margin over DIP at ×4 is not meaningful. We will add ZSSR to Table 3 and revise Section 4.2 to describe GaborCanvas as a competitive GT-free SR method. On Set14, its performance is close to ZSSR at ×4, and it outperforms ZSSR on all three metrics at the larger upscaling factor of ×8.

(b) Learned-frame and Gabor-dictionary baselines. We added DDTF (Cai et al., 2014), which learns a tight frame from a single noisy image without external training data and supports exact reconstruction when coefficients are unchanged. We implemented Algorithm 1 with its published denoising settings, applied separately to each RGB channel.

We also fit the authors’ GDLNet implementation per image using our R2R loss, noise realizations, and known noise level. We use MoG-3 with 30 stages, 169 subbands, and 11×11 filters, adapted to RGB (557,700 parameters). Starting from its own initialization, it is trained on 48×48 random crops with Adam for 400 updates. These results evaluate GDLNet adapted to our single-noisy-image setting.

Table R4-4. Same images, noise realizations, and metrics as Table 1, with additional CBSD68 evaluation on 256×256 center crops. Kodak24 and McMaster18: PSNR / SSIM; CBSD68: PSNR.

MethodKodak24 σ=10Kodak24 σ=25McMaster18 σ=10  McMaster18 σ=25  CBSD68 σ=15 / 25 / 50
DDTF33.45 / .90728.59 / .77834.37 / .92529.41 / .81730.31 / 27.63 / 24.45
GDLNet (R2R, per image)  33.84 / .92329.16 / .82133.96 / .91729.41 / .82030.92 / 28.60 / 25.05
GaborCanvas35.31 / .937  30.03 / .837  34.95 / .92929.93 / .83532.50 / 29.45 / 25.69

GaborCanvas exceeds DDTF by 0.52–2.19 dB and GDLNet by 0.52–1.58 dB across these settings.

The 400-update budget was selected among 200, 400, 800, and 1,600 updates by PSNR on Kodak24 at σ=25 (28.75, 29.16, 28.20, and 28.80 dB). GDLNet takes 57.4 s per image with 0.8 GiB peak memory, compared with 23.5 s and 2.9 GiB for GaborCanvas (W5).

These comparisons assess the methods as a whole; Table 5 isolates the contribution of spatial adaptation within our pipeline. We will add DDTF and GDLNet to Table 1 and the CBSD68 results to Appendix F.

Reviewer_yU6ROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GaborCanvas, an image-adaptive Gabor frame learning framework for self-supervised image restoration. A shared neural network predicts spatially varying Gabor parameters, including scales, orientations, frequencies, and phases. Corresponding dual frames enable perfect reconstruction under suitable full-rank conditions. Leveraging coefficient sparsity, the framework addresses denoising, super-resolution , and inpainting through learnable thresholding, recorrupted-to-recorrupted training, and sparsity regularization. The authors validate the approach through theoretical analysis, numerical stability tests, ablation studies, and experiments on multiple restoration benchmarks.

Critical strengths

1.GaborCanvas combines neural prediction of spatially adaptive Gabor atoms with dual-frame construction, enabling representations that adapt to local edges and textures while retaining an explicit analysis–synthesis structure. The k-term approximation and numerical stability analyses further support its representation efficiency and reconstruction accuracy.

  1. The paper compares predefined, learned stationary, and nonstationary Gabor frames, and examines the effects of atom types, adaptive thresholding, and dictionary size. Spatial adaptation yields approximately 0.5 dB improvement over stationary frames. Competitive GT-free denoising results on Kodak24, SIDD, and FMD further support the effectiveness of the proposed representation.
Critical weaknesses
  1. Nonstationary Gabor Frames, Data-Driven Tight Frames, and GDLNet have previously explored adaptive Gabor representations, learnable frames with reconstruction properties, and neural denoising with learned Gabor dictionaries, respectively. Thus, the main novelty lies in image-conditioned local Gabor prediction and dual-frame construction rather than these underlying concepts. Although stationary-frame and Gaussian-atom ablations are provided, comparisons with closely related learnable Gabor dictionaries or general adaptive dictionaries with comparable reconstruction properties remain insufficient, making it difficult to isolate the proposed method's contribution.

  2. Perfect reconstruction in the 2D implementation requires full-row-rank local analysis matrices and proper overlap normalization. Despite the numerical stability analysis, reconstruction consistency under different patch strides and boundary treatments remains unclear. Moreover, local dual-frame computation and per-image optimization introduce potentially substantial overhead; SR inference, for example, requires 120 reconstruction updates. Without runtime and memory comparisons, the efficiency claims remain insufficiently supported. This is particularly relevant given the marginal 0.01 dB PSNR improvement over DIP on Set14 ×4 SR.

Questions for authors
  1. Could the authors clarify the key differences from GDLNet and Data-Driven Tight Frames? Comparisons with learnable non-Gabor local dictionaries or adaptive representations supporting dual reconstruction, under matched self-supervised objectives, parameter counts, and computational budgets, would help establish the independent contributions of Gabor parameterization and spatial adaptation.

  2. How is the reconstruction identity maintained under different patch strides, boundary treatments, and overlap averaging in the 2D implementation? Could the authors report full-image reconstruction errors, end-to-end inference time, peak GPU memory, and local matrix computation costs across restoration tasks to substantiate the practical efficiency of the framework?

5 comments
AuthorsOpenReview ↗

Response to Reviewer yU6R (1/3)

We thank the reviewer for identifying the closest prior work and for concrete, testable requests.


W1 / Q1. Differences from related work; matched comparisons with non-Gabor local dictionaries.

Response. We thank the reviewer for this summary. Our contribution has two complementary aspects:

(1) Learning image-conditioned, spatially varying Gabor frames. DDTF learns one filter bank shared across an image, while GDLNet learns Gabor filters shared across spatial locations within each layer. Our network predicts atom-specific orientations, frequencies, and phases from local image content without clean training targets. Thus, the learned frame is conditioned on the target image, and its atom sets vary across locations. Table 5 isolates the benefit of this spatial adaptation: under the same fixed-threshold protocol, it improves denoising by 0.50–0.55 dB over a learned stationary Gabor frame.

(2) Frame analysis with nonuniform local frequencies. The time-adaptive nonstationary Gabor construction of Balazs et al. (2011) allows windows and frequency spacings to vary across locations, but retains uniformly spaced frequencies within each local atom set. Their painless frame proof relies on this structure and suitable support conditions to obtain a diagonal frame operator. Our nonuniform local frequencies break the required Fourier orthogonality, so their proof cannot be applied directly. We establish an explicit sufficient condition for frame preservation through additional analysis: our continuum proof combines Gaussian fiber-matrix estimates, nonharmonic Fourier bounds, and phase-error control.

Please see our response to Reviewer Yg8B, W1, for a more detailed discussion of these methodological and theoretical distinctions.

To address the requested empirical comparisons, we separately examine Gabor parameterization and spatial adaptation, including a free-form baseline with matched parameter and FLOP budgets, then compare with GDLNet and DDTF.

Gabor versus free-form local atoms. We extend Table 6 with two free-form variants. The same descriptor network predicts 96 atoms for each 8×8 patch, using either an unrestricted output head or a factorized head with approximately matched parameter and FLOP budgets. All variants use the same thresholding rule, R2R loss, dual construction, training schedule, and noisy inputs.

Table R3-1. Kodak24, Table 6 protocol; PSNR / SSIM and FLOPs for a complete run:

Atom parameterizationσ=10σ=25Frame params  TFLOPs
Gaussian32.54 / .88827.04 / .72073,584—
Free-form34.60 / .93029.32 / .826441,2647.00
Free-form, factorized head  34.43 / .92829.05 / .81960,4664.80
Gabor (GaborCanvas)35.31 / .937  30.03 / .837  61,1104.83

The factorized head predicts vec⁡(Gp)=b+Uzp\operatorname{vec}(G_p)=b+Uz_p before normalization, where zp∈R2z_p\in\mathbb{R}^2 is computed linearly from the patch descriptor. The rank-two restriction applies to this prediction map, not to the local atom matrix GpG_p.

With approximately matched parameter and FLOP budgets, GaborCanvas improves PSNR by 0.88 and 0.98 dB over the factorized-head variant. It also exceeds the unrestricted free-form model by 0.71 dB at both noise levels, despite that model having 7.2× as many frame parameters and 45% more FLOPs.

Both free-form variants have full row rank at every checked location and reconstruct unchanged coefficients with relative error below 10−1510^{-15} in float64. Thus, all three representations support dual reconstruction.

Spatial adaptation within the Gabor parameterization. The stationary-versus-nonstationary comparison in Table 5, summarized above, isolates spatial adaptation using the same Gabor parameterization and a common normalized threshold.

Comparisons with GDLNet and DDTF. We fit the authors’ GDLNet implementation per image using our R2R loss, noise realizations, and known noise level. We use MoG-3 with 30 stages, 169 subbands, and 11×11 filters, adapted to RGB (557,700 parameters). Starting from its own initialization, GDLNet is trained on 48×48 random crops with Adam for 400 updates. These results evaluate GDLNet adapted to our single-noisy-image setting.

For DDTF, we implement Algorithm 1 of Cai et al. (2014) with its published denoising settings, applied separately to each RGB channel.

AuthorsOpenReview ↗

Response to Reviewer yU6R (2/3)

Table R3-2. Same images, noise realizations, and metrics as Table 1, with additional CBSD68 evaluation on 256×256 center crops. Kodak24 and McMaster18: PSNR / SSIM; CBSD68: PSNR.

MethodKodak24 σ=10Kodak24 σ=25McMaster18 σ=10  McMaster18 σ=25  CBSD68 σ=15 / 25 / 50
DDTF33.45 / .90728.59 / .77834.37 / .92529.41 / .81730.31 / 27.63 / 24.45
GDLNet (R2R, per image)  33.84 / .92329.16 / .82133.96 / .91729.41 / .82030.92 / 28.60 / 25.05
GaborCanvas35.31 / .937  30.03 / .837  34.95 / .92929.93 / .83532.50 / 29.45 / 25.69

GaborCanvas exceeds DDTF by 0.52–2.19 dB and GDLNet by 0.52–1.58 dB across these settings. The 400-update budget for GDLNet was selected among 200, 400, 800, and 1,600 updates by PSNR on Kodak24 at σ=25 (28.75, 29.16, 28.20, and 28.80 dB).

GDLNet takes 57.4 s and 0.8 GiB per image, compared with 23.5 s and 2.9 GiB for GaborCanvas (W2). These benchmark comparisons evaluate the restoration pipelines as a whole; the ablations above separately assess atom parameterization and spatial adaptation.


W2 / Q2. 2D perfect reconstruction; runtime, memory, and component costs; 0.01 dB gain over DIP at ×4 SR.

Response. We thank the reviewer for these requests. We clarify reconstruction across strides and boundaries and report end-to-end runtime, memory, and local computation costs.

(a) Reconstruction across strides and boundaries. As discussed in our response to Reviewer GChh, PR holds when the local atoms span each patch space and every original pixel is covered.

  1. Pixel coverage. For square windows of size q×qq\times q, strides h≤qh\le q provide complete coverage with appropriate boundary extension and patch placement.
  2. Overlap normalization. When the stride divides the window size, interior pixels receive the same number of patch contributions. Otherwise, coverage may vary. The implementation always divides each pixel’s accumulated reconstruction by its actual coverage count, handling both cases and any reduced coverage near boundaries.
  3. Boundary extension. Standard extensions, including periodic and Neumann-type extensions, are compatible with the reconstruction argument. After reconstructing the covered region of the extended image, we crop back to the original domain. Provided the coverage and local rank conditions hold, the extension rule does not affect the PR identity.

The following experiment verifies reconstruction for every stride from 1 to 8, including strides with nonuniform coverage.

Table R3-3. Submitted AWGN model; 8×8 patches; float64; all coefficients retained. Maximum relative error over three images per Kodak24/McMaster18 noise setting and reflection, replication, and zero padding.

Stride  Minimum coverage  Maximum reconstruction error across boundary rules
1167.5×10⁻¹⁵
241.4×10⁻¹⁴
341.8×10⁻¹⁴
412.5×10⁻¹⁴
512.8×10⁻¹⁴
613.4×10⁻¹⁴
714.0×10⁻¹⁴
815.0×10⁻¹⁴

Reconstruction errors remain at most 5×10−145\times10^{-14} across all tested settings. We also verified full row rank at every evaluated location of the AWGN, SIDD, inpainting, and SR models; detailed diagnostics are reported in our response to Reviewer GChh (Table R1-9).

In the revision, Appendix A.3 will extend Theorem 2 from one-dimensional sequences to two-dimensional images, covering different patch strides and boundary extensions. The extension will establish exact reconstruction under the stated local rank and pixel coverage conditions, with normalization by each pixel’s actual coverage count.

AuthorsOpenReview ↗

Response to Reviewer yU6R (3/3)

(b) End-to-end runtime and memory. All methods were timed on the same idle RTX 4090, with one process per method. Times include image-specific optimization and reconstruction where applicable; memory is the peak GPU memory of the process. Diffusion methods run at 256×256, so their timings are reference results when other methods use larger images.

Table R3-4. Denoising: 256×256 Kodak24 crops, σ=25; time per image.

MethodTime (s)  Peak memory (GiB)
PD-Denoising1.10.5
Pixel2Pixel22.218.5
ScoreDVI33.31.4
MASH40.70.8
R2R46.02.4
ZS-HPD52.60.7
GDLNet (R2R, per image)  57.40.8
NLR69.51.6
GaborCanvas23.52.9

Table R3-5. Inpainting: two Set14 images at full resolution (baboon, 500×480; barbara, 720×576), 50% missing; time per image. DDNM runs at 256×256.

MethodTime (s)  Peak memory (GiB)
DDNM (pretrained diffusion)  8.42.6
DIP133.39.2
Deep Decoder (DD)90.83.0
Fourier-feature INR74.84.0
INCODE97.52.3
GaborCanvas89.98.2

Table R3-6. SR ×4: BSD100, 480×320 output; time per image, including model loading. GT-free methods were timed on three images, supervised and diffusion methods on two.

MethodTime (s)  Peak memory (GiB)
Supervised external training
SwinIR3.20.8
LIIF-EDSR3.91.0
LTE-EDSR3.81.0
ContinuousSR5.11.5
Pretrained diffusion priors: 256×256
DDNM8.42.6
DiffPIR11.83.7
DAPS15.63.7
GT-free single-image methods
DIP45.86.6
DD54.21.1
Fourier-feature INR26.31.7
INCODE15.30.9
I-INR38.41.3
SL2A-INR8.30.6
SuperF-single28.61.3
MSA12.10.6
GaborCanvas39.81.0

In these measurements, GaborCanvas is faster than DIP in inpainting and SR and faster than six of the eight timed self-supervised denoisers. Some INR methods are faster. Memory use also depends on the task: GaborCanvas uses less memory than DIP in SR, but more than several baselines in inpainting.

(c) Fitting, iterative reconstruction, and local matrix costs. Image-specific fitting is the main runtime component. In denoising, fitting takes 18.3 s and the final analysis–thresholding–synthesis pass takes 0.45 s; approximately 4.7 s is spent on setup and input/output operations. With fixed 48×48 training crops and a fixed update budget, fitting cost is largely independent of image size. Final reconstruction takes 3.25 s at 768×512.

For SR, fitting takes 30.3 s and the 120 reconstruction updates take 9.5 s, both included in the reported 39.8 s. The inpainting reconstruction takes 11.1 s and is likewise included in its end-to-end time. Gaussian denoising requires only one final reconstruction pass.

Table R3-7. We measure FLOPs using PyTorch’s FlopCounterMode, supplemented by analytic counts for dense local solves:

TaskLocal matrix computationsShare of FLOPs
DenoisingLocal dual solves0.51 of 4.8 TFLOPs, approximately 11% of the complete run
Inpainting  Local Gram matrices and solves  1.6% of the complete run on average; 0.5–2.7% across Set14
SRLocal solves0.3% of the reconstruction solver FLOPs

Inpainting fitting forms no local duals. These measurements quantify the local computations’ FLOP contributions; the runtime tables report their practical cost within the complete pipeline.

(d) Interpreting the SR results. We agree that the +0.01 dB PSNR margin over DIP on Set14 ×4 is not meaningful. GaborCanvas achieves higher SSIM and lower LPIPS in that setting: 0.7509 / 0.3412, compared with 0.7230 / 0.3616 for DIP (Table 3). We will remove the claim of the best GT-free SR method and report performance by benchmark and scale.

The revised manuscript, submitted before the discussion deadline, will incorporate these comparisons, reconstruction diagnostics, and computational measurements.

Reviewer_Yg8BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes GaborCanvas, which uses a neural network to predict spatially varying Gabor atoms and constructs corresponding dual frames for reconstruction. The method combines this representation with coefficient thresholding or sparsity regularization for self-supervised denoising, super-resolution, and inpainting.

Critical strengths

The method achieves competitive denoising results, and the ablations support the benefits of spatial adaptation and adaptive thresholding.

Critical weaknesses
  1. The methodological novelty is limited. The main change is a neural predictor for local Gabor parameters, while coefficient processing, dual-frame construction, and self-supervised training follow established approaches.
  2. The advantage of Gabor atoms over other learned representations is unclear. The Gaussian-atom ablation shows the benefit of frequency modulation, but does not compare Gabor atoms with learned dictionaries or unconstrained local filters.
  3. The perfect-reconstruction property does not explain the restoration gains. It follows from the full-rank local matrix and its dual, and is not unique to Gabor atoms. Preserving the input with all coefficients does not explain why modifying those coefficients improves restoration.
  4. The evaluation beyond denoising is limited to Set14, with only random masks for inpainting. At ×4 super-resolution, the PSNR gain over DIP is 0.01 dB. For inpainting with 50% missing pixels, the gain over the biharmonic initialization is 0.49 dB.
Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Yg8B (1/4)

We thank the reviewer for the constructive feedback. Our main contribution is the self-supervised learning of spatially varying Gabor frames with atom-specific parameters. The accompanying theoretical analysis provides a verifiable sufficient condition under which the spatially varying Gabor system with nonuniform local frequencies remains a frame.


W1. Limited methodological novelty.

Response. Our contribution in methodology has two complementary aspects: learning spatially varying Gabor frames for restoration and establishing sufficient frame conditions for nonuniform local frequencies.

Method part: Learning spatially varying Gabor frames. A closely related neural method is GDLNet (Janjušević et al., 2022), raised by Reviewer yU6R, which learns Gabor filter banks shared across spatial locations within each layer. Our network predicts atom-specific orientations, frequencies, and phases from local image content without clean training targets. Consequently, different locations can use different Gabor atom sets, providing spatial adaptation within the image.

Table 5 directly isolates this contribution: under the same fixed-threshold protocol, the spatially varying frame improves denoising by 0.50–0.55 dB over the learned stationary frame across all four settings. The additional direct comparison with GDLNet (table in our response to W2) further supports the effectiveness of the proposed method.

Theoretical part: Frame analysis with nonuniform local frequencies. We have not identified prior work establishing frame conditions for the specific combination of spatially varying Gaussian scales and atom-dependent nonuniform frequencies considered here. In the time-adaptive nonstationary Gabor construction of Balazs et al. (2011), windows and frequency spacings may vary across locations, but frequencies remain uniformly spaced within each local atom set. Their painless frame proof relies on this uniform spacing and suitable support conditions to obtain a diagonal frame operator through Fourier orthogonality. Our nonuniform local frequencies break this orthogonality, so their proof cannot be applied directly. We instead combine Gaussian fiber-matrix estimates, nonharmonic Fourier bounds, and phase-error control to derive an explicit, verifiable sufficient condition for frame preservation.

We will clarify these methodological and theoretical distinctions in the revised contribution statement and related-work discussion.

References.

  • Balazs, P., Dörfler, M., Jaillet, F., Holighaus, N., & Velasco, G. (2011). Theory, implementation and applications of nonstationary Gabor frames. Journal of Computational and Applied Mathematics.
  • Janjušević, N., Khalilian-Gourtani, A., & Wang, Y. (2022). Gabor is Enough: Interpretable Deep Denoising with a Gabor Synthesis Dictionary Prior. IEEE Image, Video, and Multidimensional Signal Processing Workshop (IVMSP).
AuthorsOpenReview ↗

Response to Reviewer Yg8B (2/4)

W2. Gabor atoms versus other learned representations.

Response. We thank the reviewer for this suggestion. We address this concern through an ablation with unconstrained local filters and comparisons with DDTF and GDLNet.

Unconstrained local filters. We extend Table 6 with a free-form atom generator: the same descriptor network predicts every entry of the 96 atoms on each 8×8 patch. The thresholding rule, R2R loss, dual construction, training schedule, and noisy inputs follow the same protocol as the Gabor model.

Table R2-1. Kodak24, Table 6 protocol; PSNR / SSIM:

Atoms predicted at every pixel  σ=10σ=25Frame params
Gaussian32.54 / .88827.04 / .72073,584
Free-form34.60 / .93029.32 / .826441,264
Gabor (GaborCanvas)35.31 / .937  30.03 / .837  61,110

Gabor atoms improve PSNR by 0.71 dB at both noise levels over the free-form baseline, using approximately 1/71/7 of the frame parameters. This supports the Gabor parameterization as an effective structural prior when learning local atoms from a single noisy image.

Learned dictionaries. DDTF learns a tight frame from the single noisy image without clean training data and supports exact reconstruction when coefficients are unchanged. Its learned filter bank is shared across spatial locations. We implemented Algorithm 1 of Cai et al. (2014) with its published denoising settings, applied separately to each RGB channel as for the classical methods in Table 1. For GDLNet, we fitted the authors' model per image with our self-supervised loss and the same noise realizations, and report it at its best number of updates; this adapts its architecture to our noisy-only protocol and is not a reproduction of its supervised results.

Table R2-2. Same images, noise realizations, and metrics as Table 1, with additional CBSD68 evaluation on 256×256 center crops. Kodak24 and McMaster18: PSNR / SSIM; CBSD68: PSNR.

MethodKodak24 σ=10Kodak24 σ=25McMaster18 σ=10  McMaster18 σ=25  CBSD68 σ=15 / 25 / 50
DDTF33.45 / .90728.59 / .77834.37 / .92529.41 / .81730.31 / 27.63 / 24.45
GDLNet (R2R, per image)  33.84 / .92329.16 / .82133.96 / .91729.41 / .82030.92 / 28.60 / 25.05
GaborCanvas35.31 / .937  30.03 / .837  34.95 / .92929.93 / .83532.50 / 29.45 / 25.69

GaborCanvas exceeds DDTF by 0.52–2.19 dB and GDLNet by 0.52–1.58 dB across these settings. This evaluates the restoration methods as a whole; the free-form ablation above compares atom parameterizations under the same training and restoration protocol.

In the revision submitted before the discussion deadline, we will add the free-form baseline to Table 6, DDTF and GDLNet to Table 1, and the CBSD68 results to Appendix F.


W3. Role of perfect reconstruction in the restoration gains.

Response. We agree that perfect reconstruction (PR) alone does not explain restoration gains. We distinguish its role from that of the image-specific prior for self-supervised image restoration:

  1. Role of PR in faithful reconstruction. As with the discrete Fourier, discrete cosine, and wavelet transforms, PR ensures that unchanged coefficients reproduce the input exactly. It ensures that coefficient-domain processing is transferred back to the image without additional reconstruction error.
  2. PR does not provide the image prior, which is critical for self-supervised image restoration as there is no ground truth for supervision. PR specifies how to recover an image from its coefficients, but does not specify how those coefficients should be modified for restoration. Restoration performance critically depends on the image prior. For example, sparsity of wavelet coefficients motivates thresholding for denoising; PR provides the reconstruction mechanism, while sparsity guides noise removal.
  3. Learning adapts the representation and its associated prior to each image. GaborCanvas learns an image-conditioned frame from the target degraded image. The restoration gains come from this image-specific optimization of the coefficient prior associated with the representation. Without PR, additional error when reconstructing the image from its coefficients can partially offset the benefits of optimized coefficient processing. PR preserves these gains by avoiding additional reconstruction error.

We will clarify these roles in the revision and attribute the restoration gains to coefficient processing in the learned adaptive frame.

AuthorsOpenReview ↗

Response to Reviewer Yg8B (3/4)

W4. Limited evaluation beyond denoising; 0.01 dB gain over DIP; 0.49 dB gain over the biharmonic initialization.

Response. We thank the reviewer for these concrete points. We expanded the evaluation to BSD100 and structured masks and tested biharmonic warm starts for the strongest inpainting baselines. We agree that the +0.01 dB margin over DIP at ×4 is not meaningful and will remove the claim that GaborCanvas is the best GT-free SR method.

(a) Beyond Set14 and random masks. We evaluated the methods in Tables 3 and 4 on BSD100 and those in Table 4 on text and block masks, using their original settings. Comparisons below focus on the GT-free single-image methods. Pretrained diffusion methods use external training data and run at 256×256; their results are included as references rather than direct comparisons with full-resolution evaluations.

Table R2-3. BSD100 inpainting: 100 images; GT-free methods at full resolution; Table 4 protocol; PSNR / SSIM / LPIPS.

Method30% missing50% missing
Pretrained diffusion priors: 256×256, reference results
DDNM34.70 / .953 / .04731.24 / .906 / .104
DiffPIR32.63 / .936 / .08129.19 / .872 / .160
DAPS29.87 / .871 / .15826.38 / .755 / .274
GT-free single-image methods
DIP32.43 / .942 / .08228.88 / .877 / .176
DD32.90 / .947 / .07429.88 / .898 / .141
Fourier-feature INR31.27 / .914 / .14427.96 / .831 / .257
INCODE33.14 / .942 / .08929.78 / .886 / .159
I-INR32.60 / .939 / .09129.07 / .872 / .185
SL2A-INR32.41 / .933 / .10929.11 / .867 / .198
MSA30.04 / .901 / .17027.42 / .827 / .277
GaborCanvas34.11 / .963 / .052  30.46 / .918 / .113

GaborCanvas leads the tested GT-free methods on all three metrics at both missing rates, with PSNR margins of 0.97 dB and 0.58 dB over the strongest baseline.

Table R2-4. Structured-mask inpainting on Set14: text masks with 14–15% missing pixels and block masks with 25% missing pixels; GT-free methods at full resolution; PSNR / SSIM / LPIPS.

All methods use the same evaluation masks. For self-supervised training, GaborCanvas additionally holds out observed pixels using text- or block-shaped masks; details will be provided in Appendix F.

MethodTextBlock
Pretrained diffusion priors: 256×256, reference results
DDNM34.92 / .969 / .032  26.58 / .886 / .147
DiffPIR33.08 / .960 / .04325.39 / .875 / .139
DAPS30.96 / .934 / .07124.24 / .833 / .147
GT-free single-image methods
DIP30.29 / .941 / .08122.30 / .817 / .185
DD31.75 / .952 / .08320.97 / .831 / .220
Fourier-feature INR27.42 / .895 / .16518.26 / .763 / .280
INCODE31.03 / .934 / .10120.89 / .792 / .240
I-INR29.18 / .919 / .13019.67 / .781 / .253
SL2A-INR28.99 / .911 / .13419.68 / .774 / .253
MSA29.10 / .923 / .11622.01 / .809 / .210
GaborCanvas31.76 / .957 / .07623.16 / .856 / .202

Among the tested GT-free methods, GaborCanvas achieves the highest SSIM for both mask types, the lowest LPIPS for text masks, and the highest PSNR for block masks. Text-mask PSNR is essentially tied with DD (+0.01 dB), while DIP achieves the lowest LPIPS for block masks.

AuthorsOpenReview ↗

Response to Reviewer Yg8B (4/4)

(b) Super-resolution. On Set14 at ×4, GaborCanvas is essentially tied with DIP in PSNR while achieving the best SSIM and LPIPS among the GT-free baselines of the submission (Table 3). It also uses 0.118M parameters, compared with DIP’s 13.4M. On BSD100, the ranking depends on the scale:

Table R2-5. BSD100 SR: 100 images; Table 3 protocol and settings; PSNR-Y / SSIM-Y / LPIPS. Bold indicates the best result among the GT-free single-image methods listed.

Method×2×4×8
Bicubic29.55 / .856 / .20225.95 / .685 / .37823.66 / .538 / .542
Supervised external training
SwinIR32.53 / .904 / .14927.92 / .749 / .33025.03 / .609 / .478
LIIF-EDSR32.18 / .909 / .15727.60 / .757 / .34624.79 / .597 / .497
LTE-EDSR32.21 / .910 / .15627.62 / .757 / .34524.82 / .598 / .498
ContinuousSR–27.48 / .754 / .32824.68 / .598 / .480
Pretrained diffusion priors: 256×256, reference results
DDNM31.25 / .899 / .15626.68 / .733 / .34124.02 / .562 / .523
DiffPIR25.38 / .596 / .44823.12 / .456 / .57620.34 / .324 / .593
DAPS30.23 / .850 / .21526.22 / .672 / .41523.54 / .490 / .590
GT-free single-image methods
DIP30.18 / .864 / .20526.18 / .688 / .38923.74 / .522 / .532
DD29.47 / .839 / .28426.68 / .721 / .383  24.22 / .569 / .530
Fourier-feature INR30.37 / .874 / .19426.29 / .695 / .38023.59 / .482 / .552
INCODE26.61 / .665 / .40021.45 / .319 / .60116.47 / .125 / .681
I-INR24.64 / .585 / .43118.75 / .275 / .60817.22 / .219 / .651
SL2A-INR24.71 / .628 / .45815.05 / .199 / .67710.72 / .066 / .721
SuperF-single27.82 / .811 / .20823.46 / .552 / .46220.06 / .323 / .624
MSA28.04 / .792 / .29923.85 / .551 / .51722.04 / .452 / .586
GaborCanvas30.48 / .889 / .183  26.38 / .717 / .36423.97 / .555 / .523

Among the GT-free baselines of the submission, GaborCanvas leads on all three metrics at ×2 and achieves the lowest LPIPS at every scale. DD achieves higher PSNR and SSIM at ×4 and ×8.

As in Table 3 on Set14, INCODE, I-INR, SL2A-INR, and MSA are below bicubic interpolation. This is a limitation of INRs that are fitted directly to the low-resolution pixels and then evaluated at the high-resolution coordinates, as in the SR script of INCODE, which we also apply to the other three: the network reproduces the low-resolution input, but nothing constrains its values between the samples.

We will report these results by benchmark and scale. SR will be presented as an application of the learned coefficient prior within a standard inverse-problem solver; our main empirical strengths are denoising and inpainting.

(c) Gain over the biharmonic initialization. Table 8 isolates the effect of the coefficient prior within our pipeline: setting its weight to zero returns the biharmonic initialization, while enabling it improves PSNR by 0.84 dB and 0.49 dB at 30% and 50% missing pixels.

We also tested whether the strongest GT-free inpainting baselines benefit from a biharmonic warm start. Before their original Table 4 optimization, DIP, DD, and INCODE were first fitted to the biharmonic completion computed from their training pixels.

Table R2-6. Set14, random masks, Table 4 settings; PSNR / SSIM. Arrows show original results → results with biharmonic warm starts.

Method30% missing50% missing
Biharmonic initialization  33.78 / .95630.31 / .914
DIP34.05 / .954 → 33.98 / .954  30.17 / .904 → 29.99 / .902
DD32.69 / .950 → 32.82 / .95029.79 / .906 → 29.92 / .905
INCODE33.74 / .947 → 33.92 / .94830.37 / .894 → 30.30 / .894
GaborCanvas34.62 / .96230.80 / .918

The warm starts change baseline PSNR by −0.18 to +0.18 dB. The strongest warm-started baseline remains 0.64 dB and 0.50 dB below GaborCanvas. Thus, providing these baselines with a biharmonic warm start does not close the observed performance gap.

Reviewer_GChhOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces GaborCanvas, an image-adaptive Gabor frame representation in which a shared neural network predicts spatially varying Gabor parameters—orientation, scales, frequencies, and phases—for local atoms at each image location. Its main contributions are: (1) a neural framework for learning non-stationary Gabor frames aligned with local image structures; (2) theoretical results for continuum and discrete adaptive Gabor systems, including frame-preservation conditions and a local dual-frame construction that yields perfect reconstruction under admissible parameters/full-rank local matrices; and (3) task-specific self-supervised formulations for denoising, single-image super-resolution (SR), and inpainting using coefficient sparsity and R2R/thresholding or measurement-fidelity objectives, without clean targets.

Critical strengths
  1. The paper connects non-stationary/non-uniform Gabor systems with neural prediction and constructs local dual frames for perfect reconstruction. This is consequential because it provides interpretable spatial-frequency atoms while retaining an exact reconstruction identity, unlike purely implicit INR/DIP representations.
  2. GaborCanvas achieves strong results among GT-free methods in several denoising, SR, and inpainting settings, with very low parameter counts (e.g., 0.592M for denoising, 0.118M for SR, 0.041M for inpainting). The ablations also support the value of non-stationary atoms and Gabor modulation over Gaussian atoms.
Critical weaknesses
  1. SR and inpainting are evaluated only on Set14. Denoising uses Kodak24, McMaster18, SIDD, and FMD, but several standard benchmarks are missing. Some SSIM results are lower than competing GT-free methods, and diffusion-prior methods remain stronger in inpainting. There is also no runtime/FLOPs comparison, despite the method requiring per-image optimization. This limits the strength of the empirical claims.
  2. Theorem 1 and Theorem 2 give sufficient conditions for frame preservation and dual construction, but the network predicts scales/frequencies via a tanh-parameterized head. It is unclear whether the predicted parameters are guaranteed or monitored to satisfy the frequency-perturbation bounds. The 2D extension is informal, and perfect reconstruction in practice relies on local full-rank matrices. The reported condition numbers are not negligible, so the practical robustness of the dual construction needs more justification.
Questions for authors
  1. Please clarify the precise technical difference from WIRE, data-driven tight frames, Gabor CNNs, and 3D Gabor splatting. Which part is fundamentally new: the adaptive non-stationary frame, the dual construction, or the self-supervised restoration formulation?
  2. How do you ensure or monitor that the predicted Gabor parameters satisfy the conditions in Theorem 1/Theorem 2? If the network does not explicitly enforce them, does perfect reconstruction still hold only because the local matrices G_p have full row rank? Please provide diagnostics or constraints used during training.
  3. Can you provide a formal 2D theorem and proof, including boundary handling, patch overlap normalization, and the exact constant-coverage argument for the 2D local dual construction? Please provide diagnostics or constraints used during training.
  4. Can you add results on standard benchmarks such as Urban100, BSD100, DIV2K, or CBSD68, and include structured/text inpainting masks and ×8 SR? This would materially affect the generality assessment.
  5. What are the runtime, FLOPs, and memory costs compared with DIP, INR-based methods, and diffusion-prior methods? Since the method involves per-image optimization, this is important for practical significance.
5 comments
AuthorsOpenReview ↗

Response to Reviewer GChh (1/5)

We thank the reviewer for the constructive feedback and for highlighting the interpretability, explicit reconstruction guarantees, and compact parameterization of our image-adaptive Gabor representation.


W1. Benchmark coverage, SSIM, diffusion priors in inpainting, and runtime/FLOPs.

Response. The additional results below will be included in the revised paper: benchmarks in Appendix F and Table 3, and computational costs in Appendix G.

(a) Benchmarks. We added BSD100 inpainting and SR, text/block masks and ×8 SR on Set14, and CBSD68 denoising. We use the original baselines and settings, with externally trained methods grouped separately.

Table R1-1. BSD100 inpainting. 100 images, full resolution; Table 4 protocol; PSNR / SSIM / LPIPS.

Method30% missing50% missing
Pretrained diffusion priors (256×256)
DDNM34.70 / .953 / .04731.24 / .906 / .104
DiffPIR32.63 / .936 / .08129.19 / .872 / .160
DAPS29.87 / .871 / .15826.38 / .755 / .274
GT-free single-image methods
DIP32.43 / .942 / .08228.88 / .877 / .176
DD32.90 / .947 / .07429.88 / .898 / .141
Fourier INR31.27 / .914 / .14427.96 / .831 / .257
INCODE33.14 / .942 / .08929.78 / .886 / .159
I-INR32.60 / .939 / .09129.07 / .872 / .185
SL2A-INR32.41 / .933 / .10929.11 / .867 / .198
MSA30.04 / .901 / .17027.42 / .827 / .277
GaborCanvas34.11 / .963 / .052  30.46 / .918 / .113

Among the listed GT-free methods, GaborCanvas leads in all six metrics, with PSNR gains of 0.97/0.58 dB over the next-best method at 30%/50% missing.

Table R1-2. Structured masks. Set14, full resolution; text: 14–15% missing; block: 25%. All methods use the same masks and Table 4 settings; PSNR / SSIM / LPIPS.

MethodTextBlock
Pretrained diffusion priors (256×256)
DDNM34.92 / .969 / .032  26.58 / .886 / .147
DiffPIR33.08 / .960 / .04325.39 / .875 / .139
DAPS30.96 / .934 / .07124.24 / .833 / .147
GT-free single-image methods
DIP30.29 / .941 / .08122.30 / .817 / .185
DD31.75 / .952 / .08320.97 / .831 / .220
Fourier INR27.42 / .895 / .16518.26 / .763 / .280
INCODE31.03 / .934 / .10120.89 / .792 / .240
I-INR29.18 / .919 / .13019.67 / .781 / .253
SL2A-INR28.99 / .911 / .13419.68 / .774 / .253
MSA29.10 / .923 / .11622.01 / .809 / .210
GaborCanvas (Ours)31.76 / .957 / .07623.16 / .856 / .202

Among the listed GT-free methods, text-mask PSNR is essentially tied with DD (+0.01 dB); DIP has better block-mask LPIPS.

Table R1-3. BSD100 SR. 100 images; Table 3 protocol; PSNR-Y / SSIM-Y / LPIPS.

Method×2×4×8
Bicubic29.55 / .856 / .20225.95 / .685 / .37823.66 / .538 / .542
Supervised external training
SwinIR32.53 / .904 / .14927.92 / .749 / .33025.03 / .609 / .478
LIIF-EDSR32.18 / .909 / .15727.60 / .757 / .34624.79 / .597 / .497
LTE-EDSR32.21 / .910 / .15627.62 / .757 / .34524.82 / .598 / .498
ContinuousSR–27.48 / .754 / .32824.68 / .598 / .480
Pretrained diffusion priors (256×256)
DDNM31.25 / .899 / .15626.68 / .733 / .34124.02 / .562 / .523
DiffPIR25.38 / .596 / .44823.12 / .456 / .57620.34 / .324 / .593
DAPS30.23 / .850 / .21526.22 / .672 / .41523.54 / .490 / .590
GT-free single-image methods
DIP30.18 / .864 / .20526.18 / .688 / .38923.74 / .522 / .532
DD29.47 / .839 / .28426.68 / .721 / .383  24.22 / .569 / .530
Fourier-feature INR30.37 / .874 / .19426.29 / .695 / .38023.59 / .482 / .552
INCODE26.61 / .665 / .40021.45 / .319 / .60116.47 / .125 / .681
I-INR24.64 / .585 / .43118.75 / .275 / .60817.22 / .219 / .651
SL2A-INR24.71 / .628 / .45815.05 / .199 / .67710.72 / .066 / .721
SuperF-single27.82 / .811 / .20823.46 / .552 / .46220.06 / .323 / .624
MSA28.04 / .792 / .29923.85 / .551 / .51722.04 / .452 / .586
GaborCanvas30.48 / .889 / .183  26.38 / .717 / .36423.97 / .555 / .523

Among the listed GT-free methods, GaborCanvas leads at ×2 and in LPIPS at all scales; DD achieves higher PSNR and SSIM at ×4/×8. As in Table 3 on Set14, INCODE, I-INR, SL2A-INR, and MSA are below bicubic interpolation. These INRs are fitted directly to the low-resolution pixels and evaluated at high-resolution coordinates, as in the SR script of INCODE, which we also apply to the other three: the network reproduces the low-resolution input, but nothing constrains its values between the samples.

AuthorsOpenReview ↗

Response to Reviewer GChh (2/5)

Table R1-4. Set14 ×8 SR. Table 3 protocol; PSNR-Y / SSIM-Y / LPIPS. This column will be added to Table 3.

Method×8
Bicubic23.19 / .560 / .517
Supervised external training
SwinIR25.46 / .657 / .431
LIIF-EDSR24.94 / .640 / .457
LTE-EDSR24.99 / .642 / .456
ContinuousSR24.85 / .641 / .440
Pretrained diffusion priors (256×256)
DDNM22.97 / .563 / .492
DiffPIR19.36 / .304 / .594
DAPS22.47 / .499 / .556
GT-free single-image methods
DIP23.65 / .554 / .522
DD23.51 / .588 / .499
Fourier-feature INR23.04 / .485 / .551
INCODE16.60 / .152 / .698
I-INR16.89 / .237 / .634
SL2A-INR10.61 / .068 / .712
SuperF-single19.05 / .300 / .623
MSA20.96 / .441 / .591
GaborCanvas23.63 / .578 / .505

GaborCanvas ranks second in all three metrics among the listed GT-free methods. DIP leads PSNR by 0.02 dB; DD leads SSIM and LPIPS.

Table R1-5. CBSD68 denoising. 68 images, 256×256 center crops; Table 1 protocol; PSNR / SSIM. DDTF is the additional baseline described in Q1.

Methodσ=15σ=25σ=50
TV29.48 / .84326.97 / .75824.13 / .612
Framelet+LDCT  29.75 / .84427.04 / .73622.75 / .527
DT-CWT28.69 / .82126.39 / .72823.53 / .562
DST29.74 / .84727.20 / .75824.05 / .599
DDTF30.31 / .86627.63 / .78224.45 / .624
Gabor27.78 / .78525.22 / .67622.88 / .550
PD-Denoising30.25 / .86828.30 / .80825.63 / .706
ScoreDVI29.61 / .82727.98 / .77824.76 / .656
MASH24.05 / .63924.28 / .64623.85 / .618
R2R30.96 / .86229.13 / .82825.41 / .655
NLR24.29 / .64623.98 / .62123.16 / .570
Pixel2Pixel29.10 / .85328.17 / .80625.78 / .693
ZS-HPD28.02 / .82525.98 / .73221.64 / .500
GaborCanvas32.50 / .914  29.45 / .845  25.69 / .705

GaborCanvas leads in both metrics at σ=15/25, with PSNR gains of 1.54/0.32 dB over R2R. At σ=50, it ranks second in both metrics, 0.09 dB behind Pixel2Pixel in PSNR.

(b) SSIM. We thank the reviewer for pointing this out, it made us recheck the SSIM values of Table 2 of our submitted paper. The values of the baselines in Table 2 are quoted from the NLR paper (Guo et al., 2025). In the SIDD benchmark column: the original SIDD benchmark platform is no longer available and was replaced by a Kaggle competition in May 2024, on which SSIM may be calculated differently. The benchmark values of CVF-SID (.917) and AP-BSN (.931) were obtained on the original platform, whereas ours (.889) and those of recent methods such as NLR (.874) were obtained on Kaggle, so the two groups are not directly comparable. This concerns only affect comparison in the SIDD benchmark column. The SIDD validation and FMD columns are not affected and remain comparable. In the revision, Table 2 will mark the SIDD benchmark values obtained on the original platform and state that their computation differs. Thanks again for the reminder.

(c) Diffusion priors. DDNM achieves higher inpainting PSNR, whereas DiffPIR and DAPS score below GaborCanvas in all metrics on random masks in Set14 and BSD100. DDNM and DiffPIR are stronger on text and block masks. These comparisons involve different training data and resolutions: diffusion methods use pretrained priors at 256×256, while GaborCanvas learns from the observed pixels of a single full-resolution image, with 0.041M parameters and no external training data.

AuthorsOpenReview ↗

Response to Reviewer GChh (3/5)

(d) Runtime and FLOPs. All measurements use an RTX 4090 and cover the complete per-image procedure. Peak process memory is measured with nvidia-smi, using one process per method. FLOPs are counted using PyTorch's FlopCounterMode, supplemented by counts for dense local solves.

Table R1-6. Denoising. Kodak24, 256×256 crops, AWGN σ=25; Table 1 protocol; end-to-end time per image.

MethodTime (s)  Peak memory (GiB)
PD-Denoising  1.10.5
Pixel2Pixel22.218.5
ScoreDVI33.31.4
MASH40.70.8
R2R46.02.4
ZS-HPD52.60.7
NLR69.51.6
GaborCanvas23.52.9

GaborCanvas is faster than five of the seven listed baselines. Its 23.5 s comprises 18.3 s fitting (4.6 TFLOPs), 0.45 s reconstruction (0.25 TFLOPs), and 4.7 s startup and I/O overhead. Fitting uses fixed 48×48 crops; at 768×512, reconstruction takes 3.25 s (1.47 TFLOPs).

Table R1-7. Inpainting. Two Set14 images at full resolution (baboon, 500×480; barbara, 720×576), 50% missing; end-to-end time per image.

MethodTime (s)  Peak memory (GiB)
DDNM (pretrained diffusion, 256×256)  8.42.6
DIP133.39.2
DD90.83.0
Fourier-feature INR74.84.0
INCODE97.52.3
GaborCanvas89.98.2

GaborCanvas is faster than DIP, DD, and INCODE; Fourier-feature INR and pretrained DDNM are faster.

Table R1-8. SR ×4. BSD100, 480×320 output; end-to-end time per image, including model loading. Measurements use three images for GT-free methods and two for supervised/diffusion methods; diffusion outputs are 256×256.

MethodTime (s)  Peak memory (GiB)
Supervised external training
SwinIR3.20.8
LIIF-EDSR3.91.0
LTE-EDSR3.81.0
ContinuousSR5.11.5
Pretrained diffusion priors (256×256)
DDNM8.42.6
DiffPIR11.83.7
DAPS15.63.7
GT-free single-image methods
DIP45.86.6
DD54.21.1
Fourier-feature INR26.31.7
INCODE15.30.9
I-INR38.41.3
SL2A-INR8.30.6
SuperF-single28.61.3
MSA12.10.6
GaborCanvas (frame fitting + solver)39.81.0

GaborCanvas is faster than DIP and uses less peak memory; several INR and pretrained methods are faster. Separately, on Set14, a complete SR run requires at most 1.9 TFLOPs per image for GaborCanvas, compared with 644 for DIP and 724 for Fourier-feature INR. These counts include fitting, reconstruction, and local matrix computations.

Comments on the paper

3 comments

We thank all reviewers for their careful reading and constructive feedback. We have posted our point-by-point responses and welcome any further questions or requests for clarification.

All additional experiments, ablation studies, and requested theoretical derivations will be included in the revised manuscript and appendices, which we will upload before the discussion deadline.

Area_Chair_UgUSOpenReview ↗

AC Reminder: Please Follow Up on the Authors’ Responses

Dear Reviewers,

Thank you for your time and thoughtful reviews. The authors have posted responses to your questions and concerns.

If you have not yet done so, please read their responses and any relevant revisions, and leave a follow-up comment visible to the authors indicating which concerns have been addressed and which remain unresolved. Please update your review and rating if warranted. If your assessment remains unchanged, a brief explanation would still be helpful.

Thank you for your continued engagement in the discussion.

Best regards,

Area Chair

AuthorsOpenReview ↗

Summary of revisions

We thank the reviewers and the Area Chair for their constructive feedback. We have uploaded the revised manuscript and appendix, incorporating the additional experiments, theoretical derivations, and clarifications presented in our responses to the reviewers. Substantive changes are highlighted in blue.

1. Novelty and comparisons. We clarified the distinction between spatially shared learned dictionaries and our image-conditioned, spatially varying Gabor frames. We added DDTF and GDLNet comparisons (Table 1) and free-form atom ablations, including a parameter- and computation-matched variant (Table 6). These complement the spatial-adaptation ablation in Table 5.

2. Two-dimensional theory and reconstruction. The new Theorem 3 is included in the main paper, giving explicit sufficient conditions and frame bounds for spatially varying scales and nonuniform local frequencies. The main text explains dual reconstruction, stride-dependent overlap normalization, and boundary extension; the proof is in Appendix A.3. Appendix C.1 reports exhaustive rank checks and reconstruction tests across strides and boundary rules.

3. Broader evaluation. We added BSD100 for inpainting and SR, CBSD68 for denoising, structured inpainting masks, ×8 SR, and ZSSR comparisons (Appendix F and Table 3). We replaced the claim of the best GT-free SR performance with a benchmark- and scale-specific assessment.

4. Computational cost. Appendix G reports runtime, peak memory, and FLOPs across all three tasks, including separate fitting, reconstruction, and local matrix costs.

Additional changes clarify the role of perfect reconstruction, expand implementation details and sensitivity studies, correct baseline labels and supervision categories, and correct the parameter count in Table 2 to 0.553M.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.