acceptodds
Under review as a conference paper at ICLR 2027

LNRR: Learnable Nonlocal Relation Regularization for Implicit Neural Representations

Abstract

Implicit neural representations (INRs) model discrete observations as samples of a continuous signal and provide compact, resolution-independent representations. However, an INR is supervised only at observed locations and generalizes poorly to unobserved ones, especially inside large missing regions. Local smoothness priors only propagate information inward from the boundary of a missing region, so the interior of a large region degrades into a blur that loses the original edges and textures. These structures can only be borrowed from similar ones elsewhere in the image, yet INRs have no built-in mechanism to exploit such nonlocal relations. We propose Learnable Nonlocal Relation Regularization (LNRR), which brings nonlocal relations between image patches into INR training to improve completion under large missing regions. LNRR learns patch relations with a patch feature network and uses them to constrain the reconstruction of missing regions through a robust graph-residual regularizer in the wavelet domain. LNRR changes only the regularization term and applies to a range of INR backbones. Across multiple natural-image datasets and missing patterns, LNRR achieves state-of-the-art completion among existing INR regularization methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Implicit neural representations, INR generalization, Nonlocal regularization, Wavelet-domain regularization, Structured missingness, Single-image restoration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kwyeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies implicit neural representations (INRs) for image representation. This is the most common application of INRs, although other modalities have been studied too (1D signals, videos, point-clouds, etc). The paper makes the observation that INRs tend to perform poorly when inpainting large missing regions (i.e., where there are no observations to act as a supervisory signal for the model). It proposes to learn a patch representation of the observed image and use that during reconstruction. In this sense it is similar to early super-resolution and energy-based image inpainting approaches. Experiments show quantitative and qualitative improvements for image completion at a modest increase in running time (53s vs 199s).

Critical strengths

The paper is well written and the approach is principled borrowing on ideas in the computer vision literature on patch-based approaches for image completion and in-painting. The technical aspects of the paper are clear with complete mathematical description of the algorithm. Experiments are promising albeit on a very small dataset and only for image completion.

Critical weaknesses
  1. The idea of using self-learned patches for image completion is not new, and the paper cites some traditional approaches. The authors may consider also citing classic approaches in energy-based image inpainting (e.g., Komodakis and Tziritas, CVPR 2006) and image super-resolution (e.g.,Glasner et al., ICCV 2009).

  2. The experiments are quite limited. Only a small number images are tested and only for image in-painting using five mask types. I would have liked to see results on other tasks such as super-resolution, which is the primary justification for INRs (i.e., resolution independence) and other modalities (e.g. 1D sound waves or 4D video).

Questions for authors

Please address the concerns raised above.

Reviewer_dkDAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a Learnable Nonlocal Relation Regularization (LNRR) to improve the completion performance of Implicit Neural Representations (INRs) over large missing regions. TV and INRs fail to generalize over large missing regions yielding blurry outputs. This paper exploits non-local self-similarity without relying on pixel distances. LNRR utilizes overlapping 2D image patches as graph nodes. The patch graph is learned via a coordinate-based patch feature network trained jointly with the INR. Both the graph adjacency and the reconstruction are optimized by minimizing a Huber penalty on the Haar coefficients of each patch’s deviation from the weighted mean of its neighbors. The evaluation is performed across 36 images and 5 backbones improving the reconstruction metrics over the baselines.

Critical strengths
  1. The idea of the regularizer is simple and backbone agnostic which broadens its aplicability.
  2. Thorough experimental setup and ablations which lead to consistent average improvement.
  3. Transparent reporting and honest framing of limitations and failure cases.
Critical weaknesses
  1. No standard deviations or confidence intervals along with statistical tests are reported in the main paper.
  2. All the images tested have the same size 256x256 which confouds the stride and resolution ablations. It would be nice to see how the method performs in other sizes.
  3. It is not obvious whether the learning rates were properly tuned and if they were it is not reported how or on which set. Since we have 2 networks with 2 different learning rates it would nice to see this in the paper.
  4. CRNL was run with the released defaults and not tuned at all.
Questions for authors
  1. Can you report standard deviations or confidence intervals along with statistical testing?
  2. Can you tune CRNL with a comparable budget and also justify the chosen learning rates?
  3. Can you discuss or potentially apply the regularizer in a non-INR domain to demonstrate its applicability?
Reviewer_AwTtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a regularizer for implicit neutral representations of images that encourages to utilize inherent structures within an image to learn regions with no supervision or weak supervision. The authors specifically tackle the task of inpainting in this paper. The proposed regularizer works as follows:

  1. Represent the image as a graph G(V,A), where the nodes V, are defined as overlapping patches of an image. The Adjacency, A, is defined by the similarity of two patches

  2. Similarity is computed via the dot product of features of any two patches ii and jj i.e $ <G_{\phi}(i), G_{\phi}(ij)> $, where $ G_{\phi} $ is a learnable feature extractor.

  3. During training, deviation of any patch from nearby patches is penalized along with the traditional regression loss.

  4. The implicit network fθf_{\theta} and feature network GϕG_{\phi} jointly optimized through the reconstruction loss and the regularizer.

Quantitative and Qualitative results show modest improvement over prior work.

Critical strengths
  1. The paper proposed is well motivated and easy to follow (for the most part).

  2. The method proposed is intuitive, works well and is seemingly simple to implement.

  3. Quantitative and Qualitative results show improvement over prior work.

Critical weaknesses
  1. The “Robust potential” and “Transfer domain” are very dense, hard to follow and poorly written. They need a complete re-write with more intuitive explanations.

  2. It would be good to see more normalizations ablated for the adjacency in addition to softmax, such as softmax with temperature, linear weighting, sigmoid etc.

Questions for authors

See weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.