acceptodds
Withdrawn from ICLR 2027

Colour-INR - Implicit Neural Representation for Image Colorization through Colour Guidance

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Colourization of black and white images can be considered as an inverse problem that searches for pixel-level colour values based on observed intensity mappings. Since the forward problem follows a many-to-one mapping, the inverse admits multiple possible chrominance solutions. However, there is a set of solutions that can be considered realistic and acceptable. When defining this set of solutions, human perception becomes an important factor. Hint-based image colourization provides guidance to constrain this set of solutions. Implicit Neural Representations (INRs) have been applied to multiple inverse tasks such as denoising, super-resolution, and inpainting. However, their use for image colourization remains underexplored. In this research, we show that INRs have the potential to perform image colourization when guided appropriately. Moving further, we introduce Colour-INR, where a coordinate-based INR predicts chrominance from spatial coordinates, sparse colour hints provide chrominance supervision, and grayscale-derived edges guide spatial regularization. On Kodak with colour hints, Colour-INR achieves a mean PSNR of dB and substantially outperforms the evaluated pretrained learning-based baselines, while remaining competitive with the classical optimization method of Levin et al. Further, we experimentally show that Colour-INR can be optimized directly on high-resolution DIV2K images at their native resolution without changing the network architecture or requiring dataset-level retraining. These results demonstrate the potential of properly guided INRs for image colourization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hint based Colourization, Implicit Neural Representations, Edge Guidance, Inverse Problems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fCJXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates sparse-hint image colorization as an image-specific inverse problem. The proposed method, Colour-INR, represents the unknown CIELAB chrominance channels using a coordinate-based SIREN, while preserving the observed luminance for final image reconstruction. The network is optimized independently for each image using two losses: an MSE loss at sparse chrominance hint locations and an L1 spatial smoothness loss over eight-connected neighboring pixels. The smoothness term excludes pairs containing pixels identified as edges by a Canny detector applied to the luminance image.

Critical strengths
  1. Simple and clearly presented formulation: The paper presents a straightforward approach to sparse-hint colorization. Predicting only chrominance while preserving the observed luminance is a reasonable design choice, and the combination of sparse hint supervision and spatial regularization is easy to understand. The architecture and optimization procedure are described clearly.
Critical weaknesses
  1. Performance: The most important limitation is that Colour-INR is consistently outperformed by the classical method of Levin et al. across all reported metrics. The paper also does not evaluate other possible advantages of the INR formulation, such as runtime, memory consumption, robustness to hint placement, or generalization across resolutions. In particular, Colour-INR requires image-specific optimization for 2,000 epochs, whereas Levin et al. solves an optimization problem directly. Without measured runtime and memory comparisons, the practical trade-off remains unclear.

  2. Effects of the implicit neural representation: The proposed loss consists of sparse hint MSE and an edge-masked L1 smoothness penalty. When formulated directly over the unknown chrominance values at each pixel, this objective is convex, and it can be optimized without introducing a neural network. Does the SIREN provide a useful implicit prior, or are the reported results primarily explained by the sparse supervision and spatial regularization? It would be beneficial to include the most direct baseline: optimizing a dense pixel-wise chrominance grid using exactly the same objective and edge mask.

  3. Effects of edge-guidance: The paper emphasizes grayscale-derived edge guidance as an important mechanism for preventing color propagation across object boundaries. However, the experiments only vary the smoothness coefficient while keeping the edge mask enabled. The method also depends on Canny edge detection, but no sensitivity analysis is provided for the edge thresholds, smoothing parameter, or missing and inaccurate boundaries. Comparisons with soft, intensity-dependent affinity weights, as used in classical color propagation, would be particularly informative.

  4. Novelty: The proposed method combines established components: image-specific coordinate-based neural representations, sparse point supervision, and edge-aware spatial smoothness. Although applying this combination to hint-based colorization is a reasonable exploration, the paper does not clearly establish what new capability emerges from using an INR rather than directly optimizing a chrominance field. The paper would benefit from a more focused discussion of related image-specific colorization and neural-field methods, together with experiments identifying the distinctive benefits of the proposed representation.

Questions for authors
  1. What is the performance without edge masking? How sensitive is the method to Canny parameters, and how does hard edge exclusion compare with soft affinity-based smoothness?
Reviewer_w4DqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Colour-INR, which combines a coordinate network, sparse colour hints, and edge-aware smoothness for image colourization. A SIREN predicts CIELAB chrominance through image-specific optimization, and the predictions are combined with the observed luminance for reconstruction. The paper evaluates smoothness weights, network backbones, and network widths on Kodak24, compares with representative hint-based methods, and demonstrates native-resolution optimization on 10 DIV2K images.

Critical strengths
  1. The method is clearly described and has a coherent formulation. Separating observed luminance from unknown chrominance allows the coordinate network to focus on recovering the missing colour information. The approach also operates without dataset-level training, providing a useful exploration of INRs for guided colourization.
  2. The experiments demonstrate that spatial regularization substantially improves reconstruction over hint supervision alone. The backbone and width ablations provide further insight into the implementation choices, while the DIV2K experiments demonstrate the feasibility of native-resolution optimization without changing the network architecture.
Critical weaknesses
  1. The specific value of INR parameterization remains unclear without a pixel-wise chrominance baseline under the same objective. Levin et al. achieves better reported metrics on both datasets, while alternative benefits in efficiency, storage, or interaction are not quantified.
  2. Comparison fairness and robustness are insufficiently established: resolution handling differs across methods, hint-collision handling and effective hint counts after resizing are not specified, and the main results use one hint density and fixed seeds. Using Kodak24 for both configuration selection and evaluation also introduces a risk of selection bias.
Questions for authors
  1. How does Colour-INR compare with directly optimizing pixel-wise chrominance under identical hints, edge masks, and losses?
  2. How do the comparative results change when all methods reconstruct their outputs using the original-resolution luminance?
  3. How robust is Colour-INR when only a few user-provided colour hints are available?
  4. Does the reported advantage over pretrained baselines persist on a test set not used for configuration selection?
  5. How does Colour-INR’s computational cost compare with Levin et al. across image resolutions?
Reviewer_aEB6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This article addresses image colorization from a grayscale image using color hints. Instead of training a global model on a large dataset, a coordinate-based neural network is fitted separately for each image to predict the color channels, while the original brightness is kept unchanged. The network is optimized to match the given color hints while encouraging smooth colors, except across image edges. The approach is evaluated on some benchmark images against some classical and learning-based colorization methods, together with ablations on the main design choices and experiments on high-resolution images.

Critical strengths
  • The article is relatively clear and easy to follow.
  • The method needs no training dataset or pretrained model, and applies directly at any resolution with a very small network. The network infers colors only from pixel coordinates, colour hints, and an edge map. Reaching reasonable colorization results with such a minimal design is interesting (though the practical benefit of this choice remains to be shown).
Critical weaknesses
  • While the minimal design is interesting, the contribution is very limited. The proposed approach follows the same well-known principle of propagating color hints according to the image structure, the only difference being that the solution is parameterized by a coordinate-based network. Yet the classical approach of Levin et al. (2004) outperforms it on all metrics (Table 4). What is the advantage of this method if a 20 year old learning-free algorithm performs better? No specific benefit is demonstrated. Notably, no computation time is reported, although it is a critical aspect for such per-image optimization, unlike the fast inference of learning-based models.
  • The choice of not providing the luminance as input to the network is neither justified nor tested experimentally. While interesting from a design perspective, such a constraint seemingly offers no practical benefit, since the luminance is available and could directly guide the color prediction.
  • An entire body of closely related literature is omitted, namely single image optimization methods such as Deep Image Prior [1], Gatys-style optimization [2], or SinGAN [3]. To validate the methodology, the approach should be compared to other single-image optimization approaches.
  • The gap with learning-based approaches is very surprising: it is unclear why a classical 2004 method would so significantly outperform recent and well-established deep learning models (Table 3). This suggests a significant evaluation bias, and I suspect it may be due to resolution handling: the learning-based methods run at a lower resolution, and their output is upscaled back to the original resolution (Appendix D.1). This is unfair, as the comparison then includes the performance loss caused by upscaling. While working at full resolution is an interesting property of the proposed approach, a fair comparison would evaluate all methods at comparable resolutions.

[1] D. Ulyanov et al., "Deep Image Prior", CVPR 2018.

[2] L. A. Gatys et al., "Image Style Transfer Using Convolutional Neural Networks", CVPR 2016.

[3] T. R. Shaham et al., "SinGAN: Learning a Generative Model from a Single Natural Image", ICCV 2019.

Questions for authors
  • The article should report the computation/training time per image for different image resolutions.
  • The article should report the performance for varying percentages of color hints.
  • Why does the proposed approach not outperform the classical method of Levin et al. (2004), and what advantage does it offer over it?
  • What motivates the choice of not using the luminance as input to the network, and how could it be validated?
  • Clarify the large performance drop of learning-based approaches, notably regarding upscaling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.