- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a new architecture for building INRs for image representation. The method consists of having multiple parallel MLPs that each get a differently scaled input, and who’s output gets merged at the end again. The authors provide an argument using NTK to justify why they’d expect their method to actually improve the representational capability of INRs. The authors claim it surpasses state-of-the-art in various tasks including just simple image representation, super resolution, image denoising, image inpainting etc. Some of the experiments are done on small datasets, others on single samples.
Critical strengths
The authors propose this multi-branch architecture with different coordinate scaling in each branch. This is a novel idea and based on the presented experiments it warrants some investigation. Figure 2 nicely explains the idea and gives a good overview on how it is designed.
Critical weaknesses
The experiments the authors provide can easily serve as a motivation for some further experiments, but are fundamentally insufficient to support the claim of “outperforming” the baselines. I’ll go into more detail why:
Experiment 1 (the main experiment) uses the kodak image dataset, consisting of 24 images. Experiments 2-6 were performed on a single sample, each of them with slightly different parameters. To actually have convincing results, each of those should ideally include larger datasets, maybe even from heterogeneous sources, to make sure the chosen hyperparameters are not just specific to this image, expecially considering the very thin improvements that are being reported. Also, the chosen settings might not be optimal for the baseline. For this we’d need to see a pipeline to carefully optimize of the hyperparameters of all of the baselines, and the authors did not report doing that. Furthermore, from what we can gather from the paper, the proposed method uses roughly 4x (in 2D) the amount of parameters compared to the baselines. This should more prominently be mentioned and taken into account with all the comparisons and ablations.
I’d like to add a few points that the authors could possibly address:
-
The NTK argument boils down to saying that for certain values of the scaling factor, the eigenvalues increases and the convergence therefore decreases. This argument appears quite post-hoc and provides no further intuition, I’d recommend shortening it.
-
An ablation is only provided in the appendix. One experiment I’m missing there is a fair (as in equal number of parameters with a suitable architecture, same amount of memory or FLOPs used or similar notion of “fair”) comparison with the baseline architectures. The authors provided the comparison with a wider SIREN network but this is on the one hand not the optimal way to scale SIRENs, but also omits the other baselines.
-
Please use larger datasets to evaluate your proposed method on and actually report some measure of spread, especially since the reported improvements are quite marginal.
-
Please first define the expression “frequency atom” before using it.
-
A few claims refer to figure 1, but without specifying to what subfigure. Similarly, figure 3 has three subfigures, but not all of them are actually relevant. E.g., 3a is not referenced anywhere and provides no additional value.
-
The resolution of the images in figure 6 are insufficient to display the changes well. Furthermore, the poses of the displayed mesh should be identical to provide a good comparison.
-
Equation 8 is referenced a lot, but it is only listed in the appendix. Consider moving it to the main paper.
Questions for authors
The experiments use mainly scaling factors around 0.5, however, the more “theoretical” parts investigate mostly scaling factors of 10, 20 or even larger. Of course I see that a scaling of e.g. (1, 0.5) or (2, 1) results in the same network (modulo learning rate), but a factor of 2 is still quite different than a factor of 10 -100. Can you comment on why there is this mismatch?