- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Invertible Neural Cellular Automata (iNCA). Its main innovation is an additive coupling layer: the cell state is split into two channel groups, each updated by a residual computed from the other group’s local neighbourhood. Parameters are shared across steps, and the group order alternates between steps. The authors use reversible backpropagation to reduce training memory. They also train on latent grids using a frozen VQ-VAE, reducing both the number of cells updated per step and the number of local interactions needed to span the grid. They demonstrate the utility of this approach for handling ambiguous inverse mappings. The authors conduct experiments on a bidirectional MNIST task, structured inversion on Edge2Icon, appearance variation on Edges2Handbags, and ambiguous inverse mappings using MNIST. Their results show performance comparable to or better than the i-RevNet, RealNVP, and Glow baselines, along with a roughly 50-fold reduction in peak memory usage relative to ordinary autograd through the use of reversible backpropagation.
Critical strengths
- The ablations are extensive and insightful: The study of neighbourhood size and parameter sharing between F and G shows which design choices matter.
- The construction is simple and sensible: The additive coupling is an elegant way to achieve invertible neural cellular automata.
Critical weaknesses
- The introduction and literature review do not adequately capture the fields they discuss.
- The introduction states that forward mappings are easy to learn “following the Universal Approximation Theorem.” However, the UAT is an existence result and says nothing about learnability.
- A central concern is that the introduction presents inverse problems without adequately acknowledging the extensive literature on them in the context of neural networks. Approaches can be broadly divided into those that improve inference beyond a forward pass and those that change the learning problem, for example, by modifying the neural network architecture and, in some cases, the associated loss function.
- There is also much more to Bayesian deep learning than Bayesian neural networks (BNNs), so the discussion of Bayesian methods is too reductive.
- It is also worth discussing the rich history of invertible cellular automata, as reviewed in “Invertible cellular automata: A review” in Physica D: Nonlinear Phenomena. The novelty of this work needs to be positioned against this history.
-
The comparison methods used in the experiments and discussed in the related work appear to date from around 2019 or earlier. It would be helpful to consider more recent benchmarks for invertible architectures, such as this work. Invertible diffusion models, such as EDICT, are also worth benchmarking against.
-
Novelty concerns of the core innovation
- It would also be worth noting a similar addictive coupling trick is also found in the NICE paper but for generative modelling. The paper cites Laurent Dinh’s later work, but NICE introduced an invertible mapping with an additive coupling mechanism and should be cited at least.
Questions for authors
A Diff-NCA paired with a reversible sampler, such as the one proposed in “Rex: A Family of Reversible Exponential (Stochastic) Runge-Kutta Solvers” from this year’s ICML, would also yield an exactly invertible, local, weight-tied dynamics with genuine stochastic generation. This would be a natural baseline for your work, as it would effectively constitute another invertible neural cellular automaton, but one based on diffusion.
Withdrawal
We thank the reviewers for their valuable feedback. After careful consideration, we conclude from the reviews that the reviewers see potential in our proposed method and basically approve its theoretical derivation, but argue that a a broader discussion and comparison with related work is needed. We agree with this evaluation and will revise our work accordingly for re-submission at a different venue.