acceptodds
Withdrawn from ICLR 2027

Invertible Neural Cellular Automata

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce invertible Neural Cellular Automata (iNCA), a novel recursive, self-organizing and space variant generative neural architecture, which is able to learn bi-directional mappings with Cellular Automata. By modeling an additional latent target distribution, iNCA allow to learn ambiguous inverse mappings and the estimation of posterior probabilities. Compared to previous invertible neural network architectures, iNCAs show a significant improvement in inverse mapping capabilities for data organized in discrete grid structures. We demonstrate the general theoretical capabilities of iNCA empirically on a series of image generation and classification tasks.

Primary area:
generative models
Keywords:
Invertible NCA, Cellular automata, Neural cellular automata, Inverse problems, Bidirectional mapping, Invertible networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_z1JgOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Invertible Neural Cellular Automata (iNCA). Its main innovation is an additive coupling layer: the cell state is split into two channel groups, each updated by a residual computed from the other group’s local neighbourhood. Parameters are shared across steps, and the group order alternates between steps. The authors use reversible backpropagation to reduce training memory. They also train on latent grids using a frozen VQ-VAE, reducing both the number of cells updated per step and the number of local interactions needed to span the grid. They demonstrate the utility of this approach for handling ambiguous inverse mappings. The authors conduct experiments on a bidirectional MNIST task, structured inversion on Edge2Icon, appearance variation on Edges2Handbags, and ambiguous inverse mappings using MNIST. Their results show performance comparable to or better than the i-RevNet, RealNVP, and Glow baselines, along with a roughly 50-fold reduction in peak memory usage relative to ordinary autograd through the use of reversible backpropagation.

Critical strengths
  1. The ablations are extensive and insightful: The study of neighbourhood size and parameter sharing between F and G shows which design choices matter.
  2. The construction is simple and sensible: The additive coupling is an elegant way to achieve invertible neural cellular automata.
Critical weaknesses
  1. The introduction and literature review do not adequately capture the fields they discuss.
  • The introduction states that forward mappings are easy to learn “following the Universal Approximation Theorem.” However, the UAT is an existence result and says nothing about learnability.
  • A central concern is that the introduction presents inverse problems without adequately acknowledging the extensive literature on them in the context of neural networks. Approaches can be broadly divided into those that improve inference beyond a forward pass and those that change the learning problem, for example, by modifying the neural network architecture and, in some cases, the associated loss function.
  • There is also much more to Bayesian deep learning than Bayesian neural networks (BNNs), so the discussion of Bayesian methods is too reductive.
  • It is also worth discussing the rich history of invertible cellular automata, as reviewed in “Invertible cellular automata: A review” in Physica D: Nonlinear Phenomena. The novelty of this work needs to be positioned against this history.
  1. The comparison methods used in the experiments and discussed in the related work appear to date from around 2019 or earlier. It would be helpful to consider more recent benchmarks for invertible architectures, such as this work. Invertible diffusion models, such as EDICT, are also worth benchmarking against.

  2. Novelty concerns of the core innovation

  • It would also be worth noting a similar addictive coupling trick is also found in the NICE paper but for generative modelling. The paper cites Laurent Dinh’s later work, but NICE introduced an invertible mapping with an additive coupling mechanism and should be cited at least.
Questions for authors

A Diff-NCA paired with a reversible sampler, such as the one proposed in “Rex: A Family of Reversible Exponential (Stochastic) Runge-Kutta Solvers” from this year’s ICML, would also yield an exactly invertible, local, weight-tied dynamics with genuine stochastic generation. This would be a natural baseline for your work, as it would effectively constitute another invertible neural cellular automaton, but one based on diffusion.

Reviewer_CJDsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Invertible Neural Cellular Automata (iNCA), a cellular automaton architecture that enforces exact bijectivity on the full cell-state space by splitting cell channels into two groups and applying alternating additive coupling updates parameterized by shared local perception and update convolutions across rollout steps. Full-state invertibility allows intermediate activations to be reconstructed in reverse during training, enabling memory-efficient reversible backpropagation. To handle partial observations, where only visible image channels are observed and hidden workspace channels are filled with constants, and to model ambiguous one-to-many inverse mappings, the model is trained with a supervised bidirectional reconstruction objective together with a distributional matching loss over designated latent channels. The method is evaluated on bidirectional digit-seed recovery on MNIST, edge-to-icon translation on Edge2Icon, rotation-ambiguous digit inversion, and multimodal edge-to-handbag synthesis.

Critical strengths

Embedding additive coupling into weight-tied local cellular updates yields an exact full-state bijection with a very compact parameter footprint across multi-step rollouts. Exploiting this invertibility for reversible backpropagation addresses a major practical bottleneck of training deep cellular automata on spatial grids, substantially reducing peak GPU memory compared to both standard backpropagation and gradient checkpointing with only a modest runtime overhead.

The authors also establish a clear theoretical boundary between full-state invertibility and partial-observation inversion, explicitly acknowledging that projecting onto visible channels breaks injectivity. Their experimental separation between real-inverse recovery and projected-round-trip recovery offers a useful diagnostic for studying how information survives the observation bottleneck.

Critical weaknesses

In all reported tasks, observations specify only a few visible channels while the majority of hidden workspace channels are discarded at inference and overwritten with zeros before running the inverse rollout. Consequently, seed recovery from observations is not guaranteed by algebraic invertibility, but is instead learned through explicit bidirectional supervision and cycle-consistency losses. However, the paper compares iNCA only against untied feedforward invertible networks (i-RevNet, RealNVP, Glow) and never compares against a standard, non-invertible NCA (or the direction-conditioned NCA mentioned in the appendix) trained with the exact same bidirectional objective. Without this baseline, it is unclear whether additive coupling actually improves inverse recovery or whether its benefit is limited to reversible backpropagation.

Furthermore, the claims for inverse modeling on complex and high-resolution images are not well supported by the natural-image experiments. On Edge2Handbag, no external baselines are compared, and the appendix diagnostics reveal that while projected roundtrip edge recovery from generated handbags is nearly perfect, real-inverse edge recovery from actual test handbags drops sharply, suggesting that the forward pass embeds subtle steganographic cues into the visible channels of generated images that do not generalize to real photographs. In addition, the RealNVP and Glow baselines on MNIST and Edge2Icon are stripped of their standard multiscale squeezing layers, forcing them to stack flat local convolutions at full resolution and artificially limiting their receptive field.

Questions for authors
  1. How does a standard non-coupling NCA (or the direction-conditioned NCA mentioned in your style-manipulation appendix) with a matched parameter budget perform on Bidirectional MNIST, Edge2Icon, and Ambiguous MNIST when trained with the exact same bidirectional reconstruction objective?

  2. Why does real-inverse edge recovery on Edge2Handbag degrade so substantially compared to projected round-trip recovery? Does adding small noise or quantization to the visible channels during the round-trip cycle prevent steganographic shortcutting and improve real-image inversion, and how do RealNVP, Glow, and i-RevNet perform under the same Edge2Handbag setup?

  3. If standard multiscale spatial squeezing operations are retained in the RealNVP and Glow baselines at a matched parameter budget, how does their inverse accuracy on MNIST and Edge2Icon change?

Reviewer_3sZTOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel invertible neural network design referred to as invertible neural cellular automata. The proposed approach splits the continuous cell state into two groups of channels, and each group is updated from a local neighbourhood of the other group using additive coupling layers. It shares the weights across all rollout steps, so that way each step has a closed form inverse over the full state. The model is trained with supervised losses in both the directions. Authors also use exact inverse for reversible backpropagation to lower the training memory. The experimental results are shown on small image sets (e.g. MNIST, Edge2Handbag).

Critical strengths
  • The problem of making local, self-organizing models invertible is interesting and relevant.
  • The core idea of combining weight-shared NCA rules with additive coupling is simple and easy to follow. It is described clearly through Equations and a figure.
  • The models are parameter-efficient (ranging in tens to a few hundred thousand parameters) as the weights are shared across the steps.
  • The memory result is strong: reversible backpropagation trick cuts peak memory significantly, with only a small overhead in the runtime cost.
Critical weaknesses

[1] Several claims made in the abstract and introduction are not supported by the results: The abstract claims a "significant improvement" over prior approaches, but on MNIST, proposed method scores within the standard deviation range of Glow (95.1 ±\pm 2.5 vs. 92.6 ±\pm 2.0 for Glow), and RealNVP (94.86 ±\pm 4.02) is extremely close to the proposed method. On Edge2Icon, RealNVP has the best real-inverse IoU while iNCA leads only on the "projected round trip" metric (which is also defined by the authors in this paper).

[2] I find the baseline comparisons inadequate. RealNVP and Glow are restricted to 3x3 kernels and very narrow hidden widths (w = 5 or 6) to match iNCA's parameter count across the untied blocks. On the other hand, iNCA uses 5x5 perception operators and a wide update network. It is unclear whether the gains come from the cellular coupling design or from the larger receptive field and wider layers. On MNIST, Glow also lacks actnorm and invertible 1x1 convolutions which are its key components. Authors should report 5x5-kernel, weight-tied baselines in the main tables.

[3] The ambiguous-inverse experiments are weak. The 6/9 MNIST task has only four classes and does not provide comparisons against any external baseline.

[4] Edge2Handbag shows a large domain gap for real vs. projected inputs: IoU is only 0.5443 for Real-inverse vs. 0.9986 on projected. It seems that the model recovers edges far better from its own outputs than from real images, so it may be infusing a “hidden” signal in the generated outputs. Also, comparisons with other image-to-image methods such as BicycleGAN or other image-to-image approaches are not provided (ref: Toward Multimodal Image-to-Image Translation, Jun-Yan Zhu et al.).

[5] The high-resolution scaling is presented as a core contribution, but it is evaluated only qualitatively. No quantitative metrics or compute / runtime numbers are provided for the 280×280 experiments.

[6] Forward MNIST accuracy is 100% for every model, so it seems that the tasks are easy or saturated and unable to distinguish the performance gaps between different methods. On the other hand the MNIST inverse accuracy (~95%) is well below a standard classifier (>99%).

[7] While I am not well-versed with the literature in this area, based on my preliminary reading, the novelty seems incremental. The main component of additive coupling with exact inverses is explored in RealNVP and iRevNet. The main novelty seems to be applying one coupling block repeatedly with local convolutions, which is close to a weight-tied iRevNet.

Questions for authors

N/A

Comments on the paper

1 comment
AuthorsOpenReview ↗

Withdrawal

We thank the reviewers for their valuable feedback. After careful consideration, we conclude from the reviews that the reviewers see potential in our proposed method and basically approve its theoretical derivation, but argue that a a broader discussion and comparison with related work is needed. We agree with this evaluation and will revise our work accordingly for re-submission at a different venue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.