acceptodds
Under review as a conference paper at ICLR 2027

Reasoning with Neural Cellular Automata

Abstract

Modern AI architectures used to solve visual reasoning tasks typically rely heavily on global connectivity and synchronization. As biological systems demonstrate, though, sophisticated computation can be performed in a more decentralized fashion. In this work, we test the reasoning capabilities of Neural Cellular Automata (NCAs), networks of recurrent cells that use strictly local connectivity and asynchronous updates. NCAs have been extensively studied in artificial life experiments, but it is unclear whether they can perform complex multi-step reasoning. We show that NCAs produce spatio-temporal dynamics capable of solving challenging visual reasoning tasks, including large mazes, Sudoku, and ARC-AGI-1. Furthermore, we provide evidence that NCAs generalize out-of-distribution when running with larger grids, longer rollouts, or parallel trials; and that the latter can be made more efficient via pruning of redundant trajectories. We find that these generalization capabilities depend on training with sample replay and stochastic perturbations, and that stochasticity remains beneficial at test time. Finally, we show that NCAs are robust reasoners capable of dynamically modulating compute to recover efficiently from damage, and that they can scale to solve reasoning in raw pixel space.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Neural Cellular Automata, Self-Organizing Systems, Visual Reasoning

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_urSmOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether neural cellular automata with strictly local neighborhood communication and recurrent stochastic updates can solve spatial reasoning tasks. It evaluates sudoku, maze solving, ARC-AGI-1 and studies how performance changes with longer rollouts, larger grids, and multiple stochastic solution attempts. The results provide evidence that local update rules can support reasoning beyond immediate spatial neighborhoods. In particular, a model trained on 9-by-9 mazes achieves 100% accuracy on the reported 201-by-201 Maze-OOD test set.

Critical strengths

The paper demonstrates successful local recurrent computation across several tasks, including maze solving and Sudoku. Generalization from 9-by-9 to 201-by-201 mazes is really impressive! The experiments explore spatial generalization, rollout depth, and parallel stochastic trials.

Critical weaknesses
  1. The main limitation is that training-run variability is not clearly reported for Table 1. For example, it is unclear whether the Maze-Hard accuracies come from one trained checkpoint per configuration or are averages across independently trained models. The table does not report a corresponding standard deviation or the number of training runs, and I sadly could not find an explicit checkpoint-selection rule for these results. I consider this a reporting and reproducibility concern that can be clarified, rather than a reason on its own to reject the paper.

The related-work discussion does not include "2D Spatial Reasoning with Adaptive Neural Cellular Automata" (accepted @ NeurIPS 2026), which also studies Sudoku and maze solving. Discussing its use of learned adaptive neighborhoods in relation to the strictly local communication studied here would help contextualize the contribution. As this is recent work, my suggestion is intended to improve the discussion.

Questions for authors

Q1. Are the results in Table 1 based on a single checkpoint or averaged across independent training runs? Please clarify checkpoint selection and report variability across training seeds separately from test seeds.

Q2. Why use all-step supervision for Maze and Sudoku-OOD but last-step supervision for Sudoku-Extreme? Have you compared both schemes on the same benchmark?

Q3. Have you explored larger neighborhood variants? It would be interesting to know whether widening the locality improves accuracy or reduces the required number of rollout steps.

5 comments
AuthorsOpenReview ↗

General Response to Reviewer urSm

We thank the reviewer for their positive assessment and highly constructive feedback. We attempt to address each point below. The reviewer's feedback has enabled us to improve the manuscript with additional results and discussion. We believe these changes improve both the reproducibility of our results and the positioning of the paper with respect to related work.

AuthorsOpenReview ↗

Specific Response 1 to Reviewer urSm

Item 1. Training-run variability and checkpoints

Weakness #1: The main limitation is that training-run variability is not clearly reported for Table 1. For example, it is unclear whether the Maze-Hard accuracies come from one trained checkpoint per configuration or are averages across independently trained models. The table does not report a corresponding standard deviation or the number of training runs, and I sadly could not find an explicit checkpoint-selection rule for these results. I consider this a reporting and reproducibility concern that can be clarified, rather than a reason on its own to reject the paper.

Question #1: Are the results in Table 1 based on a single checkpoint or averaged across independent training runs? Please clarify checkpoint selection and report variability across training seeds separately from test seeds.

This is an important point, and we fully agree that being clear on the checkpoint selection and reported variability are critical for reproducibility.

First, we do not use early stopping or best-checkpoint selection. Every run uses the final checkpoint at the end of training, with Exponential Moving Average (EMA) weights. We now state this explicitly in Appendix C (Experimental Settings).

Second, In the original submission, Table 1 reported the best of 3 independently trained runs, which is also the checkpoint released with our codebase. This was done to limit the cost of the full test-time scaling evaluation, and is in line with baselines such as TRM and LoopViT, which report single-checkpoint results.

We agree, however, that variance across independent training runs gives a more complete and transparent picture. We have now run the full test-time scaling evaluation for all three training seeds, and Table 1 reports the mean ± std across training runs, alongside the best-seed result for comparison with single-checkpoint baselines. The only exception is Visual Sudoku, which we report for a single training run because of its high training cost (see Appendix C.4), and because this experiment is intended as a proof of concept that NCAs can scale to large pixel-space reasoning tasks, rather than as a benchmark comparison.

AuthorsOpenReview ↗

Specific Response 2 to Reviewer urSm

Item 2. Relevant related work

Weakness #2: The related-work discussion does not include "2D Spatial Reasoning with Adaptive Neural Cellular Automata" (accepted @ NeurIPS 2026), which also studies Sudoku and maze solving. Discussing its use of learned adaptive neighborhoods in relation to the strictly local communication studied here would help contextualize the contribution. As this is recent work, my suggestion is intended to improve the discussion.

We thank the reviewer for pointing us to this highly relevant work by Spitznagel and Keuper, which we were unaware of at the time of submission (since it was made public on arXiv after the ICLR submission deadline). We have added it to the Related Work section and mention the use of adaptive neighborhoods as a potential direction for future work in the revised manuscript. Here, we will also now comment on the differences and complementarity of our contribution and theirs.

With respect to the differences between NCA and adaptive NCA (aNCA), aNCA's connectivity is more dense, (each cell perceives up to N=27N=27 cells, across the three heads), and the cell's connections are data-dependent and can be rewired at every step to any other cell in the grid, with no spatial constraint. In contrast, we impose strict, fixed, locality from the ground up in order to ask how far local-only computation can take us. One of the motivations is that answers under these exact constraints could help inform the design and research of fully decentralized, low-power computing substrates.

With respect to the complementarity of the contributions, Spitznagel and Keuper show that more effective connectivity patterns can be learned directly from data, adapting to each task and even to each cell, without knowing in advance which connections will be useful. We agree that different tasks should benefit from different inductive biases and neighborhood topologies. Intuitively, a task like Sudoku should benefit greatly from full visibility over rows, columns and blocks, which a 3×3 neighborhood does not directly provide. Automatically discovering such inductive biases could thus lead to more efficient training and potentially better generalization. Overall, we see a middle ground between our work and theirs as a promising direction for future research: architectures that remain mostly local to preserve low-power, decentralized communication, but add a few (possibly adaptive) long-range connections that provide the right inductive biases for the task at hand. We now mention this possibility in the Discussion section.

Reviewer_WiRyOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Neural cellular automata, where each cell on a grid updates its state based on neighboring cells using a shared neural network, are well studied in artificial life, but their capabilities on complex reasoning tasks remain less explored. The paper shows that a locally connected network can solve global combinatorial problems like Sudoku and ARC-AGI-1. Specifically, the network only sees the adjacent cells in a 3×3 neighborhood and iteratively updates the state vectors. The paper also shows that the model generalizes well in OOD settings with test-time scaling (Maze-OOD, Sudoku-OOD) and uses fewer parameters than models that use global attention.

Critical strengths
  • The paper tests the NCA idea on a variety of reasoning tasks.
  • The paper compares the model with strong existing methods, and the proposed local network achieves comparable or better performance in some settings.
  • The paper provides extensive analysis. I find Section 4.4 particularly interesting, as it examines adaptive compute based on local uncertainty and recovery from damage. D.6 and D.7 also show that techniques and properties from prior NCA research remain useful for reasoning tasks.
Critical weaknesses
  • I don’t see any particularly critical weaknesses in the paper. But it would be interesting to see the performance on more difficult tasks (such as language modeling or sequential prediction).
Questions for authors

I don't have any particular questions that affect my assessment.

1 comment
AuthorsOpenReview ↗

Response to Reviewer WiRy

General Response

We thank the reviewer for the very positive assessment and for highlighting the analysis of adaptive compute and damage recovery (Section 4.4). These properties can now also be explored interactively in the web demos of our accompanying website (https://reasoning-nca.github.io/). We also thank the reviewer for pointing out Appendices D.6 and D.7, which we also consider as important results of the paper. Given the extended page limit, we have moved these results back into the main text (resulting in a new Figure 5 and extended Figure 6). We respond to your specific comment below.

Specific Responses

Weakness #1: It would be interesting to see the performance on more difficult tasks (such as language modeling or sequential prediction).

We agree that extending NCAs beyond grid-structured reasoning tasks to applications of broader practical relevance, such as language modeling, sequential prediction and image generation, remains an important open question for future work. Unfortunately, we did not have sufficient time within the review rebuttal period to modify the architecture significantly (which such tasks would require) and collect the data. Nonetheless, we are working on these other non-grid-structured tasks (as are other groups, we suspect), and we imagine that the field will have demonstrations of NCAs operating well for such tasks in the near future.

Reviewer_iyL9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether neural cellular automata with local connectivity and stochastic updates can solve visual reasoning tasks. It evaluates mazes, Sudoku, ARC-AGI-1, and pixel-space Sudoku, examining generalization through larger grids, longer rollouts, and parallel trials. It also introduces diversity-based trajectory pruning and confidence-based local update rates to improve computational efficiency and recovery from damage.

Critical strengths
  1. The maze generalization is compelling. A small NCA trained on 9×9 mazes solves much larger 201×201 instances without changing its weights. This provides evidence that scalable computation can emerge under local communication constraints.

  2. The study extends beyond pathfinding to global constraint satisfaction and joint perception–reasoning. The Sudoku and Visual Sudoku results strengthen the central capability claim, while replay, perturbation, and update-rule ablations help explain which training ingredients support generalization. The damage-recovery experiments add a meaningful demonstration of robustness.

Critical weaknesses
  1. The pruning experiments do not isolate the benefit of the proposed diversity criterion. Comparisons mainly use unpruned rollouts, leaving out random halving, confidence-only halving, and smaller unpruned ensembles with longer trajectories at matched compute. The reported gains could therefore largely reflect reallocating computation from breadth to depth. Establishing the added value of niche-based selection is important for assessing this methodological contribution.

  2. The ARC results leave the source of cross-task generalization unclear. Each evaluation task receives 10,000 optimization steps that update both task embeddings and all model weights. This is a valid adaptation protocol, but the results do not separate reusable reasoning in the shared NCA from task-specific rules learned during adaptation. Consequently, they provide stronger evidence for local execution after adaptation than for a shared local rule-induction mechanism.

Questions for authors
  1. Does niche-capped pruning outperform random and confidence-only halving under the same schedule, and the best unpruned breadth–depth allocation at equal compute?

  2. On ARC, how does updating only task embeddings with frozen NCA weights compare with full test-time training? This would clarify what transfers across tasks.

  3. What is the total per-task ARC compute, including test-time training and candidate aggregation? How does including these costs change the efficiency comparison?

5 comments
AuthorsOpenReview ↗

General response to Reviewer iyL9

General Response

We thank Reviewer iyL9 for really engaging with our paper and asking very insightful questions. As we outline below, the Reviewer’s comments have led us to run several new analyses, one of which has, in fact, altered our initial conclusions. This was a very productive experience for better understanding our models, and we believe the resulting manuscript is much stronger.

AuthorsOpenReview ↗

Specific Response 1 to Reviewer iyL9

Item 1. Pruning experiments

Weakness #1: The pruning experiments do not isolate the benefit of the proposed diversity criterion. Comparisons mainly use unpruned rollouts, leaving out random halving, confidence-only halving, and smaller unpruned ensembles with longer trajectories at matched compute. The reported gains could therefore largely reflect reallocating computation from breadth to depth. Establishing the added value of niche-based selection is important for assessing this methodological contribution.

Question #1: Does niche-capped pruning outperform random and confidence-only halving under the same schedule, and the best unpruned breadth–depth allocation at equal compute?

We thank the reviewer for this comment as we believe it highlighted important missing baselines. After running the proposed baselines, as the reviewer correctly intuited, our initial pruning results did in fact largely reflect reallocating computation from breadth to depth. We initially demonstrated the pareto curves of an unpruned ensemble of K=512K=512 rollouts alongside the pareto curves of the same ensemble progressively being pruned for a variety of halving iterations up to m=8m=8. We have since run an additional baseline of K=128K=128 parallel rollouts which is compute-matched with the K=512K=512 ensemble pruned with m=8m=8 (Figure 22A). We find that choosing to begin with a lower ensemble yielded a greater accuracy on the test-set than going wider and pruning. We have therefore moved the discussion around pruning into the appendix (Appendix D.6) and present it as an alternative way to shift compute from breadth to depth while highlighting that choosing the compute-optimal unpruned allocation remains the superior method. Real world hardware naturally presents a trade-off between breadth first (more compute units) and depth first (more time, or faster compute units). Each has its own advantages, costs, and scaling limits, so understanding how models of this flavour influence that tradeoff is useful.

In place of the pruning pareto curve figure and corresponding discussion in the main text, we instead discuss the breadth-depth tradeoff, which we believe significantly strengthens the discussion surrounding compute allocation (thanks to the reviewer). We find that there are two distinct regimes. Under a certain depth (D=512D=512 in the case of Sudoku-Extreme) the NCA is depth-limited (i.e., doubling depth is worth more than doubling breadth). We find this to be consistent with the locality bottleneck, which requires information to propagate across the board over many iterations. Beyond this threshold, the returns cross over and breadth becomes the better use of additional compute (Figure 4). We also report the full grid search in Figure 21.

Finally, we tested our results concerning niche-capped pruning against confidence-based pruning. While we find that niche-capped pruning does in fact lead to a slight increase in accuracy, it is marginal to the point of insignificance. Therefore, in the spirit of simplicity, we have replaced niche-capped pruning results with confidence-based pruning – the simpler alternative that yields nearly exactly the same performance. We do, however, provide results demonstrating that confidence-based pruning is superior to random halving at nearly every depth and its performance gap over random halving rises with the frequency of pruning.

We would like to thank the reviewer again for raising this issue. We actually feel that the analysis of scaling breadth versus depth is a more informative piece of data than the original niche-capped pruning results, which were a relatively minor component of the original paper anyway. With these new analyses, our work now provides greater clarity on the impact of breadth versus depth compute allocation for NCA performance.

AuthorsOpenReview ↗

Specific Response 2 to Reviewer iyL9

Item 2. Comment on task transfer for ARC-AGI-1

Weakness #2: The ARC results leave the source of cross-task generalization unclear. Each evaluation task receives 10,000 optimization steps that update both task embeddings and all model weights. This is a valid adaptation protocol, but the results do not separate reusable reasoning in the shared NCA from task-specific rules learned during adaptation. Consequently, they provide stronger evidence for local execution after adaptation than for a shared local rule-induction mechanism.

Question #2: On ARC, how does updating only task embeddings with frozen NCA weights compare with full test-time training? This would clarify what transfers across tasks.

We thank the reviewer for this comment, it highlights an important consideration for our examination of task transfer in ARC-AGI-1. We agree that a better investigation of the source of the cross-task generalization is warranted and requires additional analysis.

To address this point we have run an ablation experiment in-line with the Reviewer’s recommendation. Specifically, we have run an analysis on task transfer for ARC-AGI-1 comparing three different test time training (TTT) regimes:

  • Scratch-TTT: initializing from scratch and training all parameters (task embeddings and shared weights).
  • Frozen-NCA: initializing from frozen pre-trained NCA weights and updating only the task embeddings.
  • Full-TTT (our initial approach): initializing from pretrained weights and fine-tuning both the task embeddings and shared weights. The results are shown in the table below (which is now also included as Table 10 in Appendix D.4). | Condition | Pretrained init | NCA weights | Adapted params / task | Pass@1 | Pass@2 | Pass@64 | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | Scratch-TTT | No | Trained | 3,460,652 (100%) | 12.7 | 14.5 | 21.8 | | Frozen-NCA | Yes | Frozen | 6,528 (0.2%) | 17.8 | 21.5 | 30.2 | | Full-TTT (paper) | Yes | Trained | 3,460,652 (100%) | 43.1 | 48.3 | 59.3 |

Summarizing the results here, we find that Frozen-NCA (i.e. updating only the task embeddings of the frozen pre-trained NCA, ≈0.2% of the parameters) already outperforms Scratch-TTT (training all parameters from scratch), showing that the pretrained model itself transfers to unseen tasks. Of course, Full-TTT (fine-tuning all weights of the NCA) further improves results, but only when starting from the pretrained model. This further suggests that the shared weights of the pre-trained NCA encode priors that can be reused during fine-tuning to improve performance on novel tasks.

We have added a paragraph about these results to the main manuscript and the table above to the Appendix. Importantly, these results help to confirm that the NCAs do develop reusable reasoning dynamics, though they also demonstrate that these reusable components can be refined to achieve better task specific performance.

Comments on the paper

4 comments
~Janis_Keuper1OpenReview ↗

Contemporaneous work

Nice work and I agree with the the positive reviews. Still I would like to direct the authors and reviewers attention towards our contemporaneous work "2D SPATIAL REASONING WITH ADAPTIVE NEURAL CELLULAR AUTOMATA" which has been accepted at NeurIPS 26

https://neurips.cc/virtual/2026/loc/paris/poster/154751

https://arxiv.org/pdf/2610.08518

We also evaluate on Sudoku and Maze benchmarks - very likely outperforming the presented results here (benchmarks are very similar but not identical).

While both works pursue spatial reasoning with NCAs, the actual methods are orthogonal approaches (which probably could be combined).

Being contemporaneous work, I don't see any problems with the acceptance here, however a discussion of our method would be fair ...

Thanks for commenting and for highlighting this contemporaneous adaptive NCA (aNCA) work, which became public after submission of our work. Also, congratulations on the NeurIPS acceptance! As per our response to Reviewer urSm, we've now included this work in our Related Works section and Discussion, and we highlight the differences and complementary aspects between them. We agree that a middle ground combining aspects of both approaches is a very exciting future research direction.

We would, however, like to gently push back on the claim that their models are "very likely outperforming" ours, at least within the scope of the benchmarks in our paper. The only benchmark the two papers share is Maze-Hard. There, on the same valid-shortest-path criterion, our 41K-parameter local NCA reaches 81.3% / 92.6% (A* / BFS targets) with a single rollout and 84.7% / 94.9% with confidence-based selection over 128 rollouts, compared with 94.2% for the 256K-parameter aNCA with a single rollout. So, adaptive perception gives an edge in single-rollout accuracy, but strictly local perception remains competitive. We hypothesize the 0.08% 3×3 local NCA baseline in the aNCA paper's Table 3 thus reflects differences in the training and inference versus our work.

The Sudoku settings differ substantially: aNCA uses randomly masked SRM puzzles, mostly admitting multiple valid completions (any of which counts as solved), while we focused on unique-solution, backtrack-heavy Sudoku-Extreme puzzles with only 1K training examples. Likewise for Visual Sudoku, aNCA operates on a 9×9 latent board behind a frozen encoder–decoder, whereas our Visual-Sudoku NCA reasons end-to-end, directly, on 256×256 pixels. As such, we don't think a ranking can be inferred here.

Nonetheless, as we noted to Reviewer urSm, we think that exploring some middle-ground between the two approaches is likely to be the most effective general solution: models where connectivity is by default local, but there exists some flexibility, such that longer-range connections can be employed when the task demands it.

AuthorsOpenReview ↗

General comment on reviews and rebuttal

We want to thank all three Reviewers for their extremely constructive comments on our paper. We found the reviews to be incredibly helpful, and the manuscript is much improved as a result. Briefly, we have made the following major changes:

  • To address Reviewer iyL9’s comment about our pruning results, we ran some additional experiments. Our results show that the reviewer was correct: the reported gains largely reflected the benefit of reallocating computation from breadth to depth. In fact, we have now done a more thorough analysis, and come to the conclusion that our pruning technique itself was not important, and what is more important is determining whether one is in a regime where using additional compute for breadth or depth would be more helpful. We have therefore taken out the pruning claims from the paper, and replaced them with this analysis. We note, though, that the pruning results were not a major component of our original manuscript, and were simply one part of our examination of scaling compute at test-time. We actually feel the new analysis fits even better in this section than the original results. Please see our comments to Reviewer iyL9 for more details.

  • To answer Reviewer iyL9’s question about the ARC-AGI-1 task transfer results, we have done careful ablation experiments per the Reviewer’s recommendation. Our results show that training only the task embeddings with a frozen network still shows improvements over from scratch training. This demonstrates that there is some real task transfer, and we now include these results in the Appendix. Again, please see our comments to the Reviewer for more details.

  • Per Reviewer urSm’s request we now report clearly the training-run variability. Specifically, in Table 1 we now report the mean ± std across training runs, alongside the best-seed result for comparison with single-checkpoint baselines.

  • In recognition of Reviewer urSm’s highly relevant contemporaneous citation, we now cite and discuss the paper by Spitznagel and Keuper in our related works section.

  • We have provided responses to all of the Reviewers’ specific questions as well. Please see our responses to the Reviewers for more details.

In summary, we believe we have addressed all of the Reviewers’ concerns, and we thank them again for raising them—they were very insightful. Our experience here has been precisely how peer review should be!

AuthorsOpenReview ↗

New interactive demo

We would like to draw the Area Chair’s and Reviewers’ attention to an additional resource we’ve developed for engaging with our paper and understanding our results.

Specifically, we have added a dedicated, anonymized, website with interactive demos to accompany the paper: https://reasoning-nca.github.io/. Readers can observe trained NCAs solve Maze-Hard, Sudoku-Extreme, ARC-AGI-1 and Visual Sudoku directly in the browser. While the model runs, they can modify the task inputs to test generalization to unseen inputs (move the source/target, draw new walls in mazes, or draw new inputs for ARC tasks), or perturb the dynamics (change the fire rate, add noise or damage) and see how the NCAs react. Dedicated demos also show out-of-distribution generalization (to harder Sudokus, larger mazes) and adaptive compute (updating only active cells). We believe these demos cast a spotlight on the paper's strengths by letting readers intuit NCAs' reasoning dynamics, robustness, and generalization for themselves.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.