acceptodds
Under review as a conference paper at ICLR 2027

MetaLearnNCA: Few-Shot Offline Meta-Learning via Interacting Neural Cellular Automata

Abstract

Few-shot meta-learning traditionally formulates task adaptation either as analytical gradient descent through unrolled computational graphs or as metric-based distance comparisons over flattened 1D feature vectors, which either incur costly test-time backpropagation or discard native 2D spatial geometry. In this work, we propose MetaLearnNCA, a decentralized framework that achieves few-shot adaptation through the dynamical interaction of coupled Neural Cellular Automata (NCAs) without computing analytical gradients during inference. MetaLearnNCA decomposes task adaptation into an Active-NCA, which executes task inference conditioned on a continuous 2D spatial memory grid termed the spatial program, and a learned Meta-NCA, which acts as a decentralized cellular optimizer by diffusing spatial error residuals across local neighborhoods to dynamically update this program. MetaLearnNCA is competitive against canonical meta-learners in-distribution ( on Omniglot) with Out-Of-Distribution transfer gains on MNIST, KMNIST, and Fashion-MNIST transfer across 10 independent testing seeds across 1-, 5-, and 10-shot regimes (e.g., surpassing Prototypical Networks by on 10-shot MNIST and a gain on 10-shot Fashion-MNIST over FOMAML). Our results establish that robust, gradient-free learning-to-learn can emerge from decentralized cellular dynamics on non-von Neumann substrates.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Meta Learning, Neural Cellular Automata, NCA, Representation, Dynamic System, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Vpd2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MetaLearnNCA, a few-shot meta-learning framework based on two interacting Neural Cellular Automata. The Active-NCA performs prediction, while the Meta-NCA uses pixel-level residual errors from the support set to iteratively update a 2D spatial program. After adaptation, the program is frozen and used for query inference without explicit test-time backpropagation. The model is meta-trained only on Omniglot and evaluated on several OOD datasets, including MNIST, USPS, KMNIST, Fashion-MNIST, and MedMNIST. It outperforms several canonical meta-learning baselines on multiple target domains while keeping the model size around 118k parameters.

Critical strengths
  1. The method is novel. The Meta-NCA modifies a 2D spatial program that steers the Active-NCA. This is clearly different from both MAML-style adaptation and prototype-based meta-learning.

  2. The spatial task representation is an interesting design choice. Task-specific information is encoded in a Cs×H×WC_s \times H \times W spatial program. This preserves spatial structure and matches the local interaction bias of NCAs.

  3. OOD transfer is competitive on several datasets. MetaLearnNCA shows clear gains on MNIST, KMNIST, and Fashion-MNIST across different shot regimes, with particularly strong improvements in some 10-shot settings.

  4. The paper provides a relatively careful robustness analysis across random seeds. The adaptation trajectories are reported over 42 independent meta-training runs and 42 testing seeds, which gives more confidence that the observed dynamics are not due to a single favorable run.

Critical weaknesses
  1. Key components are not sufficiently ablated. The current experiments do not clearly isolate the contributions of the cellular updater, spatial program, foreground masking, and long recurrent unrolling. The paper explicitly acknowledges this issue.

  2. There is some concern about baseline fairness. ProtoNet, MatchingNet, RelationNet, and FOMAML all use a standard Conv4 backbone, while MetaLearnNCA uses a very different recurrent cellular architecture. It is therefore difficult to separate the effect of the proposed adaptation mechanism from the inductive bias of the NCA architecture itself.

  3. Gradient-free does not necessarily imply computational efficiency. Test-time backpropagation is removed, but both NCAs require many recurrent updates. The paper does not report quantative results to prove the computational efficiency.

  4. Some mechanistic explanations remain speculative. For example, the proposed interpretation of class-specific steering fields occupying quasi-orthogonal subspaces is explicitly described as hypothetical and is not empirically verified.

  5. The authors are suggested to improve the quality of figures, especially Figure 1.

Questions for authors
  1. Although the method avoids test-time gradients, it requires many recurrent NCA updates. How does its actual wall-clock latency, FLOPs, and memory cost compare with FOMAML and ProtoNet?

  2. MetaLearnNCA underperforms FOMAML on MedMNIST and some USPS settings. What explains these failure cases? Are they related to foreground masking, spatial centering, or the structural similarity between Omniglot and the target domains?

  3. It is unclear whether the gains mainly come from the Meta-NCA, the spatial program, foreground masking, or recurrent unrolling. Could the authors provide component-wise ablations to isolate their individual contributions?

2 comments

In accordance with your review, we have added component ablations, NCA backbone for ProtoNet, and test-time compute cost. We have also removed any speculative paragraphs and improved Figure 1. As per your question about underperformance on certain datasets, this comes down to a mix of structural similarity between training and testing, as can be seen in the table in Appendix J, and spatial centring. As MedMNIST and USPS take up the whole image, feature placements can vary much more than in smaller, more centred datasets; this is also partially supported by the results in Appendix L.

Thank you again for your detailed review, particularly your recognition of the architectural novelty and your suggestion to separate the adaptation mechanism from the NCA backbone. Following our revision update, we would like to summarise what the new experiments establish.

The revised paper includes retrained component ablations and NCA-ProtoNet as a simpler alternative. NCA-ProtoNet achieves similar average OOD accuracy at 5 and 10 shots, with no demonstrated aggregate advantage for the full model. This indicates that the backbone accounts for a substantial part of the transfer performance. The full model improves the 1-shot OOD mean by 2.2 percentage points (reported 95% CI: ±1.4), and improves 10-shot Fashion-MNIST and KMNIST by 8.6 and 4.4 points, while performing substantially worse on MedMNIST.

The remaining ablations show substantial performance reductions when spatial conditioning, masking, residual feedback, or Meta-NCA neighbourhood communication are altered within the tested architecture.

Table 3 now reports latency, FLOPs, and memory. The overhead is substantial. For example, approximately 66× FOMAML’s adaptation FLOPs on a 10-way 10-shot task. Therefore, we make no efficiency claim. We have also removed the speculative explanation based on quasi-orthogonal class-specific steering fields and revised Figure 1.

Regarding the failure cases, spatial sensitivity and domain structure are plausible contributing factors, but we do not regard them as established causal explanations. The NCA-ProtoNet comparison indicates that limitations of the adaptation mechanism also require further investigation.

We consequently frame the contribution as a proof of concept for biologically inspired, local recurrent adaptation of spatial task memory, with explicitly measured benefits and limitations. The backbone comparison also opens a complementary direction for exploring NCAs within other meta-learning methods.

We would welcome your updated assessment of whether this evidence addresses the concerns raised in your review.

Reviewer_zcjtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a novel few-shot meta-learning framework called MetaLearnNCA by building two interacting Neural Cellular Automata (NCAs). MetaLearnNCA decomposes task adaptation into an Active-NCA, which executes task inference conditioned on a continuous 2D spatial memory grid termed the spatial program, and a learned Meta-NCA, which acts as a decentralized cellular optimizer by diffusing spatial error residuals across local neighborhoods to dynamically update this program. Experiments on the in-distribution Omniglot dataset and several out-of-distribution datasets, including MNIST, KMNIST, and Fashion-MNIST, demonstrate competitive performance on some benchmarks and improvements over the evaluated baselines.

Critical strengths
  1. The paper is generally well written and easy to follow.

  2. The related work is well organized and provides useful background on few-shot meta-learning.

  3. The proposed method enables task adaptation without test-time backpropagation.

Critical weaknesses
  1. Although the specific coupling of two NCAs offers some novelty, the overall methodological contribution appears incremental, which is its biggest limitation. The framework combines established ideas from existing studies. Stronger comparisons with simpler alternative adaptation mechanisms are needed to establish the added value of this design.

  2. While the technical contribution is small, the experimental scope is limited and does not sufficiently establish broad applicability to few-shot learning. Specifically, this method applies only to simple MNIST-class datasets and has little potential to extend to other datasets.

  3. Meta NCA generates learning-based state updates without guaranteed convergence. When the model encounters stronger distribution shifts, it is likely to fail.

Questions for authors
  1. What benefits does the dual-NCA design offer over simpler alternatives?

  2. Does MetaLearnNCA generalize to natural image benchmarks such as cifar100 and miniImageNet?

  3. Are the updates stable under stronger distribution shifts?

3 comments

In accordance with your review, we have added an NCA backbone for Protonet as a simpler alternative and component ablations. We have also added results for CIFAR-100.

Thank you for the author's response. I will increase my score.

Thank you for reconsidering the paper and updating your recommendation. Your suggestions motivated the simpler NCA-ProtoNet comparison and the natural-image experiments, which helped us distinguish the contribution of the cellular adaptation mechanism from that of the NCA backbone and clarify the method’s benefits and limitations. We appreciate your constructive engagement.

Reviewer_fXVDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates meta-learning without computing gradients during inference. It proposes MetaLearnNCA, which uses two interacting Neural Cellular Automata (NCAs) to adapt a spatial program based on support-set errors. It achieves competitive results against canonical meta-learning methods, both in-distribution and OOD. Benchmarks include Omniglot, MNIST, USPS, KMNIST, Fashion-MNIST, and MedMNIST.

Critical strengths
  1. This work investigates a very important question at the heart of modern AI: meta-learning without gradient descent during test-time adaptation.
  2. The idea of using interacting NCAs to perform adaptation is interesting. Results on several OOD benchmarks are promising, especially considering the small number of parameters.
Critical weaknesses
  1. The paper is extremely hard to follow without prior knowledge of NCAs. An accessible introduction (needed in the Preliminaries) to NCAs is missing, and letters and notation are often used without prior introduction.
  2. Most cited papers date to around 2020. The literature should discuss more recent work to better contextualise the contribution. For instance, very recent related in-context learning gradient-free frameworks such as WARP [1]. Overall, this limited literature makes it difficult to assess the novelty of the contribution.
  3. The robustness claims (L28) are not fully supported by the results. In Table 1 for instance, MetaLearnNCA performs substantially worse than FOMAML on MedMNIST (19.76% versus 35.91% at 5-shot), and also underperforms on USPS at 5- and 10-shot.
  4. Furthermore, in terms of experiments, ablation studies are insufficient. The experiments do not isolate the benefits of the cellular updater, spatial program, and foreground masking. It is therefore unclear which components explain the reported gains.

Minor weaknesses: no effect on rating

  • L20: The gradient-free framing needs clarification. NCAs still require differentiable components during meta-training, which limits the scope of this claim.
  • L67: Missing full stop.
  • L249–255: Hypotheses about the learned representations are discussed in the method section, obscuring the presentation. Such claims are also not empirically tested.
  • L319: The USPS results may be wrongly reported? FOMAML outperforms MetaLearnNCA at 5- and 10-shot, not 1- and 5-shot.

References:

[1] Nzoyem et al. "Weight-Space Linear Recurrent Neural Networks", ICLR 2026

Questions for authors
  1. What is the test-time computational cost compared with FOMAML? Although gradients are avoided, the method still requires multiple NCA iterations. Can you provide wall-clock and memory comparisons?
  2. Can you provide ablations isolating the contributions of the Meta-NCA, spatial program, and foreground masking? This would help establish which components are responsible for the improvements.
  3. How do the authors explain the substantially worse results on MedMNIST and the mixed results on USPS, given the claims about OOD generalisation?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Paper Revision

We have uploaded the revised paper. We note a few major changes:

  1. An error in training the canonical models was found (they got to see about 2.5X the examples of MetaLearnNCA); the training regime was different from MetaLearnNCA; they have been retrained under the same regime and tested under the same regime

  2. MatchingNet was not properly set up, which explains the poor performance; the cosine temperature was set to 1 instead of the literature-recommended 100. We have changed this and added the new improved scores.

  3. Ablation Studies, the natural image dataset, and test-time compute cost have been added in both the main text and appendix.

  4. Abstract and all related text to the changes have also been adapted.

AuthorsOpenReview ↗

Final comment to all

We thank the reviewers for their constructive feedback, which has substantially strengthened the paper. The revision adds retrained component ablations, an NCA-ProtoNet comparison, test-time computational measurements, natural-image experiments on FC100, clearer NCA preliminaries, and revised figures and claims.

We would like to clarify the conceptual contribution and what the new evidence establishes. MetaLearnNCA is a proof of concept for few-shot adaptation through interacting local, recurrent cellular systems. Inspired by the distributed dynamics underlying biological morphogenesis, the Meta-NCA converts support-set residuals into updates of a persistent spatial program that conditions the Active-NCA. Network weights remain fixed during adaptation, which proceeds without test-time backpropagation, using local cellular updates and explicit support-set aggregation. Both networks are trained offline by gradient descent.

The new experiments identify both benefits and limitations. NCA-ProtoNet achieves similar average OOD accuracy at 5 and 10 shots, with no demonstrated aggregate advantage for the full model. This indicates that the NCA backbone accounts for a substantial part of the transfer performance. The learned adaptation nevertheless provides measurable benefits: the full model improves the 1-shot OOD mean by 2.2 percentage points (reported 95% CI: ±1.4), and improves 10-shot Fashion-MNIST and KMNIST by 8.6 and 4.4 points over NCA-ProtoNet. It performs substantially worse on MedMNIST. The remaining ablations show substantial performance reductions when spatial conditioning, masking, residual feedback, or Meta-NCA neighbourhood communication are altered within the tested architecture.

We therefore claim a viable, domain-dependent adaptation mechanism, rather than general performance superiority. We make no computational-efficiency claim: the recurrent dynamics are substantially more expensive than the canonical baselines. In a separate experiment trained on FC100, an image-conditioned variant learns on natural images but remains below ProtoNet, MatchingNet, and FOMAML.

The NCA-ProtoNet result also suggests a complementary research direction: investigating NCAs as feature-extraction backbones for other meta-learning methods, independently of our cellular optimizer. Understanding which reusable spatial features these backbones learn remains an open question.

We hope the paper’s value will be assessed as an architectural demonstration with explicitly measured benefits and limitations, motivating further investigation of biologically inspired meta-learning and adaptation on non-von Neumann computing substrates.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.