acceptodds
Under review as a conference paper at ICLR 2027

Simulate, Don't Interpolate: Recovering Continuous Spatial Dynamics from Discrete, Unpaired Snapshots

13 views

Abstract

Spatiotemporal processes are central to many scientific questions, from how structure emerges in physical systems to how organisms form, or diseases develop. However, measuring such systems often relies on destructive experiments that preclude repeated time-series measurements, especially in biology and medicine. The alternative, unpaired snapshots, generally cannot fully determine the underlying dynamical processes. Recovering them requires constraining the solution space; implementing this at the level of the rules governing the system remains an open problem. We introduce REGIS (REcovering Generative dynamics from Independent Snapshots), a framework that recovers spatial dynamics from time-labelled images by learning a unified update rule that is spatially local, Markovian, and time-invariant, and that is iterated to reach every observed time. We implement these constraints with neural cellular automata, trained adversarially by matching generated rollouts to the observed distributions. On synthetic data, REGIS outperforms state-of-the-art baselines in trajectory quality within and beyond the training horizon, data efficiency, and recovery of the underlying dynamics. Applied to a real biological system, a spatial atlas of zebrafish heart regeneration, REGIS recovers a rule that simulates regenerative dynamics. When used in an in silico functional screen, REGIS identifies critical cell types and hierarchical dependencies consistent with known biology, giving the learned rule a mechanistic interpretation. The result is a general recipe for building simulators from unpaired spatial snapshot data by sharing weights across space and time under a locality constraint. The resulting simulators are mechanistic and perturbable, offering a practical route to digital twins for biological systems.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
trajectory inference, spatiotemporal dynamics, neural cellular automata, adversarial training, AI for science, spatial transcriptomics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_t4hHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors proposed a cell automata based method to learn spatial-temporal processes using unpaired spatial snapshots only, via GAN type training. They test their method on image and cell datasets.

Critical strengths
  • A clear mechanism to handle spatial dependencies.
  • Performance seems good. Esp it seems being able to predict perturbation.
Critical weaknesses
  • The mechanism handling temporal dependencies is quite unclear to me. Since the observations are not paired, the data contains no information about the dependency structure in time. Thus the information has to come from somewhere else. I suspect the cell-automata as well as a shared MLP is doing some of that work but I am not sure.
  • A DDPM condition on time can be trained to do marginal interpolation. It just won't be good for predicting a hypothetical paired trajectory.
Questions for authors
  • My biggest question is where the temporal information is from. I think it is fine wherever but since we never had paired training it cannot be from the model.
  • In table 5 I am s bit unsure about the bolding rule, smaller the better or larger the better.
3 comments

We thank the reviewer for the comments.

We agree that unpaired snapshots do not contain any information about the dependency structure in time. As the reviewer notes, a DDPM conditioned on time can reproduce and interpolate the marginals, but nothing in its training links samples at different times. Trajectories produced that way therefore wouldn’t capture the true underlying dynamics – as we showed for other transport-based baselines (MMSFM and MMtSBM). For clarity, the DDPM in our Ising comparison is a different model, conditioned on the previous state and trained on true pairs.

As the reviewer suspects, the temporal information in REGIS comes from the model’s formulation rather than from the data alone. There are two factors in play:

  1. The marginals are ordered, and the discriminator learns each corresponding data distribution. The generator is guided by discriminator’s gradients to learn to transfer samples from one marginal to the next. Informally speaking, this gives a general shape of the possible temporal dependencies.
  2. The generator itself is defined as NCA, so the transfer is achieved by iteratively applying a local, time-independent update rule that is shared across all pixels. These are strong constraints – the same rule must reproduce every snapshot over every interval. Because one rule must turn the statistics at each observed time into those at the next, the snapshots constrain it even without explicit pairs in training.

Empirically, this is sufficient for the problems we tested the framework on. In Ising model experiments, REGIS recovers spin-flip probabilities close to the simulator’s. In the zebrafish heart regeneration dataset, it reproduces known dependencies between cell types under knock-outs. Consistent with the intuition above, ablated models with relaxed constraints also fit the marginals but perform worse at recovering the mechanism.

We have added a paragraph at the end of section 2.3 to communicate this intuition more clearly.

“Note that observed and generated images are compared only at the distribution level, so the data do not specify which state at one time corresponds to which state at a later time. Many different trajectories are consistent with the observations. Restricting the generator to a single local, time-invariant rule that must reach every observed distribution through repeated application narrows this space to trajectories that such a rule can produce. Provided the true trajectory satisfies the chosen constraints and is thus retained while many alternatives are removed, recovering this trajectory from the observations becomes more likely.”

Regarding the bolding rule in Table 5: we bold all values where the mean absolute deviation from 1 (the ground truth value) across all seeds does not exceed 0.025. We have clarified this in the table legend.

Please let us know if we interpreted your concerns about the temporal dependencies correctly and whether our response addresses them.

AuthorsOpenReview ↗

Response to reviewer t4hH

We would like to check whether our clarifications and the amendments to the manuscript have resolved the outstanding questions. If any concerns remain, we would be grateful if the reviewer could raise them soon, so that we have time to respond before the discussion deadline.

Thank the authors for responding. I do not have remaining concerns. I would usually increase score but I don’t feel the current paper makes general ML contribution for a top score. So I kept my 3, which is better to be seen as a 3.5.

Reviewer_ZwN5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces REGIS, a framework for learning stochastic spatial dynamics from unpaired observations collected at discrete time points. The method combines a shared local update rule, implemented as a neural cellular automaton, with adversarial training to match the observed marginal distributions. Its central insight is that the learned rule defines a simulator that can be run repeatedly, extended beyond the observed time points, and subjected to perturbations. The experiments examine different aspects of this approach. The cycling MNIST experiment tests whether the model can sustain repeated transformations while maintaining image quality. The Ising experiment evaluates domain coarsening and local transition statistics against a known simulator. The zebrafish heart regeneration experiment examines whether the learned dynamics reproduce the observed changes in tissue composition, and whether in silico knockouts recover dependencies between cell types consistent with known biology.

Overall, I find the approach compelling and the experiments thoughtfully designed. The compact models, controlled comparisons, and evaluation beyond marginal distribution matching are particular strengths. My comments mainly concern relations with prior work, clarifying recovery of the underlying dynamics and some empirical obstacles in the learning procedure.

Critical strengths
  1. Well-motivated formulation. Instead of choosing an interpolant between marginals, the paper constrains the dynamics through structural priors on the update rule: spatial locality, Markovianity, and time-homogeneity. The result is a simulator rather than an interpolation. It can be run beyond the observed horizon, applied to larger domains, and perturbed for interventional analyses.
  2. Compact models. With the NCA kernels, REGIS only uses 17K–88K parameters, compared with 3.6M–41.7M for the transport baselines. Despite this, it achieves the best long-horizon performance in MNIST transition experiments, while the baselines and the non-local control degrade or diverge.
  3. Data efficiency. REGIS recovers the correct Ising growth trend from as few as 50 unpaired snapshots, whereas the paired DDPM baseline deteriorates as data decrease.
Critical weaknesses
  1. Identifiability and the scope of recovering the underlying dynamics. Inference from unpaired snapshots is underdetermined, and the paper does not state precisely which ambiguities its structural restriction resolves.
  2. Positioning relative to discrete diffusion and local transition models. Iterated local stochastic updates are a well-established route to global structure, and discrete diffusion frameworks can parameterize local categorical transition kernels. The paper does not discussion the advance from NCA. It also lacks a baseline that would isolate these factors, such as a local categorical transition model trained on the same unpaired marginals.
  3. Robustness of adversarial training. The paper provides evidence that training works in the selected configurations. However, it does not report sensitivity to key settings.
Questions for authors
  1. Relation to Discrete diffusion

    The idea that iterated local updates can generate global structure invites comparison with discrete diffusion models, which also build samples through sequences of categorical transitions. D3PM (Austin et al., 2021) specifies categorical corruption matrices, including uniform replacement, absorbing-state masking, and discretized Gaussian transitions between nearby ordinal values. These matrices act independently across spatial sites. Campbell et al. (2022) formulate discrete denoising as a continuous-time Markov chain whose reverse jump rates are parameterized by a shared network conditioned on the current state. Restricting that network’s receptive field would yield a local reverse kernel. The same parameterization could instead define a time-homogeneous transition rule in physical time, fitted directly to the observed marginals. In the Ising setting, either option amounts to learning neighborhood-dependent spin-flip probabilities, or jump rates in a continuous-time formulation. Moreover, it is well established that iterating local stochastic rules produces large-scale structure, from kinetic Ising models (Glauber, 1963; Bray, 1994) to neural cellular automata (Mordvintsev et al., 2020). Locality alone therefore does not distinguish REGIS.

    I would therefore ask the authors to clarify which advantages are specific to the NCA parameterization, and which arise from the broader combination of spatial locality, temporal homogeneity, and marginal-distribution supervision. In particular, how should REGIS’s adversarial learning of an evolution rule be distinguished from learning the reverse of a prescribed corruption process? One difference seems central. A discrete-diffusion reverse kernel is indexed by noise level and transports noise to data. The REGIS rule, by contrast, is homogeneous in physical time and transports one observed marginal to the next. The comparison should identify the respective roles of the state representation, the transition parameterization, and the training objective. A focused Ising comparison would help isolate these contributions. One option is a local categorical transition model with the same neighborhood and checkerboard update schedule, trained on the same unpaired marginals.

  2. Identifiability and the meaning of “recovering dynamics”

    The paper appropriately acknowledges that inference from unpaired snapshots is generally underdetermined. Nevertheless, the meaning of “recovering the underlying dynamics” could be stated more explicitly. Matching the observed marginals, generating possible trajectories, and identifying the true transition mechanism are distinct goals, each requiring stronger assumptions than the last.

    First, the marginals do not determine the path. Many processes share the same time-marginals, and selecting one requires an additional principle. For example, Benamou–Brenier dynamic OT (Benamou & Brenier, 2000) minimizes the kinetic action 12∫0T∫ρt(x)∣vt(x)∣2dxdt\frac12\int_0^T\int \rho_t(x)|v_t(x)|^2 \mathrm{d}x \mathrm{d}t subject to the continuity equation and prescribed marginals. A Schrödinger bridge (Léonard, 2014) instead minimizes path-space relative entropy DKL(P∣∣R)D_{\mathrm{KL}}(P||R) over path measures PP with the prescribed marginals, for a reference process RR. When RR is a diffusion with fixed diffusivity σ2\sigma^2, Girsanov's theorem expresses this objective, up to an initial-marginal term, as an expected control energy. These approaches therefore select dynamics through explicit evolution constraints and variational principles.

    Second, selecting a path does not amount to recovering the mechanism that generated the data. gWOT (Lavenant et al., 2024) provide a useful example. Their consistency result for trajectory inference requires a known diffusivity, a smooth gradient drift, and observation times that become dense. Comparable guarantees do not follow merely from the existence of a unique OT or SB solution.

    REGIS imposes a different structural restriction: an update rule that is spatially local, Markovian, and shared across space and time. To clarify what this restriction contributes, could the authors state which ambiguities it excludes beyond marginal matching, which remain, and how its role differs from the action or reference-process assumptions above? In particular, does "recovering the underlying dynamics" refer to identifying the transition law within the proposed model class, or to learning a simulator that reproduces selected dynamical and perturbation properties?

    An explicit account of these distinctions would sharpen both the novelty and the interpretation of the results. It would also explain why the proposed architectural assumptions suit the studied systems, and clarify which experimental results support the claim of mechanism recovery.

  3. Stability of GAN training

    REGIS uses an adversarial generator–discriminator objective to match simulated and observed marginal distributions. The regularized objective, exponential moving averages, generated-state pools, and results across multiple random seeds provide useful evidence that the proposed training procedure is workable. Could the authors clarify how sensitive training is to its main settings, particularly the strength of the discriminator gradient penalty and the length of the backpropagation window for long rollout segments?

    A compact sensitivity analysis, ideally tracking an appropriate validation metric during training, would help establish how reliably the method can be trained beyond the selected configuration.

  4. Data requirements for the biological application

    The zebrafish results are encouraging, and the Ising data-efficiency experiments and the whole-fish holdout provide useful supporting evidence. For the biological application, however, the 12–18 sections per stage come from only 4–6 fish, as the authors note in Appendix C.1. This raises an important practical question: how much independent biological variation is needed for reliable distribution matching?

    Could the authors assess how the heart results change when fewer fish are available, for example through a modest fish-level subsampling study with evaluation on held-out fish? All sections from a given fish should remain together in these splits. Tracking both the marginal fit and the inferred dependency structure would show whether the biological conclusions remain stable as sample diversity decreases.

  5. Adaptation to non-regular grids

    REGIS represents observations on a regular lattice, and the heart experiment uses grid-based binning and pooling. Many real-world datasets lack this regularity. For example, geographic data are often partitioned into regions with irregular borders. Similarly, some spatial transcriptomics techniques yields irregularly positioned cells, which are more naturally represented as point configurations or cell-contact graphs. Could the authors clarify how the framework would apply to these settings? Would the intended approach be to rasterize the observations onto a regular grid, or to extend the local update rule to a graph-based formulation? In either case, how would the spatial representation preserve the physical meaning of a local interaction, particularly when sampling density or inter-cell distances vary? A brief discussion of the required preprocessing or architectural changes, including any changes to the discriminator, would clarify the method's scope and its applicability across spatial measurement platforms.

  • Austin, J., Johnson, D. D., Ho, J., Tarlow, D., & van den Berg, R. (2021). Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34.
  • Benamou, J.-D., & Brenier, Y. (2000). A computational fluid mechanics solution to the Monge–Kantorovich mass transfer problem. Numerische Mathematik, 84(3), 375–393.
  • Glauber, R. J. (1963). Time-dependent statistics of the Ising model. Journal of Mathematical Physics, 4(2), 294–307.
  • Bray, A. J. (1994). Theory of phase-ordering kinetics. Advances in Physics, 43(3), 357–459.
  • Mordvintsev, A., Randazzo, E., Niklasson, E., & Levin, M. (2020). Growing neural cellular automata. Distill.
  • Campbell, A., Benton, J., De Bortoli, V., Rainforth, T., Deligiannidis, G., & Doucet, A. (2022). A continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems, 35.
  • Léonard, C. (2014). A survey of the Schrödinger problem and some of its connections with optimal transport. Discrete and Continuous Dynamical Systems – Series A, 34(4), 1533–1574.
  • Lavenant, H., Zhang, S., Kim, Y.-H., & Schiebinger, G. (2024). Toward a mathematical theory of trajectory inference. The Annals of Applied Probability, 34(1A), 428–500.
5 comments
AuthorsOpenReview ↗

Response to Reviewer ZwN5 (1/3)

We thank the reviewer for a thoughtful and detailed review. Below are point by point responses to the raised questions. The responses will be split in several sections due to the limitation on the number of characters set by the platform.

1. Relation to Discrete diffusion

Comparison to D3PM. The reviewer already addresses the most critical point: “One difference seems central. A discrete-diffusion reverse kernel is indexed by noise level and transports noise to data. The REGIS rule, by contrast, is homogeneous in physical time and transports one observed marginal to the next.” As the reviewer noted, the main motivation of our work is not to generate end states of a system’s evolution but to model the process itself. Methods that cannot transport between multiple unpaired marginals therefore do not meet our requirements, although we use them in the paper for illustration, as with our DDPM baseline.
If paired observations were available, the task would be much simpler. Each step would then have a direct training target, and a wide range of parametrisations of a local, time-homogeneous update rule could learn it. While we use an NCA, one can imagine alternatives. For example, as the reviewer noted, learning Ising dynamics amounts to learning neighbourhood-dependent spin-flip probabilities, which we would expect various local parametrisations to learn, potentially even within frameworks such as D3PM. Our DDPM baseline, trained on true pairs, illustrates this (Table 5). However, without pairs, or a pre-defined reference process, the task becomes highly non-trivial. No step has a direct target, so the rule can only be learned by simulating it over many steps and comparing the result with the next observed marginal.

The comparison should identify the respective roles of the state representation, the transition parameterization, and the training objective.

We would first like to shift the focus from the specific architecture to the general conceptual value of the paper. We see our main contribution not as the combination of an NCA with R3GAN but as an approach to modelling trajectories when only unpaired multi-marginal snapshots are available. We chose each component because it is well established and has the properties we needed: an NCA is local and time-invariant by construction (Mordvintsev et al., 2020), and R3GAN trains stably (Huang et al., 2024). The latter we confirmed with our new analysis on parameter sweeps and comparing to the default GAN (Appendix D). We do not consider this exact combination essential.

State representation. The visible channels are states of the system itself, not noised data, so every intermediate step of a rollout can be read out, compared with observations and perturbed. Hidden channels further increase a capacity for modelling processes where not all information is directly observable, which is common for biological processes.

Transition parameterisation. Locality and time-invariance are the structural priors that regularise the problem; our controls show that relaxing either degrades recovery (Tables 6, 11 and 12). We use an NCA because it is a well-studied architecture with exactly these properties (Mordvintsev et al., 2020; Richardson et al., 2024). Any local, time-invariant parameterisation would fit the framework, and we adapt its inductive biases to each system: a checkerboard schedule and spin symmetries for Ising, and a non-increasing damage channel for the heart. Continuous states also directly allow back-propagation, unlike discrete transition models that would require additional tricks like continuous relaxations or use of high-variance estimators.

Training objective. With unpaired data no step has a direct target, so the rule must be trained by matching the distributions it produces to the observed ones. In high-dimensional spatial data, distances with a fixed kernel or cost, such as MMD, lose power as dimension grows and give a weak training signal. We therefore use an adversarial objective.

AuthorsOpenReview ↗

Response to Reviewer ZwN5 (2/3)

2. Identifiability and the meaning of “recovering dynamics”.

Meaning of “recovering dynamics”. We agree with the reviewer that our use of “recovering the underlying dynamics” was underspecified, and we now define it more explicitly. In the reviewer’s terms, formally, REGIS generates possible trajectories: those that are consistent with the observed marginals and produced by an update rule that is spatially local, Markovian, and shared across space and time, as many physical mechanisms are. Given the large class of problems we aim to work with, we can not give any theoretical guarantees that the learned rule is the true transition mechanism. However, the experiments in the paper show that beyond just generating plausible trajectories, both for the Ising and heart regeneration datasets, REGIS is able to extract scientifically valuable information about the modelled systems. In the reviewer’s words, we learn a simulator that reproduces selected dynamical and perturbation properties.

The role of our assumptions, and how they differ from the least-action and reference-process assumptions. The approaches mentioned by the reviewer link observations at successive times via a coupling and use a variational principle to break ties among the processes that fit. This gives a convenient way of training and a unique solution; however, these principles carry little information about the particular system being modelled, beyond the choice of reference process. We work with high-dimensional problems in a low-data regime, where coupling the observed samples is unlikely to produce reasonable pairs. In the Ising model, for example, two independent fields have entirely different domains, so a model trained on such pairs learns transitions that are extremely improbable under the actual physical law. We want to highlight that this effect also undermines the influence of architectural priors on FM methods – one can formulate a local time-independent model, but if it is trained to approximate unrealistic trajectories, it is improbable that a realistic simulator emerges.

We therefore link the marginals in a different way – by explicit simulation. Our assumptions restrict the family of rules that are simulated, and they reflect what is reasonable for the processes that generate such images in biology and physics: interactions are local, and the law does not change over time.

Ambiguities excluded. Each of our two assumptions removes an ambiguity that the marginals leave open. The first is time homogeneity. With sparse time points, a rule that depends on time could in effect learn a separate rule for each transition between marginals, and nothing would connect them. Requiring the same rule at every step forces the model to use the state of the system itself as the only source of information about what should happen next. In the heart experiment, giving the model a stage label weakens the dependency between cardiomyocyte states (63% retained after knock-out against 31%), and the ranking of cell types by prior biological groups falls to near chance (60% of pairs against 77%).

The second is spatial locality. A rule could otherwise link arbitrarily distant sites. Requiring each site to be updated from its neighbourhood forces the rule to explain how local interactions create global behaviour, which reasonably describes many biological and physical systems. It also improves data efficiency, since each sample contains many neighbourhoods to learn from. Our ablations show that relaxing this assumption moves the learned rule away from the correct one: in the Ising model, wider neighbourhoods give less accurate transition probabilities and domain growth (0.92–0.94 of the true value for 5×5 and 7×7, against 1.00–1.01 for 3×3), and in the heart, a non-local model retains more of the cascade after knock-out (74% against 31%) and ranks the cell-type groups at chance (52% of pairs). Naturally, both assumptions help only when the modelled system is itself governed by laws that are local and do not change over time.

Ambiguities that remain. Dynamics that leave the marginals unchanged, such as cycling through states or the overall rate of a process at equilibrium, are invisible in snapshots; REGIS can only model processes with a defined arrow of time. Locality requires that new material arise from the neighbourhood, but does not separate proliferation, conversion and migration (important for spatial biology data modeling).

We thank the reviewer for these questions. We have realised that the main points differentiating REGIS from transport-based methods such as FM haven’t been made sufficiently clear. To remove this gap, we have added a new Appendix section E, “Relation to flow-matching models”, where we have gathered our responses to flow-matching-related questions and also questions about the nature of tasks that we envision for REGIS – including this one.

AuthorsOpenReview ↗

Response to Reviewer ZwN5 (3/3)

3. Stability of GAN training.

We thank the reviewer for this comment. We have performed additional experiments to evaluate REGIS’s stability with respect to the hyperparameters the reviewer suggested - namely, gradient penalty strength, length of the backpropagation window, learning rate, and EMA averaging strategy. For all the experiments, we provide training curves that track the change in performance over time, expressed as FID for MNIST and ratio of domain lengths for Ising – our main metrics that we use throughout the paper.

We have found that REGIS achieved good results across a wide range of training configurations. One notable exception is the backpropagation window length – while for Ising as little as 5 steps is sufficient, training on MNIST benefits from backpropagating through the entire rollout (30 steps). We speculate that this is due to the fact that for MNIST trajectories, there are different phases – digits and their intermediate stages – while an Ising trajectory is much more homogeneous over time. Hence, gradients obtained from backpropagation through a short window would be more representative of the full rollout for Ising compared to MNIST.

We have included the quantitative details of these experiments, along with semantically similar requests from reviewer Pp38, in the paper as an additional Appendix section D and referenced it throughout the main text. The section contains: 1.Sensitivity analysis w.r.t. relativistic penalty strength, learning rate, backpropagation window length and EMA 2.Comparison of R3GAN vs vanilla GAN 3.Breakdown of where the time goes during every training step (rollout, discriminator forward/backward, relativistic penalty, etc) 4. Comparison of training and inference times for REGIS and the competitors

4. Data requirements for the biological application

We thank the reviewer for this suggestion.

We retrained REGIS on progressively fewer fish (34, 32, 24, 16 and 8 of the 35), with three fish subsamples and two training seeds at each size (6 reruns per data regime). All sections of a fish were kept together, and every fish left out of training was used for evaluation. Each model was scored on training and held-out, covering both the marginal fit (temporal correlation) and the inferred dependency structure (knock-out group recovery and cascade retention).

Reported metrics are largely stable down to two fish per stage (16 fish, 48 sections). BZm temporal r (marginal reconstruction) is 0.84 ± 0.14 (full data: 0.88 ± 0.13), and both pre-specified group contrasts are recovered (78 ± 9% and 85 ± 4% of pairs, against 77 ± 9% and 84 ± 11%). Only downstream cascade retention weakens (49 ± 25% against 31 ± 10%). With only one fish per stage (8 fish, 24 sections) results substantially degrade: BZm r falls to 0.63 ± 0.19 and the group contrasts approach chance (63 ± 13% and 58 ± 20%).

These results show that REGIS is data-efficient on this task, which is encouraging for other processes that are similarly conserved between individuals, such as development. We would nonetheless expect larger data requirements for systems with greater biological or technical variability, such as spatial tumour dynamics, where evolutionary trajectories and the resulting heterogeneity are far larger.

We have added the full design and quantitative results of these experiments, with a summary table and figure, as a new Appendix C.6 (“Data requirements”), referenced from Section 3.3.

5. Adaptation to non-regular grids

For generalising to non-regular grids, we find switching to a graph-convolution-based architecture the most straightforward approach. There are several ways of preserving the locality of interactions: either by defining the graph such that only immediately adjacent cells or regions are connected by edges, or by specifying a distance-dependent weighting function, as is commonly done in the field. Graph convolutions are well studied for biological spatial data, so many options are available, even for situations with varying sampling density (Fischer et al., 2023; Dong & Zhang, 2022; Monti et al., 2017). This is a relatively minor alteration of the current architecture, so we expect it to work out of the box.

The discriminator requires a more substantial architectural modification, as it currently has a global view, which is more difficult to define for graph-based data. To explore whether a discriminator consisting of local convolutions followed by a single global pooling provides enough training signal, we trained a model of this type on the MNIST dataset.

The results show that not only training the described model in possible, but once trained, during inference it also generalises to different grids and even to grids with moving nodes.

We have added Section A.7 that describes the experiment setup and the results. We have also added the videos of the rollouts to our project page (https://anon03752.github.io/regis-demo/#lattices).

Reviewer_Pp38OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this submission, the method REGIS is proposed: an extension of NCA that uses an adversarial training objective in order to simulate trajectories that obey prescribed evolutionary dynamics. The prior notions of dynamics are encoded by the way the NCA does its predictions, i.e. how it generates the next state vector given the current one. The method is tested on three different image datasets and successfully recovers the underlying dynamics, when known.

Critical strengths

Learning evolutionary dynamics with encoded prior information is an important and highly relevant topic of study, and the authors do a great job motivating the research. Moreover, the figures are outstandingly well curated and appealing, and the experiments conducted are non-trivial and ambitious. For instance, the authors extended the rotating MNIST dataset to be rotatable indefinitely, which allowed them to experiment with long-range roll-out capabilities. This was interesting to see. Furthermore, the introduction to NCAs was good for non-experts, like me.

Critical weaknesses

The following two points affect my soundness score, the third point concerns presentation and the last point concerns informativeness.

(1) Baselines I appreciate that the authors compare their NCA-based REGIS against multimarginal FM and SB methods in Sec. 3.1, but I am wondering what motivated the choice of MMSFM? The method does not appear to have strong results in its publication and is not standard. Two more natural competitors are either OT-CFM (Tong et al. (2024)) which is the go-to FM method, or the adversarial one referenced in your manuscript (Kviman et al. (2026)) as it shares the same marginal-distribution fitting engine. I am not saying that the results of the baselines would change drastically, but I believe that the choice requires a motivation.

(2) Interpolant vs. prior For transparency, I am not an expert on NCAs (the authors did a good job of introducing the methodology in Sec. 2.2, however). I was surprised that REGIS wasn't more extensively compared to the ALI-CFM multimarginal flow matching (MMFM) method (Kviman et al. (2026)). In the intro, the argumentation is that the choice of the interpolation scheme in MMFM methods bias the ability to study the underlying evolutionary system, but in a sense, an interpolant is a prior that can be corrected based on the observations in the marginals.

To make the statement that distances REGIS to MMFM more exact, I would thus use ALI-CFM to concretely explain how the methodologies differ. That is, I would use ALI-CFM as it also uses the adversarial objective. As the manuscript reads now, I find the distancing of REGIS from e.g. ALI-CFM hand-wavy -- yes, the learning frameworks are different, but what fundamentally separates these methods in terms of what you can infer from the learned dynamics? ALI-CFM doesn't interpolate between marginal samples, it fits a distribution of interpolants to the marginal distributions.

Computational burden A standard question when seeing a GAN-like method being proposed is of course wrt the hyperparameter tuning efforts required; do you need many to do hyperparameter sweeps to find a working parameterization? Although standard, I think addressing this is relevant.

A more critical and measurable question that has not been investigated in the paper is the complexity of using R3GAN. From my experience this GAN version is considerably slower to run and adds additional hyperparameters to tune. I think the following two studies are missing:

  1. An ablation where R3GAN is compared to the vanilla GAN in REGIS
  2. A computational complexity study of REGIS in contrast to it's competitors

(3) The experiment subsections and the learning rule There are a lot of analyses of the experimental results, which is positive of course, however I find the information difficult to parse as the text is rolled out at lengths without headings. I would suggest introducing some paragraph headings that capture what aspects of the experiment are discussed in that particular paragraph. This would improve the exposition of the paper.

Regarding the learning rule, this seemingly key object is discussed in the abstract and at multiple places in the text but I am still unsure exactly what it is. Is the learning rule Eq. (5)?

These two concerns render a lower presentation score.

(4) Informativeness This is probably my main concern sits. I agree with the narrative of the paper in terms of the need for methods to learn evolutionary dynamics and how priors are important to guide the learning. However, in the end I found all argumentation for REGIS as a superior approach to learning dynamics to be merely supported empirically.

There are indeed many experimental details provided from the three conducted experiments, but without a fundamental break-down of what REGIS is actually doing in contrast to, for example, multimarginal flow matching methods, I find the evidence supporting the method quite thin and ad hoc -- why is REGIS inherently a better option than the used baselines? Or, why is an MMFM method inherently not able to solve the experimental problems considered in the paper? Right now I read the results here as REGIS outperforming the implementations of MMSFM and MMtSBM etc., not the other methodologies.

This resonates with my earlier concerns, and is fixable by a more in-depth non-empirical comparison of the methodological differences, however the required work might be quite significant.

Minors: Fig. 1 bottom left plot: the ∼pobs\sim p_\text{obs} and "gen" are difficult to map to any shape in the figure. I.e. it is not clear what is simulated from these distributions.

In line 189 xtbobsx_{t_b}^\text{obs} is simulated from ptbp_{t_b} --- this seems to be inconsistent notation from the use of pobsp_\text{obs} in Fig. 1.

Questions for authors

There are some questions naturally integrated in the weaknesses cell above, but here is another question I have:

Would you see REGIS as an extension of NCA with the addition of an adversarial loss, or is there a deeper methodological contribution that I failed to catch? This might very well be the case, since I am not an expert on NCAs. Note that I have not weighed in any notion of "method novelty" in my evaluation above at this point.

Could REGIS be applied to non-image data? Such as spatial biology data such as spatial proteomics or spatial transcriptomics?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Pp38 (1/3)

We thank the reviewer for a thoughtful and detailed review. Below are point-by-point responses to the raised questions. The responses are split into several sections due to the character limit set by the platform.

(2) Interpolant vs. prior and (4) Informativeness

We agree that an interpolant is a type of prior and can be fitted to the data, and that in ALI-CFM it has substantially more flexibility than in other MMFM methods. However, we would like to emphasise that our method has several key differences, motivated by a different objective: we aim not just to transport between marginals, but to extract the system’s underlying evolution principles for further scientific investigation. Within the FM framework, ALI-CFM included, this is not easily achieved due to the following factors.

Coupling. Fundamentally, all flow matching methods are trained by regression onto a path supplied by the interpolant. This requires a coupling of initial and target points, typically mini-batch OT or independent. In our domain of interest – experimental spatiotemporal data – samples are high-dimensional (thousands of dimensions) and datasets are small, so obtaining a meaningful coupling is challenging in practice: each snapshot comes from a different trajectory, and in high dimensions even the OT partner is barely closer than a random one (Beyer et al., 1999). For example, in our Ising experiment, the paired fields therefore have entirely different magnetic domains, and a path between them must dissolve one domain pattern and build another, which coarsening under the true dynamics practically never does. Naturally, a flow regressed onto such paths would produce correct marginals at the observed times, but the individual trajectories would poorly represent the original data-generating process. REGIS, on the other hand, does not require coupling and does not have this problem.

Intermediate states. Further, efficient training of FM models requires the interpolant to be evaluable at an arbitrary point in flow time. While it is possible to parametrise the interpolant flexibly, for many systems with complex dynamics learning an interpolant that produces realistic intermediate states directly from the initial and final states is challenging; in fact, such hypothetical interpolant would have to already encode the dynamics we aim to learn (and would still depend on a coupling, which, as discussed above, is not always easy to obtain). This problem is tackled by another line of work, in which the path is, as in our approach, simulated iteratively rather than predicted instantaneously. However, training such systems requires backpropagating through the simulation, which makes it prohibitively expensive to work with large models and high-dimensional data: TrajectoryNet (Tong et al., 2020) is run on 5 principal components “due to computational constraints”, and MIOFlow (Huguet et al., 2022) learns its trajectories in the latent space of an autoencoder. We overcome this by using an NCA as our simulation model, which is orders of magnitude smaller than typical FM models: in our experiments, NCAs typically have tens of thousands of parameters, compared with millions for the transport baselines.

Transition function definition. Finally, a simulation-based approach gives us more freedom in defining the model and in adding trajectory-level constraints. For example, maintaining a pixel-wise hidden state, which is important for modelling biological processes where not all information is observable, has no direct counterpart in a classical FM formulation. Another example: in our heart regeneration experiment, we add a trajectory-level auxiliary loss that preserves morphological invariants, such as an individual organ’s shape.

Therefore, while REGIS isn’t inherently better than FM models when focusing exclusively on restoring the marginals, we argue that REGIS is better suited to tasks where the aim is to recover the mechanism or to train a simulator, as is the case for many biological processes.

We thank the reviewer for these questions and hope that the difference between REGIS and transport-based methods is more clear now. We have included the points above into the manuscript as a new Appendix section E, with references from the main text in appropriate locations.

AuthorsOpenReview ↗

Response to Reviewer Pp38 (2/3)

(1) Baselines

We agree that the choice of baselines requires clear motivation. When selecting the FM and SB methods for comparison, we focused on methods that solve the multi-marginal task and have been demonstrated on images. MMSFM and MMtSBM were the only ones we are aware of that satisfy both requirements.

OT-CFM. OT-CFM is a two-marginal method. We agree that if it were trained to perform transport between all pairs of consecutive marginals independently, as a go-to FM method, it could have performed on par or better than MMSFM in terms of the quality of reconstruction of marginals. However, we are not aware of multimarginal image-targeting implementation and therefore excluded it from the baseline category.

ALI-CFM. A comparison to ALI-CFM (Kviman et al. (2026)) would have been extremely interesting. However, the strongest results in the original paper were obtained on 2D points. The authors note that even on 50-dimensional data, ALI-CFM performs worse than baselines with non-trainable interpolants; the gap further increases for 100-dimensional data. Our data of interest is at least an order of magnitude more high-dimensional and also consists of images, for which no ALI-CFM recipes were provided. Therefore, we did not include this method in our experiments.

We now state our baseline selection criteria directly in Section 3.1. We also reference Appendix A.3 with a detailed reasoning, including why OT-CFM and ALI-CFM were not our primary baselines

Computational burden

We thank the reviewer for highlighting these gaps.

We initially used a REGIS formulation with a vanilla GAN. However, we found that it suffered from serious convergence problems, which could not be addressed by carefully tuning the hyperparameters. On the contrary, once we switched to R3GAN, the training recipe worked essentially out of the box and remained stable for a wide range of configurations – including varied strength of the gradient penalty, which is absent from the vanilla GAN formulation.

Furthermore, in our case, the slowdown introduced by R3GAN is limited – the computational bottleneck is the generator rollout, therefore the additional backward pass through the discriminator only increases training step time by 7-28%, depending on the dataset.

In terms of computational complexity, the training of REGIS is faster or comparable to the competitors, while inference becomes 1-2 orders of magnitude faster due to the smaller network size.

We have included the quantitative details of these analyses into the paper as an additional Appendix section D and referenced it throughout the main text. The section contains:

  1. Sensitivity analysis w.r.t. gradient penalty strength, learning rate, backpropagation window length and EMA
  2. Comparison of R3GAN vs vanilla GAN
  3. Breakdown of where the time goes during every training step (rollout, discriminator forward/backward, gradient penalty, etc)
  4. Comparison of training and inference times for REGIS and alternative methods

(3) The experiment subsections and the learning rule

Following the suggestion, we have added paragraph headings throughout the main text. Regarding the “learning rule” – thank you for pointing out this ambiguity. The update rule is the map NCAθ\mathrm{NCA}_\theta in Eq. (2). Eqs. (3)-(5) are its three stages, so Eq. (5) is its final step. We have added a statement after Eq. (2) to disambiguate.

Minors

We have colour coded pgenp_{gen} and pobsp_{obs} for clarity. We also added a note to the captions of Fig. 1 relating the simplified notation to the detailed notation defined in Section 2 and used throughout the manuscript.

AuthorsOpenReview ↗

Response to Reviewer Pp38 (3/3)

Would you see REGIS as an extension of NCA with the addition of an adversarial loss, or is there a deeper methodological contribution...

On the surface, yes, REGIS is an NCA trained with an adversarial loss. We see our main contribution not as the combination of an NCA with R3GAN but as an approach to modelling trajectories when only unpaired multi-marginal snapshots are available. Previously NCA have been used to model either static or stationary targets (Mordvintsev et al., 2020, Pajouheshgar et al., 2023, Richardson et al., 2024). The ability to learn dynamics from unpaired snapshots is an exciting result on its own; it is also instrumental for modeling biological data.

Spiritually, the original NCA was introduced as a nature-inspired trainable model of how collective dynamics can arise from simple local rules (Mordvintsev et al., 2020). Since then, several works have trained NCAs on biological or physical data, but they have either learned to reproduce single static targets or required access to full trajectories (Richardson et al., 2024; Milite et al., 2025). There was a mismatch between the type of data that spatial biological assays produce, which are unpaired multi-marginal snapshots of destructive measurements, and the training procedures available for NCA. REGIS addresses this, and we demonstrate it on spatial transcriptomics data of heart regeneration. We hope this will let the field use NCAs as simulation engines for the growing range of spatio-temporal snapshot datasets in biology.

Could REGIS be applied to non-image data? Such as spatial biology data such as spatial proteomics or spatial transcriptomics?

Yes. First of all, we want to note that spatial biology data are, in fact, routinely represented as images: in sequencing-based platforms the measurements are binned onto a regular grid by construction (Stereo-seq, Chen et al., 2022; Visium HD, Oliveira et al., 2025), and several analysis methods convert spatial transcriptomics into images so that standard image-processing tools can be applied (Martin et al., 2022, Hu et al., 2023). Such rasterised images can be used with REGIS directly.

Our zebrafish heart regeneration dataset (Section 3.3) is exactly of this type - it is real-world spatial transcriptomics data produced using the Stereo-seq technology.

Where rasterisation is undesirable, REGIS can also operate directly on irregular grids by replacing the 2D convolution in the perception module of the NCA by a graph convolution. We have successfully trained proof-of-concept models on our cycling MNIST dataset adapted to such grids; beyond just accomplishing the task, REGIS is able to generalise to types of grids not seen during training (section A.7) and even moving grids (https://anon03752.github.io/regis-demo/#lattices)

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers – summary of the revisions

We deeply thank all the reviewers for constructive discussion and suggestions. We appreciate the time and effort they put into reviewing our paper.

We are grateful for the positive reception of several key aspects of our work:

Reviewers ZwN5 and Pp38 highlighted our presentation, noting that the paper is interesting, enjoyable to read, and accessible to non-experts with no prior knowledge of NCA. Moreover, we a glad that the reviewers found our figures well-designed and visually appealing.

Speaking of the scientific content, Pp38 characterised our research goal as an “important and highly relevant topic of study”, which ”the authors did a great job motivating”, and ZwN5 emphasised that “the idea that local updates can compose paths between unpaired marginals is fascinating”.

All reviewers highlighted our empirical contributions, such as the interesting and ambitious experimental design (Pp38) and the strong performance of our method, particularly its ability to predict perturbations (t4hH). Finally, the reviewers valued the contributions that we see as central to our work – namely the formulation of a small, data-efficient model that can easily incorporate desired inductive biases (t4hH, ZwN5).

We found the discussion with the reviewer very informative. Thanks to their suggestions, we have conducted further experiments and improved the manuscript.

New experiments:

  • Training stability and cost. We explored stability of the adversarial training summarised in Added Appendix D. It covers new experiments on sensitivity to training settings and a comparison of R3GAN with a vanilla GAN (Pp38, ZwN5). Also added Table 14, comparing generation and training cost between methods (Pp38).

  • Data requirements for the biological application. Beyond demonstrating data efficiency on our Ising example, based on the input ZwN5 we also investigated data requirements on our heart regeneration dataset that showed high stability for the recovery of the metrics to as little as 3 tissue slides per time point. This experiment is summarised in Appendix C.6.

  • Irregular grids. Following the reviewers ZwN5, Pp38 interest in applications to non-regular grids, we added an experiment demonstrating methods flexibility on irregularly lattices, described in Appendix A.7 and visualized at our project page (https://anon03752.github.io/regis-demo/#lattices).

Additional discussions:

  • Based on the input of reviewer Pp38, we provided a more clear motivation behind selection of baselines. The selection criteria are stated in Section 3.1 and explained in detail in Appendix A.3.

  • Based on the in depth discussion a new section covering the relation to other methods. Added Appendix E, which compares REGIS with flow matching, including ALI-CFM (Pp38). The section also touches on practical obstacles that make development of the framework working on the same assumption as REGIS challenging within the FM framework. This section also expands on what we mean by recovering the dynamics and which ambiguities remain (ZwN5). Finally, based on the reviewers' feedback, we emphasise that both FM and REGIS can coexist, as each method has its strengths (ease of incorporating biases for REGIS and stronger theoretical guarantees for FM).

  • Added a limitations paragraph to the Discussion. And generally improved readability based on the inputs of all reviewers.

Thank you again for your valuable feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.