acceptodds
Under review as a conference paper at ICLR 2027

Shared-Generator Neural Flows for Zero-Support Transition Prediction

Abstract

Generalizing learned state-transition mechanisms to unobserved condition pairs is a fundamental problem in scientific machine learning, dynamical modeling, and perturbation prediction. Yet learning how a state changes under observed conditions does not determine how to reverse that change or extend it to a larger shift. We study zero-support transition prediction: given a source state and the source and target conditions, the task is to predict the resulting state without examples of the requested transition. Mechanism-Preserving Transport (MPT) ties these predictions together through shared state-dependent vector fields: reversing the condition difference runs the exact flow backward, while doubling it follows the same dynamics for twice as long. To establish when these shared fields also admit a unique interpretation, we characterize the activation patterns that identify their directions up to scaling and permutation, given sufficiently rich grouped short-time observations in fixed coordinates. Our prediction experiments show that the two constraints need not help together. On PDEBench Burgers, MPT reduces direct prediction MSE by 37.8% relative to a zero-preserving conditional ODE over ten seeds. On Shapes3D, keeping reversal but relaxing linear scaling reduces latent MSE by 59.5% relative to MPT. On Norman Perturb-seq, the more flexible zero-preserving model remains more accurate after mean-matching retraining. These controlled comparisons show that preserving an inverse relation can be useful even when extending a change by linear control scaling is not. Reversal and scaling should therefore be chosen separately for unseen-transition prediction, using endpoint accuracy to determine which structure the task supports.

Primary area:
generative models
Keywords:
neural ordinary differential equations, zero-support prediction, compositional generalization, inductive biases, identifiability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZkuDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies zero-support transition prediction: given a source state and the source and target conditions, the goal is to predict the target state without examples of the requested transition. The authors propose Mechanism-Preserving Transport (MPT), which uses the proposed condition difference to linearly combine shared vector fields, called generators. Integrating the resulting autonomous ODE defines the transition map. Wherever the required exact flows exist, this construction satisfies zero-change identity, reversal, and same-direction composition. The authors compare neural ODE controls with different structural constraints to examine their predictive benefits across tasks. MPT performs well on PDEBench Burgers and controlled periodic advection. On Shapes3D, however, Autonomous-odd reduces latent MSE by 59.5% relative to MPT, while on Norman Perturb-seq, Zero-NODE remains more accurate than MPT after mean-matching training. The theoretical analysis studies when shared generator directions are identifiable in fixed coordinates. Under the stated observation assumptions, it characterizes the activation patterns that permit identification up to scaling and permutation, as well as the mixtures that remain indistinguishable when identification fails.

Critical strengths
  1. MPT uses shared vector fields and linear control to enforce reversal and same-direction composition in the exact flow, without requiring an additional consistency loss. On Burgers, its direct prediction MSE is 37.8% lower than that of Zero-NODE.
  2. In fixed coordinates and under the stated observation assumptions, the theoretical analysis characterizes the activation patterns that identify directions up to scaling and permutation, as well as the mixtures that remain indistinguishable when identification fails.
Critical weaknesses
  1. The evaluation covers PDEBench Burgers and a generated periodic advection task. Although MPT performs well on their primary direct prediction metrics, these settings do not adequately test generalization when multiple generator directions act together. Its ability to handle multiple varying parameters and their interactions in more systems therefore remains unclear.
  2. In the Norman Perturb-seq experiment, MPT is compared only with Zero-NODE, while Autonomous-zero and Autonomous-odd are omitted. This makes it difficult to determine which structural properties of MPT contribute to its performance.
Questions for authors
  1. On Burgers (Table 2), why does NCF+/alpha outperform FE-NODE+/alpha and SharedLowRank in direct prediction, while FE-NODE+ /alpha achieves a similar routed endpoint error?
  2. Could the authors include Autonomous-zero and Autonomous-odd in the Norman experiment to better distinguish the effects of the individual structural constraints (Table 4)?
  3. Could the authors evaluate MPT on a broader range of PDE tasks to assess whether its benefits extend beyond Burgers and periodic advection?
6 comments
AuthorsOpenReview ↗

Response to Reviewer ZkuD

Thank you for reviewing our paper and helping us distinguish between two different extrapolations: predicting a larger change along one physical parameter, and predicting the result of two parameters changing at the same time. Based on this, we added a two-parameter reaction–diffusion experiment. We also ran the two additional controls you suggested on Norman.

1. Why is NCF more accurate than FE-NODE on direct Burgers predictions, while their path-based predictions are close?

Table 2 compares direct and path-based predictions on the same test queries. The direct prediction MSEs of NCF + α, FE-NODE + α, and SharedLowRank were 1.884, 10.01, and 16.56, respectively. For path-based predictions, they were 1.255, 1.206, and 1.348, in the same order. All six values are in units of 10⁻⁴. The gap is mainly in the direct prediction. For NCF and FE-NODE, we evaluated each predictor with its adapter, so this comparison does not separate the adapter’s contribution from that of the underlying dynamics.

Taking the unseen query 0 → 4 as an example, a direct predictor has to predict the entire change in one call from the source field and the condition metadata. For a path-based prediction, we apply the four learned local maps in order, 0 → 1 → 2 → 3 → 4. The prediction at each step is the input to the next; we do not use the true intermediate fields. On this query, FE-NODE’s mean MSE across the three seeds was 0.002442 for direct prediction and 0.000153 for path-based prediction. Its local maps worked well when applied in sequence. The difficulty was predicting the same change in one call.

2. After adding Autonomous-Zero and Autonomous-Odd to Norman, what can we say about reversal and scaling?

We trained four models with the same condition-mean objective and matched training and validation budgets. We tested them on held-out pairs of gene perturbations. We first averaged the errors over three training seeds within each split, then calculated the mean and sample standard deviation across the five overlapping condition splits. The DE20 condition-mean MSEs were 0.07192 ± 0.02147 for MPT, 0.05870 ± 0.01027 for Zero-NODE, 0.06269 ± 0.01184 for Autonomous-Zero, and 0.06112 ± 0.00955 for Autonomous-Odd.

Each of the three controls had lower error than MPT on four of the five splits. In particular, Odd keeps the inverse relation but relaxes the linear scaling of the control. It was still more accurate than MPT on average. So the original result cannot be explained simply by saying that reversal is harmful for gene response prediction. MPT also had the smallest disagreement between direct and sequential predictions: 0.01275, compared with 0.02431–0.02713 for the three controls. On this dataset, the model whose two predictions agreed most closely was not the one closest to the measured mean gene response.

3. Can MPT predict the interaction between two physical parameters when they change at the same time?

We tested this question using the Gray–Scott reaction–diffusion system. The two parameters were the feed rate and the kill rate. For each initial field, we simulated all nine parameter combinations from exactly the same starting point. Training included only single-parameter changes. We tested all 36 directed two-parameter transitions on 128 held-out initial fields. The task was to predict both concentration fields under the new parameters at the same physical time, t = 40.

With only adjacent single-axis changes in training, the mean endpoint MSE across five seeds on the joint transitions was 0.00516 for MPT, 0.10435 for Odd, and 0.04289 for Auto-Zero. A more direct comparison comes from the same trained MPT. When we predicted the two single-parameter displacements separately from the same source and added them to the source field, the endpoint MSE was 0.04902. When we asked MPT to predict the joint change in one call, it was 0.00516.

We also measured the interaction separately. We subtracted the two true single-parameter displacements from the true joint displacement. What remains is the part that simple addition misses. We made the same calculation using the model’s three predictions. Compared with predicting zero interaction, MPT’s interaction MSE in physical units was 91.1% lower with adjacent single-axis training and 88.1% lower when training also included longer single-axis changes. In both settings, MPT had lower interaction error than the zero-interaction baseline on each of the 36 test edges in every seed. The full results are in Appendix D. In this system, MPT learned from single-parameter changes alone and still predicted much of the joint response that simple addition missed.

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you.

Thank you for the additional experiments, which address my partial concerns. However, my remaining concern is when the proposed structural constraints are beneficial. In particular, it is still unclear why different structural constraints lead to different prediction accuracies across tasks, and what properties of the underlying dynamics or data might guide the choice of these constraints for a new task?

Reviewer_bcvjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies zero-support transition prediction, where a model must predict a state change conditioned on a source state and the source and target conditions without examples of that particular transition. The paper proposes Mechanism-Preserving Transport (MPT), a neural ODE whose vector field combines shared state-dependent generators using the difference between condition codes. This construction enforces reversal and same-direction composition for the exact flow. Architectural controls relax these constraints separately to investigate their effects on prediction accuracy.

Critical strengths
  • A useful architectural comparison of structural constraints. The paper extensively tests structural constraints and distinguishes autonomy, reversal and linear control scaling, while query information remains fixed (Table 1). This provides a clear framework for investigating which relations between predictions improve unseen-transition accuracy. The authors also report cases where MPT is outperformed, including Shapes3D and Norman, alongside the absence of a detected leakage advantage in the fit-matched dSprites comparison. This balanced reporting strengthens the paper’s contribution as an investigation of when these constraints are useful.

  • A clear distinction between internal agreement and predictive accuracy. The identity baseline demonstrates that path disagreement and leakage can both be zero without correctly predicting nonzero changes. The direct-versus-routed comparisons further show that stronger agreement between predictions need not imply lower endpoint error. This is a useful methodological contribution because it clarifies why consistency metrics alone cannot establish accurate transport or mechanism recovery.

Critical weaknesses
  • The experiments do not fully isolate the effect of linear control scaling. On Shapes3D, the headline comparison between MPT and autonomous-odd uses different optimization protocols. MPT reuses historical fits with a fixed learning rate of 3×10−43\times10^{-4}, whereas the autonomous controls receive a learning-rate search and select 10−310^{-3} in every run. Gradient clipping, initialization, and sampling procedures also differ (Appendix B.9). Consequently, the reported 59.5% improvement cannot be attributed solely to relaxing linear scaling. The authors acknowledge this limitation, but a comparison under matched optimization and model-selection procedures is important because this attribution is central to the paper.

  • The dependence of the conclusions on control coverage and parameterization remains insufficiently characterized. The positive PDE results concern restricted control regimes: advection observes a single positive control value, while Burgers tests control magnitudes beyond the training range. On Shapes3D, MPT’s aggregate advantage over Zero-NODE is driven by reverse queries; whereas Zero-NODE is more accurate on the reported forward queries. These are legitimate tests of unseen-transition prediction, but they leave unclear how the relative benefits of reversal and linear scaling change as the observed controls become more diverse.

Questions for authors
  • Does autonomous-odd’s advantage on Shapes3D persist when MPT receives the same learning-rate search, gradient clipping, and validation-based selection protocol? For Burgers, how stable is the ranking across additional seeds and longer training, given that the autonomous controls’ validation errors are still decreasing at the reported budget limit? These checks would help determine whether the contrasting rankings reflect the imposed constraints or differences in optimization.

  • Can the existing validation data identify which model family performs best on each task? Specifically, if you select the family with the lowest validation error, which family is chosen, and what is its test error? This would help assess whether validation performance provides useful guidance for choosing the structural constraints.

4 comments
AuthorsOpenReview ↗

Response to Reviewer bcvj

Thank you for reviewing our paper and for pointing out that the training protocols for the original Shapes3D comparison were not the same. The 59.5% difference came from comparing two existing sets of runs. To see whether Odd still had an advantage after relaxing linear scaling, we needed to give MPT the same opportunity for tuning and model selection. We redid the comparison in this way. We also tried your suggestion of choosing the model family by validation error.

1. Does Odd still win on Shapes3D under the same training protocol? Is the Burgers ranking stable with more seeds and a longer training budget?

On Shapes3D, we used the same budget of 12,000 updates for both models. For each of ten new seeds, MPT and Autonomous-Odd used the same frozen representation. We also used the same initialization rules, sampling procedure, gradient clipping, three candidate learning rates, and validation procedure. Both models selected the last checkpoint in every run. On the held-out transitions, mean latent MSE was 0.001043 for Odd and 0.004615 for MPT. Odd’s error was 77.4% lower on average, and it had lower error in all ten pairs. When we decoded the predictions back to images, the MSEs were 0.001271 and 0.002431, respectively. Odd had lower image error in nine of the ten pairs. So the Shapes3D result still holds when MPT gets the same tuning budget as Odd.

For Burgers, we used ten paired seeds. The viscosity grid was built from the public files, with 0.020 in place of 0.008 in the original Table 2 grid. Our question here was whether doubling the budget on the same grid would change the ranking. At 24,000 updates, validation error was still decreasing in eight of ten runs for each model. At 12,000 updates, mean direct MSE was 2.123 for MPT, 2.631 for Odd, and 2.878 for Auto-Zero, in units of 10⁻⁴. At 24,000 updates, the values were 1.979, 3.684, and 3.099 in the same order. At both budgets, MPT had lower direct error than either control in all ten paired runs.

2. What changes when the models see a wider range of controls during training?

As you pointed out, MPT’s advantage over Zero-NODE on Shapes3D comes mainly from the reverse queries. This is a different comparison from MPT versus Odd, where both models keep the inverse relation. To look at control coverage directly, we used the two-parameter Gray–Scott system. We kept the parameter coordinates, training budget, and 36 joint test queries fixed. One training set had 24 adjacent single-axis edges. In the other, we added longer single-axis changes, giving 36 training edges in total. Neither training set contained joint changes.

With the longer shifts included, Odd’s endpoint MSE went from 0.10435 to 0.01531, and Auto-Zero’s from 0.04289 to 0.01191. MPT still had the lowest error, but its own MSE went up slightly, from 0.00516 to 0.00635. In this experiment, MPT had its largest advantage when training covered only local shifts. The longer single-axis examples helped Odd and Auto-Zero much more than they helped MPT.

3. If we choose the model using validation error alone, which model is selected and what is its test error?

In each run, we first chose the family with the lowest validation error. We then evaluated that model on the test transitions. The results below use each task’s primary test metric.

TaskFamilies chosen by validationMean test MSE after selection
Shapes3DOdd 10 times0.001043
Burgers, 12,000 updatesMPT 6 times, Auto-Zero 3 times, Odd once0.0002237
Burgers, 24,000 updatesMPT 9 times, Odd once0.0002008
NormanOdd 6 times, Auto-Zero 5 times, Zero-NODE 4 times0.06160

On Shapes3D, validation chose Odd in all ten runs, and Odd was also better on the test data in all ten. On Norman, choosing by latent condition-mean error gave a test DE20 MSE of 0.06160. If we used Zero-NODE every time, it was 0.05870. Here the two metrics look at different things: validation measures the whole latent mean, while DE20 measures only the twenty genes with the strongest responses. For this task, the model with the lowest validation error was not always the one we wanted for DE20. The full results are in Appendix D.

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you.

Thank you for the thorough response and for running the comparisons I asked for. Under a matched training and selection protocol, Autonomous-Odd still outperforms MPT on Shapes3D in all ten paired runs, and MPT outperforms both autonomous controls on Burgers in all ten paired seeds at both budgets. Together with the validation-based selection results and the Gray–Scott coverage experiment, this addresses the two weaknesses I raised, so I am raising my score to 3.

Dear Reviewer bcvj,

Thank you very much for your time, your constructive feedback, and for updating your score. We are pleased to see that we have successfully addressed your concerns. Your rigorous suggestions have greatly strengthened the empirical evaluation of our paper. Thank you again for your support!

Best regards, The Authors

Reviewer_Mja8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Mechanism-Preserving Transport (MPT) that allows learning transitions between states given a source and the set of conditions. It does so by composing a state-dependent vector operating over a graph where nodes are the condition allowing forward and reverse transport. The claims are validated through experiments on PDEs, Shapes3D images, and single-cell Norman Perturb-seq dataset.

Critical strengths

Problem presentation: The problem is well presented and the example of using 0 ->1 -> 2 transitions in the introduction to construct reversed and scaled jumps is easy to understand

Scope of experiments: The method is tested on several domains and tasks including PDEs, images and single-cell data with best results on images and cells, authors investigate what choices matter the most for each task.

Critical weaknesses

Reversibility guarantee: It is slightly unclear to me how reversal is achieved since architectures such as flow matching models are not invertible (see question below). It would be great to add analysis if states are recovered exactly by doing backward steps.

Scalability: Since the method uses a graph to represent state conditions, it would be great to understand scalability limitations if there are any.

Questions for authors
  • How is reversal guaranteed? For example flow matching models that also use Neural ODE to learn velocity field are not invertible

  • Could you provide some analysis on computational budget across experiments and to what extent the method can scale?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Mja8

Thank you for reviewing our paper and raising questions about reversibility and computational cost. We have revised Section 3 to explain why swapping the two conditions reverses the flow. We also added two measurements in Appendix D: how closely the numerical solver returns to the starting point, and how node count and generator count affect the time required for direct prediction.

1. In what sense is reversibility guaranteed? Can the numerical prediction return to the initial state?

Our guarantee applies only to the exact flow in the state space used by MPT. Taking the prediction from condition 0 to condition 1 as an example, integrating the fitted dynamics backward and asking a conditional model “what happens from 1 to 0?” are generally not the same thing. In MPT, swapping the conditions negates the code difference. Since the shared fields are weighted linearly by this difference, the reverse query uses the negative of the forward velocity at every state. If the forward trajectory is z(t)z(t), then the reverse trajectory starting at the predicted endpoint is z(1−t)z(1-t). As long as the required trajectories exist and are unique, the exact flow returns to the starting point. In the Shapes3D experiment, this starting point is the encoded state, not the original image.

RK4 can only approximate this return, so we measured the error on held-out Shapes3D transitions. With 2, 8, and 32 RK4 steps per direction, the latent-space round-trip RMSEs were 2.27 × 10⁻⁴, 2.51 × 10⁻⁷, and 1.61 × 10⁻⁸, respectively. Here, we first took the median within each seed, then averaged over ten seeds. With eight steps per direction, the model’s forward RMS displacement was 0.118, compared with 0.109 in the data. That is, the model moved a distance comparable to the observed change, and the reverse query brought it back close to the starting point. We evaluated whether the forward prediction reached the correct target separately, using the measured endpoint.

2. What computational budget was used? What costs increase as the graph grows larger?

The table below shows the budgets for the additional training comparisons. One training run at one learning rate counts as one candidate; these candidates are not independent statistical replicates.

ExperimentTraining candidatesUpdates per candidate
Shapes3D, matched optimization settings6012,000
Burgers, training budget comparison90 per budget12,000 or 24,000
Norman, mean matching1206,000
Gray–Scott, two control-coverage settings60 in total3,000

For inference, we compared direct and routed prediction. Direct prediction runs the solver once; routed prediction runs it once for each edge along the path. On the same four Burgers queries, MPT took 8.01 ms per batch for direct prediction and 23.92 ms per batch for routed prediction, with nearly identical endpoint MSE (batch size 128, eight RK4 steps per solve, three seeds; Appendix B.16).

We also used the Burgers network architecture on an RTX 5090 to vary the node count and generator count separately, keeping the solver fixed at eight RK4 steps. The inputs and node codes were already on the GPU. We measured each setting in five rounds, with 30 timed calls per round. We first took the median within each round, then the median of the five rounds. With four generators fixed, increasing the number of nodes from 32 to 16,384 changed the time per single query from 6.769 ms to 6.780 ms. With a batch size of 128, the corresponding batch times were 7.750 ms and 7.932 ms. The FP32 node-code table grew from 512 bytes to 256 KiB. Even with 16,384 nodes in the graph, a query from A to B reads only the codes for A and B.

Generators differ from nodes: their fields are evaluated and combined at each solver step. With the node count fixed at 2,048 and the generator count set to 1, 2, 4, 8, and 16, single-query latency was 8.745, 6.703, 6.770, 6.784, and 6.833 ms, respectively. We found that within this test range, latency did not increase in proportion to the number of generators. For a query with given source and target conditions, the main cost of adding nodes was storing more codes. Following a longer path, by contrast, required more solver calls.

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you.

Thank you for providing clarifications regarding reversibility as well as computational budget. These have satisfied points raised in the review and therefore I will be raising my score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions

Revised Draft and Updated Experimental Code

We thank the three reviewers for their comments on our paper. We have uploaded the revised paper and the updated experimental code package. Changes in the paper are highlighted in blue. The main text uses the same 0 → 1 → 2 example to illustrate the difference between direct prediction and stepwise prediction along an observed path. We have also explained why exchanging the source and target conditions reverses the exact MPT flow, and how closely the numerical solver returns to the starting point in practice. We also changed some examples and transition sentences to make the article more readable.

Appendix D summarizes the experiments added in response to the reviews:

  • Shapes3D and Burgers. With the same hyperparameter tuning and model selection procedure, Autonomous-Odd had lower latent-space MSE than MPT on held-out Shapes3D transitions in all ten new paired runs. On the Burgers viscosity grid constructed from publicly available data files, MPT had lower direct prediction MSE than either autonomous control in all ten paired runs, at both 12,000 and 24,000 training updates. We also report which model family was selected by validation error alone and its test error after selection.

  • Norman Perturb-seq. We added Autonomous-Zero and Autonomous-Odd to the comparison, using the same condition-mean objective and training budget. Each model had lower DE20 MSE than MPT on four of the five condition splits. Since Odd retains the inverse relation, the difference cannot be explained by the reversal constraint alone.

  • Two-parameter variation. In the independently simulated Gray–Scott system, each training transition changed one parameter at a time, while each test transition changed both parameters together. Compared with predicting zero interaction, MPT’s interaction MSE was 91.1% lower when training used only adjacent single-axis changes. With longer single-axis changes included in training, it was 88.1% lower than the same baseline.

  • Reversibility and inference overhead. We measured round-trip error on held-out Shapes3D transitions. We also timed direct queries, varying the node count and generator count separately. With four generators fixed, increasing the number of nodes from 32 to 16,384 changed single-query latency from 6.769 ms to 6.780 ms on one RTX 5090. The node-code table grew from 512 bytes to 256 KiB.

The code package retains the originally submitted code and adds the runners, configurations, and result summaries for the new experiments. The README lists the commands for each experiment and explains how to reproduce the paper’s inference timing table from the recorded measurements without using a GPU.

AuthorsOpenReview ↗

Summary of Revisions

We thank the reviewers for their comments and discussion. We have uploaded an updated revised manuscript. Building on the first revision, we have added the explanations from our responses to the paper: Section 3.3 (page 4) further explains the difference between reversal and scaling constraints; Appendix D (page 47) explains how training coverage affects the Gray–Scott results; and page 48 reports a post-hoc analysis of validation-based model selection. Previous revisions and the new additions are marked in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.