acceptodds
Under review as a conference paper at ICLR 2027

Learning to Control on the Fly: Online Meta Flow Matching across Unknown Dynamics

Abstract

The control of complex physical systems is a fundamental challenge in science and engineering, requiring effective finite-horizon planning under unknown and varying dynamics. While deep learning provides a promising framework for optimizing control sequences, most learning-based approaches assume a fixed distribution of system dynamics. This assumption limits their ability to adapt efficiently when system dynamics are either unknown in advance or vary across environments. To address this limitation, we propose OnlineMeta Flow Matching (OMFM), a generative control framework that continuously adapts control generation from streaming interactions without requiring explicit knowledge of governing equations or dynamical parameters. For each encountered environment, OMFM encodes observed trajectories into a shared context that adapts both control generation and candidate evaluation to the underlying dynamics. The generator learns to produce control sequences from a Gibbs-weighted empirical distribution of observed trajectories, while the predictor estimates future state evolution for candidate evaluation. During deployment, multiple candidates are generated, evaluated by a learned dynamics predictor, and ranked by their predicted objectives without additional environment interaction. We evaluate OMFM on five diverse control tasks involving dynamical systems of varying complexity. The results indicate that OMFM consistently outperforms strong baselines, achieving 25.01% improvement in control performance and converging significantly faster during online adaptation. Our code is available at https://anonymous.4open.science/r/OMFM-473B

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Physical Systems, Online Meta-Learning, Generative Control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FFmjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Online Meta Flow Matching (OMFM) for control under unknown and varying dynamics. Environments are presented sequentially, and each environment is characterized by unobserved dynamics parameters. The proposed methodology uses a Transition-Aware Context Encoder (TACE) to map the observed trajectories from an environment into a latent context vector. This context vector conditions both 1) a flow-matching model that generates complete candidate control sequences and 2) a learned neural surrogate of the dynamics.

The control actions generator is trained on a Gibbs-weighted empirical distribution of observed trajectories, in order to assign a larger training weight to lower-cost control sequences. An autoregressive neural surrogate of the dynamics is trained with one-step and recursive multi-step prediction objectives. At inference time the model parameters are frozen and a single context trajectory is encoded. Multiple candidate controls are then generated and their outcomes are recursively predicted by the learned dynamics surrogate and, among those many candidate control, only the candidate with the lowest predicted objective is selected and executed.

This methodology is evaluated on a variety of benchmarks: Pendulum, CartPole, KS1D, 2D Kolmogorov flow, and Smoke2D, including parameter-shifted OOD environments and the paper reports lower control costs than the considered baselines across all main experimental settings.

Critical strengths

First, the empirical analysis is broad and nontrivial since the method is evaluated across both low-dimensional systems and high-dimensional PDE and transport control problems, and the reported improvements are consistent across the main benchmark settings rather than being limited to a single task. Furthermore, the paper also includes useful ablations of the context encoder, replay, Gibbs weighting, and predictor-based candidate ranking. The experimental details are reported with attention to details.

Also, the proposed system is operationally coherent. In particular, adaptation to a new environment requires only a context trajectory and a forward pass through the encoder rather than test-time parameter optimization, while candidate generation and model-based ranking avoid additional physical interactions. It is also a strength that the authors explicitly train the dynamics predictor with recursive multi-step losses rather than only one-step teacher-forced prediction to mitigate autoregressive rollout error accumulation.

Critical weaknesses
  1. The conceptual novelty appears limited, and the current experimental setup does not sufficiently isolate what is new in OMFM

All the main ingredients of this method have close precedents: online/meta-adaptive control across varying environment-dependent dynamics and shared representations (e.g. [1, 2]), latent context inference followed by context-conditioned dynamics prediction (e.g. [3]), generative trajectory/control modeling (e.g. [4] and related diffusion/flow policies), and amortization of finite-horizon predictive control using learned models or policies (e.g. [6], as well as subsequent amortized-MPC work). Reward-/cost-weighted supervised policy fitting is also an established mechanism (see [5]).

The identifiable novelty therefore appears primarily to be the particular composition of these ideas: transition-based context encoding + Gibbs-weighted flow matching + a context-conditioned dynamics surrogate + candidate ranking under sequential replay. I do not currently see sufficient evidence that this composition constitutes a substantial methodological advance rather than a useful engineering synthesis of pre-existing ideas.

Relatedly, I think the paper should position OMFM more explicitly with respect to the amortized predictive-control literature. A substantial line of work replaces or accelerates repeated finite-horizon optimization with learned policies or models. OMFM similarly learns a mapping from system/task information to finite-horizon control sequences in order to avoid iterative optimization at deployment. The paper should therefore clarify what is fundamentally gained by conditional flow matching relative to these approaches, beyond providing a different amortized parameterization of finite-horizon control.

More importantly, the baselines do not cleanly isolate the contribution of flow matching. For example, this work would benefit from ablations with a context-conditioned sequence behavior-cloning model and/or CVAE, or another non-flow sequence generator, given the same cost weighting, the same context representation, the same number of candidate samples, and the same learned-dynamics ranking procedure. A context-conditioned amortized-MPC or differentiable-predictive-control baseline would also be highly informative. Without such controls, it is difficult to determine whether the reported gains come specifically from the proposed flow-matching formulation or primarily from environment conditioning plus model-based candidate selection. This concern is reinforced by the Pendulum ablation, where removing the encoder or candidate ranking is substantially more damaging than removing Gibbs weighting. It would also be helpful to characterize the distribution of generated action candidates.

  1. The paper's central claim of adaptation/generalization to unknown dynamics is supported mainly by relatively local parameter shifts within unchanged dynamics families, while both the empirical and theoretical characterization of adaptation remain limited.

The OOD environments preserve the governing equations, geometry, and qualitative task structure, and generally extend one or a small number of physical parameters only slightly beyond the training support. For example, the Pendulum natural-frequency support is extended from [0.75, 1.25] to adjacent intervals reaching 0.65 and 1.35; KS1D extends κ from [0.15, 0.45] to [0.10, 0.15) ∪ (0.45, 0.50]; and Smoke2D explicitly shifts one hidden parameter by 15% of the width of its training range. These experiments demonstrate local parametric extrapolation, but in my view they do not yet substantiate a broader claim of adaptation to OOD dynamics.

This concern is especially important because control selection relies on recursively rolling out a learned dynamics surrogate over the entire horizon. The authors appropriately include recursive multi-step prediction losses during training, so the issue is not that compounding error is ignored. However, I did not find direct diagnostics of the quantities that matter for control selection under OOD dynamics, such as recursive rollout error versus horizon, predicted-versus-true candidate cost, or candidate-ranking accuracy. Since only the surrogate-selected candidate is executed, ranking fidelity is particularly important.

A further concern is the limited theoretical characterization of the central adaptation problem. This is particularly noticeable relative to related work on online meta-learning and meta-adaptive control. For example, Online Meta-Learning [7] provides regret guarantees under appropriate assumptions, while Shi et al. [1] study Online Meta-Adaptive Control in sequential environments with unknown environment-dependent nonlinear dynamics and develop both learning- and control-theoretic guarantees. Related control-oriented meta-learning work also explicitly considers closed-loop tracking performance [2].

In contrast, the theoretical material in this submission mainly characterizes the conditional flow-matching regression target and the finite-sample loss construction. In particular, the analysis states that the desired environment-conditioned field is recovered when the learned context c preserves the relevant environment-dependent information, but it does not establish when or why the proposed trajectory encoder should satisfy this property, especially when c is inferred from a single trajectory in an unseen environment.

Thus, the analysis effectively assumes the difficult part of the proposed adaptation mechanism: that c is a sufficiently informative representation of the unknown dynamics. I did not find a result relating context quality to dynamics-prediction error, candidate-ranking error, control suboptimality, regret, robustness, or generalization beyond the training dynamics distribution. Given both the paper's framing as an online meta-control method and the stronger theoretical characterization available in some closely related antecedents, I view this as a meaningful weakness rather than merely an optional extension.

[1] Shi, Guanya, et al. "Meta-adaptive nonlinear control: Theory and algorithms." Advances in Neural Information Processing Systems 34 (2021): 10013-10025. [2] Richards, Spencer M., et al. "Control-oriented meta-learning." The International Journal of Robotics Research 42.10 (2023): 777-797. [3] Lee, Kimin, et al. "Context-aware dynamics model for generalization in model-based reinforcement learning." International Conference on Machine Learning. PMLR, 2020. [4] Wei, Long, et al. "Diffphycon: A generative approach to control complex physical systems." Advances in Neural Information Processing Systems 37 (2024): 4090-4147. [5] Peters, Jan, and Stefan Schaal. "Reinforcement learning by reward-weighted regression for operational space control." Proceedings of the 24th international conference on Machine learning. 2007. [6] Drgoňa, Ján, et al. "Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear systems." Journal of Process Control 116 (2022): 80-92. [7] Finn, Chelsea, et al. "Online meta-learning." International conference on machine learning. PMLR, 2019.

Questions for authors
  1. Novelty and controlled baseline: Can the authors compare OMFM against a non-flow, context-conditioned sequence generator trained using the same context trajectories and cost weighting, with exactly the same number of sampled candidates and exactly the same learned-dynamics ranking procedure? This would help determine whether flow matching itself contributes materially beyond context conditioning + sampling + model-based ranking. A context-conditioned amortized-MPC or differentiable predictive-control baseline would also strengthen the paper.
  2. Surrogate reliability: Can the authors report recursive dynamics-prediction error as a function of rollout horizon, separately for ID and OOD environments, particularly for KS1D/K2D/Smoke2D? More importantly, can they report metrics between predicted and true candidate costs or top-k/ranking accuracy of the surrogate? Since only the surrogate-selected candidate is physically evaluated, prediction accuracy alone is less informative than ranking fidelity.
  3. OOD severity: Can the authors evaluate materially larger or structurally different dynamics shifts, rather than only adjacent parameter-range extensions within the same governing system? Examples could include simultaneous multi-parameter shifts, changed actuator effectiveness/layout, changed forcing/boundary conditions, or substantially farther parameter extrapolation. If performance degrades, a quantitative curve versus distance from the training support would be much more informative than a binary ID/OOD split.
  4. K2D clarification: Appendix C states that OOD environments are divided into viscosity-shift and forcing-amplitude-shift groups and describes these as single-parameter shifts, but it also states that kfk_f is in {3,4} for ID and {2.5,4.5} for OOD. Does every OOD environment therefore also have an OOD kfk_f? If so, these are not single-parameter shifts. Please clarify the construction and report results broken down by the actual shifted parameters.
  5. Theoretical characterization: Several closely related online/meta-adaptive control methods provide regret, convergence, stability, or tracking guarantees, whereas the present analysis appears to characterize flow matching conditional on an already-informative environment representation. Can the authors provide a result that addresses the central adaptation step itself? In particular, under what assumptions does the finite trajectory context yield a sufficient or approximately sufficient representation of the unknown dynamics for control? Can error in c be related to dynamics-prediction error, candidate-ranking error, and/or final control suboptimality? A result of this form, even under a simplified dynamics class, would strengthen the paper.
  6. Role of the formalism: What technical result or capability of OMFM requires the empirical-measure / measure-conditioned formulation of the context in Sec. 3.1? As currently written, it seems that the implemented method could be described equivalently as a finite-set trajectory encoder conditioning a flow model. If the measure-theoretic formulation is important beyond interpretation, could the authors clarify what result depends on it? Otherwise, simplifying this part may make the actual methodology more straightforward.
7 comments
AuthorsOpenReview ↗

Official Response to Reviewer FFmj (1)

Thank you for your careful reading of the paper and your constructive comments. We address each point below.

Q1: The conceptual novelty appears limited.

A1: Thank you for this assessment. We acknowledge precedents in online/meta-adaptive control, context-conditioned dynamics modeling, generative planning, cost-weighted learning, and amortized predictive control. OMFM's methodological contribution lies in coupling environment-conditioned control distribution learning with independent dynamics-based candidate evaluation under sequentially arriving unknown dynamics. The trajectory context jointly conditions generation and evaluation; Gibbs weighting shapes candidate quality, while dynamics-based ranking selects controls suitable for the inferred environment.

1. Conditional distribution learning beyond deterministic amortization. Unlike deterministic regression (s0,c)→u1:H(\mathbf{s}_0,\mathbf c)\rightarrow\mathbf u_{1:H}, OMFM learns pθ(u1:H∣s0,c)p_\theta(\mathbf u_{1:H}\mid\mathbf s_0,\mathbf c). For the same initial condition and context, multiple near-optimal strategies Uϵ=U:Jξ(U;g)≤J∗+ϵ\mathcal U_\epsilon=\\{\mathbf U:J_\xi(\mathbf U;\mathbf g)\leq J^*+\epsilon\\} may exist. Regressing distinct modes to one sequence risks mode averaging; furthermore, identical tasks may require different controls under different dynamics. Context conditioning preserves environment-dependent strategies, while within-initial-condition Gibbs weighting favors low-cost solutions without over-weighting easier initial states. Flow matching thus models alternative control strategies rather than only amortizing a single optimizer output. In revised Appendix D.3 (Table 13), OMFM achieves J=0.1358/0.1217/0.1879J=0.1358/0.1217/0.1879 on Pendulum/KS1D(Dissipation)/K2D, compared with 0.1651/0.1436/0.24350.1651/0.1436/0.2435 for Context+FM and 0.1785/0.2102/0.43510.1785/0.2102/0.4351 for Context+FNO under the same one-trajectory context, demonstrating the benefits of OMFM's integrated formulation beyond context conditioning alone.

2. Decoupled generation and dynamics-based evaluation. A context inferred from limited observations need not fully identify unknown dynamics; a single proposal U∗=Gθ(c,s0,ϵ)\mathbf U^*=G_\theta(\mathbf c,\mathbf s_0,\boldsymbol\epsilon) can therefore be sensitive to context uncertainty, OOD shifts, approximation errors, or sampling variability. OMFM samples multiple candidates and uses a separately trained, context-conditioned predictor to estimate trajectories X^(m)\hat{\mathbf X}^{(m)} and costs J^(m)\hat J^{(m)}, selecting the lowest predicted cost. Generation explores alternative strategies without gradients from the predictor; independent evaluation ranks their suitability under the inferred dynamics. Since selection requires relative cost ordering, not exact absolute predictions, it can remain reliable under increased OOD prediction error when rankings are preserved. This differs from directly optimizing one control sequence through a learned model.

3. Sequential online meta-learning and gradient-free adaptation. As environments arrive, OMFM updates its shared encoder, generator, and predictor with bounded replay and no access to future environments. At deployment, one observed trajectory conditions both generation and ranking through forward inference, without gradient-based parameter updates. The framework thereby links cumulative knowledge acquisition to low-overhead adaptation to unseen dynamics.

4. Evidence for complementary components. The revised Appendix D.3 reports:

  • Gibbs weighting (Table 16): Removing it increases JJ by 3.24%/29.91%/12.45% on Pendulum/KS1D(Dissipation)/K2D. Your observation that removing the encoder (+32.62%) or ranking (+21.21%) has larger Pendulum effects (Table 3) reflects distinct roles: context distinguishes dynamics, weighting improves proposals, and ranking selects them. Their ablation magnitudes need not match.
  • Ranking fidelity (Table 18): Despite higher OOD rollout errors (Figure 11), OOD Spearman correlations remain 0.775/0.910/0.764 across Pendulum/KS1D(Dissipation)/K2D. On K2D OOD, predictor selection reduces JJ from 0.2526 (random) to 0.1895.
  • Context mismatch (Table 19): Mismatched contexts increase oracle candidate costs by 20.08% (ID)/16.85% (OOD) and selected costs by 17.05% (ID)/11.96% (OOD), establishing that context affects both proposal quality and final selection.

Together, these controlled analyses support the complementary roles of context-conditioned distribution learning, cost-weighted generation, and predictor-based selection—not novelty claims for their individual ingredients.

AuthorsOpenReview ↗

Official Response to Reviewer FFmj (2)

Q2: Compare OMFM with matched non-flow generators and predictive-control baselines.

A2: Thank you for this suggestion. We added controlled comparisons with non-flow generators, sequence baselines, and differentiable predictive control in the revised Appendix D.3 (Tables 14, 17, and 26). These experiments evaluate the contributions of conditional flow matching and amortized control generation under matched context and evaluation protocols.

1. Matched non-flow generators (Table 17). We replace OMFM's conditional flow matching (CFM) generator with a conditional variational autoencoder (CVAE) or diffusion model (50 DDIM steps), while keeping the same context encoder, Gibbs cost weighting, dynamics predictor, and 16-candidate ranking procedure.

GeneratorPendulum JTime (s)KS1D(Dissipation) JTime (s)K2D JTime (s)
CVAE0.14200.0130.12940.0220.19110.905
Diffusion (50 DDIM)0.15040.14520.27390.16320.41103.544
OMFM (Ours)0.13580.0240.12170.0400.18791.206

OMFM achieves the lowest control cost on all three systems, outperforming both non-flow alternatives under identical conditioning, weighting, and selection mechanisms. It also provides substantially faster inference than diffusion, while CVAE offers a faster but less accurate alternative. These results isolate the contribution of the flow-matching generator beyond context conditioning and model-based ranking.

2. Matched sequence-generation baselines (Table 14). We additionally equip OFM, FMPhyCon, and Sequence BC with the same one-trajectory test context and 16-candidate budget. Under this matched setting, OFM is equivalent to OMFM without Gibbs weighting, directly isolating the contribution of cost-weighted learning. OFM and Sequence BC use predictor-based ranking, whereas FMPhyCon ranks jointly generated state–control trajectories.

MethodPendulumKS1D(Dissipation)K2D
OFM0.14020.15810.2113
FMPhyCon†0.16940.26440.3263
Sequence BC0.17910.14960.2117
OMFM (Ours)0.13580.12170.1879

OMFM achieves the lowest control cost across all three systems, outperforming both non-flow alternatives under identical conditioning, weighting, and selection mechanisms. It offers a favorable accuracy–efficiency trade-off: CFM achieves higher control accuracy with substantially faster inference than diffusion, while outperforming the faster but less accurate CVAE. These results isolate the contribution of flow matching beyond context conditioning and model-based ranking.

3. Context-conditioned differentiable predictive control (Table 26). We introduce CC-DPC, which shares OMFM's TACE encoder, dynamics predictor, and control objective, but directly optimizes 16 independently initialized control sequences through gradients. We evaluate optimization budgets of 1–100 steps.

MethodPendulum JTime (s)KS1D(Dissipation) JTime (s)K2D JTime (s)
CC-DPC-10.34180.0742.04590.2230.59666.940
CC-DPC-100.29090.6301.85011.9550.557860.672
CC-DPC-500.21282.9481.150810.5820.4266449.280
CC-DPC-1000.18465.8510.516619.3970.3296592.246
OMFM (Ours)0.13580.0240.12170.0400.18791.206

OMFM achieves lower control costs and faster inference across all three systems, even compared with 100-step CC-DPC. On KS1D, OMFM achieves J=0.1217J=0.1217 in 0.040 s, versus J=0.5166J=0.5166 in 19.397 s for CC-DPC-100. The higher costs of CC-DPC may reflect the risk of over-optimizing controls against an imperfect learned dynamics model. By avoiding test-time gradient optimization, OMFM reduces computational overhead and exposure to surrogate-model exploitation, demonstrating the accuracy and efficiency of amortized, context-conditioned control generation.

AuthorsOpenReview ↗

Official Response to Reviewer FFmj (3)

Q3: How reliable are recursive dynamics predictions and candidate rankings under ID/OOD shifts?

A3: Thank you for this question. We added recursive prediction and candidate-ranking diagnostics in the revised Appendix D.3 (Figure 11 and Table 18).

1. Recursive prediction error (Figure 11). We measure relative ℓ2\ell_2 errors over the full rollout horizon on KS1D(Dissipation) and K2D (H=20H=20) and Smoke2D Domain (H=64H=64), separately for ID/OOD environments. Predictions recursively use estimated states without ground-truth feedback.

Horizon hh (KS1D/K2D/Smoke2D)KS1D ID / OODK2D ID / OODSmoke2D ID / OOD
5 / 5 / 16 (25%)0.024 / 0.0210.035 / 0.0380.079 / 0.093
10 / 10 / 32 (50%)0.049 / 0.0500.070 / 0.0870.114 / 0.137
15 / 15 / 48 (75%)0.095 / 0.1080.104 / 0.1320.155 / 0.194
20 / 20 / 64 (100%)0.182 / 0.2290.130 / 0.1640.190 / 0.247

Errors accumulate with horizon, with larger OOD errors at longer horizons. The predictor uses both one-step and multi-horizon training losses (Section 3.2; Appendix B.3). Importantly, its role in OMFM is to rank candidate controls rather than provide exact state predictions for execution. Since selection depends on relative predicted costs rather than absolute prediction accuracy, reliable candidate rankings can mitigate the impact of OOD prediction errors on control performance, as further supported by Table 18.

2. Candidate-ranking fidelity (Table 18). For the same 16 candidates, we compare predicted and true simulator costs using Spearman correlation ρ\rho and Recall@3. We also report true costs of predictor selection (JselJ_{\rm sel}), oracle selection (JoracleJ_{\rm oracle}), and random selection (JrandomJ_{\rm random}).

SystemSplitρ\rhoRecall@3JselJ_{\rm sel}JoracleJ_{\rm oracle}JrandomJ_{\rm random}
PendulumID0.8590.8400.12960.12050.1649
OOD0.7750.7600.14210.13060.1775
KS1DID0.9520.9600.07850.07680.1020
OOD0.9100.9200.16500.16430.1857
K2DID0.8540.8800.18640.16200.2463
OOD0.7640.8400.18950.16590.2526
Smoke2DID0.7150.8000.09220.08730.2257
OOD0.6990.6600.11450.10590.3380

Despite increased OOD prediction errors, candidate ranking remains effective, consistently outperforming random selection and approaching oracle performance in several settings. True costs of all candidates are computed only for this diagnostic; deployment selects controls using predicted costs and executes a single physical rollout.

Q4: K2D problem setting clarification.

A4: Thank you for identifying this inconsistency. The original statement kf∈2.5,4.5k_f\in\\{2.5,4.5\\} for standard OOD environments was a reporting error that conflated two separate K2D experiments. We have corrected and clarified both settings in the revised Appendix C.4.

1. Standard ID/OOD evaluation (Tables 1 and 9). The relevant parameter space is (νvis,F0)(\nu_{\rm vis},F_0), with kf∈3,4k_f\in\\{3,4\\} for both training and testing. ID environments use νvis∈[5,8]×10−4\nu_{\rm vis}\in[5,8]\times10^{-4} and F0∈[0.75,1.0]F_0\in[0.75,1.0]. OOD environments are equally divided into two groups:

  • Viscosity shift: νvis∈[4.5,5)×10−4∪(8,8.5]×10−4\nu_{\rm vis}\in[4.5,5)\times10^{-4}\cup(8,8.5]\times10^{-4}, with F0F_0 within its training range.
  • Forcing-amplitude shift: F0∈[0.65,0.75)∪(1.0,1.1]F_0\in[0.65,0.75)\cup(1.0,1.1], with νvis\nu_{\rm vis} within its training range.

Therefore, no standard OOD environment has an out-of-distribution kfk_f. The OOD evaluation extends one parameter at a time beyond the training region in the (νvis,F0)(\nu_{\rm vis},F_0) space, while retaining the same admissible wavenumbers. Table 9 reports aggregate K2D control costs of 0.1864 (ID) and 0.1895 (OOD) for OMFM.

2. Separate parameter-space evaluation (Figure 4(b)). This experiment uses independently trained models with continuous training distributions F0∼U(0.75,1.0)F_0\sim\mathcal U(0.75,1.0) and kf∼U(3,4)k_f\sim\mathcal U(3,4), fixing νvis=5×10−4\nu_{\rm vis}=5\times10^{-4}. Evaluation covers a 31×31 grid over F0∈[0.625,1.125]F_0\in[0.625,1.125] and kf∈[2.5,4.5]k_f\in[2.5,4.5], including joint extrapolation in forcing amplitude and wavenumber.

Thus, kf∈[2.5,4.5]k_f\in[2.5,4.5] applies exclusively to the separate parameter-space evaluation, not the standard ID/OOD tests. The revised Appendix C.4 now explicitly distinguishes these experimental protocols.

Reviewer_Lc6ZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Online Meta Flow Matching (OMFM) to address control tasks where the underlying unknown dynamics vary across different environments. OMFM encodes observed trajectories into an environment representation, which conditions both a control generator and a dynamics predictor. The generator learns a Gibbs-weighted distribution that prioritizes low-cost control sequences, while the predictor evaluates and ranks candidate control trajectories. Training is carried out on sequentially incoming environments with bounded replay. At test time, the model can adapt using only a single context trajectory, requiring no parameter updates. The authors validate their method across five dynamical systems and seven distinct control setups.

Critical strengths
  1. The paper targets a relevant problem and puts forward a coherent framework. Learning to generate effective control actions for unfamiliar dynamics from limited observations is a worthwhile research challenge. OMFM combines environment inference, control generation, and predictive evaluation, relying on a shared context representation to drive both candidate control generation and selection.
  2. The design choices are well justified. The TACE module preserves temporal information within individual trajectories and aggregates cross-trajectory information in a permutation-invariant way. Decoupling control generation from dynamics prediction lets the generator focus on producing control sequences, while the predictor handles candidate selection and ranking.
  3. The empirical evaluation is broad. The method is benchmarked against ten baselines, covering both low-dimensional systems and high-dimensional spatiotemporal processes. The experiments also assess performance under distribution shift, partial observations, measurement noise, and computational overhead. OMFM achieves the lowest mean cost across all seven primary test settings.
Critical weaknesses

There is an inconsistency in the K2D out-of-distribution experimental setup. Appendix C.4 (page 25) describes the OOD configurations as single-parameter shifts in viscosity or forcing amplitude. However, the forcing wavenumber is also modified: kf∈3,4k_f\in\\{3,4\\} for in-distribution environments and $k_f \in \{2.5,4.5\} $ for OOD environments. As currently written, multiple parameters are changed at once. The authors should clarify the exact parameter combinations and update the description accordingly.

Equation (38) uses the forcing term F0cos⁡(kfy)F_0\cos(k_f y) defined on a doubly periodic domain. For half-integer wavenumbers, this function is not continuous across the periodic boundary. The authors need to explain how this forcing term is constructed, whether it is periodically extended, and how numerical simulations resolve the resulting discontinuity.

Questions for authors
  1. What exact wavenumbers and parameter combinations were used in the K2D OOD experiments? How does the periodic solver handle forcing with half-integer wavenumbers?
  2. Equation (12) weights trajectories from different task conditions by their absolute costs. How do the authors separate performance improvements attributable to better control policies from those stemming from easier initial conditions? Would additional comparisons with fixed initial conditions or difficulty-normalized costs help disentangle this effect?
4 comments
AuthorsOpenReview ↗

Official Response to Reviewer Lc6Z (1)

Thank you for your positive feedback and for recognizing the significance and potential of our method.

Q1: Clarification of K2D OOD parameter shifts and periodic forcing with half-integer wavenumbers.

A1: Thank you for identifying these inconsistencies. We have corrected the K2D parameter settings and clarified the treatment of noninteger forcing wavenumbers in the revised Appendix C.4. Importantly, the standard ID/OOD evaluation and the separate 31×31 parameter-space evaluation use different forcing configurations.

1. Standard K2D ID/OOD evaluation. The original statement kf∈2.5,4.5k_f\in\\{2.5,4.5\\} for OOD environments was a reporting error that conflated two distinct experiments. In the standard experiments (Tables 1 and 9), kf∈3,4k_f\in\\{3,4\\} for both training and testing, as corrected in Eq. (57). The implementation explicitly requires integer wavenumbers, ensuring that F0cos⁡(kfy)F_0\cos(k_fy) is periodic on [0,2π)2[0,2\pi)^2.

The parameter space is (νvis,F0)(\nu_{\rm vis},F_0), with ID ranges νvis∈[5,8]×10−4\nu_{\rm vis}\in[5,8]\times10^{-4} and F0∈[0.75,1.0]F_0\in[0.75,1.0]. OOD environments are equally divided into:

  • Viscosity shifts: νvis∈[4.5,5)×10−4∪(8,8.5]×10−4\nu_{\rm vis}\in[4.5,5)\times10^{-4}\cup(8,8.5]\times10^{-4}, with F0F_0 in its ID range.
  • Forcing-amplitude shifts: F0∈[0.65,0.75)∪(1.0,1.1]F_0\in[0.65,0.75)\cup(1.0,1.1], with νvis\nu_{\rm vis} in its ID range.

Thus, standard OOD environments involve no noninteger or OOD kfk_f. The periodicity concern does not arise in these experiments.

2. Separate continuous-wavenumber evaluation (Figure 4(b)). The 31×31 experiment uses independently trained models with continuous F0∼U(0.75,1.0)F_0\sim\mathcal U(0.75,1.0) and kf∼U(3,4)k_f\sim\mathcal U(3,4), fixing νvis=5×10−4\nu_{\rm vis}=5\times10^{-4}. Its evaluation grid spans F0∈[0.625,1.125]F_0\in[0.625,1.125] and kf∈[2.5,4.5]k_f\in[2.5,4.5].

For noninteger kfk_f, we do not treat the literal F0cos⁡(kfy)F_0\cos(k_fy) as a periodic Fourier mode. Instead, the forcing is sampled on the 64×64 grid and represented through its discrete Fourier coefficients. The pseudo-spectral solver applies the standard two-thirds-rule mask (∣kx∣,∣ky∣≤21|k_x|,|k_y|\leq21) to the full right-hand side, including forcing. Consequently, the effective numerical forcing is

Fper=Pmask[F0cos⁡(kfy)],F_{\rm per}=P_{\rm mask}[F_0\cos(k_fy)],

where PmaskP_{\rm mask} denotes projection of the sampled field onto the retained integer Fourier modes.

This defines a smooth, band-limited periodic forcing field for any sampled kfk_f. For integer kf=3,4k_f=3,4, the original cosine is already a retained periodic mode. For noninteger kfk_f, the projected forcing contains a mixture of integer Fourier modes rather than a single fractional-wavenumber mode. The solver therefore evolves the projected periodic field, not the literal nonperiodic cosine or its discontinuous periodic extension.

Accordingly, the 31×31 experiment evaluates generalization to continuous variations in forcing amplitude and projected spatial structure, rather than to nonperiodic forcing. The revised Appendix C.4 (“Parameter-space evaluation”) distinguishes this diagnostic from the standard K2D evaluation and specifies the Fourier-projection procedure.

AuthorsOpenReview ↗

Official Response to Reviewer Lc6Z (2)

Q2: Does Gibbs weighting favor easier initial conditions rather than genuinely better control policies?

A2: Thank you for this insightful observation. We investigated this concern and found that global absolute-cost Gibbs weighting indeed favors easier initial conditions (ICs). We therefore replaced the original formulation with within-IC Gibbs weighting (Fixed-IC) throughout OMFM and added controlled analyses in the revised Section 3.2 (Eq. 10) and Appendix D.3 (Tables 15–16).

1. Correcting initial-condition bias. The original formulation (Eq. 12) normalized Gibbs weights globally across trajectories with different ICs:

w~i,jβ=exp⁡[−β(Ji,j−Ji,min⁡train)]∑j′∈Aiexp⁡[−β(Ji,j′−Ji,min⁡train)].\widetilde{w}_{i,j}^{\beta} = \frac{\exp[-\beta(J_{i,j}-J_{i,\min}^{\mathrm{train}})]}{\sum_{j'\in\mathcal{A}_i}\exp[-\beta(J_{i,j'}-J_{i,\min}^{\mathrm{train}})]}.

Because absolute costs depend on IC difficulty, this formulation may favor easier initial conditions rather than better controls for a given IC.

The revised Eq. (10) groups trajectories by IC and applies Gibbs weighting within each group:

w~i,jβ=1Giexp⁡[−β(Ji,j−Ji,r,min⁡train)]∑j′∈Ai,rexp⁡[−β(Ji,j′−Ji,r,min⁡train)],j∈Ai,r.\widetilde{w}_{i,j}^{\beta} = \frac{1}{G_i}\frac{\exp[-\beta(J_{i,j}-J_{i,r,\min}^{\mathrm{train}})]}{\sum_{j'\in\mathcal{A}_{i,r}}\exp[-\beta(J_{i,j'}-J_{i,r,\min}^{\mathrm{train}})]},\quad j\in\mathcal{A}_{i,r}.

Here, GiG_i is the number of distinct ICs, and Ai,r\mathcal A_{i,r} contains trajectories sharing IC rr. Each IC receives equal total weight 1/Gi1/G_i, while lower-cost controls within the same IC are favored. This separates control quality from differences in initial-condition difficulty without changing OMFM's architecture or deployment procedure.

2. Performance across all seven control settings. Comparing the original absolute-cost formulation with the revised Fixed-IC formulation yields:

SettingFixed-IC JJAbsolute JJΔJ\Delta J
Pendulum0.13580.1366+0.59%
CartPole10.261510.3238+0.61%
KS1D Dissipation0.12170.1282+5.34%
KS1D Spectrum0.03230.0324+0.31%
K2D0.18790.1881+0.11%
Smoke2D Boundary0.19100.1915+0.26%
Smoke2D Domain0.10340.1036+0.19%

Fixed-IC achieves lower control costs in all seven settings, with the largest reduction on KS1D Dissipation (5.34%). The revised main results use Fixed-IC weighting.

3. Initial-condition bias and difficulty-stratified evaluation (Table 15). We measure IC difficulty using its zero-control cost Ji0J_i^0 and examine the total absolute Gibbs weight WiW_i assigned to each IC.

Datasetρ(Wi,Ji0)\rho(W_i,J_i^0)Easy/HardEasy ΔJ\Delta JMedium ΔJ\Delta JHard ΔJ\Delta J
Pendulum−0.7305.50×+0.66%+0.13%+0.89%
KS1D Dissipation−0.6401.39×+4.80%+4.96%+5.89%
K2D−0.6911.44×−0.69%+0.33%+0.68%

The negative Spearman correlations confirm that absolute-cost weighting systematically assigns greater weight to easier ICs. On Pendulum, easy ICs receive 5.50 times the aggregate weight of hard ICs. Difficulty-stratified evaluation further shows that Fixed-IC achieves lower control costs in eight of nine groups. All variants are evaluated on identical ICs; positive ΔJ\Delta J indicates higher cost under Absolute weighting.

4. Comparison with difficulty-normalized costs (Table 16). We additionally compare Fixed-IC with Absolute, zero-control-cost Normalized weighting, and uniform weighting without Gibbs. Normalized weighting uses Jˉi,j=(Ji,j−Ji,r0)/σtrain\bar J_{i,j}=(J_{i,j}-J^0_{i,r})/\sigma_{\rm train}, with a shared training-set interquartile-range scale.

DatasetAbsolute ΔJ\Delta JNormalized ΔJ\Delta Jw/o Gibbs ΔJ\Delta J
Pendulum+0.59%0.00%+3.24%
KS1D Dissipation+5.34%+10.60%+29.91%
K2D+0.11%+2.77%+12.45%

All differences are relative to Fixed-IC, with positive values indicating higher costs. Cost normalization alone does not enforce equal IC-level training weights: Normalized matches Fixed-IC on Pendulum but yields higher costs on KS1D and K2D. Removing Gibbs weighting produces substantially larger degradations, confirming the importance of retaining cost-based preferences within each IC.

These results directly validate the concern and motivate the revised formulation: Fixed-IC removes unequal aggregate weighting across initial conditions while preserving preference for better controls under the same initial condition.

Thank you for the comprehensive reply. Your clarification on the K2D setup and periodic forcing has addressed my concerns. I am grateful for the revised within-IC Gibbs weighting as well as the newly added controlled comparisons. Such revisions enhance the reliability of this work, and my final recommendation stays unchanged.

AuthorsOpenReview ↗

Appreciation for the constructive comments

We truly appreciate your valuable feedback, which has helped us strengthen our work. Thank you for your time and effort!

Reviewer_MppTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes OMFM, a generative controller for finite-horizon control under unknown dynamics that change between environments. Within each system, environments share the same task and objective but differ in a few hidden physical parameters. The model trains on a stream of such environments with a replay buffer of up to 256 past environments. It has three parts: 1) a context encoder (TACE) maps a few trajectories to an environment embedding, 2) a flow-matching generator, conditioned on this embedding and the task, outputs full control sequences, and 3) a context-conditioned dynamics predictor predicts state rollouts. At test time, the model encodes one trajectory from the new environment, samples 16 candidate control sequences, ranks them by predicted cost, and executes the best one open-loop. The authors evaluate on five systems, from low-dimensional nonlinear systems to high-dimensional PDE control tasks, with in-distribution and out-of-distribution parameter splits. OMFM reports the lowest control cost in all settings, improving 11.7%–43.2% over the best baseline. It keeps this advantage under partial observation and noise, and ablations attribute most of the gain to the context encoder and candidate ranking.

Critical strengths
  • The paper is easy to ready and understand.

  • The deployment recipe is simple and practical. It encodes one trajectory, samples candidates, ranks them with a learned model, and needs no gradient updates or extra environment interaction at test time. The ablations show that both the context encoder and the ranking step matter, which gives a useful data point that this combination carries over to PDE control.

  • The evaluation is broad. It covers five systems, from low-dimensional nonlinear control to high-dimensional fluid and PDE control with thousands of control dimensions. It also tests out-of-distribution parameters, partial observation, and noise. This breadth makes the results more than a single-benchmark effect.

Critical weaknesses
  • The novelty is limited. Context encoders, cost-weighted generative imitation, and model-based candidate ranking all exist in prior work. However, to be fair, their combination is novel to the best of my knowledge. The online setting could be a contribution, but the paper does not show it matters. The stream has no distribution shift, removing replay costs only 4%, and there is no comparison against offline training on all environments at once. The paper also never adapts online at test time: the weights stay frozen and control runs open-loop.
  • Baselines do not test adaptation. Most baselines get no context trajectory, so the main table mostly compares the use of context versus the lack of it. This does not show that OMFM adapts better than other adaptive methods, but it shows that adaptation helps and OMFM indeed adapts.
  • The comparison is uneven. Namely, OMFM selects the best of 16 candidates with a learned model, while the generative baselines return a single sample. Without ranking, OMFM is worse than FTML on Pendulum (0.1651 vs. 0.1547), and FTML also runs without its test-time adaptation step.
  • The paper misses some important baselines, most importantly comparisons to adaptive methods such as MetaDiffuser, CaDM with MPC, MBOP, or context-based offline meta-RL such as CORRO or UNICORN.
Questions for authors
  • If every generative baseline (OFM, FMPhyCon, Sequence BC) gets the same best-of-16 ranking with a learned dynamics model, and the strongest baselines get the same one-trajectory context, does OMFM keep its advantage?
  • How does OMFM compare with methods that explicitly target online adaptation (I suggest comparing under the same one-trajectory adaptation protocol)?
  • How does OMFM trained online compare with the same model trained offline on all environments at once? If the two perform about the same, what does the online formulation contribute?
6 comments
AuthorsOpenReview ↗

Official Response to Reviewer MppT (1)

Thank you for your constructive comments. We address each point below.

Q1: Limited methodological novelty beyond combining existing components.

A1: Thank you for this insightful assessment. We agree that context encoding, cost-weighted generative learning, and model-based candidate ranking have been studied individually. OMFM addresses the coupled challenges of learning transferable control distributions and selecting reliable controls under unknown dynamics and sequentially arriving environments through a unified, context-driven mechanism.

1. Conditional distribution learning. Unlike deterministic control regression, which learns a mapping (s0,c)⟶u1:H(\mathbf{s}_0,\mathbf{c}) \longrightarrow \mathbf{u}_{1:H} from context and initial conditions to a single control sequence, OMFM learns a conditional distribution pθ(u1:H∣s0,c)p_\theta\left(\mathbf{u}_{1:H}\mid\mathbf{s}_0,\mathbf{c}\right) over control sequences. For the same initial condition and environment context, multiple near-optimal control strategies \mathcal U_\epsilon=\left\\{\mathbf U:J_\xi(\mathbf U;\mathbf g)\leq J^*+\epsilon\right\\} may exist. A deterministic regressor may suffer from mode averaging when trained on different control modes, potentially producing a sequence that corresponds to none of them. Moreover, different environments may induce distinct dynamics and require different control strategies even under identical initial and task conditions. By conditioning generation on the inferred environment context, OMFM preserves these environment-dependent control strategies rather than averaging them, while Gibbs weighting favors low-cost solutions.

2. Decoupled control generation and independent dynamics-based evaluation. In unknown-dynamics environments, a context inferred from limited trajectories cannot fully identify the underlying dynamics. Consequently, a single generated sequence U∗=Gθ(c,s0,ϵ)\mathbf U^*=G_\theta(\mathbf c,\mathbf s_0,\boldsymbol\epsilon) may be unreliable, particularly under OOD conditions, due to incomplete context, unseen-dynamics distribution shifts, model approximation errors, and sampling variability.

OMFM addresses this issue by generating multiple plausible candidates and using a separately context-conditioned dynamics predictor to estimate their future state trajectories X^(m)\hat{\mathbf X}^{(m)} and corresponding costs J^(m)\hat J^{(m)}, selecting the candidate with the lowest predicted cost. This decoupling allows the generator to explore diverse control strategies without identifying the optimum in a single forward pass, while the predictor independently evaluates their suitability under the inferred dynamics. Although absolute predictions may become less accurate under OOD dynamics, candidate selection relies on relative cost comparisons rather than precise absolute predictions, which can mitigate the impact of prediction errors when relative rankings remain reliable. Thus, candidate exploration and environment-aware evaluation jointly reduce the risk associated with individual generated samples.

3. Sequential online meta-learning and gradient-free context adaptation. OMFM further integrates these mechanisms into a sequential learning framework, incrementally updating the shared encoder, generator, and predictor as new environments arrive, using bounded replay without access to future training environments. At deployment, a single observed trajectory conditions both generation and evaluation through forward inference, without test-time parameter updates. This connects cumulative knowledge acquisition across environments with efficient adaptation to previously unseen dynamics.

4. Additional experimental evidence. The revised manuscript provides further evidence for these complementary mechanisms in Appendix D.3:

  • Balancing Initial Conditions with Gibbs Weighting (Table 16): Removing Gibbs weighting increases control costs by 3.24%, 29.91%, and 12.45% on Pendulum, KS1D(Dissipation), and K2D, respectively, demonstrating its contribution to learning effective control candidates.
  • Surrogate Ranking Fidelity (Table 18): Despite higher OOD rollout prediction errors (Figure 11), the predictor retains strong relative ranking ability, with OOD Spearman correlations of 0.775, 0.910, and 0.764 on the three systems. On K2D OOD, predictor-based selection reduces control cost from 0.2526 (random selection) to 0.1895.
  • Effect of Test-Time Context Mismatch (Table 19): Replacing the correct environment context increases oracle candidate costs by 20.08% (ID) and 16.85% (OOD), and selected costs by 17.05% and 11.96%, confirming that environment-specific context improves both candidate quality and final control performance.

Together, these results support the complementary roles of context-conditioned generation, cost-weighted learning, and predictive selection, rather than attributing OMFM's performance to any single component.

AuthorsOpenReview ↗

Official Response to Reviewer MppT (2)

Q2: The benefits of online learning under distribution shifts and the role of replay.

A2: Thank you for highlighting the importance of distribution drift. The original training stream consists of randomly ordered environments without systematic distribution drift, which may explain the relatively modest benefit of replay. To more directly evaluate replay under non-stationary dynamics, we added three KS1D training streams with gradual, abrupt, and recurring shifts in the dynamics parameter κ\kappa (revised Appendix D.3, “Sequential Online Learning and Replay under Distribution Drift,” Table 23). Each stream contains 3,000 environments and is trained separately with and without replay under identical optimization budgets.

DriftOOD JJ (w/o Replay)OOD JJ (w/ Replay)Forgetting (w/o → w/ Replay)
Gradual0.32780.2953+164.0% → +16.0%
Abrupt0.26210.2228+98.3% → +7.7%
Recurring0.21200.1713+92.6% → +15.8%

Replay consistently reduces OOD control costs and forgetting across all three drift patterns, supporting its role in retaining knowledge under non-stationary dynamics.

Replay is not claimed as a novel component or OMFM's primary contribution; it serves to mitigate forgetting and is applied consistently to all applicable online baselines.

Q3: Does online training offer advantages over offline training?

A3: Thank you for this important question. Online learning addresses practical settings where environments arrive sequentially without access to future data, such as tokamak plasma control across discharges [1], incremental plasma trajectory optimization [2], and autonomous underwater vehicle control under time-varying dynamics [3]. These applications motivate accumulating transferable control knowledge from successive runs, as further emphasized in the revised Introduction (Section 1, first paragraph).

We added a matched online/offline comparison (Appendix D.3, “Sequential Online Learning and Replay under Distribution Drift,” Table 22). Both variants use identical architectures, training environments, and optimization budgets. Offline training accesses all environments simultaneously; online training processes them sequentially with a replay buffer of at most 256 environments.

Control settingOnlineOffline
Pendulum0.13580.1351
CartPole10.261510.0324
KS1D Dissipation0.12170.1297
KS1D Spectrum0.03230.0331
K2D0.18790.1812
Smoke2D Boundary0.19100.1884
Smoke2D Domain0.10340.1044

Online OMFM achieves comparable performance across all seven settings and outperforms offline training in three, including a 6.17% lower cost on KS1D Dissipation. These results demonstrate its ability to maintain competitive control performance under sequential data access and bounded memory, without requiring all environments in advance.

References

[1] Degrave, Jonas, et al. "Magnetic control of tokamak plasmas through deep reinforcement learning." Nature 602.7897 (2022): 414–419.

[2] Wang, Allen M., et al. "Learning plasma dynamics and robust rampdown trajectories with predict-first experiments at TCV." Nature Communications 16.1 (2025): 8877.

[3] Jiang, Peng, Shiji Song, and Gao Huang. "Attention-based meta-reinforcement learning for tracking control of AUV with time-varying dynamics." IEEE Transactions on Neural Networks and Learning Systems 33.11 (2022): 6388–6401.

Q4: No test-time parameter updates or closed-loop control during deployment.

A4: Thank you for this important point. OMFM distinguishes sequential online meta-training from test-time context adaptation. At deployment, a single observed trajectory conditions both control generation and dynamics evaluation, enabling adaptation to unseen dynamics without parameter updates.

1. Efficient gradient-free adaptation. Avoiding test-time gradient updates reduces computational overhead and mitigates overfitting risk when only one context trajectory is available. In the revised Appendix D.3 (Tables 25–26), FTML-Policy's Pendulum cost rises from 0.1547 (0 steps) to 0.2075 (5 steps), whereas OMFM achieves 0.1358 without updates. CC-DPC shares OMFM's encoder, predictor, and objective but gradient-optimizes 16 controls. On KS1D(Dissipation), OMFM achieves J=0.1217J=0.1217 in 0.040 s, versus J=0.5166J=0.5166 in 19.397 s for CC-DPC (100 steps), demonstrating accurate, efficient amortized adaptation.

2. Open-loop setting. This work focuses on finite-horizon open-loop trajectory optimization, in which complete control sequences are selected before physical execution. Extending OMFM to closed-loop control with online feedback and replanning is an important direction for future work.

AuthorsOpenReview ↗

Official Response to Reviewer MppT (3)

Q5: Fair comparison under identical context and best-of-16 candidate selection.

A5: Thank you for the suggestion. We added controlled comparisons under matched context and candidate budgets. All methods follow the same sequential training stream, bounded replay protocol, and one physical rollout per control instance.

1. Matched context and candidate ranking. We equip OFM, FMPhyCon, and Sequence BC with the same one-trajectory test context and 16-candidate selection budget as OMFM. OFM and Sequence BC use learned dynamics-based ranking, while FMPhyCon ranks its jointly generated state–control trajectories. The results are reported in the revised Appendix D.3 (“Matched Context and Candidate Selection,” Table 14).

MethodPendulumKS1D(Dissipation)K2D
OFM0.14020.15810.2113
FMPhyCon†0.16940.26440.3263
Sequence BC0.17910.14960.2117
OMFM (Ours)0.13580.12170.1879

OMFM achieves the lowest cost across all three systems, outperforming the strongest competing baseline by 3.14%, 18.65%, and 11.07%, respectively. Notably, OFM corresponds to OMFM without Gibbs weighting under the matched setting, directly isolating the contribution of cost-weighted generative learning.

2. Matched test-time gradient adaptation. We further evaluate FTML-Policy with 0, 1, 5, and 10 gradient updates using the same single context trajectory per test environment (Appendix D.3, “Gradient-Based Test-Time Adaptation and Control Optimization,” Table 25).

MethodPendulumCartPoleKS1D(Dissipation)
FTML (0 step)0.154714.76740.2834
FTML (1 step)0.164620.82200.3418
FTML (5 steps)0.207581.30930.4097
FTML (10 steps)0.458173.85180.4228
OMFM (Ours)0.135810.26150.1217

OMFM outperforms FTML at every evaluated update budget. FTML's best result uses zero updates, while additional gradient steps degrade its performance, potentially reflecting overfitting to the limited information provided by a single context trajectory. In contrast, OMFM adapts through context-conditioned inference without test-time gradient updates, avoiding this potential overfitting risk and its associated computational overhead. Together, these results demonstrate OMFM's advantage under matched context availability and test-time adaptation budgets.

Q6: Comparison with established adaptive control and meta-RL methods under the same protocol.

A6: Thank you for the suggestion. We added four adaptive/meta-RL baselines (revised Section 4.1, Appendix D.1):

  • MetaDiffuser: Context-conditioned diffusion planning with objective and dynamics-consistency guidance.
  • CaDM-MPC: Context-aware dynamics prediction and model-based trajectory optimization.
  • C-MBOP: Model-Based Offline Planning augmented with OMFM's TACE encoder.
  • UNICORN: Context-based offline meta-RL with self-supervised task representations.

All methods follow the same sequential training stream, bounded replay protocol, and single-rollout evaluation. Each test environment provides one context trajectory; complete control sequences are constructed before physical execution.

Control cost JJ across seven settings (revised Table 1):

MethodPendulumCartPoleKS1D-DissipationKS1D-SpectrumK2DSmoke-BoundarySmoke-Domain
MetaDiffuser0.403427.07671.05820.13170.6183––
CaDM-MPC0.160521.57751.10020.04790.53510.48450.5081
C-MBOP0.192714.90810.67920.12580.27110.34050.2216
UNICORN0.157513.21230.16580.04530.24320.27150.1684
OMFM (Ours)0.135810.26150.12170.03230.18790.19100.1034

OMFM achieves the lowest control cost in all seven settings, improving by 12.22%–35.42% over the strongest baseline in Table 1. We have also updated Figures 3 and 4(a) to incorporate additional baseline comparisons in the trajectory visualizations and online learning curves, respectively. Additional ID/OOD results (Appendix D.3) further support its generalization across varying dynamics. Together, these comparisons demonstrate the effectiveness of context-conditioned control distribution learning and independent candidate evaluation under a shared deployment protocol.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback. All three recognize OMFM's coherent, practical framework and broad evaluation across low-dimensional and high-dimensional PDE control tasks. They also appreciate the clear presentation and informative ablations (Reviewer MppT), well-motivated TACE and decoupled design (Reviewer Lc6z), and recursive multi-step prediction and detailed experiments (Reviewer FFmj). Efficient gradient-free deployment is also highlighted (Reviewers MppT and FFmj). In response, we added extensive experiments and baselines, refined Gibbs weighting, and established new theoretical guarantees. All revisions are highlighted in blue. The major additional experiments and improvements are as follows.

  1. Methodological novelty and component contributions. We clarify OMFM's contribution: integrating context-conditioned control distribution learning, cost-weighted generation, and independent dynamics-based candidate ranking under sequentially arriving unknown dynamics. Unlike deterministic regression, OMFM generates multiple candidate controls. New ablations and context-mismatch analyses validate the complementary components (Section 3; Appendix D.3, Tables 13, 16, 18–19).

  2. Matched generative and predictive-control baselines. We add controlled comparisons with CVAE, diffusion, OFM, FMPhyCon, and Sequence BC, aligning context and candidate budgets. We also introduce CC-DPC, which directly optimizes controls through the learned predictor, and evaluate FTML under different test-time update budgets. OMFM achieves lower costs with faster inference than diffusion and gradient-based predictive optimization (Appendix D.3, Tables 14, 17, 25–26).

  3. Additional adaptive-control and meta-RL baselines. We include MetaDiffuser, CaDM-MPC, C-MBOP, and UNICORN under the shared training and evaluation protocol. OMFM achieves the lowest costs across all seven settings, improving upon the strongest baseline by 12.22%–35.42% (Section 4.1, Table 1; Appendix D.1).

  4. Online learning and distribution drift. We strengthen the motivation for sequential learning and add matched online/offline comparisons. Three non-stationary KS1D streams with gradual, abrupt, and recurring shifts show that replay reduces forgetting and improves OOD performance. Replay is treated as a standard retention mechanism, not a novel component (Section 1; Appendix D.3, Tables 22–23).

  5. Prediction and ranking reliability. We analyze recursive prediction errors across ID/OOD environments and rollout horizons on KS1D, K2D, and Smoke2D. Spearman correlation, Recall@3, and oracle/random comparisons show that candidate ranking remains effective despite accumulating prediction errors (Appendix D.3, Figure 11 and Table 18).

  6. Stronger OOD generalization. We evaluate larger parameter extrapolations on Pendulum and KS1D, a 31×31 K2D joint parameter-space grid, and unseen structural forcing shifts. OMFM consistently achieves the lowest costs across the evaluated shift severities, extending validation beyond adjacent OOD ranges (Section 4.3, Figure 4(b); Appendix D.3, Figure 10 and Table 10).

  7. Finite-context theoretical guarantees. We introduce Theorem 1 (Finite-context control suboptimality), connecting single-trajectory identifiability, context estimation error, candidate generation, predictive ranking, and final control performance under explicit assumptions. Complete proofs are provided in Section 3.3 and Appendix B.5.

  8. Initial-condition-balanced Gibbs weighting. Addressing the identified bias toward easier initial conditions, we replace global absolute-cost weighting with within-IC Gibbs weighting, assigning equal total weight to each IC while favoring better controls within that IC. This improves performance across all seven settings. Additional difficulty-stratified and alternative-weighting experiments validate the correction (Section 3.2, Eq. 10; Appendix D.3, Tables 15–16).

  9. K2D and mathematical clarifications. We correct the standard K2D setting to integer wavenumbers 3 or 4 for both training and testing, with OOD shifts in viscosity or forcing amplitude. The separate continuous-wavenumber experiment uses a band-limited periodic Fourier projection. We also replace the unnecessary empirical-measure formulation with explicit finite-set context encoding (Section 3.1; Appendix B.1 and C.4).

Together, these revisions provide stronger theoretical and empirical support for OMFM's ability to learn transferable control distributions and select environment-adapted controls from limited context, without test-time gradient updates or additional physical interactions for candidate selection. We appreciate the reviewers' valuable suggestions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.