acceptodds
Under review as a conference paper at ICLR 2027

Source-Lifted Flow Matching for Intervenable Multimodal Imitation

Abstract

Flow-matching policies are promising for imitation learning because they model complex multimodal action distributions. However, their stochasticity is largely passive: repeated sampling may yield diverse behaviors, but users cannot directly choose among valid continuations from the same state. We propose Source-Lifted Flow Matching (SL-FM), a sourceintervenable flow-matching policy that exposes such a handle while keeping the velocity field shared and latent-free. The handle selects only the source endpoint of the conditional flow, not a mode-specific field, preserving the standard formulation while avoiding decomposition into separate modeconditioned dynamics. The core mechanism is Orthogonal Source Lifting, designed to prevent path-crossing ambiguity. Instead of partitioning target actions by mode, SL-FM lifts handle-specific sources into auxiliary orthogonal coordinates and keeps targets in the original action subspace. This preserves the demonstrated action distribution while allowing one shared field to carry different branches without merging at crossings. To keep handles usable across states, we learn a state-dependent source mixture end to end and use a responsibility floor, giving each handle weak supervision and mitigating dead modes. Experiments on crossing-flow diagnostics and robot-control benchmarks show that SL-FM converts passive source randomness into an actionable intervention variable. It removes crossing-induced composite trajectories, changes future routes in 91.1% of matched-prefix interventions, and achieves strong free-deployment performance, with improvements in several benchmark settings. Overall, source geometry provides actionable multimodal control without conditioning the velocity field on the selected mode.

Primary area:
interpretability and explainable AI
Keywords:
Flow matching, Multi-modal distribution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JWcnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Source-Lifted Flow Matching for controllable multimodal imitation. It introduces selectable source components and auxiliary coordinates to preserve source identity during generation, allowing behavior to be influenced through the starting point of a shared flow model.

Critical strengths
  • The paper addresses a meaningful problem: generating diverse behaviors does not necessarily make those behaviors easy to select.

  • The proposed source handles offer a potentially useful interface between high-level decisions and continuous action generation, without requiring manual behavior labels.

  • The matched-prefix intervention setup is a useful way to examine whether changing a handle actually changes subsequent behavior.

Critical weaknesses
  • I am not fully convinced why control needs to be restricted to the source. The auxiliary coordinates still carry information about (z), and directly conditioning a shared network on (z) would also be a natural choice. Although this baseline is included, the practical benefit of the source-only restriction remains unclear. The restriction feels somewhat self-imposed, and its added complexity needs a clearer justification.

  • I also wonder whether the learned action groups correspond to meaningful behavior strategies. Differences in local actions do not necessarily translate into consistent long-term intentions. The connection between selecting a source and choosing a coherent strategy needs more discussion.

  • The paper includes ablations, but they could focus more directly on controllability. It would be useful to see how removing lifting or the responsibility floor, or using non-orthogonal tags, affects successful behavior switching. Task success and route entropy alone do not fully establish whether these components preserve branch identity as intended.

Questions for authors

see weaknesses.

2 comments

W1: Why restrict control to the source?

Thank you for your comments!

We designed our method this way and distinguish it from zz-conditioning because the augmentation serves two purposes:

  1. It distinguishes different handles and provides label or conditioning information. As you noted, this role is similar to that of zz.
  2. It provides orthogonal path separation through augmentation, ensuring that paths remain nonintersecting during training and inference. In comparison, zz and other conditioning inputs are used as labels and stay outside the noising and denoising process.

The main benefit is that lifting substantially alleviates velocity conflicts in the flow field. For example, as illustrated in Figure R2-1 and Figure R2-2, provided in our response to Reviewer 2, the distinction between zz-conditioning and SL-FM is whether lifting is used. The zz-conditioned model produces less straight trajectories with greater curvature, whereas SL-FM produces straighter trajectories with fewer conflicts after lifting. This is consistent with the second purpose described above.

Reducing these conflicts lowers the complexity of denoising and enables the model to fit the target distribution more accurately. It also improves the interpretability of the flow field, making its properties easier to analyze.

W2: Do the action groups correspond to meaningful long-term behavioral strategies?

Your question addresses the most important issue in moving from the algorithm to practical downstream applications.

Our initial goal was to observe differences in overall behavior styles. However, as you pointed out, identifying and making use of long-term intentions is a difficult problem. If this can be achieved, FM would be well suited to learning and invoking multiple skills. However, identifying coherent behavioral strategies requires introducing the temporal dimension into clustering. Questions such as how to segment time and identify when to switch handles require further in-depth study. As shown in Figure 7 of our paper, our current solution is straightforward: we specify an execution window and then switch handles to achieve local behavioral changes. However, this approach is highly unreliable for complex tasks.

We are also exploring whether a metric can indicate when the current state permits a skill switch, but we do not yet have a stable method. We will explicitly acknowledge this limitation and discuss it further in the appendix.

W3: Ablations that more directly evaluate controllability

We agree that more direct measures are needed to assess whether the model has learned effective modes. However, as you noted above, task-level evaluation often reflects coherent behavioral strategies.

We primarily compared task success and route entropy in the paper because these are commonly used metrics and allow us to assess whether our method maintains task performance.

We have also been considering the issue you raised. The paper reports route-change rates, both-success rates, and successful route-switching rates under same-prefix interventions (Table 2), as well as mutual information between handles and route choices (Figure 5) and associations between handles and subtasks in Kitchen (Figure 3(b)). In addition, with the low-level policy frozen, a high-level CEM/PPO selector achieves 91.7% joint route-and-task success over 24 target routes from a fixed initial state (Figure 3(c)). These results provide evidence from interventions, correlations, and downstream usability, respectively. Among them, successful route switching under same-prefix interventions most directly addresses your ablation question.

Following your suggestion, we supplemented the experiments with same-prefix intervention evaluations of the three modifications:

  1. Without lifting: We use the SL-FM w/o lift configuration from the paper, retaining the source handle while removing the auxiliary coordinates.
  2. Without the responsibility floor: We set the responsibility floor to zero during training.
  3. Nonorthogonal tags: We retain the lifting coordinates and use preset unit-norm tags with pairwise cosine similarity 0.5. The original method instead learns state-dependent tags regularized toward orthogonal anchors.
ConfigurationRoute-code change (including failures, %)Both-success rate (%)Successful full-route switching (%)
SL-FM91.163.957.8
Without lifting93.863.058.3
Without responsibility floor99.044.844.3
Nonorthogonal tags80.257.840.6

One notable observation is that removing the responsibility floor can produce a very high route-code change rate, meaning that the behavioral mode “certainly” changes, while the both-success rate drops substantially, from 63.9% to 44.8%. The value of the responsibility floor therefore needs to balance behavioral differentiation and accuracy.

We consider the nonorthogonal-tag ablation particularly important. Together with the ablation that removes lifting, it provides two complementary validations of the necessity of orthogonality. We will prioritize adding this ablation to the main text. We sincerely appreciate this suggestion! We surely believe that including this ablation will help address this gap in our evaluation and strengthen the paper.

Reviewer_54tEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses controllable multimodal imitation learning by proposing Source-Lifted Flow Matching (SL-FM). The method introduces a state-conditioned source mixture and orthogonal auxiliary coordinates to distinguish different behavioral modes without explicitly conditioning the velocity field on discrete mode labels. Experiments demonstrate competitive imitation performance and improved controllability through source interventions.

Critical strengths

1.The paper presents an interesting approach to controllable multimodal imitation learning by introducing an intervenable source distribution. 2.The method is clearly presented, and the experiments include useful intervention evaluations and ablation studies.

Critical weaknesses

1.The authors argue against directly conditioning the velocity field on (z), instead embedding mode information into the flow integration process through additional coordinates. However, this is essentially a transformation of the mode variable and noise, which a sufficiently expressive neural network conditioned on both (z) and noise could potentially learn. Therefore, the proposed approach appears more like an inductive bias or regularization mechanism rather than a fundamentally different solution. Could a properly tuned (z)-conditioned baseline achieve comparable performance? In Table 1, the directly conditioned baseline even performs worse than standard Flow Matching, raising concerns about whether this baseline has been sufficiently optimized. A more carefully controlled comparison is needed. 2.The proposed framework is conceptually similar to hierarchical reinforcement learning and latent skill learning, where a high-level latent variable is selected to guide low-level action generation. Therefore, the authors should include comparisons with representative state-of-the-art methods from these fields, particularly latent skill-based imitation learning approaches, to better demonstrate the advantages of the proposed framework. 3.The authors introduce auxiliary orthogonal coordinates to prevent path-crossing ambiguity. However, these coordinates are learned through a soft regularization constraint, which does not guarantee that different behavioral modes remain sufficiently separated. It is unclear whether the learned representations can reliably distinguish different modes, or whether the additional dimensions may negatively affect performance when mode separation is unnecessary. More visualizations and targeted ablation studies are needed to validate the effectiveness and robustness of this design.

Questions for authors

Please refer to the Weaknesses section for my questions and concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 54tE

W1: Comparison with direct conditioning and the advantages of our method

Thank you for your question! This concern is similar to one raised by another reviewer.

We designed our method this way and distinguish it from zz-conditioning because the augmentation serves two purposes:

  1. It distinguishes different handles and provides label or conditioning information. As you noted, this role is similar to that of zz.
  2. It provides orthogonal path separation through augmentation, ensuring that paths remain nonintersecting during training and inference. In comparison, zz and other conditioning inputs are used as labels and stay outside the noising and denoising process.

As you pointed out, the zz-conditioned baseline in our submitted paper strictly followed the hyperparameter settings of SL-FM without sufficient tuning. Following your suggestion, we performed a hyperparameter search and tuning, obtaining a result of 75.44% ± 3.13% for zz-conditioning.

In this task environment, zz-conditioning performs worse than SL-FM. We believe the main difference lies in the flow-field mechanism. During denoising, the zz-conditioned model produces more curved trajectories, which we attribute to velocity averaging caused by crossing paths. SL-FM introduces lifting during denoising and produces comparatively straighter trajectories, suggesting that it is less affected by velocity averaging, as illustrated in Figure R2-1 and Figure R2-2.

This can shorten the paths and increase the distinction between inputs corresponding to different action distributions, making the mapping easier for the model to learn.

W2: Comparison with hierarchical reinforcement learning and latent skill learning

Because our setting does not provide rewards, we cannot directly apply reinforcement learning training procedures. The main difference from earlier established models is that our method builds on Flow Matching and modifies the model through close integration with its denoising process. We also aim to explore the capabilities and interpretability of FM.

We additionally present the official D3IL results below. The methods in our paper use MLP architectures, which leaves a performance gap compared with Transformer-based methods.

SourceMethodAvoiding ↑Aligning ↑PushT* ↑Avg* ↑
Original paperFM0.7980.8080.8010.802
Original paperCPD0.6920.7150.8140.740
Original paperMM-FM0.2250.0000.2790.168
Original paperModal cpl.0.3350.6230.8390.599
Original paperSL-FM0.8250.7920.8520.823
W2QueST0.715 ± 0.1140.588 ± 0.1820.814 ± 0.0040.706 ± 0.027
W2STAR0.385 ± 0.3670.126 ± 0.0440.783 ± 0.0030.432 ± 0.109
W2VFP adaptation0.338 ± 0.4150.059 ± 0.0750.278 ± 0.0310.225 ± 0.155
W2Z-cond0.760 ± 0.0360.750 ± 0.0320.848 ± 0.0150.774 ± 0.025
Official D3ILBC-MLP0.666 ± 0.5120.708 ± 0.052——
Official D3ILVAE-State0.716 ± 0.2650.579 ± 0.138——
Official D3ILBeT-MLP0.665 ± 0.1360.507 ± 0.081——
Official D3ILIBC0.760 ± 0.0460.638 ± 0.027——
Official D3ILDDPM-MLP0.637 ± 0.0550.763 ± 0.039——
Official D3ILVAE-ACT0.851 ± 0.1090.891 ± 0.022——
Official D3ILBC-GPT0.833 ± 0.4080.833 ± 0.043——
Official D3ILBeT0.747 ± 0.0300.645 ± 0.069——
Official D3ILDDPM-GPT0.927 ± 0.0220.839 ± 0.020——
Official D3ILBESO0.950 ± 0.0150.861 ± 0.016——
Official D3ILDDPM-ACT0.899 ± 0.0060.849 ± 0.023——

As shown, our model remains relatively competitive among MLP-based methods. In future work, we also plan to incorporate reinforcement learning to control and select zz online, enabling online changes to the behavior style. As you noted, this follows the idea of hierarchical reinforcement learning. Thank you for drawing our attention to these baselines!

W3: Separation and robustness of orthogonal auxiliary coordinates

Thank you for raising this concern!

Although the coordinates are learned through soft regularization, the training procedure is simple, and we observe very little deviation in the tasks we consider. Based on our preliminary training experience, learned and fixed coordinates perform almost identically.

The measured regularization loss, before multiplication by the regularization weight, is 2.0027×10−52.0027 \times 10^{-5}.

Nevertheless, your concern remains valid. We will add a discussion of the soft regularization constraint to the main text. Alongside the reported loss, we will remind readers that relying on a soft constraint remains a potential limitation.

For a simple downstream problem, such as navigating from the origin to four corners, the separation between regions is highly reliable, as illustrated in the four-mode visualization provided for Reviewer 1 (Figure R1-1). However, when the distribution to be fitted is particularly complex, such as the distribution shown in our response to W1 (Figure R2-1), the separation is not entirely reliable and can only distinguish clearly identifiable clusters.

We believe that the additional dimensions impose a small learning burden. Our zz-conditioned baseline also serves as an ablation without lifting. So far, we have not observed an obvious difference in this learning burden. However, as KK increases, the dimensionality also increases, and this will affect performance. Therefore, KK should not be excessively large.

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to improve the baselines and provide further analysis.

However, my main concern remains whether the proposed source lifting offers a distinct advantage over direct Z-conditioning. A sufficiently expressive Z-conditioned velocity field should also be able to distinguish different modes and avoid cross-mode velocity averaging. Moreover, since SL-FM introduces multiple design components, it remains unclear whether the performance improvements can be specifically attributed to orthogonal lifting rather than other inductive biases or training mechanisms.

Nevertheless, the additional experiments and clarifications have addressed some of my concerns. I am therefore considering raising my rating from 2 to 3, although the above question remains open.

Thank you for your constructive feedback! We have revised the main text and added a discussion of your concern in Appendix H.4.

We understand your main concern. To be candid, one of our original motivations for lifting was to reduce velocity averaging when paths from different handles cross in action space. After introducing lifting, we gradually added components such as the responsibility floor as we worked to improve the lifted policy’s performance. Direct zz-conditioning may require fewer design components and allow more emphasis on learning an expressive representation of zz.

We believe direct zz-conditioning remains a practical and highly competitive approach. Our method explores a new way to structure flow matching by grouping actions at the source. In future work, we plan to compare SL-FM with a sufficiently expressive zz-conditioned model, focusing on differences in performance and interpretability.

Following your suggestion, we will emphasize these points in the limitations discussion. Our method offers a way to introduce interpretable handles at the source and can simplify the geometry of the velocity field. Direct zz-conditioning also has practical value and is straightforward to implement. The trade-offs between these approaches deserve further study.

Thank you very much for taking the time to discuss this distinction with us! Your willingness to consider raising your score is also very encouraging to us.

Reviewer_dob5OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors propose Source-Lifted Flow Matching (SL-FM), a flow-matching mechanism that aims to learn multimodal behaviors actively while preserving the original target distribution. Empirical evidence is provided to show its effectiveness on several multimodal imitation benchmarks. However, several critical issues remain, as detailed in the Weakness and Questions sections. Therefore, I suggest a weak rejection for this round.

Critical strengths

The paper is well written and easy to follow. The problem of learning from multimodal demonstrations and enabling active inference is important and interesting to explore. The proposed source-lifting mechanism is conceptually novel and provides a useful perspective on exposing an intervenable source interface without explicitly conditioning the velocity field on a discrete mode label.

Critical weaknesses
  1. The selection of KK. The method relies on a globally fixed number of source handles KK, while the true number of modes may vary across states. It is unclear how to choose a single global K in practice:
  • 1.1 If KK is larger than the true number of modes, the supervision signal for each handle may be diluted. In a batch of size BB, the effective number of samples assigned to each handle is roughly B/KB/K, which can increase the variance of the weighted gradient and cause training instability.

  • 1.2 If KK is smaller than the true number of modes, multiple modes may still be captured by a single Gaussian component, so path overlap and identity collapse may persist within that component. Orthogonal Source Lifting separates different handles, but it does not resolve multi-modality inside a single handle.

  1. Target endpoint preservation vs. target distribution preservation. The proposed construction appears to preserve the target endpoint distribution, i.e., all targets are still demonstrated actions projected onto the zero-lift plane. However, this does not guarantee that the generated marginal distribution in free deployment remains equal to the original data distribution. The generated distribution also depends on the learned source prior, the shared velocity field, the responsibility weights, and the ODE integration; density-level preservation is not established.
  2. The “latent-free” claim is overstated. The velocity field does not take a discrete handle z as an explicit input, but it does take the lifted coordinate xtg=(1−t)λτk(s)x_t^g=(1-t)\lambda\tau_k(s), which encodes the handle identity when t<1. Thus, the velocity field is implicitly conditioned on the source handle through a continuous geometric coordinate. The claim should be stated more precisely as “no explicit discrete mode input” rather than “latent-free” or “not conditioned on the selected mode.”
  3. Lift separation vanishes as t→1t\to1. For different handles, the lift coordinate in Eq.(12) is xtg=(1−t)λτk(s)x_t^g=(1-t)\lambda\tau_k(s), which converges to zero as t→1t\to1. Therefore, different handles become less distinguishable in the lift space near the target. If two branches overlap or cross near the end of the transport, the shared field may still average incompatible velocities. The paper does not sufficiently analyze this regime or the sensitivity to the lift scale \lambda.
  4. Missing important benchmarks. The empirical evaluation would be stronger with additional standard benchmarks, e.g., D4RL MuJoCo, to test whether the proposed intervention interface and free-deployment performance generalize beyond the current D3IL, PushT, and Kitchen settings.
Questions for authors
  1. How should KK be selected in practice? The paper fixes KK globally and reports sensitivity for KK=4-10. Could the authors provide a principled procedure for choosing K when the number of modes is unknown and may vary across states? For example, would validation likelihood, BIC, responsibility entropy, handle usage, or downstream intervention success be reliable criteria?
  2. What happens when KK is smaller than the true number of modes? If a single Gaussian component covers multiple true modes, Orthogonal Source Lifting only separates different handles, not multiple modes within the same handle. Can the authors provide a diagnostic or experiment showing how SL-FM behaves in this case? Does the shared field still average incompatible velocities inside one handle?
  3. Is the target distribution preserved at the density level? The construction keeps all target endpoints on the zero-lift plane, but free deployment samples z∼πk(s)z\sim\pi_k(s), samples from the source mixture, integrates the shared field, and projects back to action space. Can the authors clarify whether the resulting generated marginal is guaranteed, or empirically shown, to match pdata(a∣s)p_{\mathrm{data}}(a\mid s) in terms of density and mode weights, rather than only in terms of target endpoints?
  4. How should “latent-free” be interpreted? The velocity field sees xtg=(1−t)λτk(s)x_t^g=(1-t)\lambda\tau_k(s), which depends on the handle kk. Does this not constitute an implicit conditioning on the source handle? Could the authors clarify the distinction between “no explicit discrete mode input” and “not conditioned on the selected mode”?
  5. How robust is the method when branches cross near t=1t=1? Since the lift coordinate vanishes as t→1t\to1, different handles become less separated near the target. Can the authors provide analysis or experiments on crossing paths near the target region? How sensitive is the method to the lift scale λ\lambda and the number of Euler steps in this regime?
6 comments
AuthorsOpenReview ↗

Response to Reviewer dob5

W1, Q1, and Q2: The choice of K

Regarding W1:

K is indeed an important hyperparameter. Thank you for your concern, and we also encountered this issue when designing our method. We view it as similar to K in KNN clustering. Our solution in the submitted paper was to allow a certain degree of redundancy in K.

Before considering the metrics you suggested, our reasoning was as follows:

  1. When K is too small and some modes are missing, the distinctions between them are lost, and our method loses its purpose.
  2. When K is larger, we use soft assignments. As you pointed out, this requires more training and consumes additional computational resources. Our experiments show that, despite this cost, training remains relatively stable as long as K is kept within a moderate range.

Regarding Q1:

K represents the number of Gaussian components used to fit the action distribution with a Gaussian mixture. The most accurate approach would be to estimate the action distribution associated with the current state s_t from the entire dataset. In practice, this is difficult. For example, if s_t is an image, we would first need to identify sufficiently similar images and then collect statistics on their associated actions. This is already a complex task, and the wide variety of state representations adds further difficulty.

We find the metrics you suggested very meaningful. They can be used for a post hoc assessment of whether the chosen K is appropriate. We trained networks with different K values and calculated the corresponding metrics:

KValidation NLL per dimension ↓Normalized responsibility entropyEffective handle usage ratio
11.2516 ± 0.23840.0000100.00%
20.5941 ± 0.19300.057196.70%
40.1321 ± 0.12930.026792.44%
60.0024 ± 0.30360.015282.55%
8−0.1190 ± 0.13660.016180.19%
10−0.2036 ± 0.04520.014571.13%

Because it is difficult to reliably fit the action statistics associated with each state, BIC values calculated directly from the dataset are inaccurate, so we omit them here.

In this environment, the most informative metric is the effective handle usage ratio. In particular, the ratio exceeds 80% when K is insufficient, while it drops to 71.13% at K = 10, indicating excessive redundancy.

As for a reliable selection procedure, our discussion suggests using prior human knowledge, or first training an FM model to estimate the number of possible modes and then choosing K accordingly. We are also considering whether K can be selected through post hoc expectation calculations, which will be a direction for future investigation.

Regarding Q2:

We set up experiments with insufficient K and visualized the velocity field and agent behavior (Figure R1-1 and Figure R1-2).

Velocity averaging can occur. When K = 1, the method effectively reduces to the original FM. Averaging can cause the denoising trajectories to deviate from straight lines. In the four-mode toy experiment, when K < 4, a single GMM component may correspond to multiple target distributions. The degree of trajectory bending can be used to examine this behavior: a component corresponding to a single Gaussian exhibits little bending, while a component corresponding to multiple modes can exhibit more bending. This motivates comparing trajectory bending across different K values. As shown in Figure R1-1, when K is smaller than the number of target modes, a single source Gaussian component may correspond to multiple target modes, with more pronounced bending in its generated trajectories.

The K = 3 case in Figure R1-2 provides a particularly clear example. The first source component, H1, corresponds to both target modes A and B, and its denoising trajectories are visibly curved, consistent with velocity averaging between different target directions within the same handle. In comparison, H2 and H3 each correspond to a single target mode, C and D, respectively, and their denoising trajectories are nearly straight. This comparison illustrates that lifting separates different handles, while velocity averaging can persist within a handle shared by multiple target modes.

W2: Preserving target endpoints and preserving the target distribution

Our understanding is that adjusting the handle distribution necessarily makes the final marginal distribution different from the true distribution. The difference concerns sampling probabilities, while our method aims to achieve as much coverage as possible.

A different distribution can also be beneficial. Some valid alternative actions have very low sampling probabilities, and we want to increase their chances of being sampled.

We used the one-dimensional experiment designed for Q5 to examine how the distribution changes during denoising, as shown in Figure R1-3 and Figure R1-4. We find that our method may reduce velocity averaging more effectively when crossings occur: the peak around zero, where averaging occurs, is lower. The resulting distribution nevertheless differs from the original distribution.

W3 and Q4: The meaning of “latent-free”

We sincerely appreciate your concern. The augmentation in our method serves two purposes:

  1. It distinguishes different handles and provides label or conditioning information. As you noted, this role is similar to that of z.
  2. It provides orthogonal path separation through augmentation, ensuring that paths remain nonintersecting during training and inference. In comparison, z and other conditioning inputs are used as labels and stay outside the noising and denoising process.

We intended to emphasize that our augmentation acts directly on x and participates in the noising and denoising process. This differs from conditioning variables z, or latents, that serve only as conditioning inputs in other methods.

Thus, our model has “no explicit discrete mode input,” while it is indeed “conditioned on the selected mode.” Your understanding is correct. We will add an explanation in the main text to clarify this distinction.

W5: Additional benchmarks

Your suggestion is reasonable. We had considered D4RL, but our preference for scenarios with multimodal actions led us to set it aside initially.

We have now conducted an experiment on D4RL maze2d-umaze-v1. Training directly uses the officially released D4RL data, and we designed the two-endpoint evaluation ourselves. During both training and testing, the policy observes only position and velocity; goal coordinates are omitted.

During evaluation, two branches start from the same state in the middle of the corridor. FM resamples its noise, while SL-FM switches handles. The proportion of pairs in which both branches reach an endpoint and arrive at different endpoints is 28.0% for FM and 41.4% for SL-FM. Other D4RL tasks require further experimental design and validation.

W4 and Q5: Late crossings and sensitivity to λ and Euler steps

Thank you for pointing out this issue. We agree that, as t approaches 1, the lifting coordinates gradually shrink, potentially making different handles harder to distinguish. We therefore added a two-branch Gaussian transport experiment with a controllable crossing time. We construct source–target pairs so that the two prescribed training paths intersect at a specified time t_c, then train the shared velocity field and evaluate the actual generated results.

The following experiments use a fixed prescribed path speed and fixed orthogonal tags. We report the mean and standard deviation across three training seeds. The hit rate measures the proportion of generated endpoints falling within ±3σ of the designated target center.

First, we examine the effect of crossing time. With lifting scale λ = 1 and N = 100 Euler steps, the results are as follows (Figure R1-5a):

Crossing time t_cλDesignated-target hit rate (%)
0.50199.66 ± 0.08
0.80199.46 ± 0.22
0.90198.34 ± 0.55
0.95193.68 ± 3.29
0.98185.86 ± 7.86

Under this setting, as the crossing time approaches the endpoint, the hit rate gradually decreases and variation across seeds increases. This demonstrates performance degradation under extremely late crossings and supports your concern about practical robustness. In this experiment, delaying the crossing also reduces the target separation and noise scale, so these changes must be considered together with the shrinking lifting distance when interpreting the degradation.

Next, we examine whether the lifting scale can mitigate this degradation. We fix t_c = 0.98 and N = 100 and train models separately for different λ values (Figure R1-5b):

λDesignated-target hit rate (%)
0.566.47 ± 7.54
185.86 ± 7.86
298.68 ± 1.99
398.11 ± 2.56

Increasing λ from 1 to 2 raises the hit rate from 85.86% to 98.68%. This shows that appropriately increasing the lifting scale can mitigate late-crossing degradation in this experiment. This is consistent with a larger λ providing greater separation in the lifting coordinates at the same time. Increasing λ further to 3 yields no additional average benefit, indicating that the scale should be chosen appropriately.

Finally, we examine the relationship between lifting scale and Euler steps. At the same t_c = 0.98, we reuse the models trained at each scale and change only the number of integration steps during generation. Alongside hit rate, we report normalized endpoint error, NRMSE, to capture large deviations affecting a small number of samples (Figure R1-6):

λEuler steps NDesignated-target hit rate (%)NRMSE ↓
11089.68 ± 3.830.243 ± 0.030
110085.86 ± 7.860.264 ± 0.104
21082.89 ± 27.9912.230 ± 20.980
210098.68 ± 1.990.057 ± 0.051
31081.48 ± 27.237.410 ± 12.626
310098.11 ± 2.560.080 ± 0.064

For λ = 2 and 3, 10-step integration produces large endpoint errors and substantial variation across seeds. Increasing the number of steps to 100 substantially alleviates these issues. This shows that the benefits of a larger lifting scale should be evaluated together with a sufficient integration budget. Meanwhile, λ = 1 shows no improvement with more steps, indicating that the effectiveness of increasing the integration budget depends on the setting.

These results show that appropriate choices of lifting scale and integration steps can mitigate degradation under extremely late crossings. Geometric separation before the endpoint and consistent numerical robustness remain distinct properties. We will refine the corresponding statements and specify that these results come from controlled synthetic mechanism experiments, with robustness in full robotic tasks requiring further validation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.