- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MOSS (Mode-Oriented Source Shaping), a framework for controllable multimodal behavior generation in flow-matching-based robotic policies.
The key idea is to control the generated behavior by modifying the initial source distribution rather than explicitly conditioning the velocity field on a desired behavior mode. The authors first analyze the implicit mode-dependent partitioning of the source space under deterministic flow transport and discuss how practical approximation errors can weaken mode separation.
Based on this observation, MOSS constructs mode-specific Gaussian source distributions whose centers are aligned with semantic differences between behavioral modes. A modality-strength estimator predicts the local degree of behavioral multimodality and adaptively adjusts the separation and covariance of these distributions. A single shared flow-matching network is then trained to generate actions from the corresponding sources.
The method is evaluated on six simulated and three real-world robotic manipulation tasks. The experiments demonstrate improved mode controllability, particularly under imbalanced demonstrations, while maintaining competitive task success rates.
Critical strengths
- Effective and practical multimodal behavior control.
The paper addresses an important problem in generative robotic policies: reliably selecting a desired behavioral mode instead of relying on stochastic sampling. MOSS provides a relatively simple solution by encoding mode information into the initial source distribution while retaining a shared velocity field. This formulation avoids introducing an additional mode condition directly into the flow network and provides a useful alternative to conventional conditional generation.
- Encouraging experimental results across simulation and real-world tasks.
The paper evaluates the proposed method on six simulation tasks and three real-world robotic manipulation tasks. The real-world experiments show substantial improvements in average guidance rate over mode-conditioned FM (84.2% vs. 40.8%), together with higher average task success (85.3% vs. 73.3%). The experiments on imbalanced demonstrations are also encouraging, suggesting that the shared flow network can leverage common motion structures to improve minority-mode learning. Furthermore, the ablation results show that adaptive source shaping consistently improves controllability over fixed source separation on the evaluated simulation tasks.
Critical weaknesses
- Limited methodological novelty relative to existing source-shaping approaches.
My primary concern is the novelty of the proposed framework. Controlling generative behavior by manipulating the initial noise or source distribution is not a fundamentally new idea and has been explored in prior diffusion- and flow-based generative models. In the context of flow matching, related approaches such as SL-FM, LAFM, and Gaussian-mixture-based source modeling have already investigated structured source distributions for multimodal generation.
The main methodological distinction of MOSS appears to be the combination of semantically aligned Gaussian components and modality-dependent adaptive separation. While this is a reasonable design, the underlying idea remains closely related to existing source-space control methods. The paper would benefit from a more convincing demonstration of what cannot be achieved by simpler existing source-shaping strategies.
I also find the theoretical contribution somewhat limited. Theorem 1 largely follows from the properties of preimages under deterministic mappings and the exact pushforward condition. Theorem 2 establishes local separation under additional regularity assumptions. However, these results do not directly justify the particular Gaussian parameterization or the adaptive separation schedule proposed in MOSS. Consequently, the connection between the theoretical analysis and the specific methodological contribution is not sufficiently strong.
- Insufficient evidence isolating the contribution of adaptive source shaping.
Although the overall experimental results are promising, it remains unclear how much of the improvement comes from the proposed modality-adaptive mechanism rather than simply using mode-specific Gaussian sources.
The ablation experiments in Table 3 show that fixed mode-specific Gaussian separation already achieves relatively strong controllability, and adaptive shaping provides an additional improvement. However, these comparisons are limited to three simulation tasks. The real-world experiments and imbalanced-data evaluations do not include the corresponding fixed-separation baseline, making it difficult to determine whether the most significant reported improvements genuinely require the adaptive component.
Furthermore, the GMM-sourced baseline does not explicitly incorporate semantic mode supervision, whereas MOSS uses known mode labels to construct its sources. This difference in supervision complicates the interpretation of the comparison. A more direct comparison with semantically aligned but non-adaptive source distributions, using the same supervision, would better establish the incremental contribution.
Finally, the real-world evaluation contains only 20 trials per requested mode, without reported confidence intervals or repeated training-seed evaluations. Additional statistical evidence would help establish the robustness of the reported improvements.
Questions for authors
- Novelty relative to existing source-shaping methods.
Could the authors clarify the fundamental methodological difference between MOSS and existing approaches that control flow matching through structured or learned source distributions, particularly SL-FM?
Beyond explicitly associating Gaussian components with semantic mode labels, what is the main new technical mechanism that distinguishes MOSS from a supervised mixture-of-Gaussians source model?
A more direct comparison with existing source-shaping methods under matched supervision would significantly strengthen the novelty claim.
- Contribution of adaptive separation.
Table 3 shows that fixed mode-specific Gaussian separation already provides strong controllability. Could the authors extend this comparison to the three real-world tasks and the 10:90 imbalanced-data experiments?
In particular, how much of the reported improvement over mode-conditioned FM comes from semantic source separation itself, and how much comes specifically from the modality-strength estimator and adaptive source geometry?
It would also be informative to compare the proposed action-aligned Gaussian centers with simpler alternatives, such as fixed or randomly assigned separated Gaussian centers, to clarify the importance of the proposed geometric construction.
- Theoretical justification.
The theoretical analysis establishes that deterministic flow transport induces mode-dependent source-space preimages. However, it is less clear how these results motivate the specific Gaussian source parameterization in Section 4.
Could the authors clarify whether the theory provides any guarantee or quantitative insight into the effectiveness of the proposed adaptive separation strategy, beyond establishing the existence of mode-dependent source regions?
In particular, how does the theoretical analysis inform the choice of source-center displacement and covariance along the mode-discriminative subspace?
- Robustness and baseline comparisons.
The real-world results show substantial differences between MOSS and mode-conditioned FM, but the latter exhibits severe mode collapse in some tasks. How sensitive are these results to the conditioning implementation, training balance, and classifier-free guidance scale?
Although Appendix F explores stronger guidance, a more systematic comparison and statistical uncertainty estimates would help determine whether the observed advantage is robust across different baseline configurations and training runs.
Additionally, how sensitive is MOSS to inaccuracies in the estimated modality strength, especially when the local neighborhood statistics become unreliable under limited or imbalanced demonstrations?
9 comments
Response to Reviewer qSfb (2/5)
Q2: How much does adaptation improve over fixed semantic Gaussian sources? Can you extend the comparison to 10:90 data and three real-robot tasks?
Answer: Thank you for this suggestion. Original Table 3 compares fixed semantic Gaussian sources with adaptive MOSS on three balanced simulation tasks; Appendices H.1–H.3 provide training curves and source–action displacement statistics. Following your advice, we added fixed-source ablations for 10:90 data and three real-robot tasks.
1. 10:90 imbalance
Mode 1 is the minority (10%) and Mode 2 the majority (90%). Fixed-separation results are:
| Task | Mode 1 GR / SR | Mode 2 GR / SR |
|---|---|---|
| Push Cube | 44% / 84% | 98% / 100% |
| Grasp Eyeglasses | 56% / 100% | 98% / 98% |
| Stack Cube | 70% / 96% | 98% / 98% |
Fixed separation reaches only 44%–70% GR for minority modes. Adaptive MOSS provides better minority-mode control in the original 10:90 experiments, indicating that semantic pairing alone does not resolve imbalance and adaptive separation further improves minority-mode generation.
2. Real-robot fixed-source ablation
we have included additional fixed-separation experiments conducted previously but not reported in the original manuscript:
| Task | Method | SR | GR1 | GR2 |
|---|---|---|---|---|
| Place Cups | Fixed Separation | 0.98 | 0.85 | 1.00 |
| MOSS | 0.98 | 0.95 | 1.00 | |
| Pick Veg | Fixed Separation | 0.70 | 0.60 | 0.50 |
| MOSS | 0.75 | 0.80 | 0.65 | |
| Open Drawer | Fixed Separation | 0.80 | 0.60 | 0.70 |
| MOSS | 0.83 | 0.90 | 0.75 |
MOSS improves GR in five of six modes and ties in the remaining one, showing that adaptation improves real-robot control even with the same semantic pairing. Balanced simulation, 10:90 experiments, and real-robot ablations all support adaptive separation.
Unfortunately, the original platform is no longer available for further tests; we have fully reported the existing results and evaluation settings in the revision.
Revisions: Table 3 and Appendix G - fixed-source ablations.
Q3: What do action-aligned centers contribute? Can fixed or random centers separate the roles of semantic pairing and adaptive geometry?
Answer: Thank you for this suggestion. We agree that the original manuscript did not clearly distinguish these components, and we appreciate the opportunity to clarify their respective roles.
Following your advice, we added random centers: given MOSS's mode axis , we sample with a fixed seed and construct a random direction orthogonal to . This retains semantic pairing but removes action alignment. Fixed Aligned separates along ; full MOSS additionally adapts centers and covariance using local modality strength.
Results on three simulation tasks are:
| Task | Source Design | Mode 1 GR / SR | Mode 2 GR / SR | Avg. GR |
|---|---|---|---|---|
| Push Cube | Random Centers | 62% / 100% | 42% / 96% | 52% |
| Fixed Separation | 86% / 100% | 85% / 98% | 85.5% | |
| MOSS | 99% / 99% | 100% / 100% | 99.5% | |
| Grasp Eyeglasses | Random Centers | 50% / 100% | 38% / 100% | 44% |
| Fixed Separation | 86% / 100% | 90% / 100% | 88% | |
| MOSS | 100% / 100% | 100% / 100% | 100% | |
| Stack Cube | Random Centers | 60% / 96% | 52% / 94% | 56% |
| Fixed Separation | 78% / 90% | 88% / 96% | 83% | |
| MOSS | 98% / 98% | 96% / 96% | 97% |
Replacing random centers with fixed aligned sources increases average GR by 33.5, 44, and 27 percentage points; adaptation adds 14, 12, and 14 points, respectively.
Source design affects both initial samples and the interpolation paths and velocity supervision in FM [3]. Overlapping paths with different target velocities can create conflicting supervision for a shared network. Rectified Flow and OT-based FM also show that source–target coupling and transport geometry affect learning [4,5]. MOSS therefore separates centers along action differences to reduce inter-mode velocity conflicts, while adaptation preserves sharing during common motion.
These comparisons distinguish the contributions: semantic pairing enables explicit selection, action alignment organizes source geometry, and adaptive separation further improves mode control.
Revisions: Appendix H - random-center comparison and action-alignment analysis.
Response to Reviewer qSfb (3/5)
Q4: How does the theory support Gaussian parameterization, center shifts, and subspace covariance? Does it guarantee or quantitatively explain adaptation?
Answer: We sincerely appreciate reviewer qSfb for raising this important question. We agree that the connection between our theoretical analysis and method design could have been explained more clearly. While the specific design of MOSS is not directly derived from the theory, our analysis provides insights into the relationship between initial noise and final behavioral modes in FM, which serves as the motivation for our method.
Under assumptions including continuity and exact transport, deterministic FM associates different behavioral modes with distinct source-space preimages. Theorem 1 relates these regions to target-mode probabilities; Theorem 2 analyzes mode preservation under source perturbations. Although based on established mathematical tools, these results connect continuity and pushforward properties to behavioral modes in robotic policies, identifying initial noise as an entry point for control.
We first conducted a simple source-shift experiment on 2D Navigation. Keeping standard FM weights fixed, we adjusted only the Gaussian mean using initial-noise statistics for the target mode and drew 100 samples per setting, as shown in Appendix A:
| Sampling setting | Path 1 | Path 2 | Path 3 | Path 4 | Stuck |
|---|---|---|---|---|---|
| Standard Gaussian | 14 | 19 | 32 | 26 | 9 |
| Mean shifted to −0.303043 | 62 | 27 | 8 | 2 | 1 |
Shifting the noise mean increases Path 1 occurrences from 14 to 62 without retraining. This shows that the initial source distribution influences final behavioral modes and directly motivates MOSS's adaptive source separation.
Building on this observation, MOSS uses action-aligned centers to encode inter-mode action differences and subspace covariance to control overlap along mode-discriminative directions. Adaptive separation then adjusts this geometry according to local modality strength: sources remain close during shared motions and separate near behavioral branches, as illustrated by Stack Cube.
This design is motivated by the theory and preliminary experiment, but its effectiveness is established empirically rather than theoretically guaranteed. Table 3, the 10:90 imbalance experiments, and real-robot results demonstrate the benefits of adaptive separation.
We sincerely appreciate your constructive suggestion and have made corresponding clarifications in the revised manuscript to better explain this distinction.
Revisions: Section 4.1 - connection between theory and method design.
Response to Reviewer qSfb (1/5)
We sincerely thank reviewer qSfb for the thoughtful and constructive feedback, as well as for recognizing the practical value of MOSS in multimodal behavior control and real-robot tasks. Your insightful suggestions on methodological distinctions, supervision, and adaptive gains have been invaluable in helping us strengthen our work. Following your advice, we have carefully conducted additional experiments, including comparisons of source designs, fixed-separation experiments under 10:90 data imbalance, SL-FM semantic calibration, conditioning variants, and CFG sweeps. We sincerely appreciate your time and consideration, and hope that our revisions adequately address your concerns. Our detailed responses are provided below.
Q1: What distinguishes MOSS technically from SL-FM, LAFM, and existing structured-source methods? Beyond associating semantic labels with Gaussian components, how does it differ from a supervised Gaussian-mixture source?
Answer: We sincerely appreciate the your suggestion to better distinguish our work from prior studies. We fully acknowledge that structured source distributions have been explored in FM, Rectified Flow, and optimal transport research [3–5]. However, MOSS directly associates user-specified behavioral modes with source components and adapts the source distribution to the degree of behavioral branching for the first time.
MOSS comprises three designs:
Comparison with SL-FM: SL-FM constructs intervenable source handles through orthogonal source lifting, but its base training does not directly associate them with user-specified semantics. Section 5.4 uses CEM search and a PPO high-level selector for target-route control[1]. MOSS establishes mode–component correspondence during training, enabling direct mode selection without additional search or selector training.
Comparison with LAFM: LAFM learns motion primitives through a latent action model and selects learned source priors from observations, aiming to better match sources to action structure[2]. MOSS addresses user-specified multimodal behavior control: modes can be actively selected at inference, while separation adapts to task progress. The objectives and source designs therefore differ.
Comparison with ordinary Gaussian-mixture sources: An ordinary GMM source could not necessarily establish stable component–semantics correspondence. We observed, for example, one component strongly favoring one behavior while another still generated both behaviors randomly. MOSS establishes supervised mode–Gaussian correspondence, organizes source geometry using action differences, and adapts separation to local multimodality. Its design therefore extends beyond attaching semantic labels to components.
Following your advice, we also added a direct SL-FM comparison. After training, we fixed each of source handles and ran 20 rollouts per handle to establish semantic correspondence. We then froze the mapping and evaluated each target mode for 50 trials:
SL-FM achieves high SR on some tasks but shows substantial per-mode GR differences, particularly 34% for Stack Cube Mode 2. MOSS provides more consistent control of user-requested modes.
To distinguish the contributions, we also added fixed-source and random-center ablations, both retaining semantic pairing, to test adaptive separation and action alignment respectively. Results appear in Q2 and Q3.
Revisions: Section 2 - distinctions from related methods; Table 1 and Appendix E.2 - SL-FM implementation and results.