acceptodds
Under review as a conference paper at ICLR 2027

Omni-Mono: Mutually Informed Reasoning across Vision, Language, and Action

Abstract

Vision-language-action (VLA) models increasingly use intermediate reasoning to guide robotic manipulation. However, existing approaches often restrict reasoning to one modality or a predefined sequence, limiting mutual feedback across modalities. We propose Omni-Mono, a framework combining joint multimodal reasoning with modality-specific refinement. The Omni Reasoner jointly generates textual, visual, and action representations in a shared latent space through grouped attention with modality relation masks, enabling information exchange across reasoning streams. The Mono Reasoner then refines these representations through dedicated experts, adaptively fusing guidance from Omni representations and preceding experts according to the current reasoning state. Together with the task context, the refined representations condition a flow-matching head to generate action chunks. Experiments on LIBERO, LIBERO-Plus, SimplerEnv, RoboTwin 2.0, and real-world manipulation tasks demonstrate improved performance and robustness to environmental perturbations. Ablations and representation analyses further support the complementary benefits of joint reasoning and modality-specific refinement. The code repository is available at https://anonymous.4open.science/r/omni-mono/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, VLA Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AJsVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Omni-Mono, a vision-language-action framework that integrates joint multimodal reasoning with modality-specific refinement for robotic manipulation. Motivated by the limitations of existing reasoning-based VLA approaches, which often process textual, visual, and action reasoning independently or in a predefined sequence, the proposed method aims to enable mutual information exchange across different reasoning modalities.

The framework consists of two main components. First, the Omni Reasoner jointly produces textual, visual, and action latent representations through grouped attention with modality relation masks, enabling structured information exchange across reasoning streams. Second, the Mono Reasoner employs dedicated modality-specific experts with adaptive guidance fusion to progressively refine these representations in text–vision–action order. The refined representations, together with the original task context, condition a flow-matching action head for action-chunk generation. The framework is trained with auxiliary supervision derived from textual plans, future visual features, and sparse reference actions.

The authors evaluate Omni-Mono on LIBERO, LIBERO-Plus, SimplerEnv, RoboTwin 2.0, and two real-world robotic manipulation tasks. The proposed method demonstrates strong performance across these benchmarks, with additional ablations and controlled interventions investigating the contributions of multimodal communication and modality-specific refinement.

Critical strengths
  1. The paper addresses a meaningful challenge in reasoning-based robotic manipulation: enabling different reasoning modalities to exchange information rather than treating them as independent or strictly sequential processes. The proposed combination of an Omni Reasoner for joint multimodal information exchange and a Mono Reasoner for modality-specific refinement is conceptually well motivated. The grouped attention mechanism provides an explicit architectural structure for cross-modal communication, while the dedicated refinement experts aim to preserve modality-specific information. I find the overall design interesting, particularly the attempt to reconcile shared multimodal representations with specialized reasoning for semantic planning, future-state prediction, and motion generation.

  2. The experimental results are impressive across several established robotic manipulation benchmarks. In particular, Omni-Mono achieves 92.4% success on LIBERO-Plus and 77.2% on RoboTwin Clean-to-Random, outperforming the strongest compared reasoning baseline by 5.2 and 6.0 percentage points, respectively. The improvements under distribution shifts are especially encouraging, given the importance of robustness for generalizable robot manipulation. The paper also includes real-world evaluations on a Franka robot, where Omni-Mono achieves an average success rate of 86.0%, compared with 75.0% for π0.5 under the reported evaluation protocol. Together, these results provide promising evidence for the effectiveness of the proposed framework across simulation and physical manipulation settings.

  3. A notable strength of this paper is the effort to experimentally validate the proposed reasoning architecture beyond overall task success rates. In addition to standard Omni-only and Mono-only ablations, the authors provide capacity-controlled comparisons that retain four independent language backbones, helping distinguish architectural contributions from increased model capacity. The directional information-flow experiments further examine the effects of restricting cross-modal communication, while the controlled obstacle experiments evaluate task success, collision rates, and textual plan validity under conflicting spatial constraints. The visual-to-text intervention is particularly interesting, as it provides evidence that modifying visual guidance can influence textual representations and downstream execution. Although such interventions have limitations, I appreciate the authors' effort to investigate the proposed mechanism more directly.

Critical weaknesses
  1. My primary concern is the computational complexity of the proposed framework and whether it is sufficiently justified by the additional performance gains. As described in Appendix A.1, the Omni Reasoner and three Mono experts employ four independent language backbones initialized from Qwen3-VL-4B-Instruct, with their language parameters independently fine-tuned. This results in a substantially larger architecture than conventional single-backbone VLA approaches. The authors have made a valuable effort to address the capacity concern through controlled comparisons. However, the improvements over the capacity-matched generic refinement baseline remain relatively modest: 1.7, 1.4, and 1.8 percentage points on LIBERO-Plus, SimplerEnv, and RoboTwin C2R, respectively. Furthermore, the reported inference latency of Omni-Mono is 680 ms on an A100 GPU, compared with 170 ms for the direct-action baseline under the same internal measurement protocol. This fourfold increase raises practical questions about the scalability and efficiency of the proposed architecture. While the controlled ablations provide evidence that the reasoning structure contributes beyond additional model capacity, the paper does not sufficiently explore whether comparable benefits could be achieved using a more parameter-efficient architecture, such as shared reasoning backbones or lightweight modality-specific refinement modules. A more explicit analysis of performance relative to model size and computational requirements would help establish the practical significance of the proposed approach.

  2. Although the paper includes physical robot experiments, the evaluation is restricted to two relatively simple tabletop pick-and-place tasks: placing a milk box onto a plate and placing a bowl onto a plate.

These experiments demonstrate that the proposed framework can successfully execute physical manipulation tasks and improve success rates over the compared baselines. However, they do not directly evaluate the primary motivation of the paper: resolving inconsistencies or conflicts between textual, visual, and action reasoning during complex execution.

For example, the real-world tasks do not explicitly require obstacle-aware replanning, reasoning over multiple competing constraints, or revising an initial plan in response to unexpected environmental changes. The controlled obstacle experiments in simulation partially address this concern and represent a useful contribution. Nevertheless, extending such evaluations to physical robotic settings would provide stronger evidence that the proposed mutually informed reasoning mechanism remains effective under real-world perception uncertainty and execution errors. I believe that more challenging real-world tasks involving explicit reasoning dependencies or conflicting spatial constraints would substantially strengthen the paper's central claims.

  1. The proposed framework enables structured cross-modal communication through relation-masked attention and adaptive guidance fusion. However, the extent to which these mechanisms implement meaningful reasoning refinement, rather than improved multimodal feature integration, remains somewhat unclear. The directional information-flow ablations and visual-to-text intervention provide encouraging evidence that information from one modality can influence other representations and affect downstream behavior. Nevertheless, these experiments do not fully characterize how reasoning representations are revised when different modalities provide conflicting information. In particular, the visual-to-text intervention introduces representations from a different scene, which may create out-of-distribution feature combinations. Although the authors appropriately acknowledge this limitation, additional analyses would help clarify the causal relationship between cross-modal communication, plan consistency, and action generation. Furthermore, the paper jointly supervises textual, visual, and action representations at both reasoning stages, but does not sufficiently isolate the contribution of each individual auxiliary supervision objective. Such ablations would help determine whether the observed benefits arise primarily from the proposed information-flow architecture, the richer intermediate supervision, or their combination.
Questions for authors
  1. Could the authors further justify the use of four independent Qwen3-VL-4B language backbones? Although the capacity-controlled experiments are appreciated, the improvements over generic refinement are relatively modest. Would it be possible to evaluate a shared-backbone architecture or a lighter modality-specific refinement design? A comparison of success rates, inference latency, and memory consumption across different model capacities would help clarify whether the proposed benefits require the full architectural complexity.

  2. Could the authors extend the real-world experiments to tasks that more directly require cross-modal reasoning? For example, tasks involving obstacle avoidance, competing spatial constraints, multi-step planning, or unexpected environmental changes would be particularly relevant to the motivation presented in Figure 1. It would also be valuable to compare the full model with unidirectional reasoning or generic refinement on these tasks, to directly validate the benefits of mutual information exchange during physical execution.

  3. Could the authors provide additional analyses demonstrating how the reasoning representations are modified in response to conflicting information from different modalities? For example, in the controlled obstacle scenarios, it would be interesting to examine whether the textual plan representation changes consistently when visual reasoning identifies an obstacle, and how such changes affect the generated actions. Additionally, could the authors independently ablate the textual, visual, and action auxiliary supervision losses to clarify their respective contributions to the overall performance gains? These experiments would help better distinguish the benefits of the cross-modal reasoning architecture from those of intermediate multimodal supervision.

7 comments
AuthorsOpenReview ↗

Response to Reviewer AJsV 1/4

We sincerely thank the reviewer for the thoughtful and constructive feedback. We have organized the reviewer's comments into the following questions and responses.

Q1: Do the performance benefits of four independent backbones justify their computational cost? Can shared-backbone or lighter refinement architectures be evaluated?

"Could the authors further justify the use of four independent Qwen3-VL-4B language backbones? Although the capacity-controlled experiments are appreciated, the improvements over generic refinement are relatively modest. Would it be possible to evaluate a shared-backbone architecture or a lighter modality-specific refinement design? A comparison of success rates, inference latency, and memory consumption across different model capacities would help clarify whether the proposed benefits require the full architectural complexity."

A1:

  1. Rationale for independent backbones

    Omni and Mono serve different modeling purposes. Omni organizes information exchange among textual, visual, and reference-action states through relation masks, allowing each modality to incorporate constraints from the others to form coordinated reasoning states. The three Mono experts refine semantic plans, future-scene information, and motion information, respectively, and adaptively select guidance from Omni and preceding experts based on their current query states. As described in Appendix A.1, the four reasoning backbones use the same Qwen3-VL-4B-Instruct initialization, but their language parameters are fine-tuned independently; the input visual encoder and task context are shared. Independent parameters allow each expert to learn transformations specific to its modality's objectives. This is the current implementation's design choice for preserving modality-specific details, while whether shared backbones or lightweight modules can achieve similar benefits requires direct comparison.

  2. Latency and performance analysis

    MethodLIBERO-Plus Avg.(%)↑Inference latency (ms) ↓
    DualCoT-VLA87.283.2
    ACoT-VLA86.6112
    LaRA-VLA83.2135
    Base62.1170
    Omni-only87.4247
    Mono-only88.0494
    Omni–Mono92.4680
    Fast-ThinkAct64.8805
    ThinkAct50.27,513

    Compared with other methods, Omni–Mono achieves the best performance while maintaining relatively low latency. In the internal ablations, Omni–Mono has higher latency than Omni-only at 247 ms and Mono-only at 494 ms, but also increases the average LIBERO-Plus success rate from 87.4% and 88.0%, respectively, to 92.4%, gains of 5.0 and 4.4 percentage points. This shows that the additional computation from joint reasoning and modality-specific refinement is accompanied by improved task performance, making Omni–Mono an option that balances efficiency and performance.

AuthorsOpenReview ↗

Response to Reviewer AJsV 2/4

  1. Controlled comparisons with a shared backbone and lightweight refinement

    We have added controlled experiments with a shared backbone and lightweight modality-specific refinement to evaluate the performance benefits and computational costs of four independent backbones. Specifically, we retain the full Omni Reasoner and construct two alternative configurations. The first has all three Mono experts share a single language backbone while retaining independent modality queries and adaptive guidance modules. The second replaces each of the three Mono experts with an independent 12-layer Transformer to reduce the parameter count and computational cost of refinement. Both variants retain the text→vision→action execution order, all auxiliary supervision objectives, and the final action head.

    Model configurationLIBERO-Plus Avg.(%)↑SimplerEnv Avg.(%)↑Inference latency (ms) ↓Relative peak GPU memory at inference ↓
    Full Omni–Mono92.478.36801.00
    Shared Mono backbone90.575.65440.58
    Lightweight Mono88.774.54300.37

    The results show that the full Omni–Mono model achieves the highest task success rates among the compared configurations. Sharing the Mono backbone reduces parameter storage overhead, but provides limited improvement in inference latency; lightweight Mono reduces computation while lowering success rates to some extent. We believe that sharing the language backbone restricts each expert's ability to independently adapt its parameters to different modalities, while reducing the number of refinement layers limits the processing of fine-grained information. Both simplifications may affect Mono's dedicated refinement of semantic plans, future-scene information, and motion details. These results therefore support using independent modality experts with sufficient capacity: among the architectures compared in this experiment, the full Omni–Mono design with joint cross-modal reasoning and modality-specific refinement achieves the highest task success rates.

Q2: Can the method be validated on real-robot tasks with greater cross-modal reasoning demands and compared with unidirectional reasoning or generic refinement?

"Could the authors extend the real-world experiments to tasks that more directly require cross-modal reasoning? For example, tasks involving obstacle avoidance, competing spatial constraints, multi-step planning, or unexpected environmental changes would be particularly relevant to the motivation presented in Figure 1. It would also be valuable to compare the full model with unidirectional reasoning or generic refinement on these tasks, to directly validate the benefits of mutual information exchange during physical execution."

A2: To strengthen the comparison in real-world settings, we have added two real-robot tasks involving object retrieval and placement and sequential stacking. The first task is Microwave-to-Plate Transfer: the robot must retrieve an object from inside a microwave and place it on a plate. The second task is Sequential Fruit Stacking: the robot must first place a banana on a plate and then place a peach on top of the banana. Real-world illustrations of these complex tasks are provided in Figure 7 of Appendix F.5 in the revised manuscript. These tasks involve object retrieval and placement and fruit stacking in a specified order, respectively. We include two structural controls on these tasks: Unidirectional and Generic refinement. Both retain four backbones, auxiliary supervision at both stages, and the final action head. Unidirectional restricts cross-modal information flow to text→vision, text→action, and vision→action, while Generic refinement replaces modality-specific experts with mixed-modality refinement modules.

MethodMethod type / control configurationSequential Fruit Stacking: SR(%)↑Microwave-to-Plate Transfer: SR(%)↑
π0.5General-purpose VLA3854
UnidirectionalFour backbones, unidirectional cross-modal reasoning4464
Generic refinementFour backbones, generic mixed-modality refinement4268
Omni–MonoFull cross-modal interaction and modality-specific refinement5676

The added unidirectional and generic-refinement controls further examine whether full cross-modal interaction and modality-specific refinement continue to improve real-world robot performance when the number of backbones and auxiliary supervision are held constant.

AuthorsOpenReview ↗

Response to Reviewer AJsV 3/4

Q3: How do reasoning representations adapt to cross-modal conflicts, and how does this affect plan consistency and action execution?

"Could the authors provide additional analyses demonstrating how the reasoning representations are modified in response to conflicting information from different modalities? For example, in the controlled obstacle scenarios, it would be interesting to examine whether the textual plan representation changes consistently when visual reasoning identifies an obstacle, and how such changes affect the generated actions."

A3: To further explain how cross-modal constraints affect reasoning representations and action execution, we have added an analysis of plan adaptation and execution consistency using the paired obstacle scenes in Appendix D.5. The experiment holds the task instruction, manipulated object, target location, and robot state fixed, changing only whether an obstacle blocks the transport path. We decode plans from the Omni and Mono textual states under clear-path and blocked-path conditions and record the actual execution trajectories. Decoding is used only for diagnosis, and the generated text is not fed back to the control policy. The experiment compares the full Omni–Mono model, Unidirectional, and LaRA-VLA, with LaRA-VLA serving as an external baseline that organizes textual latent reasoning and future visual prediction through unidirectional dependencies. We focus on whether the model can form a new plan that satisfies the obstacle constraints when the original transport plan no longer meets the current spatial constraints, and whether the corresponding adjustment is realized in action execution. For example, when a direct transport path is blocked, we check whether the plan introduces a feasible lifting or detouring operation and whether the actual trajectory reflects that adjustment.

MethodOmni blocked-scene plan validity rate (%) ↑Final blocked-scene plan validity rate (%) ↑Final plan adaptation rate (%) ↑Plan–action consistency rate (%) ↑
Unidirectional60656674
LaRA-VLA—666472
Omni–Mono66747884

The metrics are calculated as follows and are all reported as percentages:

  1. Blocked-scene plan validity rate = number of blocked-scene samples with valid decoded plans / total number of blocked-scene samples × 100%. A valid plan must satisfy the instructed operation, object, and target location, and specify a feasible path adjustment under the current obstacle constraints. Omitting a required avoidance operation or proposing an infeasible path is counted as invalid.

  2. Final plan adaptation rate = number of scene pairs requiring adjustment in which recomputation produces a valid final plan / total number of scene pairs requiring adjustment × 100%. Specifically, we first decode the original plan from the final textual state in the clear-path scene. A sample is counted as requiring adjustment only if this plan is valid in the clear-path scene but becomes invalid due to obstacle constraints when directly applied to the paired blocked-path scene. We then check whether the final plan obtained by recomputing the model from the blocked observation is valid. Samples whose original plan remains valid in both scenes are excluded from the denominator, and the model is not required to change its plan unnecessarily.

  3. Plan–action consistency rate = number of blocked-scene samples in which the path adjustment required by a valid final plan is reflected in actual execution / number of blocked-scene samples with valid final plans × 100%. For example, if a plan requires lifting before transport, the trajectory should reflect the height adjustment needed to clear the obstacle; if a plan requires a detour, the trajectory should pass around the obstacle. An action change that does not satisfy the avoidance adjustment required by the plan is not counted as consistent. This metric evaluates the correspondence between valid plans and actual actions, rather than overall task success.

The results show that when an obstacle invalidates the original plan, Omni–Mono's textual reasoning representations can adapt: its decoded plans more effectively reflect avoidance requirements such as lifting or detouring, and the corresponding adjustments are reflected in actual actions. Compared with the unidirectional control, the higher plan adaptation and plan–action consistency rates support the interpretation that cross-modal interaction helps coordinate environmental constraints, plan updates, and action execution.

Reviewer_DbqPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  1. Results are consistently strong, especially under distribution shift. Omni-Mono outperforms a broad set of action-only and reasoning VLAs on four benchmarks. The largest margins appear under perturbation: +5.2 on LIBERO-Plus zero-shot and +6.0 on RoboTwin C2R, with a smaller clean-to-randomized drop than the strongest baseline (Table 15). A small real-robot comparison against π0/π0.5 points in the same direction. The consistency across benchmarks and evaluation conditions is the main argument for acceptance.

  2. The controlled ablations are unusually thorough. The authors go beyond simple component removal. They compare against a generic mixed-modality refiner at matched backbone count, parameters, and FLOPs (Table 9). They test unidirectional, intra-modal, and Omni-intra communication structures (Tables 8–9). They build a blocked-path diagnostic that directly tests the motivating obstacle scenario of Fig. 1, including plan-validity scoring with blind annotators and a visual-to-text latent swap (Tables 10–11). They also report matched-probe representation analyses averaged over three training seeds. The cross-modal exchange consistently helps in these controls, which gives real support to the paper's central claim.

Critical strengths
  1. Results are consistently strong, especially under distribution shift. Omni-Mono outperforms a broad set of action-only and reasoning VLAs on four benchmarks. The largest margins appear under perturbation: +5.2 on LIBERO-Plus zero-shot and +6.0 on RoboTwin C2R, with a smaller clean-to-randomized drop than the strongest baseline (Table 15). A small real-robot comparison against π0/π0.5 points in the same direction. The consistency across benchmarks and evaluation conditions is the main argument for acceptance.

  2. The controlled ablations are unusually thorough. The authors go beyond simple component removal. They compare against a generic mixed-modality refiner at matched backbone count, parameters, and FLOPs (Table 9). They test unidirectional, intra-modal, and Omni-intra communication structures (Tables 8–9). They build a blocked-path diagnostic that directly tests the motivating obstacle scenario of Fig. 1, including plan-validity scoring with blind annotators and a visual-to-text latent swap (Tables 10–11). They also report matched-probe representation analyses averaged over three training seeds. The cross-modal exchange consistently helps in these controls, which gives real support to the paper's central claim.

Critical weaknesses
  1. The gains are not cleanly separated from model capacity and compute. Omni-Mono runs four independently fine-tuned Qwen3-VL-4B language backbones, roughly 4× the language-model parameters of its own single-backbone Base, with 4× its latency (680 ms vs. 170 ms) and 8×A100 training. Most baselines in Table 1 are smaller single-backbone models (π0.5, OpenVLA-OFT, DualCoT-VLA), and their numbers appear to be copied from papers with heterogeneous backbones and training protocols. The paper never reports the success rate of its own Base (same Qwen3-VL-4B, direct action prediction, same data and budget), nor of a four-backbone model without auxiliary reasoning supervision. Without these numbers, the reader cannot tell how much of the headline margin comes from reasoning at all, as opposed to the stronger backbone and extra capacity. The internal controls that do hold capacity fixed show much smaller effects: Omni-Mono beats generic refinement by only 1.4–1.8 points and the unidirectional variant by 1.9–2.7 points (Table 9). In addition, the Omni-only and Mono-only ablations in Table 2 change the number of backbones as well as the reasoning structure. The specific claim that mutual cross-modal reasoning drives the improvement is therefore supported, but only with modest effect sizes that are small compared with the overall complexity.

  2. The conceptual novelty is incremental, and the "reasoning" mostly amounts to auxiliary latent supervision. Joint latent reasoning over multiple modalities in a VLA has close precedents. DualCoT-VLA already produces visual and linguistic reasoning with parallel latent queries in one pass, LaRA-VLA internalizes textual reasoning together with future visual prediction, and HALO and ManualVLA combine textual and visual reasoning experts. The claim of being "the first to study" this problem is therefore overstated. The main additions are action-modality queries, masks that assign head groups to modality relations, and a second refinement stage with softmax source weighting. These are reasonable architectural choices, but they offer limited new insight, and the resulting system has many components and tuned hyperparameters (query budgets, horizons, Δ/s, six auxiliary losses, a cold-start stage). In addition, the "reasoning" is never decoded or acted on at inference. It amounts to latent queries shaped by auxiliary text, future-feature, and future-action prediction losses. The evaluation that would best show the value of reasoning is also limited: the obstacle-avoidance scenario in Fig. 1 is tested only in a simulated diagnostic, and the real-world study uses two simple pick-and-place tasks that require no cross-modal replanning.

Questions for authors

What are the success rates of (a) the single-backbone Base (direct action prediction, same data and budget) and (b) a four-backbone model trained with the final action loss only, on LIBERO-Plus, SimplerEnv, and RoboTwin C2R? These numbers are needed to attribute the gains to reasoning rather than to capacity.

7 comments
AuthorsOpenReview ↗

Response to Reviewer DbqP 1/4

We sincerely thank the reviewer for the thoughtful and constructive feedback. We have organized the reviewer's comments into the following questions and responses:

Q1: Are the performance gains primarily due to greater model capacity? What are the success rates of a single-backbone direct-action model and a four-backbone model trained with action supervision only?

" What are the success rates of (a) the single-backbone Base (direct action prediction, same data and budget) and (b) a four-backbone model trained with the final action loss only, on LIBERO-Plus, SimplerEnv, and RoboTwin C2R? These numbers are needed to attribute the gains to reasoning rather than to capacity. "

A1: We have added two experimental controls to further distinguish the contributions of model capacity, auxiliary supervision, and reasoning structure. The first uses the same Qwen3-VL initialization and action generation head, retains one language backbone, and is trained using only the final action loss. The second retains all four backbones, all reasoning latents, and interaction paths of the full model, but disables all auxiliary losses for text, future vision, and reference actions in both stages, retaining only the final action loss. All configurations use the same training data, input/output interfaces, and training update budget, and are evaluated on LIBERO-Plus, SimplerEnv, and RoboTwin C2R.

Experimental settingLIBERO-PlusSimplerEnvRoboTwin C2R
Single backbone, final action supervision only78.0%63.3%59.4%
Four backbones, final action supervision only81.7%65.6%61.4%
Omni–Mono92.4%78.3%77.2%

The results show that increasing the number of backbones improves task performance when using only final action supervision, but a gap remains relative to the full Omni–Mono model. With the four-backbone architecture and interaction paths held fixed, multimodal auxiliary supervision further improves performance across benchmarks. Together, these controls support that Omni–Mono's performance advantage does not arise solely from a larger parameter count; its supervision design and reasoning structure both contribute to the final results.

Q2: Is the conceptual novelty limited compared with DualCoT-VLA, LaRA-VLA, HALO, and ManualVLA? Is the claim of being "the first to study" overstated?

" Joint latent reasoning over multiple modalities in a VLA has close precedents. DualCoT-VLA already produces visual and linguistic reasoning with parallel latent queries in one pass, LaRA-VLA internalizes textual reasoning together with future visual prediction, and HALO and ManualVLA combine textual and visual reasoning experts. The claim of being "the first to study" this problem is therefore overstated. The main additions are action-modality queries, masks that assign head groups to modality relations, and a second refinement stage with softmax source weighting. These are reasonable architectural choices, but they offer limited new insight "

A2:

  1. Research positioning and core insight

    Thank you for pointing out these related studies. DualCoT-VLA, LaRA-VLA, HALO, and ManualVLA have already explored multimodal reasoning. We further investigate how intermediate reasoning states from different modalities can inform one another to form a coordinated understanding of the task while retaining the fine-grained reasoning information needed by each modality. Textual planning, visual prediction, and action reasoning address the same task but provide different constraints. For example, a textual plan may specify a straight path while visual prediction indicates a collision along that path, so visual information needs to participate in forming the textual plan. At the same time, shared reasoning must preserve the details of semantic plans, spatial predictions, and motion information. Our core insight is to first form coordinated reasoning states through cross-modal interaction, then use these states to guide modality-specific experts in refining textual, visual, and action reasoning, so that global coordination and modality-specific details complement one another. The Omni stage allows the three types of reasoning states to read information from one another, while the Mono stage adaptively uses this information according to each modality's needs for dedicated refinement, ultimately guiding action generation jointly.

AuthorsOpenReview ↗

Response to Reviewer DbqP 2/4

  1. Specific differences from related work

    Difference from DualCoT-VLA. DualCoT-VLA uses visual and linguistic latent queries to form two types of reasoning representations in a single backbone forward pass, with the visual branch learning three-dimensional spatial information from the current observation. Omni–Mono supervises its visual states with future visual information and additionally introduces reference-action states. The Omni stage organizes information exchange among the three types of states, followed by modality-specific refinement in the Mono stage.

    Difference from LaRA-VLA. LaRA-VLA progressively internalizes textual CoT into latent representations and combines them with future visual prediction. Its shared context already contains action-related information, and the original paper explicitly states that no additional independent action latent is introduced. Omni–Mono instead defines intermediate action states supervised by reference actions, enabling motion information to participate in forming textual and visual reasoning states, and further refines each modality's representations after joint interaction.

    Difference from HALO. HALO connects experts for multimodal understanding, visual generation, and action prediction through shared attention, completing tasks in the order of textual planning, visual subgoal prediction, and action generation. Omni–Mono introduces a joint reasoning stage before modality-specific refinement, allowing textual, visual, and reference-action states to inform one another, and then uses the coordinated states to guide refinement by each expert.

    Difference from ManualVLA. ManualVLA's planning expert generates manuals containing text, position cues, and subgoal images, and guides the action expert through both explicit prompts and latent representations. Omni–Mono instead models text, vision, and reference actions as separately supervised intermediate states, forms coordinated reasoning representations through joint cross-modal interaction, and then refines them with dedicated experts to jointly condition action generation.

  2. The wording of "the first to study"

    The original wording of the first contribution in our introduction was: "As far as we know, we are the first to study a novel problem in the VLA domain: how to generate reasoning across multiple modalities within a unified model while allowing the reasoning streams to attend to one another". This sentence specifically emphasizes mutual information exchange among reasoning states from different modalities during their generation. Together with the immediately following contribution introducing the Omni–Mono framework, our intended overall positioning is to allow reasoning states from different modalities to inform one another within a unified framework, and to use the coordinated states to guide modality-specific experts in further refining textual, visual, and action reasoning, so that cross-modal coordination and modality-specific details complement one another. The original sentence highlights joint interaction and dedicated refinement. We have also revised the "first to study" wording in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer DbqP 3/4

Q3: Is the so-called "reasoning" merely latents shaped by auxiliary losses? Do reasoning outputs actually affect action execution if they are not explicitly decoded? The existing experiments also do not adequately evaluate the value of reasoning

In addition, the "reasoning" is never decoded or acted on at inference. It amounts to latent queries shaped by auxiliary text, future-feature, and future-action prediction losses. The evaluation that would best show the value of reasoning is also limited: the obstacle-avoidance scenario in Fig. 1 is tested only in a simulated diagnostic, and the real-world study uses two simple pick-and-place tasks that require no cross-modal replanning.

A3:

  1. Advantages and interpretability of latent reasoning

    We adopt latent reasoning to avoid the overhead of generating intermediate text or images. Explicit textual reasoning typically requires token-by-token decoding, while visual subgoal generation also introduces additional computation, increasing the robot's waiting time before updating its actions. Omni–Mono directly represents textual plans, future visual states, and reference-action information as continuous latents, and uses the refined states as conditions for the final action head without first decoding them into text or images. These reasoning states therefore directly participate in action generation during execution.

    To verify whether the internal states contain the corresponding reasoning content, we conducted representation analysis in Section 3.4 and Table 3 of the main paper. We freeze the policy representations and use probes with matched configurations to recover the corresponding content from textual, visual, and action latents and compare it with reference targets. On the text side, we check whether the operations, objects, and target locations in decoded plans match manual annotations; on the visual side, we compare predicted future visual features with features of actual future observations; on the action side, we compare predicted reference actions with demonstrated actions. Each probe reads only the latent of its corresponding modality, without additional access to the current observation or other modality states. The results show that Mono refinement improves textual plan recovery accuracy and further reduces prediction errors for future vision and reference actions. This indicates that the model internally forms intermediate states containing task plans, future-scene information, and motion information, and that this information can be recovered and verified. Latent reasoning avoids explicit decoding during execution while preserving a means of analyzing the reasoning content.

  2. Changes in intermediate reasoning states affect action execution

    The visual-to-text intervention experiment in Appendix D.5 further examines the connection between intermediate states and execution outcomes. We construct clear-path and blocked-path scenes with identical task instructions, manipulated objects, and target locations. In the blocked-path scene, we replace the visual guidance received by the textual expert with the visual state from the corresponding clear-path scene and recompute the textual latent, while keeping the action head's other conditioning inputs and action noise unchanged. The robot executes the first intervened action chunk and then resumes the normal policy.

    Change relative to normal blocked-scene executionChange (percentage points)
    Plan validity rate−7.0
    Task success rate−4.0
    Obstacle collision rate+3.0

    Changing the visual guidance reduces textual plan validity, lowers task success, and increases collisions. This shows that visual information can affect actual execution through the intermediate textual state. Text decoding in this experiment is used only to analyze plan content; decoded outputs are not fed back to the policy, and execution changes arise from the latent states themselves. Because swapping states across scenes may produce out-of-distribution representation combinations, we also consider the retrained structural controls in Appendices D.3–D.4. With four backbones, identical auxiliary supervision, and the final action head retained, restricting cross-modal information flow or replacing modality-specific refinement reduces task performance. Together, these results support a practical role for intermediate states and their interactions in action execution.

Reviewer_6nFXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Omni-Mono, a vision-language-action framework that combines joint multimodal reasoning with modality-specific refinement. The Omni Reasoner updates textual, visual, and action latent representations through grouped attention with modality relation masks. Three Mono experts subsequently refine these representations in text–vision–action order using adaptive guidance fusion. Supervision comes from textual plans, future visual features, and sparse action references, while a conditional flow-matching head generates executable action chunks. Experiments across four simulation benchmarks and two real-world manipulation tasks report improved performance.

Critical strengths
  1. The motivation is clear and relevant.
  2. The model design is closely aligned with the goal of multimodal latent reasoning, and the method achieves the highest success rates among the methods compared across multiple simulation benchmarks.
Critical weaknesses
  1. The main paper does not contain a Related Work section. Earlier approaches, including dVLA [1], DualCoT-VLA [2], LaRA-VLA [3], etc., have already explored textual and visual reasoning for action generation. The authors should clarify how the proposed problem differs from these prior formulations, and justify the advantages of this latent reasoning approach over explicit reasoning approaches.

  2. Current experiments do not fully address concerns about the resource–performance tradeoff and the benefits of reasoning. (1) The model utilizes four independent language backbones and requires a much larger latency. The practical benefit of this additional complexity remains unclear, so a further analysis of the resource–performance tradeoff is needed. (2) The simulation benchmarks (LIBERO, LIBERO-Plus, SimplerEnv, and RoboTwin 2.0) do not clearly require the reasoning capabilities beyond improved multimodal learning, and real-world evaluation is also limited to two relatively simple pick-and-place tasks. Consequently, the necessity and distinctive benefits of the proposed multimodal latent reasoning formulation remain insufficiently established in these experiments.

[1] dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought [2] DualCoT-VLA: Visual-Linguistic Chain of Thought via Parallel Reasoning for Vision-Language-Action Models [3] Latent Reasoning VLA: Latent Thinking and Prediction for Vision-Language-Action Models

Questions for authors
  1. An additional reasoning-based VLA baseline should be included in the real-world experiments to better assess the contribution of the proposed approach. How inference latency affects the real-world task performance?
  2. What are the individual contributions of textual, future-visual, and reference-action supervision, as assessed by removing each auxiliary objective while keeping the architecture unchanged?
  3. I would be willing to raise my rating if my concerns are adequately addressed.
9 comments
AuthorsOpenReview ↗

Response to Reviewer 6nFX 1/6

We sincerely thank the reviewer for the thoughtful and constructive feedback. We have organized the reviewer's comments into the following questions and responses, and the corresponding additions have been incorporated into the revised manuscript:

Q1: A reasoning-based VLA baseline should be added to the real-world experiments to better evaluate the contribution of the proposed method

"An additional reasoning-based VLA baseline should be included in the real-world experiments to better assess the contribution of the proposed approach."

A1: To strengthen the comparison in real-world settings, we have added two real-robot tasks involving object retrieval and placement and sequential stacking, and evaluated two mainstream latent-reasoning baselines, LaRA-VLA and DualCoT-VLA. The first task is Microwave-to-Plate Transfer: the robot must retrieve an object from inside a microwave and place it on a plate. The second task is Sequential Fruit Stacking: the robot must first place a banana on a plate and then place a peach on top of the banana. These tasks involve object retrieval and placement and fruit stacking in a specified order, respectively. Real-world illustrations of these complex tasks are provided in Figure 7 of Appendix F.5 in the revised manuscript.

MethodMethod typeMilk Box on Plate: SR(%)↑Bowl on Plate: SR(%)↑Sequential Fruit Stacking: SR(%)↑Microwave-to-Plate Transfer: SR(%)↑
π0.5General-purpose82683854
LaRA-VLALatent-reasoning VLA76603658
DualCoT-VLALatent-reasoning VLA82724466
Omni–MonoLatent-reasoning VLA90825676

The results show that Omni–Mono outperforms both the general-purpose VLA and existing latent-reasoning baselines on all four real-world tasks, indicating that its advantages extend to real-world manipulation scenarios involving environmental constraints and multi-step coordination.

Q2: How does inference latency affect real-world robot performance?

" How inference latency affects the real-world task performance? "

A2: The robot pauses briefly during model inference, but the overall operation remains relatively continuous, with no impact on task completion or accuracy. The reported 680 ms is the average time required for a complete policy call to generate an action chunk containing eight actions. Real-world deployment uses receding-horizon execution: after executing the first four actions, the robot acquires a new observation and updates its prediction. We view this additional overhead as a tradeoff between performance and efficiency to improve performance on complex, multi-step tasks. We have also added a real-world video in Appendix F.5: https://anonymous.4open.science/r/omni-mono/2-web.mp4, allowing readers to directly observe the robot's real-world performance.

Q3: What are the individual contributions of textual, future-visual, and reference-action supervision when each auxiliary objective is removed while keeping the architecture unchanged?

" What are the individual contributions of textual, future-visual, and reference-action supervision, as assessed by removing each auxiliary objective while keeping the architecture unchanged? "

A3: We have added auxiliary-supervision ablations to evaluate the individual contributions of the three types of supervision. Specifically, we retain all four backbones, all reasoning latents, cross-modal interaction paths, and the final action head, while disabling the auxiliary losses for the corresponding modality in both the Omni and Mono stages. Evaluation covers all benchmarks in the main table: LIBERO, LIBERO-Plus, SimplerEnv, and RoboTwin 2.0. Values are success rates (%), with the decrease relative to the full model shown in parentheses in percentage points.

Experimental settingLIBERO Avg.LIBERO-Plus Avg.SimplerEnv Avg.RoboTwin C2CRoboTwin C2R
Full model99.392.478.390.677.2
w/o textual supervision98.2(−1.1)88.8(−3.6)74.7(−3.6)87.6(−3.0)73.4(−3.8)
w/o future-visual supervision98.4(−0.9)89.6(−2.8)75.4(−2.9)88.2(−2.4)74.3(−2.9)
w/o reference-action supervision98.6(−0.7)90.2(−2.2)75.8(−2.5)88.0(−2.6)74.0(−3.2)

The ablation results show that removing any auxiliary supervision reduces performance, indicating that all three types provide useful learning signals and play complementary roles.

AuthorsOpenReview ↗

Response to Reviewer 6nFX 2/6

Q4: Connections and differences with dVLA, DualCoT-VLA, and LaRA-VLA

" The main paper does not contain a Related Work section. Earlier approaches, including dVLA [1], DualCoT-VLA [2], LaRA-VLA [3], etc., have already explored textual and visual reasoning for action generation. The authors should clarify how the proposed problem differs from these prior formulations, "

A4:

  1. Location of Related Work and additional clarification

    Thank you for pointing this out. Due to space constraints in the main paper, our Related Work section was placed in Appendix G. We have now moved it to Section 4 of the main paper and added a discussion of dVLA. We agree that the original manuscript did not sufficiently discuss related methods such as dVLA, DualCoT-VLA, and LaRA-VLA or adequately explain their connections and differences with Omni–Mono. We have strengthened these discussions in the revised manuscript to make the positioning of our work clearer. Specifically, these studies have already explored multimodal reasoning or reasoning in latent space. We will further clarify our contribution: modeling text, vision, and reference actions as separately supervised intermediate reasoning states, exchanging information across modalities in the Omni stage, and then performing modality-specific refinement in the Mono stage to support action prediction.

  2. Specific differences between methods

    The following table compares the four methods in terms of textual reasoning, visual reasoning, action reasoning, latent reasoning, and whether they combine multimodal fusion with modality-specific refinement. Beyond these differences, our core insight is to first form coordinated reasoning states through cross-modal interaction, then use these states to guide modality-specific experts in refining textual, visual, and action reasoning, so that global coordination and modality-specific details complement one another.

    MethodTextual reasoningVisual reasoningAction reasoningLatent reasoningMultimodal fusion and modality-specific refinement
    dVLA✓✓✓✗✗
    DualCoT-VLA✓✓✗✓✗
    LaRA-VLA✓✓✗✓✗
    Omni–Mono✓✓✓✓✓
    • Difference from dVLA. dVLA jointly generates textual reasoning, visual subgoals, and action sequences through a unified discrete diffusion objective. Omni–Mono maintains textual, visual, and reference-action states in continuous latent space and organizes reasoning through joint interaction and modality-specific refinement. The distinction lies in the reasoning representations and their interaction mechanisms; joint modeling of all three modalities already exists in dVLA.
    • Difference from DualCoT-VLA. DualCoT-VLA uses visual and linguistic latent queries to form two types of reasoning representations in a single backbone forward pass, with the visual branch learning three-dimensional spatial information from the current observation. Omni–Mono supervises its visual states with future visual information and additionally introduces reference-action states and two-stage Omni→Mono interaction and refinement.
    • Difference from LaRA-VLA. LaRA-VLA progressively internalizes textual CoT into latent representations and combines them with future visual prediction. Its shared context already contains action-related information, and the original paper explicitly states that no additional independent action latent is introduced. Omni–Mono instead defines intermediate action states supervised by reference actions, enabling them to participate in information exchange among text, vision, and action.
  3. Experimental comparisons with the three related methods

    To provide further direct empirical comparisons, we have added experiments with dVLA and summarize the results of these three related methods alongside Omni–Mono below. All values are success rates (%, higher is better).

    MethodLIBERO Avg.LIBERO-Plus Avg.SimplerEnv Avg.RoboTwin 2.0 C2CRoboTwin 2.0 C2R
    dVLA(added)96.480.666.281.864.6
    DualCoT-VLA98.887.273.686.871.2
    LaRA-VLA97.983.268.883.667.4
    Omni–Mono99.392.478.390.677.2
AuthorsOpenReview ↗

Response to Reviewer 6nFX 3/6

Q5: Why use latent reasoning rather than explicit reasoning?

" justify the advantages of this latent reasoning approach over explicit reasoning approaches. "

A5: We adopt latent reasoning primarily for the following two reasons.

  1. Avoiding the decoding overhead of explicit intermediate content so that reasoning states directly support action generation.

    Robots need to update their actions continuously based on the current observation, so the computational overhead of intermediate reasoning affects how promptly the policy can respond. Explicit reasoning requires generating textual plans or visual subgoals; autoregressive text generation also involves token-by-token decoding. In Omni–Mono, textual, visual, and action reasoning states are all represented as continuous latents. The final action module directly conditions on these states to generate actions without first decoding the intermediate states into text or images. Latent reasoning therefore avoids the overhead of generating explicit CoT outputs.

  2. Allowing continuous spatial and motion information to participate directly in cross-modal interaction, reducing potential information bottlenecks from discrete intermediate outputs.

    Robotic manipulation requires both understanding task semantics and considering spatial relationships between objects, future scene changes, and specific motion information when making decisions. For example, "move the object to the target location" expresses the task goal, but execution must also account for details such as relative positions, movement direction, and path constraints. Passing all this information through intermediate text or discrete visual outputs may introduce additional representational bottlenecks. Continuous latents provide a way to transmit and fuse this information directly, without requiring all control-relevant details to first be converted into decodable intermediate content. In Omni–Mono, this consideration is reflected in the three separately supervised reasoning states and their interactions: the Omni stage allows textual, visual, and reference-action states to read information from one another, enabling task plans to use spatial and motion cues while visual and action states also incorporate task semantics; the Mono stage further refines these states for each modality, and they jointly condition action generation. Continuous representations thus provide a unified interface for exchanging task semantics, future-scene information, and reference-motion information. Reducing bottlenecks from discrete outputs is a design motivation for this representation.

Recent work also provides practical support for this choice. DualCoT-VLA learns intermediate reasoning representations through visual and linguistic latent queries and omits auxiliary decoding during inference; LaRA-VLA internalizes explicit textual CoT into continuous latent representations through curriculum training and combines them with future visual prediction to guide action generation. These studies demonstrate that using supervised continuous latent representations to support VLA reasoning and control is an approach with existing practical support.

Comments on the paper

1 comment
Area_Chair_dAiKOpenReview ↗

Please Review the Author Rebuttal and Participate in the Discussion

Dear Reviewers,

Thank you for your efforts in reviewing this submission. The authors have now provided their rebuttal. I kindly encourage everyone to carefully read the responses, actively participate in the discussion, and engage with the authors and fellow reviewers to help reach a well-informed assessment.

Thank you again for your time and contributions!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.