- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper introduces a monitor for a frozen mixture-of-experts (MoE) VLA model based on its expert-routing patterns. The monitor tracks changes in these patterns during execution. When the patterns suggest that the policy may be stuck, it raises an alarm and triggers a recovery action.
Critical strengths
1.The paper examines its expert-routing patterns to decide intervention, without retraining the policy or requiring extra policy forward passes. Thus, the proposed method is general and easy to apply. 2. It evaluates whether responding to an alarm actually helps, accounting for both failures that are rescued and otherwise successful runs that are disrupted.
Critical weaknesses
- The proposed method is largely an engineering solution, and the paper reads more like a technical report than a research paper presenting a novel contribution.
- The intervention is triggered by observed expert-routing patterns and does not require gradients from the VLA model. However, the paper offers limited theoretical justification for why these patterns should indicate when intervention is beneficial. The intervention can both rescue failing executions and disrupt otherwise successful ones, so the evidence for its effectiveness is primarily empirical.
- The intervention is essentially a simple, reset-like action triggered by observed expert-routing patterns. Its practical utility is therefore unclear, and a more adaptive recovery strategy would make the contribution more compelling.
- The model’s internal routing is difficult to interpret, the paper should provide stronger evidence that this association is robust rather than coincidental.
- The proposed method is evaluated only in simulation, which limits its practical significance.
- The paper should describe its statistical analysis and explain how paired continuations were generated from the same simulator state to ensure a fair comparison. Potential failure cases should also be discussed more carefully.
Questions for authors
See weakness.
4 comments
W5: Real-robot applicability and broader internal signals
We thank the reviewer for highlighting real-robot risks from release and lift. Simulation allows state-matched intervention comparisons, while NAR itself only reads routing produced by the policy, without simulator ground-truth inputs (§3; Appendix D.5). On hardware, the current grasp and collision conditions should determine whether release or lift is permitted; recovery actions must satisfy robot-specific safety constraints.
Monitoring internal dynamics through neuron activations offers a path to extending this idea to dense models. NAD (Neuron Agreement Decoding) selects LLM responses based on agreement in neuron activations, while RAD (Routing Agreement Decoding) uses agreement in MoE routing to select candidates [3,4]. Both motivate investigating internal signals for execution monitoring and recovery in dense and MoE embodied policies.
Advances in MoE LLMs, together with embodied architectures such as HiMoE-VLA and AdaMoE [1,2], suggest that sparse computation will be an important direction for scaling future embodied models, further motivating our study of runtime monitoring for frozen MoE policies.
W6: Fair paired continuations, statistical analysis, and failure cases
The paired protocol and statistical analysis are detailed in §4, §5.1, and Appendices D.3 and D.5, including prefix replay, camera/state checks, same-state branches with shared noise, and configuration-level bootstrapping. The paper also quantifies three adverse patterns: disruption of otherwise successful continuations (Table 11), lower rescue rates with little budget remaining (Table 12), and repeated recovery crowding out policy actions (Table 4; Appendix E.3).
References
[1] Du et al. HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies. https://arxiv.org/abs/2512.05693
[2] Shen et al. Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learning (AdaMoE). https://arxiv.org/abs/2510.14300
[3] Chen et al. Do LLMs Signal When They're Right? Evidence from Neuron Agreement (NAD). https://arxiv.org/abs/2510.26277
[4] Chen et al. Does the Same Token Mean the Same State? MoE Routing as Signal for Reasoning Control (RAD). https://arxiv.org/abs/2606.22798
I thank the reviewers for their detailed responses. It address most of my concerns, so I have decided to raise the score by one point. I will respond to each point individually to explain how the responses address my concerns.
W1/W2/W3: The clarifications provide a better understanding of the contributions. I recommend that the authors revise the abstract and introduction to highlight the research problem more clearly, as the current writing style is somewhat too empirical. Some concerns remain regarding the connection between the MoE architecture and the proposed interventions beyond the empirical results, as well as the generalizability of the method. I also encourage the authors to include a more detailed discussion of failure cases in the introduction and provide more insights than results. Although the interventions are simple, I appreciate their simplicity and effectiveness. However, the work could be more meaningful if it went beyond empirical evidence. W4: Addressed. W5: The explanations provided are acceptable. W6: Addressed.
Thank you for your detailed follow-up and for updating your assessment. We are glad that our responses have addressed most of your concerns, and we sincerely appreciate your careful consideration and constructive feedback on our work.
Thank you for recognizing the routing interface that requires neither policy retraining nor additional policy forward passes, and the joint evaluation of recovery benefits and disruption costs. We clarify the contribution, intervention rationale, dynamical evidence, and evaluation protocol below.
W1: Research contribution and novelty
The contribution is to turn the internal routing dynamics of frozen MoE VLAs into a monitoring signal transferable across tasks, and to evaluate its practical control value through interventions paired at the same states. In frozen HiMoE-VLA [1], NAR reads routing already produced during normal action generation, without policy retraining or additional policy forward passes (§3).
The research has three connected components. First, we characterize differences in the temporal routing structure of successful and failed executions, and test their monitoring value through comparisons of observation signals and temporal statistics (§5.2; Tables 1 and 3). Second, we investigate transfer of successful reference dynamics: in Table 2(a), monitoring F1 is 82.7% on unseen initial states, 87.6% on held-out tasks, and 84.2% on a held-out task suite; Table 2(c) further evaluates LIBERO-Plus perturbations. Each suite uses its own frozen policy checkpoint; the monitoring representation and successful-source references are transferred, with native and perturbed successes included in the reference data. Third, paired continuations connect recovery benefits to the cost of disrupted successes, testing whether the internal signal can help improve task outcomes (§4; §5.4; Table 11).
These studies provide empirical evidence for understanding the relationship between internal computation and execution behavior in MoE VLAs, together with a concrete monitoring and intervention approach for improving task execution while keeping the model frozen.
W2/W3: Fixed recovery, net utility, and successful execution quality
The release-and-lift primitive changes local contact and end-effector conditions before returning control to the same frozen policy. Rather than resetting the environment, it may create an opportunity for learned approach and grasp skills to resume (§4; Appendix E.3).
Using the rescue–harm decomposition in §4, Eq. (2), the 1,608 state-matched pairs yield 285 rescues and 110 harms, or 175 net additional successes (+10.88 pp per completed alarm, 95% CI [4.67, 16.83]; §5.4; Table 11).
Eventual success need not imply smooth execution. In §5.2, 27 of 41 alarms on ultimately successful runs correspond to stalls followed by spontaneous recovery. In supplementary analysis, four successful but alarmed episodes require 27.6%–49.2% more actions than the median of their same-task successful controls; two exhibit sustained commanded motion with insufficient actual displacement before the alarm. Thus, a success label can conceal execution difficulty. Harms from the fixed intervention also show that recovery must respect opportunities for spontaneous recovery and the current manipulation state; all harms remain counted in our net estimate. Other response explorations are in Table 14.
W4: Why routing recurrence relates to execution difficulty
NAR does not equate repeated expert selection with failure. It compares recent routing dynamics with the episode's opening and evaluates deviations from successful-reference dynamics (§3; Eq. (1); Appendix A). In a perception–action loop, repeated unsuccessful interaction under similar conditions can induce persistent computational recurrence; ordinary expert reuse is also present in successful executions.
Three existing controls support this interpretation: (i) successes and failures use similar numbers of experts per inference (27.59 vs. 27.32), yet differ in temporal evolution (§5.2; Fig. 3); (ii) at a matched 55 false alarms, lag-three contraction achieves 94.3% recall, compared with 58.2% for adjacent-inference routing change (descriptive; Table 3a); and (iii) episode-consistent expert renaming retains 96.81% recall, whereas independently renaming experts at each inference breaks temporal correspondence and reduces recall to ≤0.17% without recalibration (§5.3; Fig. 6).
Together, these results support routing temporal dynamics as an internal observable of closed-loop execution difficulty, rather than a simple count of repeated experts.
References
[1] Du et al. HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies. https://arxiv.org/abs/2512.05693