acceptodds
Under review as a conference paper at ICLR 2027

Less Fusion, Same Action: Revisiting Redundant Connectivity in Mixture-of-Transformer based Robot Policy Models

Abstract

Mixture-of-Transformers (MoT) robot policies typically maintain dense expert connectivity across network depth, yet it remains unclear how much of this connectivity is actually necessary for downstream adaptation. We conduct a systematic empirical study of expert connectivity redundancy by sparsifying pretrained MoT policies during fine-tuning across multiple architectures and robot manipulation benchmarks. We find that substantial portions of dense expert connectivity can be removed while largely preserving downstream performance, and that different sparse masks with matched budgets achieve similar results, suggesting that no particular subset of expert connections is uniquely indispensable. Motivated by this flexibility, we introduce an early-half fusion pattern that concentrates cross-expert connectivity in the first half of the network and applies structured pruning to the remaining layers. This design maintains comparable task success while enabling practical deployment simplifications, including truncation of unused conditioning layers and reduction of historical KV-cache storage in autoregressive policies. Deployment measurements further show reductions in loaded parameters, GPU memory, and per-chunk latency. Together, our results suggest that dense expert connectivity in pretrained MoT robot policies is substantially redundant, and that structured pruning provides a simple path toward more efficient deployment without training a new model from scratch. Our code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Mixture-of-Transformers, Vision-Language-Action Models, World-Action Models, Model Sparsification, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FX1hOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Current transformer models for robotics encode images with text, video, and actions. Pruning attention connectivity between these streams randomly, even at 50%, has little effect on performance. Moreover, if the authors cut all communication past 50% depth, then they can significantly improve latency and peak memory usage, with the second half of certain expert stream entirely unnecessary. This also means that historical KV states are required only for the lower half of the network. Speed-ups range from 13-42% and results are evaluated over four different pre-trained models, with both simulated and real-world tasks, and one evaluation that includes pretraining.

Critical strengths

This paper is clearly written, seeks to answer a simple question, and does so thoroughly and straightforwardly.

Critical weaknesses

I don’t believe there are any critical witnesses. However, I am not as familiar with the related work, and I wonder whether there are existing sparsification methods that should have been compared.

Questions for authors

A lot of related work is cited, but there is almost no commentary on how it relates to this work, and why it was or was not evaluated. Could you explain further?

Minor: What are WAM, Joint Denoising, and Cross-Chunk in Table 1?

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1/2)

We thank the reviewer for the insightful comments and valuable feedback. Please see our responses to your concerns point-by-point below.

Q1: I don’t believe there are any critical witnesses. However, I am not as familiar with the related work, and I wonder whether there are existing sparsification methods that should have been compared. A lot of related work is cited, but there is almost no commentary on how it relates to this work, and why it was or was not evaluated. Could you explain further?

A1: Thank you for raising this point. Our related work is organized into three parts, covering the architectural background, robot foundation policies, and efficiency-related methods.

The first part introduces the Mixture-of-Transformers (MoT) architecture and its applications to multimodal modeling. It discusses how specialized Transformer streams exchange information through shared attention, with examples such as LMFusion [1] and BAGEL [2], which combine multimodal understanding and generation. These works provide the architectural background for our study of expert connectivity. The second part reviews recent robot foundation policies that connect pretrained vision-language or video representations to action generation, covering both vision-language-action (VLA) models and world-action models (WAMs), including the four representative policies examined in our experiments.

The reviewer may be particularly interested in the third part, which discusses efficient robot policies. We summarize their key technical designs and the components they modify below.

Table R1: Comparison with representative robot-policy efficiency methods.

MethodPolicy TypeKey Technical DesignWhat Is Modified
TinyVLA [3]VLACompact pretrained multimodal backbone with a diffusion-based action decoderBackbone and action-decoder architecture
SmolVLA [4]VLALightweight VLM, reduced visual tokens, and asynchronous inferenceBackbone, action module, and inference pipeline
BitVLA [5]VLATernary weight representations with distillation-aware low-bit vision encodingParameter precision
QVLA [6]VLAAction-sensitivity-guided, channel-wise bit allocation, including 0-bit channel pruningChannel precision and sparsity
Fast-WAM [7]WAMVideo-prediction co-training with future-video generation omitted at inferenceTraining–inference computation
Faster-WAM (DoT) [8]WAMSingle-layer action head accessing multi-depth video representations through learned KV-FusionAction-expert depth and fusion interface
OursVLA & WAMSparsification of native expert attention connections in existing pretrained MoT policies, followed by downstream adaptationNative expert connectivity, while preserving the pretrained expert-block architecture during adaptation

These methods improve efficiency by using special model architectures, parameter precision, or inference computation, whereas our work directly investigates the redundancy of native expert attention connectivity in pretrained MoT policies. In contrast, our primary research question is how much expert-level fusion is necessary in pretrained MoT robot policies after downstream adaptation. To investigate this, we apply attention masks to pretrained architectures under matched fine-tuning protocols. Dense provides the original connectivity reference, Random-cut examines whether carefully selected connections are necessary, and Early-half evaluates a structured pattern that enables deployment simplification. These controlled comparisons allow us to study connectivity redundancy without redesigning or pretraining a new robot foundation policy.

We have also considered efficient policy designs from the related work in our experiments. In our original submission (Section 5.2), we evaluated our sparsification framework on Fast-WAM [7] using LIBERO, achieving average success rates of 96.8%, 94.4%, and 97.1% for Dense, Random, and Early-half, respectively. To further address the reviewer's concern, we additionally conducted a DoT-inspired experiment during the rebuttal period, following Faster-WAM [8]. Specifically, we reduced InternVLA-A1's 28-layer action expert to a single layer through parameter averaging and DoT-style connections. Under the same RoboTwin fine-tuning protocol, this variant achieves 78.41% average success (77.44% Easy, 79.38% Hard), compared with 88.66% for the original Dense model. Unlike Faster-WAM, which trains a dedicated DoT architecture, our experiment directly compresses an existing pretrained policy, suggesting that such compression may not preserve performance under our setting.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2/2)

Continue on A1:

Furthermore, following the suggestion from Reviewer Y1CS, we conducted additional experiments on InternVLA-A1 with Alternating, Middle-half, and Late-half fusion across RoboTwin and LIBERO. These experiments provide a more comprehensive comparison of connectivity patterns under the same pretrained architecture and downstream training protocol. Detailed results are provided in our response to Reviewer Y1CS. We hope these clarifications and additional experiments help explain the relationship between our work and existing efficiency methods.

Q2: Minor: What are WAM, Joint Denoising, and Cross-Chunk in Table 1?

A2: Thank you for raising this point. In Table 1, we summarize representative pretrained MoT robot policies along four architectural dimensions: expert composition, denoising mechanism, cross-chunk dependency, and backbone parameterization.

WAM (World-Action Model) [9, 10] refers to policies that explicitly incorporate world modeling through future-state or video prediction to support action generation, whereas VLA (Vision-Language-Action) models [11, 12] emphasize pretrained vision-language representations for semantic understanding and action conditioning.

Regarding denoising mechanisms, Joint Denoising [9, 13] refers to simultaneously denoising multiple diffusion experts, which exchange information through joint attention across Transformer layers. In contrast, Causal Denoising [10, 11, 12] conditions action denoising on upstream representations computed beforehand.

Finally, Cross-Chunk describes temporal dependencies across predicted chunks. Specifically, autoregressive (AR) models [10, 13] condition the current chunk on cached keys and values from preceding chunks, whereas non-autoregressive (Non-AR) models [9, 11, 12] operate without such cross-chunk caching.

Further details on these architectural distinctions are provided in Appendix A.

Thank you again for your valuable feedback and hope that all your concerns are properly addressed.

Reference

[1] Weijia Shi, et al. Lmfusion: Adapting pretrained language models for multimodal generation. Neurips, 2026.

[2] Chaorui Deng, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025

[3] Junjie Wen, et al. Tinyvla: toward fast, data-efficient vision-language-action models for robotic manipulation. IEEE Robotics and Automation Letters, 10(4):3988–3995, 2025.

[4] Mustafa Shukor, et al. Smolvla: A vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844, 2025.

[5] Hongyu Wang, et al. Bitvla: 1-bit vision-language-action models for robotics manipulation. arXiv preprint arXiv:2506.07530, 2025b.

[6] Yuhao Xu, et al. Qvla: Not all channels are equal in vision-language-action model’s quantization. arXiv preprint arXiv:2602.03782, 2026b.

[7] Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-wam: Do world action models need test-time future imagination?, 2026.

[8] Liheng Ma, et al. Faster-wam: Do world action models need deep action modules? arXiv preprint arXiv:2608.02365, 2026a.

[9] Hongzhe Bi, et al. Motus: A unified latent action world model. CVPR, 2026.

[10] Lin Li, et al. Causal world modeling for robot control. arXiv preprint arXiv:2601.21998, 2026.

[11] Physical Intelligence, et al. π0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025.

[12] Junhao Cai, et al. Internvla-a1: Unifying understanding, generation and action for robotic manipulation. arXiv preprint arXiv:2601.02456, 2026.

[13] Seonghyeon Ye, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026.

Thank you for the clarification and DoT experiment.

An additional question: Looking back at the paper, I noticed few mentions of standard errors and confidence intervals. Is uncertainty quantified across the results tables?

Reviewer_E3CuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the redundancy of expert connectivity in MoT robot policies: during fine-tuning, either randomly sparsifying 50% of cross-expert attention or applying early-half fusion (retaining cross-expert fusion only in the first half of layers) largely preserves performance, while enabling truncation of upper conditioning layers and halving of the historical KV cache, with measured reductions of about 28% in parameters and 1.13–1.5× speedup.

Critical strengths
  1. Easy to follow: the paper is clearly written, and the unified mask formalism makes the cross-architecture study easy to parse.
  2. Broad coverage: a single formalization covers four architectures.
  3. Rigorous measurement: numerical equivalence between the masked and structurally pruned implementations is verified; latency is reported separately for eager and compiled execution, with TF32 deviations disclosed.
Critical weaknesses
  1. No methodological advantage: early-half fusion underperforms the random baseline in nearly half of the settings (−1.61 for π0.5 and −1.60 for Motus on RoboTwin; −2.75 for A1 on LIBERO), and its remaining leads are mostly within 1 point and concentrated on the benchmarks with best-checkpoint selection. The method's value reduces to deployment convenience and does not hold up in terms of performance.
  2. Transplantation of a general method: expert-level attention masking is simply structured pruning of cross-attention in general multimodal Transformers; that cross-modal fusion concentrates in early layers while later layers are redundant is a known phenomenon in general multimodal models; and the deployment gains (layer truncation, KV cache) are likewise generic serving techniques. In essence, this is a re-verification of general pruning findings on robot policies.
  3. Missing key control: all sparse variants receive 40k–200k more fine-tuning steps than Dense, and some even surpass Dense (Motus +0.53, LingBot +1.36), so the effects of "redundancy" and "additional training" cannot be disentangled.
Questions for authors
  1. Beyond deployment convenience, how do you justify the necessity of early-half fusion over random sparsification? Please provide further analysis.
  2. What distinguishes this work from running the same experiments on general multimodal Transformers?
  3. Please provide further comparisons under matched training steps and hyperparameters to demonstrate the effectiveness of the method.
14 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1/3)

We thank the reviewer for the insightful comments and valuable feedback. Please see our responses to your concerns point-by-point below.

Q1: No methodological advantage: early-half fusion underperforms the random baseline in nearly half of the settings (−1.61 for π0.5 and −1.60 for Motus on RoboTwin; −2.75 for A1 on LIBERO), and its remaining leads are mostly within 1 point and concentrated on the benchmarks with best-checkpoint selection. The method's value reduces to deployment convenience and does not hold up in terms of performance. Beyond deployment convenience, how do you justify the necessity of early-half fusion over random sparsification? Please provide further analysis.

A1: Thank you for raising this question. Our study first demonstrates that substantial expert connectivity can be removed through Random sparsification while largely preserving downstream performance (Section 3). Building on this finding, we investigate whether such redundancy can be organized into a structured pattern that enables practical efficiency gains (Section 4). Unlike Random sparsification, which removes scattered connections, Early-half fusion concentrates cross-expert interactions in the first half of the network, enabling unused conditioning layers to be truncated or historical KV-cache storage to be reduced. This structured design maintains comparable downstream performance while reducing inference latency, deployed parameters, and memory usage.

Indeed, both Early-half fusion and Random sparsification maintain performance close to the Dense baseline. Across the eight main model–benchmark settings in Tables 2 and 3 of the original submission, Early-half retains at least 96.4% of the corresponding Dense success rate. Compared with Random, Early-half performs better in four settings and worse in four, indicating no systematic performance disadvantage. Notably, its improvements are not limited to margins within one point: Early-half outperforms Random by 5.19 points on LingBot-VA RoboTwin and 2.70 points on LingBot-VA LIBERO. Our Fast-WAM experiments (Section 5.2) also show a 2.70-point improvement on LIBERO. These results indicate that the relative performance varies across models and benchmarks, rather than consistently favoring either sparsification pattern.

Regarding the training and evaluation protocols, we follow the official fine-tuning configurations whenever available. For settings without directly applicable training recipes, we establish our training configurations based on two considerations: (1) achieving performance comparable to official published results where available, and (2) ensuring that training has stabilized without meaningful further improvement. Within each model–benchmark setting, Random and Early-half generally use the same initialization and fine-tuning configurations. In particular, the improvements highlighted above are evaluated at fixed training steps under these protocols, rather than from checkpoints selected based on their achieved performance. In Table 11 of Appendix D, we reports both the configured maximum training steps and the checkpoints used for evaluation. These may differ because the learning-rate schedule can extend beyond the actual training steps, which may have caused confusion regarding our checkpoint selection protocol.

Q2: Transplantation of a general method: expert-level attention masking is simply structured pruning of cross-attention in general multimodal Transformers; that cross-modal fusion concentrates in early layers while later layers are redundant is a known phenomenon in general multimodal models; and the deployment gains (layer truncation, KV cache) are likewise generic serving techniques. In essence, this is a re-verification of general pruning findings on robot policies. What distinguishes this work from running the same experiments on general multimodal Transformers?

A2: Thank you for allowing us to clarify this point. Our work focuses on understanding the redundancy of native expert connectivity in pretrained MoT robot policies during downstream adaptation, rather than simply transferring a general pruning technique to robotic models.

We would first like to clarify that our findings do not assume that early-layer fusion is inherently more important. In Section 3, we show that substantial native attention connectivity can be randomly removed while largely preserving downstream performance after fine-tuning, without relying on any specific layer-importance assumption. Early-half fusion is subsequently explored in Section 4 as a structured way to translate this observed redundancy into practical efficiency gains.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2/3)

Continue on A2:

Moreover, we compare findings from recent multimodal studies, as summarized below.

Table R1: Comparison with prior multimodal studies.

WorkApproachMain FindingResearch Focus
Zhang et al. (2025) [1]Cross-modal information-flow analysisLower and middle layers serve distinct roles in visual-language integrationVisual-language information propagation
Kaduri et al. (2025) [2]Visual-feature interventionsMiddle layers play an important role in visual information processingVisual information utilization in VLMs
Song et al. (2026) [3]Contrastive attention analysis and visual-feature maskingCertain models revisit visual information in later layersDepth-dependent visual information usage
Zhang et al. (2025) [4]Visual-token injection depth ablationAll-layer injection outperforms early-only injection in the tested settingsVisual information injection design
OursExpert-connectivity masks with downstream fine-tuningDifferent sparse patterns preserve comparable closed-loop performance after adaptationNative expert connectivity redundancy in robot policies, and model deployment efficiency

These studies provide varied findings for the importance of cross-modal interactions at different depths, rather than establishing a consistent pattern of early-layer dominance.

Instead, our study investigates the redundancy of native expert connectivity in pretrained MoT robot policies, which integrate specialized experts for understanding, world modeling, and continuous action generation, with diverse attention structures, denoising mechanisms, and temporal dependencies. Across both VLA and WAM architectures, we find that different sparse connectivity patterns can preserve comparable closed-loop control performance after downstream adaptation, suggesting that successful adaptation does not depend on a particular fixed subset of expert connections. Building on this finding, Early-half fusion provides a practical realization of the observed redundancy, concentrating cross-expert interactions in early layers to enable conditioning-layer truncation and historical KV-cache reduction while largely preserving downstream performance.

Together, our work goes beyond general multimodal fusion analysis by systematically studying connectivity redundancy during downstream adaptation across pretrained MoT robot policies and translating these findings into measurable deployment efficiency gains.

Q3: Missing key control: all sparse variants receive 40k–200k more fine-tuning steps than Dense, and some even surpass Dense (Motus +0.53, LingBot +1.36), so the effects of "redundancy" and "additional training" cannot be disentangled. Please provide further comparisons under matched training steps and hyperparameters to demonstrate the effectiveness of the method.

A3: Thank you for the valuable suggestion. To disentangle the effects of sparsification and additional training, we conducted additional Dense continued fine-tuning experiments on InternVLA-A1 and Motus across both RoboTwin and LIBERO.

Specifically, we continued fine-tuning the Dense checkpoints without applying any masks, using the same initialization, training data, hyperparameters, and learning-rate schedules as the sparse variants. As shown below, Continued Dense remains close to Original Dense, with differences within 0.98 percentage points, consistent with the original Dense models having largely converged. Meanwhile, Random and Early-half also maintain performance close to Continued Dense under matched training conditions. These results further support our finding that pretrained MoT policies can adapt to substantially sparser expert connectivity while preserving performance comparable to Dense, rather than the observed performance being explained simply by additional fine-tuning.

Table R2: Performance comparison under matched additional fine-tuning budgets. Success rates (%).

ModelBenchmarkOriginal DenseContinued DenseRandomEarly-half
InternVLA-A1RoboTwin88.6688.7287.0787.69
InternVLA-A1LIBERO94.4593.7593.8591.10
MotusRoboTwin86.5687.5487.0985.49
MotusLIBERO96.8597.0096.1096.15

Original Dense refers to the dense checkpoints in Tables 2 and 3. Continued Dense and the sparse variants receive matched additional fine-tuning.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 3/3)

Continue on A3:

For completeness, we provide the detailed results of the additional Continued Dense experiments below.

Table R3: Continued Dense fine-tuning results on RoboTwin. Success rates (%).

ModelEasyHardAvg.
InternVLA-A188.3089.1488.72
Motus87.6287.4687.54

Table R4: Continued Dense fine-tuning results on LIBERO. Success rates (%).

ModelSpatialObjectGoalLongAvg.
InternVLA-A193.099.893.688.693.75
Motus99.699.695.093.897.00

Thank you again for your valuable feedback and hope that all your concerns are properly addressed.

Reference

[1] Zhang, Zhi, et al. "Cross-modal information flow in multimodal large language models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

[2] Kaduri, Omri, et al. "What’s in the Imageƒ A Deep-Dive into the Vision of Vision Language Models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

[3] Song, Shezheng, et al. "Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

[4] Zhang, Xuange, et al. "Layer-wise vision injection with disentangled attention for efficient LVLMs." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.

Reviewer_Y1CSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies redundancy in expert connectivity in MoT robot policies. The authors randomly remove about 50% of expert-level attention connections during fine-tuning and find that task performance remains close to the dense baselines. They then propose Early-half Fusion, which keeps cross-expert attention in the first half of the Transformer layers and removes it from the second half. This reduces deployed parameters, GPU memory usage, and inference latency. The method is evaluated on several VLA and WAM models, including π0.5, InternVLA-A1, Motus, and LingBot-VA, across RoboTwin, LIBERO, and real-world manipulation tasks. The results suggest that much of the expert connectivity can be removed with limited performance loss while improving inference efficiency.

Critical strengths
  • The paper examines whether dense expert connectivity is necessary in MoT-based robot policies, which is relevant to reducing deployment costs. I appreciate that the authors evaluate deployment efficiency rather than relying only on task success rates. The parameter and latency reductions on π0.5 and InternVLA-A1 are particularly useful.
  • The proposed method does not require pretraining from scratch or complex connectivity search. Beyond task success rates, the authors report deployed parameters, GPU memory usage, and inference latency. The real-world experiments also show that the method can retain task performance on physical robots.
Critical weaknesses

My main concern is the choice of early Transformer layers for cross-expert attention. While the authors show that Early-half Fusion preserves performance, they do not explain why fusion should be retained specifically in these layers. The existing ablations mainly compare different fusion depths, without structured baselines such as Late-half Fusion or Alternating Fusion under matched connectivity budgets. Without these comparisons, it is difficult to tell whether early-layer fusion offers a specific advantage or whether other sparse connectivity patterns would work equally well.

Questions for authors
  1. Can the authors compare Early-half Fusion against structured baselines such as Late-half Fusion and Alternating Fusion under matched connectivity budgets?
  2. How does applying the proposed Early-half Fusion mask directly to a dense checkpoint, without additional fine-tuning, affect performance?
2 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

We thank the reviewer for the insightful comments and valuable feedback. Please see our responses to your concerns point-by-point below.

Q1: My main concern is the choice of early Transformer layers for cross-expert attention. While the authors show that Early-half Fusion preserves performance, they do not explain why fusion should be retained specifically in these layers. …… Can the authors compare Early-half Fusion against structured baselines such as Late-half Fusion and Alternating Fusion under matched connectivity budgets?

A1: Thank you for your thoughtful consideration. Our choice of Early-half fusion is motivated by the connectivity redundancy observed through random sparsification, where different sparse masks achieve comparable downstream performance. Building on this finding, we explore whether such redundancy can be translated into practical deployment benefits through a structured fusion pattern. Specifically, Early-half fusion removes cross-expert dependencies from the upper half of the network, enabling unused conditioning layers to be truncated or historical KV-cache storage to be reduced, while largely preserving downstream performance.

Following your suggestion, we further evaluated Alternating, Middle-half, and Late-half fusion on InternVLA-A1 across both RoboTwin and LIBERO, with the same fusion-layer budget of 14 out of 28 layers as Early-half. The results show that different fusion placements generally maintain performance close to Dense, with varying relative advantages across benchmarks. On RoboTwin, Early-half achieves the highest average success rate among the structured patterns (87.69%), while on LIBERO, Late-half performs best (94.70%), slightly exceeding Dense (94.45%). These results further support the flexibility of sparse expert connectivity during downstream adaptation, while Early-half offers a particularly straightforward structure for deployment simplification.

Table R1: Additional structured fusion baselines on InternVLA-A1 (RoboTwin).

VariantEasyHardAvg.
Alternating87.2887.4887.38
Middle-half87.0886.9687.02
Late-half86.0686.2286.14

Table R2: Additional structured fusion baselines on InternVLA-A1 (LIBERO).

VariantSpatialObjectGoalLongAvg.
Alternating91.098.693.689.893.25
Middle-half91.897.490.689.892.40
Late-half94.499.092.692.894.70

The Dense, Random, and Early-half results are reported in Tables 2 and 3 of the original submission.

Q2: How does applying the proposed Early-half Fusion mask directly to a dense checkpoint, without additional fine-tuning, affect performance?

A2: Thank you for the valuable suggestion. We directly applied the Early-half and Random masks to the dense checkpoints without additional fine-tuning, evaluating InternVLA-A1 and Motus on both RoboTwin and LIBERO.

As shown in Tables below, performance drops dramatically without adaptation. On RoboTwin, Early-half and Random achieve only 7.46% and 1.04% on InternVLA-A1, respectively, while both variants achieve nearly zero success on Motus. On LIBERO, all directly masked variants achieve 0% success. In contrast, after sparse fine-tuning, Early-half achieves 87.69% and 85.49% on RoboTwin, and 91.10% and 96.15% on LIBERO, for InternVLA-A1 and Motus, respectively, as reported in Tables 2 and 3 of the original submission. These results demonstrate that directly removing substantial native connectivity severely degrades the pretrained policies, while downstream fine-tuning enables them to adapt to the sparse connectivity and recover performance close to Dense.

Table R3: Direct masking without fine-tuning on RoboTwin. Success rates (%).

ModelMaskEasyHardAvg.
InternVLA-A1Early-half7.427.507.46
InternVLA-A1Random1.280.801.04
MotusEarly-half0.120.060.09
MotusRandom0.020.040.03

Table R4: Direct masking without fine-tuning on LIBERO. Success rates (%).

ModelMaskSpatialObjectGoalLongAvg.
InternVLA-A1Early-half0.00.00.00.00.00
InternVLA-A1Random0.00.00.00.00.00
MotusEarly-half0.00.00.00.00.00
MotusRandom0.00.00.00.00.00
AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

Continue on A2:

To further investigate the performance degradation without fine-tuning, we conduct an additional open-loop action prediction analysis, using the same dense checkpoints and inference procedures with masks applied directly, without updating any weights. We compare the predicted action chunks against the corresponding demonstration actions. As shown below, Direct masking increases action prediction RMSE by approximately 4.7–39× and trajectory DTW by 5.5–62× relative to Dense. These results corroborate the severe closed-loop performance degradation and further highlight the importance of downstream fine-tuning for adapting pretrained policies to sparse connectivity.

Table R5: Open-loop action prediction errors without additional fine-tuning.

ModelBenchmarkVariantRMSE (×Dense) ↓DTW (×Dense) ↓Cosine ↑
InternVLA-A1RoboTwinDense0.036 (1×)0.082 (1×)0.997
Early-half0.427 (12×)1.354 (17×)0.870
Random1.381 (39×)5.055 (62×)0.030
InternVLA-A1LIBERODense0.131 (1×)0.243 (1×)0.988
Early-half0.745 (5.7×)1.870 (7.7×)0.600
Random1.448 (11×)3.726 (15×)0.030
MotusRoboTwinDense0.014 (1×)0.030 (1×)—
Early-half0.178 (12×)0.657 (22×)—
Random0.215 (15×)0.802 (26×)—
MotusLIBERODense0.213 (1×)0.472 (1×)0.972
Early-half1.011 (4.7×)2.606 (5.5×)0.170
Random1.040 (4.9×)2.682 (5.7×)0.080

RMSE is computed in the model-specific normalized action space, and DTW reports trajectory cost normalized by alignment length. Cosine similarity is omitted for Motus RoboTwin because it is uninformative under its min–max action normalization.

We appreciate the reviewer’s valuable feedback and hope that all your concerns are properly addressed.

Comments on the paper

2 comments

Thank you for your time and thoughtful feedback throughout the review process. We truly appreciate your constructive comments and are glad that your concerns are fully addressed. We would be happy to answer any further questions!

One small note: it seems that your comment is not currently visible to the other reviewers. If convenient, would you mind making it visible to them as well?

AuthorsOpenReview ↗

Final Remarks by Authors

Dear Program Chairs, Senior Area Chairs, Area Chair, and Reviewers,

We sincerely thank all reviewers for their time and constructive feedback throughout the discussion. As the discussion draws to a close, we would like to briefly summarize the key concerns addressed and the additional experimental evidence provided.

• Reviewer Y1CS recognized the practical motivation, broad evaluation, and measurable deployment efficiency gains of our work. To address the concern about fusion placement, we conducted matched-budget structured fusion experiments (Alternating, Middle-half, and Late-half) on InternVLA-A1 across RoboTwin and LIBERO, showing that different fusion patterns can maintain competitive performance. To investigate the role of downstream adaptation, we additionally performed direct-masking experiments and open-loop action prediction analyses, demonstrating substantial performance degradation without fine-tuning. Y1CS confirmed that both concerns were addressed and maintained the rating 3.

• Reviewer FX1h recognized the clarity and thoroughness of our study and raised questions about its relationship to existing efficiency methods. We clarified the distinctions between expert-connectivity sparsification and other efficiency approaches, and conducted an additional DoT-inspired action-depth reduction experiment. To address the follow-up concern about uncertainty, we reported standard deviations across three independently sampled Random masks on RoboTwin and conducted additional LIBERO evaluations for π0.5 and Motus. FX1h acknowledged these additional contributions and raised the rating from 3 to 4.

• Reviewer E3Cu acknowledged the clear presentation, broad architectural coverage, and careful deployment measurements. The initial review raised concerns about Early-half's relative performance, additional fine-tuning, and novelty compared with general multimodal studies. We clarified that Early-half and Random did not consistently outperform one another across the evaluated settings, and that evaluation checkpoints were selected at predefined training steps rather than based on performance. We further conducted Continued Dense fine-tuning controls and clarified our contributions relative to prior multimodal studies. E3Cu explicitly withdrew the additional-training and multimodal-novelty concerns and acknowledged the system-level insight of our work.
The subsequent discussion primarily focused on the comparison between Random sparsification and Early-half fusion, particularly their different pruning budgets. We clarified that Section 3 investigates native expert connectivity redundancy, while Section 4 explores how this redundancy can enable practical deployment efficiency gains. These are complementary research questions, each primarily evaluated against Dense, rather than competing methods for accuracy superiority. We also provided detailed block-level sparsification budgets and clarified the original performance claims.
To address the reviewer's further concerns about budget matching and whether Early-half succeeds merely by preserving critical early-layer connections, we presented matched-budget structured fusion experiments (Alternating, Middle-half, and Late-half), initially conducted in response to Reviewer Y1CS, and further performed matched-budget Cross-only Random experiments on InternVLA-A1, Motus, and LingBot-VA. These results provide strong empirical evidence that effective downstream adaptation can be achieved with different expert connectivity patterns. Notably, Late-half removes all early-layer cross-expert connections while maintaining competitive performance, suggesting that preserving early-layer fusion is not strictly necessary for downstream adaptation.
In the later discussion, the reviewer questioned Early-half's performance and generality. We clarified that the relative performance varies across models and benchmarks: Early-half performs best among structured patterns on InternVLA-A1 RoboTwin, while Late-half performs best on LIBERO; Early-half also outperforms Cross-only Random on LingBot-VA LIBERO. Importantly, our original claim is to largely preserve downstream performance, not to achieve universal accuracy superiority. Across all eight main settings, Early-half retains at least 96.4% of Dense performance, with explicitly reported accuracy costs and measurable deployment gains in Section 4.2. Section 4 already presents Early-half as a deployment-oriented realization of the redundancy investigated in Section 3, and we further clarified its wording to avoid ambiguity.

We sincerely appreciate the reviewers' thoughtful discussions and valuable suggestions, which have helped us strengthen the experimental evidence and clarify the contributions of our work.

Best regards,
Authors of Submission 39872

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.