acceptodds
Under review as a conference paper at ICLR 2027

VLA-GSE: Boosting Parameter-Efficient Fine-Tuning in VLA with Generalized and Specialized Experts

Abstract

Vision-language-action (VLA) models inherit rich visual-semantic priors from pre-trained vision-language backbones, but adapting them to robotic control remains challenging. Full fine-tuning (FFT) is prone to overfitting on downstream robotic data and catastrophic forgetting of pretrained vision-language capabilities. Parameter-efficient fine-tuning (PEFT) better preserves pre-trained knowledge, yet existing PEFT methods still struggle to adapt effectively to robot control tasks. To address this gap, we propose VLA-GSE, a parameter-efficient VLA fine-tuning framework that improves control adaptation while retaining PEFT’s knowledge preservation advantage. Specifically, VLA-GSE (Generalized and Specialized Experts) is initialized by spectrally decomposing the frozen backbone, assigning leading singular components to generalized experts (shared experts) and disjoint residual components to specialized experts (routed experts). This decomposition improves adaptation capacity under a fixed trainable-parameter budget. Across comparable-budget comparison, VLA-GSE updates only 2.51% of full model parameters and consistently outperforms strong FFT and PEFT baselines. It achieves 81.2% average zero-shot success on LIBERO-Plus, preserves pre-trained VLM capability comparably to LoRA on multimodal understanding benchmarks, and improves real-world manipulation success under multiple distribution shifts.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Intelligence, VLA, Parameter-Efficient Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_QAacOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VLA-GSE, a PEFT method for adapting pre-trained VLMs, mainly Qwen3-VL-4B, into VLA policies. VLA-GSE is built from a spectral decomposition of the frozen backbone: leading singular components initialize generalized experts, while disjoint residual components initialize specialized experts. The paper evaluates VLA-GSE on LIBERO, LIBERO-Plus, general multimodal-understanding benchmarks, and real-world manipulation using an AgileX PiPER arm. The main empirical claim is that VLA-GSE improves robotic adaptation over both FFT and existing PEFT baselines while preserving the pretrained VLM's multimodal capability.

Critical strengths
  • The paper addresses an important and practically relevant VLA adaptation problem. Balancing downstream control adaptation and pretrained VLM knowledge retention is a real challenge in VLA post-training. Evaluating both manipulation success and general VLM benchmarks is good. Updating only ~2.5% of parameters while outperforming FFT and LoRA on OOD manipulation without losing VQA/OCR capabilities.
  • The empirical evaluation is relatively broad and well controlled. The paper comprehensively compares against SVD- and MoE-based PEFT baselines, with repeated training seeds and different backbones.
Critical weaknesses
  • The VLA-specific PEFT comparison is incomplete. The paper motivates VLA-GSE by arguing that existing PEFT methods under-adapt to robot control, but it does not discuss directly relevant recent work such as LoRA-SP (Kim et al., 2026), which studies adaptive rank allocation for VLA fine-tuning. Importantly, LoRA-SP reports that the rank requirement for VLA adaptation can be substantially larger than what is typical in LLM fine-tuning, and evaluates fixed-rank LoRA over (r\in{16,32,64,128}). VLA-GSE evaluated main PEFT baselines around a single low-rank budget (r=16). This is sufficient to show that VLA-GSE is strong under this budget, but not to establish that standard PEFT fundamentally lacks adaptation capacity. A rank/parameter sweep would make this claim much more convincing.

  • The method novelty appears incremental relative to prior work.GOAT (Chen et al., 2025) already combines SVD-based initialization with routed LoRA experts and scaling/optimization alignment, while shared experts are also standard in MoE architectures. The main novelty therefore seems to be how these ingredients are combined.

  • The claimed roles of the generalized and specialized experts are not convincingly demonstrated. The paper argues that the generalized expert captures common and transferable knowledge, while the specialized experts provide input-specific control refinement. However, the experiments do not provide direct evidence that the two types of experts actually learn these distinct roles.

REFERENCE

  • Kim et al., LoRA-SP: Adaptive Capacity Allocation for Vision Language Action Fine-tuning, ICRA 2026
  • Fan et al., Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment, ICML'2025
Questions for authors
  • How does VLA-GSE compare with LoRA-SP, and what happens when the PEFT rank/budget is increased? Could the authors report rank or parameter sweeps for LoRA, and VLA-GSE, rather than comparing only around (r=16)?
  • What do the specialized experts actually specialize in? Do different token types or manipulation phases route to different experts? Current routing diagnostics do not directly establish the claimed generalized/specialized functions.
  • Where does the gain over GOAT come from? GOAT already contains SVD-initialized routed LoRA experts and optimization/scaling alignment, while VLA-GSE achieves substantially better performance in several settings. A component-wise comparison would help clarify which additional design in VLA-GSE is responsible for this gain and, consequently, what the main methodological contribution is beyond GOAT.
7 comments

We thank the reviewer for recognizing the importance of balancing robotic adaptation and pretrained knowledge retention, and for the constructive comments on experimental completeness, methodological novelty, and expert specialization. We address all three weaknesses and three questions below.

W1: Incomplete VLA-specific PEFT comparisons

We agree that comparisons restricted to r=16r=16 are insufficient to establish the adaptation-capacity advantage of VLA-GSE. We have cited and discussed LoRA-SP (Kim et al., 2026) in the revised paper, including its findings on adaptive capacity allocation and the larger ranks required for VLA adaptation.

We further extend the evaluation to LoRA ranks r∈{16,32,64,128}r\in\{16,32,64,128\} and VLA-GSE ranks r∈{16,32,64}r\in\{16,32,64\}, with FFT and LoRA-SP as additional references.

The results demonstrate that increasing LoRA rank continues to improve robotic performance, reaching 74.5% at rank 128, close to FFT's 74.9%. Importantly, VLA-GSE maintains a substantial advantage, achieving 82.7% at rank 64 with fewer trainable parameters. These results clarify that our contribution is more effective allocation of adaptation capacity, rather than an assertion that conventional PEFT fundamentally lacks sufficient capacity.

Detailed comparisons are provided in Q1.

W2: Methodological novelty relative to GOAT

We acknowledge that SVD initialization, routed experts, shared experts, and optimization alignment have been explored individually. We have clarified these connections in the revised paper and explicitly credit GOAT for its prior contributions.

The main distinction is the coupling between spectral allocation and expert activation roles. Unlike GOAT, which distributes SVD-initialized components across routed experts, VLA-GSE assigns the leading spectral components to an always-active generalized expert and disjoint residual components to conditionally activated specialized experts.

This design introduces two complementary properties:

  1. Persistent spectral access: Dominant pretrained directions remain accessible to every token, independently of routing decisions.
  2. Conditional spectral adaptation: Residual directions provide input-dependent adaptation through specialized experts, supported by expert-specific scaling.

To isolate these contributions, we construct controlled comparisons of shared versus all-routed experts, leading-spectral assignment, spectral placement, scaling, and initialization correction.

In particular, D versus B matches trainable parameters and principal adapter computation to test persistent expert access, while A versus C preserves the architecture and changes spectral assignment.

The resulting Long success improvements are 5.3 and 13.8 percentage points, respectively. The original paper separately reports 85.6% OCRBench performance for the complete VLA-GSE model.

Thus, VLA-GSE's contribution is not simply combining existing components, but organizing pretrained spectral directions into persistent and conditional pathways with independently measurable effects. The complete component analysis is provided in Q3.

W3: Insufficient evidence for expert specialization

We agree that architectural distinctions alone do not establish functional specialization. We therefore supplement the original routing diagnostics with token-type analysis, manipulation-phase routing, expert interventions, and matched expert-role controls.

Across 278K layer-token observations, specialized expert-selection distributions differ substantially between visual, instruction, and action-query tokens. Manipulation-phase analysis further reveals different routing preferences during reaching, grasping, placing, and recovery.

Expert interventions confirm that these differences have functional consequences. Suppressing the generalized expert produces a substantially larger change in predicted actions than suppressing individual specialized experts, whereas specialized-expert interventions produce smaller, differentiated effects on closed-loop success.

The matched shared-versus-routed and spectral-role controls quantify the effect of persistent leading-spectral access on robotic performance. The original paper's multimodal results separately document knowledge retention by the complete method.

Together, these analyses support a persistent generalized pathway for shared adaptation and conditionally activated specialized pathways for input-dependent refinement, without assuming an exclusive one-to-one correspondence between individual experts and semantic manipulation phases.

Detailed evidence is presented in Q2 and Q3.

Q1: How does VLA-GSE compare with LoRA-SP and larger PEFT budgets?

Response. We evaluate LoRA, LoRA-SP, VLA-GSE, and FFT under the same Qwen3-VL-4B-Instruct backbone and OFT action head, with consistent training data and LIBERO-Plus evaluation settings.

Rank and parameter sweep

MethodStored rankActive rankParams (M)SpatialObjectGoalLongOverall
FFT——Full————74.9
LoRA1616105.8980.375.364.856.469.2
LoRA3232146.1783.079.367.760.472.6
LoRA6464226.7184.380.268.962.674.0
LoRA128128387.8084.880.869.462.974.5
VLA-GSE166114.0490.386.274.274.181.2
VLA-GSE3212154.3191.087.475.975.382.4
VLA-GSE6424234.8591.287.776.375.682.7

All success rates are percentages. Trainable parameter counts are in millions and include the common OFT action head. For VLA-GSE, stored rank sums all eight experts, whereas active rank sums the three activated paths per token.

1. Larger LoRA ranks continue to improve robotic adaptation. LoRA improves from 69.2% at rank 16 to 72.6% at rank 32, 74.0% at rank 64, and 74.5% at rank 128. The improvement from rank 64 to 128 is smaller but remains positive. Thus, the evaluated range demonstrates diminishing marginal gains, not a definitive performance plateau. At rank 128, LoRA approaches FFT's 74.9%, consistent with the importance of increased adaptation capacity emphasized by LoRA-SP.

2. VLA-GSE retains a substantial advantage at lower budgets. VLA-GSE achieves 81.2% at rank 16, 82.4% at rank 32, and 82.7% at rank 64. At its largest evaluated configuration, VLA-GSE outperforms rank-128 LoRA by 8.2 percentage points while using approximately 39% fewer trainable parameters. Its active rank is also substantially smaller (24 versus 128).

3. Comparison with LoRA-SP. We have added LoRA-SP to the related work and conducted a same-backbone, parameter-matched comparison.

MethodParams (M)Overall success (%)
LoRA, rank 16105.8969.2
LoRA-SP114.0476.6
VLA-GSE, rank 16114.0481.2

LoRA-SP improves over fixed-rank LoRA, confirming the importance of adaptive capacity allocation. VLA-GSE further improves performance by 4.6 percentage points over LoRA-SP at matched stored parameters. Unlike LoRA-SP's input-dependent rank selection, VLA-GSE guarantees persistent access to leading pretrained spectral directions while conditionally activating residual experts. We distinguish stored-parameter matching from active-computation matching, since the methods use different activation mechanisms.

4. Multimodal retention. The rank sweep also evaluates pretrained knowledge retention.

MethodMMMU (%)OCRBench (%)
Pretrained53.288.1
LoRA, rank 1651.883.3
LoRA, rank 3249.480.7
LoRA, rank 6447.876.0
LoRA, rank 12842.971.4
LoRA-SP51.285.0
VLA-GSE, rank 1651.185.6
VLA-GSE, rank 3248.079.5
VLA-GSE, rank 6442.973.3

VLA-GSE maintains comparatively stable retention while delivering substantially stronger robotic performance.

These results establish that VLA-GSE's advantage persists beyond low-rank comparisons and cannot be explained solely by insufficient capacity in the original LoRA baseline.

Q2: What do the specialized experts actually specialize in?

Response. We analyze routing distributions across token categories and manipulation phases, followed by expert-output interventions.

Token-type specialization

We examine 278,712 layer-token pairs from fixed observations across 252 adapted layers.

Token categoriesMedian layer-wise TV
Visual vs. action-query0.578
Visual vs. instruction0.317
Instruction vs. action-query0.522

The substantial routing-distribution differences demonstrate that specialized expert selection depends on token characteristics. Visual and action-query tokens exhibit the largest separation.

Manipulation-phase specialization

We further align routing decisions with independently annotated manipulation phases.

PhaseMost selected expertsSelection frequencies (%)
ReachingExperts 1, 244.2, 39.1
GraspingExperts 3, 445.1, 41.2
PlacingExperts 5, 643.3, 39.8
RecoveryExperts 7, 239.8, 34.9

The analysis reveals distinct expert-selection preferences across manipulation phases. Reaching, grasping, and placing exhibit different dominant expert combinations, while recovery recruits a different routing pattern.

These observations support phase-dependent specialization rather than exclusive semantic assignments: individual experts remain available across multiple phases, and their selection depends jointly on visual context, task instructions, and policy state.

Expert interventions

We suppress individual expert outputs at the trained checkpoint and evaluate action sensitivity and closed-loop success.

InterventionAction changeLong success (%)
Intact policyReference74.1
Zero shared expert0.368140.3
Zero specialized expert 10.030770.7
Zero specialized expert 20.043169.7
Zero specialized expert 30.017771.4
Zero specialized expert 40.021771.0
Zero specialized expert 50.018571.2
Zero specialized expert 60.016472.2
Zero specialized expert 70.017872.1

Suppressing the shared expert causes the largest action deviation and a substantial degradation in closed-loop success, consistent with its persistent contribution.

Individual specialized-expert interventions produce smaller but distinct effects. This is consistent with their conditional activation and differentiated contributions to action generation.

A complementary routing-identity control reduces Long success from 74.1% to 64.8% under randomized active expert identities, further confirming that learned routing assignments matter.

We emphasize that intervention magnitudes are not additive causal contributions, since experts have different activation frequencies and interact through nonlinear computations.

Together, token-type routing, phase-dependent preferences, and expert interventions provide direct evidence that specialized experts perform differentiated conditional adaptation, while the generalized expert provides a persistent pathway.

Reviewer_KxPgOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes VLA-GSE, a parameter-efficient method for adapting a pre-trained VLM (Qwen3-VL-4B-Instruct) into a VLA policy with an OpenVLA-OFT-style MLP action head. The action head is fully trained; the VLM weights are frozen and each linear layer receives a mixture of low-rank experts. The experts are initialised from the SVD of the frozen weight W0=UΣV⊤W_0 = U\Sigma V^\top: the top rg=2r_g = 2 singular components form one always-active "generalized" expert, and the next 7×27 \times 2 components form 7 rank-2 "specialized" experts selected by a top-2 router (Eqs. 1–3), with a load-balancing loss (Eq. 4). Two further components are motivated analytically: (i) a trace-inverse scaling ssi∝1/Tr(Σi)s_s^i \propto 1/\mathrm{Tr}(\Sigma_i) for each specialized expert, derived in Theorem 1 to equalise initial gradient norms under balanced routing and isotropic gradient assumptions; and (ii) a backbone weight adjustment W~0=W0−Wg−1E∑i=1EWsi\tilde W_0 = W_0 - W_g - \frac{1}{E}\sum_{i=1}^{E} W_s^i, so that the equivalent weight equals W0W_0 in expectation over routing at initialisation (Eq. 12). 114M parameters (2.51%) are trained, 48M of them in the GSE modules.

Critical strengths
  • A carefully controlled comparison against a broad set of PEFT baselines, with consistent gains. The ten baselines in Table 2 use approximately the same trainable-parameter budget (2.51–2.58%), the same backbone, data, action head and wall-clock budget (about 41.2 h on 8 A100s, Table 10), independently searched learning rates, and checkpoint selection by validation action loss rather than test success. The core comparison is repeated with three training seeds (Table 11), and the real-robot comparison with GOAT uses paired episodes with a bootstrap interval (+9.2+9.2 points, 95% CI [4.2,14.6][4.2, 14.6], Table 20). The ordering is the same on standard LIBERO (98.4 vs 96.2 for GOAT and 96.8 for FFT, with the largest gap on Long: 96.8 vs 93.0 and 94.2) and on SimplerEnv with two backbones (Table 12). This is the main reason for my positive rating: the improvement does not rest on a single benchmark or a single run.
  • The method improves control while keeping the knowledge-retention behaviour of LoRA. Table 3 shows that FFT loses much of the backbone's multimodal ability (e.g. MMStar 69.8→31.469.8 \to 31.4, DocVQA 95.3→55.195.3 \to 55.1), while VLA-GSE stays close to LoRA (67.8 vs 68.1; 92.8 vs 94.1). At the same time, VLA-GSE outperforms FFT on LIBERO-Plus and on the real robot, whereas LoRA does not. This shows that, for VLA adaptation, a structured PEFT method can be both cheaper and better than FFT, which is practically relevant. The limitations section and Appendix H also state clearly which claims the evidence does not support (e.g. the ABot-M0 comparison, single-seed real-robot runs, FLOP-unmatched role ablations).
Critical weaknesses
  • It is not clear which part of the gain over GOAT comes from the components that are new in this paper.** GOAT (Fan et al., 2025), the strongest baseline, already (a) initialises routed LoRA experts from segments of the SVD of W0W_0; (b) subtracts a constant Wres=sE∑iB0iA0iW_{res} = \frac{s}{E}\sum_{i} B_0^i A_0^i from the frozen weight, derived from Ex[wi(x)]=1/E\mathbb{E}_x[w_i(x)] = 1/E so that the initial weight matches W0W_0 in expectation over routing (GOAT, Lemma 3.3 and Theorem 3.4); and (c) uses one global scaling factor ss. Eq. 12 of this paper has the same form as (b), with the additional term WgW_g, but Section 3.3 and Appendix C do not refer to GOAT. As far as I can see, the differences from GOAT are: the always-active generalized expert; contiguous segments taken from the top of the spectrum (GOAT spaces its segments evenly over the whole spectrum, starting at offset (j−1)min⁡(m,n)/E(j-1)\min(m,n)/E); and the per-expert scaling ssis_s^i instead of a global ss. The paper does not state these differences explicitly. Observations from the paper:

    • The ablation (Table 4) is run only on the LIBERO-Plus Long suite with a single run. Removing the trace-inverse scaling changes Long success from 74.1 to 72.1, and removing the auxiliary loss changes it to 72.0. The three-seed standard deviation of the overall LIBERO-Plus score is 0.8–1.1 points (Table 11); the seed variation of the Long suite alone is not reported.
    • The larger drops come from removing SVD initialisation (60.9) or one expert type (67.2 / 63.1). As noted in Section 4.4, the expert-type variants do not keep the number of active experts per token fixed: the full model activates 1+21 + 2 experts, the "w.o. Generalized Experts" variant activates 2 of 8.
    • There is no ablation of the segment placement (top contiguous vs spread over the spectrum) or of the backbone weight adjustment (Section 3.3).
    • GOAT's per-suite LIBERO-Plus results are not reported, so the ablation variants on the Long suite cannot be compared with GOAT.

    Given these observations, is it the always-active generalized expert, the placement of the spectral segments, or the per-expert scaling that accounts for the 4.4-point improvement over GOAT?

  • I could not find very clear evidence that the two analytically motivated components behave as described in the method section.

    • Theorem 1 assumes that the routing second moment ρw=Ex[wi(x)2]\rho_w = \mathbb{E}_x[w_i(x)^2] is identical across experts and that the projected gradient moments satisfy αiL=κLTr(Σi)\alpha_i^L = \kappa_L \mathrm{Tr}(\Sigma_i) and αiR=κRTr(Σi)\alpha_i^R = \kappa_R \mathrm{Tr}(\Sigma_i) (Eq. 7). The paper's own audit (Table 16, Appendix H.5) reports a coefficient of variation of 0.69–0.71 for the routing second moments and 0.37–0.65 for the projected gradient moments across specialists, and states that routing second moments are unequal. Appendix H.6 shows lower update-norm variation with the proposed scaling on single replayed batches; the downstream effect is the 2.0-point single-run difference above.

    • The backbone adjustment makes the equivalent weight equal to W0W_0 only in expectation (Appendix C). For a given token,

      Weq(x)−W0=∑i=1E(wi(x)−1E) UiΣiVi⊤.W_{eq}(x) - W_0 = \sum_{i=1}^{E} \Big(w_i(x) - \tfrac{1}{E}\Big)\, U_i \Sigma_i V_i^\top .

      With top-2 of E=7E = 7 experts, a selected expert receives roughly +0.36+0.36 of its component and each unselected expert −0.14-0.14, and these components are singular directions 3–16 of W0W_0, i.e. among the largest ones. The paper does not report how far the model at step 0 is from the pre-trained VLM, for example the relative change of layer outputs, or multimodal benchmark scores at initialisation.

Questions for authors
  • Could the authors state explicitly how VLA-GSE differs from GOAT (segment placement, always-active expert, scaling rule), and how the backbone weight adjustment in Section 3.3 relates to GOAT's WresW_{res} (Theorem 3.4 there)? Under the Table 2 protocol, could they report GOAT's per-suite LIBERO-Plus results and variants that change one difference at a time, for example GOAT plus an always-active generalized expert, and VLA-GSE with GOAT's segment placement or with a global scaling factor?
  • For the backbone weight adjustment, what is the per-token deviation from the pre-trained backbone at step 0 (e.g. ∥Weq(x)−W0∥F/∥W0∥F\lVert W_{eq}(x) - W_0 \rVert_F / \lVert W_0 \rVert_F, or the change in layer outputs), and what are the multimodal benchmark scores of the model at initialisation? How does the method perform without the adjustment, or with an adjustment that is exact per token (e.g. subtracting the routed components using the current routing weights wi(x)w_i(x))?
  • For the real-robot study (Table 5 and Table 19), could the authors report success rates per distribution shift (lighting, language, clutter, background), and describe how π0\pi_0, π0.5\pi_{0.5} and OpenVLA-OFT were fine-tuned on the 70 demonstrations per task (steps, learning rates, whether their public pre-trained checkpoints were used)? The paper's claim concerns robustness to distribution shifts, but only the per-task averages over all four shifts are given.
6 comments

We thank the reviewer for identifying the overlap with GOAT and for requesting stronger component-level evidence. We agree that the distinction between inherited mechanisms and our contributions must be explicit. We have revised the method discussion, corrected the theoretical statement, and organized the supplementary analyses around the five points raised.

W1. Distinguishing VLA-GSE from GOAT and attributing the gain

We do not claim SVD-initialized routed experts or expectation-based residual subtraction as new. GOAT already provides both. The differences concern which spectral directions are assigned to persistent versus routed adaptation, how the shared and specialized pathways are composed, and how the specialized factors are scaled.

MechanismGOATVLA-GSE
Expert executionRouted experts onlyOne always-active generalized expert plus two selected specialized experts
Spectral assignmentDisjoint blocks distributed across the full spectrum, with offsets proportional to (j−1)min⁡(m,n)/E(j-1)\min(m,n)/ELeading singular directions in the shared expert; subsequent contiguous, disjoint directions in seven specialized experts
ScalingCommon scale in GOAT; singular-value-dependent inverse-square-root adjustment in GOAT-sFixed shared scale and expert-specific inverse-block-trace scale for specialized experts
Frozen-weight correctionSubtraction of the expected initial routed updateThe same expectation-based principle, extended by subtraction of the full always-active shared update

Specifically, GOAT's Theorem 3.4 gives Wres=∑iE[wi(x)]Di0W_{\mathrm{res}}=\sum_i\mathbb{E}[w_i(x)]D_i^0. Our initial correction is Wres=Dg0+∑iE[wi(x)]Di0W_{\mathrm{res}}=D_g^0+\sum_i\mathbb{E}[w_i(x)]D_i^0, where Dg0D_g^0 is the initial generalized-expert contribution and Di0D_i^0 is that of specialized expert ii. The coefficient of the shared term is one because this path is always active. We have added the explicit attribution to GOAT in Section 3.3 and Appendix C.

Where the overall 4.4-point gain occurs. Under the original Table 2 protocol, VLA-GSE improves the overall LIBERO-Plus success rate from 76.8% to 81.2%. The largest perturbation-specific gains are Camera (+9.9 points), Robot (+6.7), and Noise (+4.8), followed by Layout (+3.3), Background (+2.1), Language (+1.1), and Light (+1.1). These results establish where the complete-method improvement arises across evaluation settings; the controlled comparisons below identify the architectural contributions. We therefore separate this complete-method comparison from the following matched component controls.

Matched full-training controls on LIBERO-Plus Long. The expert-role comparisons preserve the same rank budget, number of active expert matrix products per token, evaluation identities, and training schedule. In particular, B and D each have 115.2M trainable parameters, including 9.3M router parameters, and the same profiled expert/router matrix-operation counts (7.561B/4.60B per call). D replaces one active routed path with a persistent shared path; B uses routed paths only. A is the original allocation; unlike D, A does not widen its router for the strict B–D parameter match. A versus C instead keeps the routing topology fixed and swaps the spectral packages assigned to the shared and first specialized roles.

VariantControlled differenceLong success (%)MMStar (%)
A: VLA-GSEOriginal shared/routed allocation; contiguous spectral blocks; inverse-trace scales74.167.8
B: All routedRouted-only allocation, matched to D in parameters and active operations67.259.4
C: Spectral role-swapSwap the leading shared spectral package with a specialized package; unchanged topology60.347.6
D: Shared, matched router widthAlways-active shared path, matched to B in parameters and active operations72.568.0
E: GOAT-style spectral placementRetain A's shared/routed topology and scaling; distribute specialized blocks over the spectrum71.364.0
F: Uniform specialized scalingRetain A's topology and spectral allocation; use a fixed specialist scale of 2 and recompute the correction72.165.0
G: GOAT-style, three routed pathsGOAT-style spectral placement and scaling, with three active routed paths66.861.8
H: G + persistent shared pathReplace one of G's three routed paths with a persistent path while matching active operations69.663.6

D versus B yields +5.3 Long points at matched trainable parameters and active matrix operations, isolating the benefit of a persistent shared path under the specified expert allocation. A versus C yields +13.8 Long points, testing whether the dominant spectral package belongs on the shared path rather than a routed path; it is a spectral-role test, not a pure test of spacing. The latter is isolated by A versus E (+2.8 Long points), which changes segment placement without changing the architecture. A versus F (+2.0 Long points) isolates the specialized scaling rule. Finally, H versus G (+2.8 Long points) repeats the shared-path comparison within a GOAT-style spectral and scaling configuration.

W2. What the theory and initialization correction actually guarantee

The originally stated assumptions are insufficient for unconditional gradient equalization. Equal routing second moments and projected gradient moments proportional to block trace are separate marginal conditions; they do not imply that routing and projected gradient energy are uncorrelated. We make the missing condition explicit and retain the dependence term in the general statement.

For specialist ii, define Zi(x)=wi(x)2Z_i(x)=w_i(x)^2, qiL(x)=Tr⁡(ΣiUi⊤gg⊤Ui)q_i^L(x)=\operatorname{Tr}(\Sigma_iU_i^\top gg^\top U_i) and qiR(x)=Tr⁡(ΣiVi⊤g⊤gVi)q_i^R(x)=\operatorname{Tr}(\Sigma_iV_i^\top g^\top gV_i). With ρi=E[Zi]\rho_i=\mathbb E[Z_i], αiL/R=E[qiL/R]\alpha_i^{L/R}=\mathbb E[q_i^{L/R}], and ciL/R=Cov⁡(Zi,qiL/R)c_i^{L/R}=\operatorname{Cov}(Z_i,q_i^{L/R}), the exact initialization identity for the factor-gradient second moment is

E∥∇Ai∥F2=si(ρiαiL+ciL),E∥∇Bi∥F2=si(ρiαiR+ciR).\mathbb E\|\nabla A_i\|_F^2=s_i\big(\rho_i\alpha_i^L+c_i^L\big),\qquad \mathbb E\|\nabla B_i\|_F^2=s_i\big(\rho_i\alpha_i^R+c_i^R\big).

The identity applies to the local factor gradients under the stated initialization parameterization, with expert dropout disabled. Consequently, the separation used in the original Eq. (5) is valid only if ciL=ciR=0c_i^L=c_i^R=0 for all specialists. A sufficient condition is E[qiL/R∣Zi]=E[qiL/R]\mathbb E[q_i^{L/R}\mid Z_i]=\mathbb E[q_i^{L/R}]; this is a conditional-mean decorrelation assumption, not a consequence of balanced routing. Theorem 1 and its proof now state this condition explicitly. The router has no direct access to the expert-specific singular projectors at initialization, which motivates decorrelation as a tractable idealization; however, its scores and the gradients still depend on the same input, so Gaussian router initialization alone does not prove the condition.

Without decorrelation, the useful result is a conditional gradient bound, not exact equalization. Set si=c/Tr⁡(Σi)s_i=c/\operatorname{Tr}(\Sigma_i), with c>0c>0. Then

E∥∇Ai∥F2=cρiαiL+ciLTr⁡(Σi),E∥∇Bi∥F2=cρiαiR+ciRTr⁡(Σi).\mathbb E\|\nabla A_i\|_F^2=c\frac{\rho_i\alpha_i^L+c_i^L}{\operatorname{Tr}(\Sigma_i)},\qquad \mathbb E\|\nabla B_i\|_F^2=c\frac{\rho_i\alpha_i^R+c_i^R}{\operatorname{Tr}(\Sigma_i)}.

Thus, when the routing-weighted projected moments satisfy E[ZiqiL/R]≤CL/RTr⁡(Σi)\mathbb E[Z_iq_i^{L/R}]\leq C_{L/R}\operatorname{Tr}(\Sigma_i), both sides obey the uniform bounds E∥∇Ai∥F2≤cCL\mathbb E\|\nabla A_i\|_F^2\leq cC_L and E∥∇Bi∥F2≤cCR\mathbb E\|\nabla B_i\|_F^2\leq cC_R. Exact equality requires the corresponding normalized routing-weighted moments to be constant across experts. A bound requiring no decorrelation follows directly from Cauchy–Schwarz: ∣ciL/R∣≤Var⁡(Zi)Var⁡(qiL/R)|c_i^{L/R}|\leq\sqrt{\operatorname{Var}(Z_i)\operatorname{Var}(q_i^{L/R})}. It explicitly accounts for the routing–gradient dependence rather than removing it. The stated uniform bound is conditional on finite, bounded normalized routing-weighted moments; it is not an unconditional equality theorem.

The empirical audit supports variance reduction, not the exact assumptions. Table 16 reports coefficients of variation of 0.69–0.71 for routing second moments and 0.37–0.65 for projected gradient moments across specialists. These deviations rule out treating the equal-moment idealization as an established property of the trained router. On matched minibatch replays, the coefficient of variation of one-step specialist update norms is 0.159 with inverse-trace scaling, versus 0.301 with uniform scaling and 0.437 with shuffled inverse-trace scales. The original paper's fixed-scale Long ablation (A versus F) gives a 2.0-point difference. This training ablation sets all specialized scales to 2; the uniform-scale update replay uses the mean of the proposed scales. The theoretical and empirical claims are therefore aligned: the rule reduces imbalance and controls gradient scale under stated conditions; it does not guarantee identical gradients for an input-dependent router.

The backbone correction is likewise an expectation statement, not a per-token identity. With μi=E[wi(x)]\mu_i=\mathbb E[w_i(x)], the initialization-time equivalent operator satisfies

Weq(x)−W0=∑i=17(wi(x)−μi)Di0.W_{\mathrm{eq}}(x)-W_0=\sum_{i=1}^{7}\big(w_i(x)-\mu_i\big)D_i^0.

Symmetric routing gives μi=1/7\mu_i=1/7, but the token-dependent residual is generally nonzero. Q2 quantifies the residual and distinguishes the expected and exact-per-token controls.

Q1. Per-suite GOAT results and one-factor-at-a-time comparisons

Per-suite LIBERO-Plus comparison under the Table 2 evaluation protocol. The four suite totals give the following matched view; the overall total follows the original benchmark aggregation.

MethodSpatial (%)Object (%)Goal (%)Long (%)Overall (%)
GOAT86.782.370.467.876.8
VLA-GSE90.386.274.274.181.2
Difference+3.6+3.9+3.8+6.3+4.4

The expert-role and one-factor controls are shown in W1. In particular, D versus B is the capacity- and active-computation-matched shared-path test; A versus C is the fixed-topology spectral role-swap; A versus E is the pure segment-placement control; and A versus F is the scaling control. G versus H further tests insertion of a persistent path under GOAT-style spectral assignment and scaling. We distinguish the aggregate GOAT comparison from the Long-suite causal controls rather than assigning the entire 4.4-point aggregate gain to one component.

The complete-method multimodal comparison also favors VLA-GSE over GOAT across seven of eight evaluated benchmarks, averaging 76.46% versus 72.34%. In this complete-method comparison, VLA-GSE uses the original paper's OCRBench result of 85.6%, versus GOAT's 79.0%, a difference of 6.6 percentage points.

Reviewer_pML1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, VLA-GSE is proposed for parameter-efficient adaptation of VLM to robotic control. In that, SVD initialization combines an always-active generalized expert with routed specialized experts. Expert-specific scaling and backbone weight adjustment support optimization. Both simulation and real-world experiments demonstrate improved control performance, while maintaining multimodal comprehension capabilities comparable to those of LoRA.

Critical strengths
  1. The paper is well written and easy to follow.
  2. Results for LIBERO-Plus demonstrate improvements over FFT and GOAT when compared under the same training time budget.
  3. Experiments in real-world environments include comparisons with GOAT as well as pairwise evaluations.
  4. Multimodal benchmark tests indicate a capacity for knowledge retention comparable to LoRA, while simultaneously achieving superior control performance.
Critical weaknesses
  1. Theorem 1 calculates routing-weighted gradient moments without assuming decorrelation (Appendix B, Equations 21 and 25). Balanced routing alone is insufficient to justify this step.
  2. GOAT [1] employed routed experts initialized via SVD, with scaling and residual subtraction applied. Removing the expert component alters the operational computation, making it difficult to isolate the incremental contribution of VLA-GSE.
  3. Knowledge-retention comparisons omit GOAT, leaving the control-versus-retention trade-off against the strongest PEFT baseline unclear.

[1] Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment. ICML, 2025.

Questions for authors

Please see and address the weaknesses above.

4 comments

Comment: We thank the reviewer for the careful reading and the positive comments on the clarity of our paper. We also appreciate the recognition that our method improves robot control while preserving multimodal knowledge. Below, we respond to each comment and question point by point.

W1: Missing decorrelation condition in Appendix B and the corrected proof

We have corrected the proof by adding a decorrelation assumption with zero covariance between each specialized expert’s squared routing weight and its projected gradient energy.

1. Additional assumption.

Eqs. (21) and (25) factor the gradient moment when each specialized expert's squared routing weight and its projected gradient energy have zero covariance. Balanced routing supplies a common marginal second moment across experts. The revised proof keeps the covariance explicitly so that the gradient identity also applies when both quantities vary together.

For specialized expert ii, let wi(x)w^i(x) be its routing weight and gg the gradient with respect to the equivalent layer weight. The projected gradient energies are qLi(x)=Tr⁡(ΣiUi⊤gg⊤Ui)q_L^i(x)=\operatorname{Tr}(\Sigma_i U_i^\top gg^\top U_i) and qRi(x)=Tr⁡(ΣiVi⊤g⊤gVi)q_R^i(x)=\operatorname{Tr}(\Sigma_i V_i^\top g^\top gV_i).

The additional condition for Eq. (5). For the theorem to hold, we additionally assume that both covariances below vanish over training inputs at initialization:

\text{for every specialized expert }i.$$ $$\operatorname{Cov}_{x}\left((w^i(x))^2,q_R^i(x)\right) = 0, \text{for every specialized expert }i.$$ **2. Corrected gradient moment.** In the revised theorem and Appendix B, the squared factor-gradient moment is $s_s^i\mathbb{E}[(w^i)^2 q_L^i]$ for $A_s^i$ and $s_s^i\mathbb{E}[(w^i)^2 q_R^i]$ for $B_s^i$. In each case the joint moment equals the product of the means plus their covariance. Under the new assumption and balanced routing, this reduces to Eq. (5). The trace-inverse equalization result additionally uses Eq. (7), which makes mean projected energy proportional to spectral trace on each side. **3. Theoretical rationale and sufficient conditions.** The rationale for the routing-gradient decorrelation assumption as a moment-factorization idealization can be seen in [Martens and Grosse (International Conference on Machine Learning (ICML) 2015), §2, Eq. (1), and §2.1, Eq. (2)](https://proceedings.mlr.press/v37/martens15.pdf#page=2). Kronecker-factored Approximate Curvature (K-FAC), a method for approximating natural-gradient optimization, approximates joint activation-gradient moments by products of marginal moments. Here the analogous factorization concerns squared routing weights and projected gradient energies. We state zero covariance explicitly as the condition for exact factorization and retain the covariance in the general identity. **The exact identity and the scaling corollary.** Let $T_i=\operatorname{Tr}(\Sigma_i)>0$, $\rho_i=\mathbb E[(w_i)^2]$ and $\alpha_i^q=\mathbb E[q_q^i]$. Then $$\begin{aligned} \mathbb E\|g_A^i\|_F^2 &=s_i[\rho_i\alpha_i^L+\operatorname{Cov}((w_i)^2,q_L^i)],\\ \mathbb E\|g_B^i\|_F^2 &=s_i[\rho_i\alpha_i^R+\operatorname{Cov}((w_i)^2,q_R^i)]. \end{aligned}$$ Zero covariance gives factorization within each expert; balanced routing makes $\rho_i$ common across experts. If $\alpha_i^q/T_i=c_q$ is also constant across experts on each side, then $s_i=s_{\mathrm{base}}\bar T/T_i$ equalizes the corresponding expected squared factor-gradient energies. The constants may differ between projection sides. Without factorization, the sufficient condition is instead that $\mathbb E[(w_i)^2q_q^i]/T_i$ be expert-independent on each side. ## W2: Distinction from GOAT and controlled evaluation We thank the reviewer for highlighting the relationship with GOAT. We acknowledge that SVD-based expert initialization, routed low-rank experts, scaling, and residual subtraction have established precedents in GOAT. **Our contribution is not these components individually, but a different allocation of pretrained spectral knowledge across persistent and routed expert paths, together with a distinct gradient-balancing objective.**

1. GOAT-s does not establish the optimality of its spectral scaling. Importantly, GOAT-s (Appendix E, Eqs. 79–83) attempts to align low-rank expert gradients with full-rank MoE gradients, but its derivation contains important gaps. First, Eq. (79) minimizes a gradient-dependent projection error, whereas Eq. (82) replaces this objective with a singular-value-only balancing condition without establishing the necessary assumptions on gradient directions and expert subspaces. Second, Eq. (81) explicitly contains the factor si2σi2 s_i^2\sigma_i^2\, while Eq. (82) instead equates si2σis_i^2\sigma_i, leaving the change in singular-value dependence unjustified. For higher-rank experts, GOAT-s further substitutes the sum of singular values without deriving its optimality for the original objective. Consequently, the proposed inverse-square-root scaling is a heuristic rather than a demonstrated general solution to its stated gradient-alignment problem.

GOAT's own results also show that GOAT-s provides no consistent empirical improvement. In its Appendix E, Table 8, GOAT-s performs below the original GOAT on three of four benchmark groups and achieves a slightly lower aggregate score (71.02 vs. 71.12). The authors characterize the results as comparable rather than demonstrating an improvement. This does not invalidate GOAT, but it highlights the limitations of its extended scaling formulation.

2. VLA-GSE addresses a different optimization objective. Instead of matching each low-rank update to its full-rank counterpart, we explicitly balance the expected squared factor-gradient magnitudes of specialized experts under the stated gradient-moment assumptions. This yields an inverse-block-trace scaling rule under those conditions, rather than GOAT-s's inverse-square-root rule. Architecturally, GOAT routes its SVD-initialized experts, whereas VLA-GSE assigns the leading spectral block to an always-active generalized expert and the following disjoint blocks to specialized experts. Accordingly, the initialization correction subtracts the full shared contribution and the expected routed contribution. These choices explicitly distinguish persistent pretrained-knowledge preservation from task-dependent adaptation.

Together, our contribution lies in the spectral-role decomposition, and the gradient-balancing objective, rather than a simple combination of existing GOAT components.

W3: Knowledge retention comparison with GOAT

We thank the reviewer for highlighting the omission of GOAT from the original retention comparison. We agree that evaluating both robotic control and multimodal knowledge retention is essential for understanding the control–retention trade-off. The following table shows the knowledge retention comparison between GOAT and VLA-GSE. For VLA-GSE, the OCRBench entry is the original paper's reported result of 85.6%.

BenchmarkPretrained (%)GOAT (%)VLA-GSE (%)LoRA (%)
MMMU53.247.451.151.8
MMStar69.863.267.868.1
OCRBench88.179.085.683.3
MMB-en85.177.282.483.0
DocVQA95.393.192.894.1
InfoVQA80.372.878.579.2
AI2D84.179.283.483.0
RealWorldQA70.966.870.169.8
Average78.3572.3476.4676.54

Analysis of the control–retention trade-off

1. Stronger robotic adaptation with improved knowledge retention. The original experiments establish that VLA-GSE improves LIBERO-Plus success over GOAT by 4.4 percentage points. In the retention comparison presented above, VLA-GSE also outperforms GOAT across 7 of 8 multimodal benchmarks, with an average advantage of 4.12 percentage points (76.46% vs. 72.34%). Particularly notable gains appear on OCRBench (+6.6 points), InfoVQA (+5.7 points), and MMStar (+4.6 points). These comparisons characterize a favorable control–retention trade-off, combining stronger robotic adaptation with better average preservation of pretrained multimodal capabilities.

2. The role of the generalized expert in knowledge preservation. A key architectural distinction is that GOAT employs exclusively routed low-rank experts, whereas VLA-GSE explicitly maintains an always-active generalized expert initialized from the dominant singular components of the pretrained backbone. This generalized expert provides a persistent pathway for pretrained spectral knowledge, independent of input-dependent routing decisions. Meanwhile, specialized experts capture task-specific adaptations, enabling stronger robotic control while maintaining access to general pretrained representations.

This architectural distinction provides a mechanistic explanation for the retention advantage described in the comparison. In GOAT, individual expert contributions depend on routing decisions, introducing variability in how pretrained knowledge is represented across inputs. In contrast, VLA-GSE consistently activates the generalized expert, preserving a stable contribution from dominant pretrained spectral components while allowing specialized experts to adapt to robotic tasks. This separation reduces the reliance on conditional routing for maintaining general multimodal capabilities and better balances knowledge preservation against task-specific specialization.

Thank you very much for your thoughtful comments and valuable feedback. Your suggestions have been very helpful in improving our work.

As the discussion period draws to a close, we would like to follow up on our responses and clarifications. We would greatly appreciate it if you could review our responses and let us know whether your remaining concerns have been adequately addressed. We hope that our explanations have sufficiently addressed your concerns, and we welcome any further feedback or suggestions.

Thank you again for your time and consideration.

Reviewer_ghHLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces VLA-GSE, a parameter-efficient method for adapting VLMs to robot control. It uses SVD to initialize low-rank experts from pretrained weights. A shared expert uses the top singular components and is always active. A router selects specialized experts built from the next components. The method uses expert-specific scaling to reduce differences in gradient magnitudes. It also adjusts the frozen weights to offset the experts' average contribution at initialization. With Qwen3-VL-4B and an OFT action head, VLA-GSE improves success rates on LIBERO-Plus. It retains general multimodal performance at a level similar to LoRA. Further tests cover four real-robot tasks.

Critical strengths
  1. GSE outperforms strong baselines on LIBERO-Plus. In Table 2, its success rate is 81.2%, compared with 76.8% for GOAT and 74.9% for full fine-tuning. The comparison uses the same Qwen3-VL backbone, and the PEFT methods have similar trainable parameter budgets. GSE also preserves general multimodal performance about as well as LoRA. These results suggest that it learns robot control more effectively while retaining much of the VLM's original ability.

  2. GSE also performs well on a real robot. With the same Qwen3-VL backbone and OFT head, it achieves 82.5% success, compared with 65.8% for full fine-tuning. Tests cover changes in lighting, instructions, clutter, and background. GSE performs better on all four tasks. These results support the gains seen in simulation, though the evaluation covers only four tasks and one training seed.

Critical weaknesses
  1. Theorem 1 seems to need an additional assumption. Eq. (5) separates the expected squared gradient norm into a routing factor and a projected gradient factor. However, routing weights and gradient energy both depend on the input and may be correlated. Equal routing second moments across experts are not enough to justify this separation. The theorem needs a condition that addresses this dependence. The scaling ablation in Table 4 suggests that it improves task performance, but this does not establish the exact equality in Eq. (5).

  2. The ablations do not clearly separate the benefit of the expert design from changes in computation. As the authors acknowledge, replacing shared or routed experts in Table 4 also changes the active computation per token. These results show that the full design works better, but they do not isolate the benefit of the proposed expert roles. Table 3 shows that GSE retains multimodal ability, but it does not show whether the shared expert is responsible. Controls with the same active computation, evaluated on both robot tasks and multimodal benchmarks, would help answer these questions.

Questions for authors
  1. What additional assumption justifies separating the routing and gradient terms in Eq. (5)? Both depend on the input and may be correlated. Does Theorem 1 still hold under the assumptions currently stated?

  2. Can you compare the shared and routed expert variants while keeping both trainable parameters and active computation matched? Please report robot success and multimodal performance for these variants. This would help show whether the expert roles explain the gains and whether the shared expert helps preserve multimodal ability.

4 comments

Comment: We thank the reviewer for the careful reading and the positive comments on the clarity of our paper. We also appreciate the recognition that our method improves robot control while preserving multimodal knowledge. Below, we respond to each comment and question point by point.

W1: Missing decorrelation condition in Appendix B and the corrected proof

We have corrected the proof by adding a decorrelation assumption with zero covariance between each specialized expert’s squared routing weight and its projected gradient energy.

1. Additional assumption.

Eqs. (21) and (25) factor the gradient moment when each specialized expert's squared routing weight and its projected gradient energy have zero covariance. Balanced routing supplies a common marginal second moment across experts. The revised proof keeps the covariance explicitly so that the gradient identity also applies when both quantities vary together.

For specialized expert ii, let wi(x)w^i(x) be its routing weight and gg the gradient with respect to the equivalent layer weight. The projected gradient energies are qLi(x)=Tr⁡(ΣiUi⊤gg⊤Ui)q_L^i(x)=\operatorname{Tr}(\Sigma_i U_i^\top gg^\top U_i) and qRi(x)=Tr⁡(ΣiVi⊤g⊤gVi)q_R^i(x)=\operatorname{Tr}(\Sigma_i V_i^\top g^\top gV_i).

The additional condition for Eq. (5). For the theorem to hold, we additionally assume that both covariances below vanish over training inputs at initialization:

\text{for every specialized expert }i.$$ $$\operatorname{Cov}_{x}\left((w^i(x))^2,q_R^i(x)\right) = 0, \text{for every specialized expert }i.$$ **2. Corrected gradient moment.** In the revised theorem and Appendix B, the squared factor-gradient moment is $s_s^i\mathbb{E}[(w^i)^2 q_L^i]$ for $A_s^i$ and $s_s^i\mathbb{E}[(w^i)^2 q_R^i]$ for $B_s^i$. In each case the joint moment equals the product of the means plus their covariance. Under the new assumption and balanced routing, this reduces to Eq. (5). The trace-inverse equalization result additionally uses Eq. (7), which makes mean projected energy proportional to spectral trace on each side. **3. Theoretical rationale and sufficient conditions.** The rationale for the routing-gradient decorrelation assumption as a moment-factorization idealization can be seen in [Martens and Grosse (International Conference on Machine Learning (ICML) 2015), §2, Eq. (1), and §2.1, Eq. (2)](https://proceedings.mlr.press/v37/martens15.pdf#page=2). Kronecker-factored Approximate Curvature (K-FAC), a method for approximating natural-gradient optimization, approximates joint activation-gradient moments by products of marginal moments. Here the analogous factorization concerns squared routing weights and projected gradient energies. We state zero covariance explicitly as the condition for exact factorization and retain the covariance in the general identity. **The exact identity and the scaling corollary.** Let $T_i=\operatorname{Tr}(\Sigma_i)>0$, $\rho_i=\mathbb E[(w_i)^2]$ and $\alpha_i^q=\mathbb E[q_q^i]$. Then $$\begin{aligned} \mathbb E\|g_A^i\|_F^2 &=s_i[\rho_i\alpha_i^L+\operatorname{Cov}((w_i)^2,q_L^i)],\\ \mathbb E\|g_B^i\|_F^2 &=s_i[\rho_i\alpha_i^R+\operatorname{Cov}((w_i)^2,q_R^i)]. \end{aligned}$$ Zero covariance gives factorization within each expert; balanced routing makes $\rho_i$ common across experts. If $\alpha_i^q/T_i=c_q$ is also constant across experts on each side, then $s_i=s_{\mathrm{base}}\bar T/T_i$ equalizes the corresponding expected squared factor-gradient energies. The constants may differ between projection sides. Without factorization, the sufficient condition is instead that $\mathbb E[(w_i)^2q_q^i]/T_i$ be expert-independent on each side. ## W2: Distinction from GOAT and controlled evaluation We thank the reviewer for highlighting the relationship with GOAT. We acknowledge that SVD-based expert initialization, routed low-rank experts, scaling, and residual subtraction have established precedents in GOAT. **Our contribution is not these components individually, but a different allocation of pretrained spectral knowledge across persistent and routed expert paths, together with a distinct gradient-balancing objective.**

Q2: Can expert roles be compared at matched parameters and active computation?

We compare four configurations, A, B, C and D, with a pair matched in trainable parameters and principal active computation.

All four use eight rank-2 experts per adapted layer and activate three experts per token, giving an active rank of 6. They start from the same eight initial factor-and-scale packages.

  • A: Original VLA-GSE allocation. One generalized expert is always active and receives the leading SVD directions at initialization. Each token also activates 2 of 7 specialized experts, initialized from disjoint residual singular blocks. The router has 7 outputs, one per specialized expert.
  • B: All-routed allocation. All eight experts are routable, and each token selects 3 of them. No expert is always active. B uses the same initial expert packages as A but has eight router outputs. A versus B therefore keeps the expert count and active rank fixed while changing both role allocation.
  • C: Spectral role-swap. C keeps A’s architecture: 1 always-active generalized expert, 2 selected specialized experts per token and seven router outputs. It exchanges the initial factors and scales of the generalized expert and the first specialized expert. A versus C tests whether assigning the leading singular directions to the persistent shared path matters, with the architecture fixed.
  • D: Shared allocation with matched router width. D uses a router output width of 8 logits per token at every adapted layer, matching B’s 8-output projection. D retains A’s always-active generalized expert and spectral assignment. Of D’s 8 logits, the 8-th is computed but masked before selection. Each token therefore still activates one generalized expert and 2 of 7 specialized experts, whereas B selects three of eight experts. D and B have identical stored trainable parameter counts and matched expert and router matrix multiplications, while D reserves one persistent shared path and B routes all three active paths.

D and B each have 115,201,031 stored parameters designated as trainable, including 9,306,112 router parameters, and perform identical principal computation: 7.561 billion expert matrix operations and 4.60 billion router matrix operations per profiled call. B versus D evaluates the contribution of a persistent shared expert under matched parameter storage and principal computation: B dynamically selects 3 of 8 experts, whereas D always activates the generalized expert and routes the remaining 2. A versus C isolates spectral-role assignment under an identical architecture, differing only in the assignment of the leading spectral factor-and-scale package to the shared or specialized expert and the corresponding frozen-weight offset.

Expert-role and spectral-role controls: Multimodal retention

The multimodal retention experiment is shown below. For OCRBench, the expert-role controls below use the same fixed 1,000-question evaluation subset.

BenchmarkA (%)B (%)C (%)D (%)
Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU)51.146.840.550.9
MMStar67.859.447.668.0
OCRBench (1,000-question subset)80.267.141.482.2
MMBench (English)82.475.662.582.7
Document Visual Question Answering (DocVQA)92.884.970.292.6
Infographic Visual Question Answering (InfoVQA)78.567.350.878.9
AI2D83.476.865.283.1
RealWorldQA70.162.752.469.8
Average75.7967.5853.8376.03

Multimodal retention analysis. Under matched stored trainable-parameter counts and principal active computation, D (shared) outperforms B (all-routed) across all eight multimodal benchmarks, improving average accuracy from 67.58% to 76.03% (+8.45 percentage points). The largest improvement occurs on OCRBench (+15.10 pp), indicating that the persistent shared path is particularly beneficial for preserving recognition-related capabilities. Furthermore, under a fixed routing architecture, A substantially outperforms C (75.79% vs. 53.83%), highlighting the importance of assigning leading spectral components to the shared expert. Notably, A and D achieve comparable average multimodal accuracy (75.79% vs. 76.03%), despite their different router widths.

Expert-role and spectral-role controls: recognition breakdown on the OCRBench subset

VariantText recognitionScene-text questionsDocument questionsInformation extractionHandwritten mathematicsOverall correct / scored (%)
Pretrained reference271 / 300178 / 200185 / 200175 / 20061 / 100870 / 1,000 (87.00%)
A: Original VLA-GSE allocation242 / 300161 / 200183 / 200165 / 20051 / 100802 / 1,000 (80.20%)
B: All routed206 / 300142 / 200162 / 200133 / 20028 / 100671 / 1,000 (67.10%)
C: Spectral role-swap202 / 30099 / 20067 / 20036 / 20010 / 100414 / 1,000 (41.40%)
D: Shared, matched router width247 / 300174 / 200181 / 200172 / 20048 / 100822 / 1,000 (82.20%)

On this 1,000-question subset, D achieves 82.20% OCRBench accuracy versus B's 67.10%, a gain of 15.10 percentage points, corresponding to 151 additional correct answers. The improvement spans all five categories, with particularly large gains in information extraction (+19.50 pp) and handwritten mathematics (+20.00 pp). This indicates that retaining a persistent shared path substantially improves recognition-related knowledge retention under matched parameter storage and principal adapter computation.

With the routing architecture fixed, A achieves 80.20% versus C's 41.40%, a gain of 38.80 percentage points. The largest improvements occur in document questions (91.50% vs. 33.50%) and information extraction (82.50% vs. 18.00%), jointly accounting for 245 of the 388 additional correct answers. These results highlight the importance of assigning the leading spectral factor-and-scale package to the persistent shared expert rather than exchanging it with a specialized expert's package.

Expert-role analysis: robot success rate

VariantLibero-Plus Long success (%)
A: Original VLA-GSE allocation74.1%
B: All routed67.2%
C: Spectral role-swap60.3%
D: Shared, matched router width72.5%

Expert-role and spectral-role analysis. The robot results further support the contributions of expert roles and spectral allocation. Under matched stored parameter counts and principal adapter computation, the shared variant D outperforms the all-routed variant B in LIBERO-Plus Long success (72.5% vs. 67.2%, +5.3 pp) and multimodal accuracy (76.03% vs. 67.58% on average; 82.20% vs. 67.10% on OCRBench). This supports the benefit of preserving a persistent shared path for both robotic adaptation and multimodal knowledge retention. Furthermore, under an identical routing architecture, the original spectral allocation A outperforms the role-swapped variant C in robot success (74.1% vs. 60.3%) and multimodal understanding (75.79% vs. 53.83%), highlighting the importance of assigning leading spectral components to the shared expert. Together, these comparisons support the importance of complementary expert roles and principled spectral allocation beyond differences in stored parameter capacity and principal adapter computation.

Thank you very much for your thoughtful comments and valuable feedback. Your suggestions have been very helpful in improving our work.

As the discussion period draws to a close, we would like to follow up on our responses and clarifications. We would greatly appreciate it if you could review our responses and let us know whether your remaining concerns have been adequately addressed. We hope that our explanations have sufficiently addressed your concerns, and we welcome any further feedback or suggestions.

Thank you again for your time and consideration.

Comments on the paper

1 comment

Dear Reviewers,

Thank you for the review. The authors have responded to the points you raised, and the response period closes on 18 November (AoE). I would be grateful if you could post a brief reply on whether your concerns are resolved, partly resolved, or still open, so that the authors have a fair chance to address them. If anything still needs clarifying, please ask now while they can still respond.

Regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.