- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposed a merging technique (Trajectory-Calibrated Regression; TCR) that creates a merged policy from multiple specialist policies. Specifically, it proposed to do the calibration process by propagating different blocks in the action decoding, caching the intermediate values, and propagating the merging process by fitting later modules on the previously merged representations. Experiments show that the proposed TCR leads to better success rate in most cases across LIBERO, LIBERO-Pro, RoboTwin, and a real world tasks using pi0.5 as the base policy.
Critical strengths
- The main idea is very intuitive, merging policies in different stages on calibrated inputs intuitively allows the policy to stay in-distribution better.
- Experiments are sufficient, although trying different base VLA models would have strengthen the findings more.
- Also tested the continual learning capability of the merged policy.
Critical weaknesses
-
The main issue with the current draft is writing clarity and many places are hard to understand. Overall there are too many math symbols in the method section and it feels convoluted. Some visual illustrations would have been nicer. More details in the questions below.
-
As TCR follows the formulation in Eq 1, it only tries to copy the expert motion superficially but is not aware of any of the semantics or interactions among the expert policies. If two policies have overlapping input states but different motions, then the objective optimizes to learn a mean behavior, which might hurt each of the expert behavior.
-
It seems quite compute-heavy even with the caching mechanism, as the intermediate representations have to be saved for each individual expert policies. This may pose a scaling issue when the base policy has a large number of parameters or there are many expert policies that we want to merge from.
-
The intermediate generation step components assumes an iterative action decoding module; without it the proposed method reduces to regression-and-replay style of policy merging.
-
Some in-depth analysis is lacking. For example, the paragraph starting at line 406 only describes the results but didn't explain why that is the case or give any general conclusion.
Questions for authors
-
It would be helpful if the authors can help clarify the following writing-oriented questions:
a) line 76: it is unclear what "These dependencies" refer to
b) line 107: the first bullet point of contribution is your methodology instead of an actual contribution?
c) line 177: what is the difference between theta and W? is W a subset of theta that is linear?
d) line 195: why do we need this equation?
e) line 199: is the definition of the specific entry in W used anywhere later?
f) line 230: taking the mean of the distribution is a little confusing, especially if the call-inputs are already somewhat hand-wavy.
g) line 276: why upweigh the experts that are already close? why not incentivize the policy to learn from from the experts that are not well-represented by the mere policy?
h) line 367: the gap between line 366 and 367 seems to be squeezed, did the authors change the provided template?
-
It seems like spatial tasks is one of the weakness for the proposed framework (Tables 1 & 11), do the authors have any intuition of why that might be the case? and what property of the TCR does this reflect?
-
Is there any ablation on using relative weighting for both passes (Equation 9)?
-
The discussion and limitation section didn't cover limitation?
-
Appendix B.1. seems to show that having a third calibration pass improves the policy even more, so why stop at two passes? why not keep going until convergence?
11 comments
[Part 2/5]
3. Critical weakness 3: Cache size, computation, and scaling
The original four-expert configuration (two passes, 5.77M regression rows) requires approximately 1.03 h on a single A100-80GB, summing the profiled collection and construction stages, with a peak memory of 32.8 GiB. Deployment uses a single static model without expert routing.
For comparison, FeatCal profiled twice under the same protocol requires 1.55 h and 1.91 h, with 86–87% of the time spent in its solve/export stage. The 42.9 GiB listed for FeatCal in our cost table was sampled per device and included memory used by another job; sampled per process, the peak is 20.5 GiB. The original TCR implementation therefore uses more memory than FeatCal, and its advantage lies mainly in construction time. We have corrected two entries in the cost table (Table 15 in Appendix B.7 of the revised manuscript; Table 13 in the submission): FeatCal's peak is now 20.5 GiB, and TCR's peak is updated from 31.75 GiB to 32.8 GiB, the process peak of the two-pass build confirmed in two independent measurements.
To examine how cost scales with the number of experts, we measured one-pass construction. For , the bank consists of repeated copies of the original four experts.
| Number of experts | One-pass construction | Peak allocated memory |
|---|---|---|
| 4 | 692 s | 18.62 GiB |
| 8 | 824 s | 26.12 GiB |
| 12 | 1,207 s | 34.80 GiB |
| 20 | 1,788 s | 52.18 GiB |
All entries were measured on an exclusive A100-80GB. The number of regression rows grows linearly (1.11M per expert). Least-squares fits give s and GiB, i.e., approximately 1.2 minutes and 2.11 GiB per additional expert; individual time increments are not uniform because weight loading and I/O vary between runs.
As the number of experts increases, the main bottleneck is the dual solve in the original implementation, whose kernel grows quadratically with the number of regression rows. For the largest module, the kernel alone occupies 10.9 GiB at and 30.2 GiB at ; the solve also requires factorization workspace alongside the cached features and model weights, and the dual build at ran out of memory on an A100-80GB. The primal form uses a kernel (1.0 GiB at ), and the 20-expert build completes.
The memory reduction consists of two parts. First, each block accesses only one prefix K/V layer, so we keep all 18 layers in host memory from load time, transfer one layer at a time to the GPU, and release tensors that are no longer needed. This reduces the four-expert peak from 30.82 GiB to 20.85 GiB while leaving the SHA-256 hash of the exported checkpoint unchanged. Second, the primal solve further reduces the peak to 18.62 GiB. The two forms are algebraically equivalent; the resulting weights differ by in relative .
In a separately profiled four-expert one-pass build of 711 s, the time divides into 178 s for the solves, 184 s for feature capture and replay, and approximately 349 s of fixed overhead for loading weights, verifying replay references, and writing the checkpoint. Since the solve accounts for about a quarter of the total, further savings would come mainly from I/O.
[Part 3/5]
4. Critical weakness 4: Dependence on an iterative action decoder
Intermediate generation calls are not specific to . The /, GR00T N1, and SmolVLA families generate continuous actions with flow matching, and RDT employs diffusion; all of these expose intermediate states that can be sampled for calibration. The multi-call component therefore applies to this broader class of iterative VLA decoders.
For a non-iterative decoder, intermediate calls are absent, but execution-stage sampling, matching of expert-module outputs, multimodal merged-prefix replay, and sequential passes remain applicable given an architecture-specific replay adapter. On , a same-request, matched-row comparison yields 79.11% for multi-call versus 74.75% for final-only (+4.36%).
We further conducted LIBERO merging experiments on GR00T N1.6 to test TCR beyond , merging publicly released Spatial and Object specialists fine-tuned from the same base (3.29B parameters). Every method produces one fixed set of weights that serves both suites; tasks are distinguished only by the language instruction, without task-specific parameters, adapters, routing, or normalization. The two specialists differ in 1.62B parameters (49.3%), including the action-head DiT (1.09B), the action encoder (0.23B), language-model layers 12–15 (0.20B), the state encoder, and the action decoder; TCR calibrates 99.9% of these parameters. All methods start from the same Soup and use the hyperparameters of the paper without tuning. Because no static demonstration set is available for GR00T, RegMean++ and FeatCal use expert-execution inputs for calibration. Each method is evaluated on 180 episodes:
| Method | Success |
|---|---|
| Specialists | 97.22% |
| Soup | 2.78% |
| RegMean++ | 8.33% |
| FeatCal | 2.78% |
| TCR | 57.22% |
TCR reaches 57.22% success, whereas no other merging method exceeds 8.33%. The low baseline scores reflect how far apart the two specialists are. We measure the action error of Soup relative to the action difference between the two specialists: this ratio is 1.08–1.22 on GR00T, meaning that Soup deviates from each specialist more than the other specialist does, compared with 0.51–0.76 for our specialists. In addition, FeatCal's anchor, , assumes that the base and the specialists share a parameterization. In GR00T, a set of action-head projection layers is randomly initialized in the base, and the deployed specialists have their action normalization folded into these layers. The anchor is therefore not informative, and FeatCal performs at the level of Soup (2.78% for both). RegMean++ does not use this anchor and reaches 8.33%, so TCR's margin over it (48.89%) is unaffected by the anchor issue.
5. Critical weakness 5: Why does action agreement relate to closed-loop success?
We have expanded the last paragraph of Sec. 4.3 (“Does action agreement accompany skill retention?”) and added Figure 2 to investigate why methods with lower action MSE sometimes succeed less often (the analysis at line 406). We held the offline MSE ranking fixed and varied which suites contributed to closed-loop success. Across nine methods, the Spearman correlation between negative MSE and success was 0.86 overall, rising to 0.95 on Spatial, Object, and Goal but falling to 0.52 on Long alone. One reversal makes this concrete. KnOTS-TIES has lower MSE than TIES-Merging and higher mean success on the first three suites (61.11% vs. 52.22%), yet lower success on Long (6.67% vs. 20.33%). This exposes a limitation of action agreement as a diagnostic: it captures behavior at expert-visited states, while a single missed interaction can leave a long-horizon policy in a state where its ability to recover was never tested offline.
[Part 1/5]
We thank the reviewer for the constructive comments on presentation, expert conflicts, efficiency, and the interpretation of action agreement.
Below we address all raised concerns. All differences in success rate are absolute.
1. Critical weakness 1: Clarity, notation, and illustrations
TCR does not introduce a new ridge solver. It combines (i) sampling execution states from the frozen experts' closed-loop runs, (ii) sampling intermediate calls within iterative action generation, and (iii) merged-prefix replay adapted to the multimodal VLA computation graph. The last component preserves the actual dependencies among vision features, language-prefix K/V states, flow-time conditioning, and the action expert. Our π0.5 implementation covers 418 linear modules (422 adapted tensors), and we verified that replay reproduces the native forward computation for identical weights and inputs. Subsequent passes use the previous merged checkpoint as the prior and fit fresh frozen-expert caches.
We have revised the presentation to make TCR’s core idea and calibration procedure easier to follow. In the revised manuscript, we have reorganized the method section around these three dependency axes: in Sec. 3.2, the paragraphs “Across task execution” and “Within action generation” cover execution time and generator calls, and Sec. 3.3 covers network propagation. We have also redrawn Figure 1 to illustrate the data flow, and removed unnecessary notation. Point-by-point clarifications are provided under Q1(a–h).
To help readers understand the method, we have also prepared a more accessible technical walkthrough on our anonymous website (https://anonymous.4open.science/r/policy-calibration-5320/index.html).
2. Critical weakness 2: Do conflicting expert actions collapse into average behavior?
TCR does not average final actions. It matches each expert's linear-module outputs on task-conditioned inputs, allowing observations, robot states, and language instructions to separate distinct target behaviors.
Table 11 in Appendix B.5 of the revised manuscript reports action MSE and cosine similarity with the assigned specialist. To directly test whether the merged action is shifted toward the other experts, we define an action-compromise projection:
t &= \frac{(a_{\mathrm{merge}}-a_{\mathrm{owner}})^\top(\bar a_{\mathrm{other}}-a_{\mathrm{owner}})}{\lVert\bar a_{\mathrm{other}}-a_{\mathrm{owner}}\rVert_2^2},\\ \bar a_{\mathrm{other}} &= \frac{1}{3}\sum_{j\neq\mathrm{owner}} a_j. \end{aligned}$$ Here $a_{\mathrm{owner}}$ is the action of the specialist assigned to the current task, and $\bar a_{\mathrm{other}}$ is the mean action of the other three specialists. $t=0$ indicates no displacement along this direction, and $t=0.75$ corresponds to the equally weighted average of all four expert actions. Over 400 matched expert-execution requests with identical initial generation noise, we obtain: | Method | Soup | RegMean++ | FeatCal | **TCR** | | ------------------------------ | ----: | --------: | ------: | --------------: | | Action-compromise projection ↓ | 0.600 | 0.471 | 0.373 | **0.341–0.345** | In 308 critical gripper decisions where the assigned expert and the other experts favor opposite open/close directions, TCR does not produce a half-open intermediate decision. Across 1,500 Goal-suite rollouts, wrong-target selections per 100 episodes are 0.3 (TCR), 5.3 (FeatCal), 12.7 (RegMean++), and 15.7 (Soup). Overall, TCR largely preserves task-conditioned actions and target selection; the remaining errors arise from high-conflict action modules and fine manipulation. We have added the paragraph “Compromise toward other experts” and Table 12 to Appendix B.5 of the revised manuscript. The paragraph gives the definition of the projection (Eq. 16), the measurement protocol, and the gripper-conflict result; Table 12 lists the compromise index and the Goal wrong-target selections for each method.