- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TrajScale-VLA brings VAR-style next-scale prediction to action tokenization. The core contribution is MTR. The cumulative reconstruction after the first k scales, D(F̂_k), is supervised against a scale-matched target. That target is built by average-pooling the expert chunk to s_k steps and linearly interpolating back to H (Eq. 3). The intermediate scales are weighted uniformly with λ_mtr/(K−1), and the final scale is supervised against the full chunk. In Stage 2, a Qwen3-VL-4B policy predicts the token groups scale by scale, with all tokens inside a scale predicted in parallel (K sequential rounds).
Results:
- 98.4% on LIBERO and 59.8% on RoboCasa-GR1, against baselines mostly copied from the StarVLA reports and other papers.
- A same-backbone ablation (Table 3) comparing w/o MTR, OAT-style and MINT-style intermediate supervision: on RoboCasa, MTR scores +9.0 pp over w/o MTR and +6.5 pp over MINT-style.
- A real-robot USB pick-and-insert task (30 trials per condition): 90.0 / 76.7% ID/OOD vs 76.7 / 50.0% for the best variant.
Critical strengths
-
The within-backbone ablation shows that the choice of intermediate target affects policy learnability, even when reconstruction is nearly identical. On RoboCasa-GR1 (1,200 trials), MTR scores 59.8% vs 50.8% without MTR and 53.3% for MINT-style. Final reconstruction MSE barely changes (3.58 vs 3.63 ×10⁻⁶). The Fig. 5/6 diagnostics are consistent with a coarse-to-fine organization: the peak correction frequency progresses 1→2→4→8 under MTR but stays at 1→1→1→8 for the other variants.
-
The idea is simple and adds no inference cost. MTR changes only Stage-1 targets and is easy to adopt in any residual-scale action tokenizer.
Critical weaknesses
- Lack of discussion of efficiency. The paper motivates TrajScale by the observation that fine-grained action tokens produce long autoregressive decoding steps in prior AR-VLAs, yet it never explicitly discusses the efficiency of its own design.
- From the reported configuration (G = 16 tokens per temporal position, dyadic scales), the token count doubles at every scale. LIBERO (H = 8, scales {1,2,4,8}) uses 16+32+64+128 = 240 tokens per chunk, i.e., more than the 8×7 = 56 raw action values. RoboCasa-GR1 (H = 16, scales {1,…,16}) uses 16+32+64+128+256 = 496 tokens per chunk.
- The paper's implicit argument is fewer sequential rounds (K = 4–5). However, each round must still emit up to 256 tokens in parallel, conditioned on a growing context. Whether this is actually faster than flat AR or compact tokenizers is unclear. I believe the authors need to discuss and quantify the efficiency advantage of TrajScale, e.g., per-chunk latency, throughput and control frequency versus flat AR with the same tokenizer and versus StarVLA-FAST/OFT.
- Mismatch between motivation and mechanistic analysis. MTR is motivated as a reconstruction objective, but Table 3 does not support reconstruction as the explanation for the success gain. On RoboCasa-GR1, w/o MTR has a lower final MSE (3.63 vs 5.18/5.13 ×10⁻⁶) and a lower intermediate MSE (6.75 vs 10.0 ×10⁻³) than both OAT-style and MINT-style, yet a lower success rate (50.8 vs 51.6/53.3%). The authors acknowledge that reconstruction "does not fully characterize downstream policy performance", but offer no further analysis of why MTR works. Their remaining diagnostic, intermediate MSE, is computed against MTR's own training targets and so is favourable by construction. Prior work (e.g., ActionCodec, cited by the paper) suggests that the structure of the token space matters more than reconstruction fidelity; analyses such as per-scale Stage-2 token prediction accuracy, codebook usage, or latent neighbourhood structure are needed before the +9 pp can be attributed to the proposed mechanism.
Questions for authors
- Latency. Is the scale-autoregressive head a separate module, or are the previous scales' tokens fed back through the VLM? What is the per-chunk latency at batch size 1, and the control frequency, compared with StarVLA-FAST/OFT/π and with flat AR using the same tokenizer?
- Inductive bias of the MTR target. MTR fixes in advance what each scale must represent: an average-pooled, linearly interpolated trajectory at a dyadic resolution. Does this not inject a strong inductive bias that constrains how the tokenizer allocates information across scales? How was average pooling + linear interpolation arrived at? Were alternatives tried, e.g., other downsampling/upsampling kernels, low-pass DCT/wavelet targets, keyframe-based targets, or learned/adaptive targets?
6 comments
Dear Reviewer 7T4z,
Thank you again for your thoughtful review and constructive feedback. We added batch-one H100 latency comparisons against Flat AR and StarVLA baselines, and clarified the independent action head, single VLM encoding, and single decoder pass. We also supplemented reconstruction diagnostics with teacher-forced per-scale token accuracy, free-running action MSE, and prefix-intervention experiments, providing evidence for coarse-scale predictability and effective prefix use during refinement. Finally, we clarified the intentional temporal prior and the rationale for average pooling with linear interpolation, supported by comparisons with nearest upsampling and low-pass DCT targets.
We would greatly appreciate it if you could let us know whether these clarifications and additional results address your concerns. We would be happy to discuss any remaining questions.
Best regards, The Authors
Thank you for the responses and additional experiments. The clarifications on the model architecture and VAR-style decoding are very helpful for understanding TrajScale-VLA's inference pipeline, and the new target ablations in Q2 are also informative.
However, for Weakness 2, reporting additional metrics (TF@s, FR MSE, and prefix-replacement MSE) is not a mechanistic explanation; it mainly replaces Stage-1 MSE with offline metrics that correlate better with task success. Moreover, Experiment 2 reveals an anomaly: although ground-truth coarse prefixes (GT1/GT2 + regen) substantially reduce the error of later scales, comparing MTR's Normal MSE (1.904) with GT1 + regen (0.204) shows that ~89% of MTR's test-time error comes from the first-scale prediction (s = 1, 16 parallel tokens), while finer scales (s = 2 … 16) cannot correct errors made at s = 1. Thus, the dominant error source lies in the first scale rather than multi-scale refinement, leaving MTR's Normal FR MSE (1.904) still very close to MINT-style (2.028). How does this reconcile with the +6.5–9.0 pp closed-loop success gain?
This also connects to Q1. Since the paper frames TrajScale-VLA as an autoregressive VLA, I expected action tokens to be generated by the VLM itself (as in OpenVLA, FAST, or OAT), rather than delegated to a separate small action head after encoding once. Why choose a separate lightweight head over native VLM generation, and given that ~89% of MTR's error comes from Scale 1 (which captures coarse motion intent), wouldn't letting the VLM itself predict the tokens (or at least Scale 1) be more effective?
Thank you for your thoughtful review and for recognizing MTR's simplicity and our controlled ablations.
1. Weakness 1: Efficiency.
Thank you for raising this concern. We report ms/chunk, as in FAST, at batch size 1 on an H100. Timing includes preprocessing, the VLM, action generation and postprocessing, excluding network and robot execution. See Table 5 of the revised manuscript.
TrajScale is 3.50–5.27× faster than Flat AR, 4.72–11.69× faster than FAST and 2.79–2.92× faster than StarVLA-π. It is close to 4-step GR00T and faster than 10-step GR00T; OFT remains faster with its MLP head. As in VAR, tokens within each scale are predicted in parallel: 240/496 codes need only 4/5 rounds (LIBERO/RoboCasa).
2. Weakness 2: Mechanistic analysis.
Thank you for highlighting this issue. Similar average MSE need not imply similar control success: small alignment or insertion errors may cause failure, while larger deviations in less sensitive segments may have little effect.
ActionCodec examines token structure and prefix errors beyond reconstruction. We examine coarse prediction and prefix use.
Experiment 1: Predictability.
On RoboCasa-GR1, TF@s is token accuracy (%) with expert prefixes. FR MSE uses predicted tokens throughout; Recon is from Table 3.
MTR has higher TF accuracy at the first two scales and 11.3% lower FR MSE than w/o MTR.
Experiment 2: Prefix use.
We replace the first one or two scales with expert tokens (GT1/GT2), then regenerate later scales (+ regen) or keep their original predictions (only). Normal is FR; MSE is in normalized action space (×10⁻³).
With the GT prefix fixed within each model, regeneration lowers mean MSE by 89.4%/93.8% for MTR (GT1/GT2), versus 36.1%/74.2% for w/o MTR, compared with retaining later tokens.
Together, TF supports coarse predictability, the prefix comparison shows how later scales use coarse information, and FR shows lower action error with policy-predicted tokens, consistent with the control gains. These analyses appear in revised Table 4 and Appendix C.1 (Tables 8–9).
3. Question 1: Architecture and control frequency.
Thank you for asking. The VLM encodes images and language once. Earlier-scale embeddings enter a separate action head; the tokenizer decoder runs once. We clarify this in revised Section 3.3.
W1 reports batch-one latencies. TrajScale makes 13.6/12.7 chunk queries/s (LIBERO/RoboCasa), executing 8/12 actions per query; StarVLA-π (4 steps) makes 4.65/4.57 queries/s.
4. Question 2: Target inductive bias.
Thank you for this question. MTR intentionally imposes a temporal prior: intermediate targets progress from coarse to fine, and the final target is the full trajectory.
Table 3 examines the benefit of scale-matched supervision. Here we retain this principle and vary how the targets are constructed.
We chose a local, parameter-free construction for its kinematic meaning: window means summarize motion trends, and linear interpolation connects them over time. Appendix A shows that means are L2-optimal constant summaries and interpolation minimizes first-derivative energy through fixed mean knots. W2 examines its role in policy prediction.
With both training stages otherwise unchanged, we tested nearest upsampling and low-pass DCT (retain s_k coefficients including DC, then inverse DCT). See revised Appendix C.2 (Table 10). Success (%):
Both alternatives exceed w/o MTR (96.2/50.8, Table 3), supporting scale-matched supervision beyond one construction. AvgPool + linear's further gains suggest that construction also matters. Nearest can introduce jumps; DCT uses a global basis. Local summaries with gradual transitions may explain our higher success.
We hope these responses address your concerns and welcome further discussion. Thank you again.