- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how future-representation supervision should be integrated into VLA policies. It compares four representative integrations and six frozen visual encoders, primarily using a Qwen3-VL-based policy, with additional pi0.5 experiments. Evaluation spans in-domain and perturbed manipulation, task horizons, moving objects, reduced demonstration budgets, and four physical-robot tasks. The main contribution is an empirical comparison and its resulting design guidance, rather than a new world-modeling architecture.
Critical strengths
-
The comparison addresses an important practical question that individual architecture papers leave difficult to answer. The representation-by-integration matrix reveals useful interactions, including the opposite preferences of WM-MoT and WM-Ext for V-JEPA versus DINOv3. The evaluation also reports failures, such as WM-Query under limited data and WM-Align when adapting pi0.5, rather than presenting only favorable settings. These observations could guide model design if the evaluated implementations are unambiguously specified.
-
The appendix contains meaningful controls beyond aggregate benchmark scores. Table B.1 repeats central positive and negative findings across three training seeds. Table B.2 separates additional DINOv3 input, model capacity, current-target supervision, and future-target supervision: within WM-MoT, the latter reaches 76.2 percent OOD success versus 73.6 without the auxiliary loss and 74.0 with current-target supervision. Table B.3 further separates input and target encoders. Together with measured deployment costs and real-robot results, these strengthen the empirical contribution substantially.
Critical weaknesses
-
The central representation comparison cannot currently be reconciled with the supplied extraction code. Section 3.1 and Appendix A.2 state that all six encoders receive the same four future frames at offsets {4, 8, 12, 16}. However, examples/Foresight/extract_vjepa21_features.py explicitly constructs the target from the two-frame pair [t, t+16], and its extraction call uses encode_pair on precisely these two images; the online extractor follows the same procedure. Thus, the supplied V-JEPA pipeline changes the temporal information and target construction as well as the encoder. This matters directly to the V-JEPA ranking reversal in Figure 3 and the factorial attribution in Table B.3. The issue is not that different encoders produce different token counts, but that the asserted common temporal input is not what the provided implementation uses. The exact evaluated pipeline needs to be established before these results support the stated controlled comparison.
-
The supplied action-conditioning topology also differs materially from the formulation being compared. Equation 9 and Appendix A.3 describe WM-MoT actions attending to both VLM and Future Expert context at each layer. In starVLA/model/framework/VLM4A/MoTFutureExpert.py, the registered implementation instead uses AlternatingRoutingMixin, which masks future keys on one block and VLM keys on the next, with an interval of two in the supplied configuration. The external predictor likewise alternates context access in its action head, unlike the combined context in Equation 12. These are consequential routing choices for a paper whose contribution is to compare integrations under a unified policy framework. The capacity and supervision controls in B.2 are valuable, but do not resolve which routing implementation generated the cross-paradigm rankings. This discrepancy does not demonstrate that the measured gains are false; it leaves the experimental object and the scope of the resulting design guidance unclear.
Questions for authors
-
Which exact feature-extraction pipeline generated the V-JEPA results in Figure 3 and Tables B.1/B.3/C.2: the four future frames specified in the manuscript or the supplied [t, t+16] pair? Please provide the evaluated configuration and clarify which conclusions remain supported if temporal inputs differ.
-
Were WM-MoT and WM-Ext evaluated using the simultaneous conditioning described by Equations 9/12, or the alternating routing in the supplied code? Please identify the evaluated implementation and explain whether correcting the manuscript or replacing the supplementary implementation is sufficient, or whether any comparisons require reevaluation.
-
Can you provide the actual run configurations linking the reported tables to backbone checkpoints, training steps, encoder profiles, and routing settings? The supplied RoboTwin defaults select Qwen3-VL-4B and 150,000 steps, whereas the main testbed is described as Qwen3-VL-2B and B.1 describes full 50,000-step replications. I recognize that A.4 permits benchmark-specific schedules and the launcher accepts overrides; the missing information is which overrides produced the reported experiments.