- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how future prediction supervision in world-action models can better support action learning. The method uses future latent residuals relative to the current observation as prediction targets. It constructs an Action Tangent Subspace (ATF) by perturbing demonstrated actions and comparing the futures generated by an action-conditioned teacher model. While retaining supervision from the ground-truth future, the method assigns additional weight to prediction errors within this subspace. The teacher is only used to construct supervision offline, and actions are generated through simple forward passes during deployment.
The authors evaluate the method on multiple simulation benchmarks and two real-world robot platforms. The method provides a concrete way to use the teacher's local responses to action changes for policy training. However, the reliability of these teacher responses and the independent contribution of the proposed supervision structure still need further validation.
Critical strengths
-
The way teacher knowledge is incorporated is reasonable.
The method keeps the ground-truth future from demonstrations as the prediction target and only uses teacher responses to adjust the weights of prediction errors. This reduces the risk of directly learning from incorrect synthetic futures while preserving supervision over the full future latent representation. -
The method has initial empirical support and a relatively simple deployment process.
The ablation without ATF suggests that the auxiliary objective may be useful, and the experiments cover different manipulation settings. Teacher queries and subspace construction are performed offline, so the 16B teacher is not needed during deployment.
Critical weaknesses
-
The reliability and task relevance of the teacher's local responses are not sufficiently verified, and the impact of incorrect supervision directions remains unclear.
ATF captures the teacher's responses to action perturbations, but these responses are not guaranteed to reflect real environment dynamics or changes that matter for task success. In particular, during contact establishment or slipping, the teacher may predict plausible futures but provide incorrect local responses. Fixing the generation conditions helps reduce sampling noise, but does not verify physical correctness. Although the ground-truth future remains the prediction target, incorrect subspaces may still cause the model to overemphasize irrelevant directions and affect action learning through shared representations. The paper does not directly verify whether the teacher's directions match real action responses or explain how unreliable directions can be identified and suppressed. -
The current comparisons are insufficient to isolate the contribution of the core designs.
The improvements from ATF may also come from stronger auxiliary supervision or general motion priors. The paper lacks comparisons with mismatched subspaces and standard residual prediction losses with comparable supervision strength. In addition, the contributions of residual parameterization and additional robot video adaptation are not clearly separated. The residual representation is recoverable, but this property alone does not demonstrate that the supervision is more action-selective. Therefore, the current evidence is insufficient to attribute the overall improvements to the proposed local action structure. A more careful attribution analysis would help address my concerns and could improve my score.
Questions for authors
Key Questions That May Affect My Evaluation
-
Are the teacher's directions physically correct and useful for control?
Could the authors execute perturbed actions from the same initial simulation states to check whether ATF captures real changes in the environment, especially around contact establishment and slipping? I would also like to know whether unreliable teacher directions can cause performance to drop below the version without ATF, and whether effective-rank filtering or loss weighting can reduce this risk. Are there any safeguards in the method to prevent such problems? -
Do the improvements depend on local action structures matched to each sample?
Under the same model, data, and training settings, does the correct ATF outperform ATF subspaces exchanged across samples and standard residual prediction losses with comparable supervision strength? These comparisons would directly affect my assessment of the core method's contribution. -
How much do residual representations and additional training each contribute?
Under the same video adaptation data and training budget, how do absolute and residual latent representations perform with and without ATF? Please also clarify the teacher adaptation data and its relationship to the policy training and test sets. -
What are the actual deployment latency and offline preparation costs?
Does the reported 0.43 seconds in Figure 4 include the full inference process, including image encoding, VideoDiT, and four ActionDiT steps? Please clarify the inference hardware, input settings, real-world action execution frequency, and replanning frequency. A baseline comparison under the same conditions would also be helpful. In addition, what are the costs of adapting the 16B teacher and generating ATF for each training window? These details would help assess the efficiency advantages and practical applicability of the method.
Separate Reporting Checks: Not Used in My Current Score, but Require Clarification
The following issues concern the statistical definitions and traceability of the reported results. I do not use them as reasons for my current score or rejection recommendation. However, they are more important than minor presentation issues, and I ask the authors to check each one and correct them if necessary.
-
Aggregation rules for RoboTwin 2.0-Plus.
Please clarify whether the Total column in Table 2 includes Original and whether the same rule is applied to all models. The Total for ΔWAM matches the average of all eight columns, while the Total for Fast-WAM matches the average of the seven perturbation categories. Under a consistent aggregation rule, the difference between the two models would be approximately 4.3 or 4.8 percentage points, rather than the reported 6.6 percentage points. Please verify the aggregation rules and check the reported improvements and baseline rankings. -
Task categories and the definition of Original in LIBERO-Plus.
Please explain how the 1,564 Original instances in Table 5 relate to the seven official perturbation categories, and whether this metric is directly comparable to the Original results on standard LIBERO reported in baseline papers. Please provide the task IDs, category mappings, and per-instance results. Also, please check the difference between the total success counts of 8,804 and 8,802 in Tables 4 and 5. -
Actual scoring rules for Move Ingredients.
Page 14 states that each correct step contributes one-third of the total score, but Table 6 includes a single-run score of 0.6. Please clarify whether this is due to truncated display or different weights for different stages. Also, please confirm whether Table 3 is calculated using the original unrounded scores.