- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper builds upon Cosmos Policy to improve future prediction-based policy optimization in World Action Models. The core motivation is that the absolute value of a predicted future may not accurately reflect the improvement caused by an action, as environmental evolution should also be considered. The method introduces two main modifications: (1) During training, the GRPO advantage is computed by subtracting the predicted future value under a stationary robot (HOLD) condition, instead of the group mean of candidate future values (CAG). (2) During inference, when CAG falls below a threshold, predicted future features are used to generate a revised action, which is then blended with the original action (Future Revision). Experiments cover RoboCasa, LIBERO, and real-world manipulation tasks, but the overall performance improvements are relatively limited, while inference latency is high.
Critical strengths
-
Clear motivation and a coherent framework. The paper highlights the distinction between the absolute value of a predicted future and the improvement attributable to an action, and proposes using the predicted future under a HOLD condition (stationary robot) as a reference. This idea is applied to both policy optimization and inference-time action revision. The overall framework is clear and easy to follow.
-
The experiments cover two simulation benchmarks, RoboCasa and LIBERO, as well as six real-world manipulation tasks. Ablation studies also demonstrate the individual performance gains from CAG-GRPO and Future Revision.
Critical weaknesses
-
CAG essentially replaces the group-mean baseline in standard GRPO with a shared HOLD value, while removing group normalization. Since the HOLD value is independent of the sampled candidate actions, subtracting this baseline does not change the expected policy gradient under standard policy-gradient theory. Although this modification may affect finite-sample gradient estimation and PPO clipping, such differences primarily arise from the choice of advantage baseline rather than a new causal modeling mechanism. Therefore, the causal contribution implied by the term Causal Action Gain appears overclaim. Furthermore, according to Table 6, CAG-GRPO improves over standard GRPO by only +1.5 and +0.7 on RoboCasa and LIBERO, respectively. These gains are relatively small and may even fall within the variability commonly observed in robotic manipulation benchmarks.
-
Future Revision uses CAG to determine whether an initial action requires correction, but the correction mechanism is largely heuristic. First, only represents the feature difference between two predicted futures and does not necessarily indicate a direction toward a better action. Second, the newly generated action is blended with the original action using fixed weights, without reevaluating the quality of the resulting action. Therefore, there is no guarantee that the revised action is better. According to Table 6, Future Revision provides only +0.9 and +0.4 improvements on RoboCasa and LIBERO, respectively. These gains are marginal, while inference latency reaches 5.77 s/action, which is prohibitively expensive for robots requiring real-time control. In comparison, VLA models can achieve inference latency as low as 0.05 s.
Questions for authors
1. About CAG. As discussed in Weakness 1, CAG essentially replaces the group-mean baseline in standard GRPO with the HOLD value. Could the authors further explain the theoretical advantages of this replacement? Its unique contribution remains unclear from both the theoretical analysis and experimental results.
2. About Future Revision. As discussed in Weakness 2, several design choices in Future Revision lack sufficient justification, including using $z_{action}-z_{HOLD} $ to construct future feedback and directly blending the initial and revised actions. Why should these operations produce better actions? Additionally, please report the frequency of Future Revision, as well as changes in predicted values and actual success rates before and after revision.
3. Computational overhead. Please provide a direct comparison with the original Cosmos Policy under the same hardware and experimental settings, including training speed, inference latency, and GPU memory usage.
10 comments
Response for Reviewer Ummw
Response for Weakness 2 & Question 2
We thank the reviewer for this thoughtful comment. We would like to clarify the motivation behind Future Revision. Future Revision uses the difference between two predicted future representations as additional conditioning information for action generation. Specifically, CAG evaluates the initial action by comparing its predicted future with the future under zero-increment control. When the initial CAG falls below the revision threshold, we use the difference between these two future representations to construct feedback. This provides information about how executing the initial action changes the predicted outcome compared with issuing no additional manipulation commands. The feedback is then used as conditioning information by the same Action DiT to generate a revised action. The following experiments examine the effectiveness of this feedback in improving action generation.
Regarding action fusion, we combine the regenerated action with the initial action as . This allows the regenerated action to adjust the initial policy output without completely replacing it. The coefficient is selected based on the sensitivity analysis in Appendix I.5. We agree that the fusion does not guarantee an improvement for every action. The final fused action is not re-evaluated during online inference to avoid additional future prediction and evaluation. We instead examine its effect by comparing the initial and fused actions under matched simulator conditions.
To examine whether the future feedback improves action generation beyond simply generating another action, we compare Difference-Based Feedback with Re-generation Only and Future-Only Feedback (). All revision variants use the same trained policy, revision trigger, additional generation budget, and fusion rule. The results are shown below.
| Revision Mechanism | RoboCasa Avg. (%) | LIBERO Avg. (%) |
|---|---|---|
| Without Future Revision | 73.3 | 99.4 |
| Re-generation Only | 73.3 | 99.4 |
| Future-Only Feedback () | 73.5 | 99.5 |
| Difference-Based Feedback (Ours) | 74.2 | 99.8 |
These results provide empirical support for the contribution of difference-based future feedback beyond simply performing another action generation or using the initial predicted future alone.
We further examine whether Future Revision improves the final executed actions. We report the revision frequency and evaluate the initial and fused actions from the same simulator states where revision is triggered, using the same subsequent evaluation protocol. The predicted-value changes and success rates are measured on these triggered-state evaluations. The results are shown below.
| Benchmark | Revision Frequency (%) | Predicted Value (%) | Success (Initial, %) | Success (Fused, %) |
|---|---|---|---|---|
| RoboCasa | 10.96 | 19.30 | 54.41 | 60.29 |
| LIBERO | 18.19 | 12.58 | 59.9 | 77.3 |
These results provide further empirical support for the effectiveness of Future Revision in improving the predicted evaluation and actual task performance of the final fused actions.
Regarding computational cost, we agree that the additional benefit of Future Revision should be considered alongside its inference overhead. The latency reported in Appendix I.6 was measured through sequential execution of the complete inference components. To provide a direct comparison, we additionally benchmark the original Cosmos Policy and different CAG-WAM inference configurations on the same H100 GPU under matched settings. The latency is measured per action-chunk decision, with 16 low-level actions executed from each generated chunk. The results are shown below.
| Inference Setting | Latency (s/decision) | Peak GPU Memory (GiB) |
|---|---|---|
| Original Cosmos Policy (Action Only) | 0.745 | 5.433 |
| CAG-GRPO Policy (Action Only) | 0.736 | 5.433 |
| CAG-WAM (Without Revision) | 1.325 | 5.433 |
| CAG-WAM (With Revision) | 1.897 | 5.433 |
These results show that CAG-GRPO maintains nearly identical action-only inference latency to the original Cosmos Policy. The additional computation comes from paired-future evaluation and selectively triggered action regeneration, while the measured peak GPU memory remains unchanged. We consider the additional computational cost acceptable for the action-chunk-based manipulation setting studied here.
Response for Reviewer Ummw
Response for Question 3
We thank the reviewer for this suggestion. We provide a direct comparison of training speed, inference latency, and GPU memory usage under matched hardware and experimental settings. For training, we compare standard GRPO and CAG-GRPO to examine the additional cost introduced by the proposed optimization signal. We report the wall-clock time per update and peak GPU memory usage below.
| Training Method | Training Time (s/update) | GPU Memory (GiB) |
|---|---|---|
| Cosmos Policy + Standard GRPO | 37.3 | 31.7 |
| CAG-GRPO (Ours) | 40.8 | 32.3 |
Regarding inference, Appendix I.6 reports latency measurements obtained by sequentially executing the inference components, including initial action generation, paired-future prediction, future evaluation, and conditional action regeneration. This setting allows the computational cost of the complete inference process to be examined without overlapping computations. The reported latencies are 4.82 s without Future Revision and 5.77 s with Future Revision per action-chunk decision.
To provide a direct comparison of actual inference latency, we additionally report the original Cosmos Policy, the CAG-GRPO policy, and the complete CAG-WAM with and without Future Revision. The action-only CAG-GRPO policy does not require future prediction or evaluation during inference. In contrast, CAG-WAM without revision includes paired-future evaluation but does not perform additional action generation. All configurations use matched inference settings and timing protocols. The results are shown below.
| Inference Setting | Latency (s/decision) | GPU Memory (GiB) |
|---|---|---|
| Original Cosmos Policy (Action Only) | 0.745 | 5.433 |
| CAG-GRPO Policy (Action Only) | 0.736 | 5.433 |
| CAG-WAM (Without Revision) | 1.325 | 5.433 |
| CAG-WAM (With Revision) | 1.897 | 5.433 |
These results show that CAG-GRPO introduces a modest additional training cost while maintaining nearly identical action-only inference latency to the original Cosmos Policy. The additional inference computation mainly comes from paired-future evaluation and selectively triggered Future Revision, while the measured peak GPU memory remains unchanged. Since each decision generates an action chunk containing 16 executable low-level actions, we consider this additional computational cost acceptable for the action-chunk-based manipulation setting studied here, while recognizing that high-frequency real-time control may require stricter latency constraints.
Response for Reviewer Ummw
We sincerely appreciate Reviewer Ummw's thoughtful comments and the time and effort devoted to the review process. We hope that our responses can clarify the concerns.
Response for Weakness 1 & Question 1
We thank the reviewer for this thoughtful comment. CAG-GRPO builds on the GRPO optimization framework by introducing an action-evaluation signal derived from our causal analysis. In the SCM presented in Section 3, conditioning on the current observation blocks the identified backdoor path, leading to the relationship in Eq. (30). This relationship allows us to compare the expected outcomes of different control choices under the same observation. In CAG-WAM, we aim to measure how executing a candidate action changes the expected task outcome compared with issuing no additional manipulation commands. Since our policy uses incremental motion commands, HOLD represents this condition by setting all motion increments to zero while maintaining the current gripper state. Eq. (32) expresses the effect of the candidate action as the difference between these two expected future evaluations. CAG-WAM then uses paired-future prediction to approximate this effect, as described in Eqs. (31) and (35), and incorporates the resulting CAG into GRPO as the optimization advantage. Thus, the causal analysis guides the construction of the optimization signal, while GRPO provides the framework for using it in policy learning.
Regarding the theoretical distinction from standard GRPO, we agree that subtracting the shared HOLD evaluation does not change the relative ranking of candidate actions or the ideal expected score-function policy gradient relative to uncentered future evaluations, as discussed in Appendices C.2 and C.3. The key difference lies in how the optimization advantage is defined. Standard GRPO evaluates candidates relative to the sampled group, whereas CAG evaluates each candidate based on its expected future-evaluation change relative to the zero-increment condition. Consequently, CAG does not depend on the other candidates sampled in the group, and its sign indicates whether executing the candidate action is expected to be beneficial or detrimental to the task compared with issuing no additional manipulation commands. For example, when all sampled candidates are worse than the zero-increment condition, standard GRPO still assigns positive advantages to relatively better candidates, whereas their CAG values remain negative. Since group centering would cancel the shared evaluation term, as shown in Appendix C.2, we use uncentered CAG to preserve the sign and magnitude of the estimated action effect in policy optimization.
In practical finite-sample Flow-GRPO with PPO-style clipping, different advantage formulations can affect the actual policy updates. To assess the contribution of the proposed action-effect evaluation, we compare CAG-GRPO with standard GRPO, uncentered raw future evaluations, a constant baseline, and a current-state reference under matched training and evaluation settings. The results shown below provide empirical support for the contribution of the proposed optimization signal.
We also appreciate the reviewer's concern regarding the magnitude and variability of the reported improvements. To further examine performance consistency, we report more results, including their mean success rates and standard deviations.