acceptodds
Under review as a conference paper at ICLR 2027

CAG-WAM: Causal Action-Gain World Action Models for Robot Manipulation

Abstract

World Action Models (WAMs) combine action generation with future prediction, allowing predicted futures to support action evaluation and robot decision making. However, the future associated with a candidate action may contain action-induced and environment-driven changes. Directly evaluating the predicted future may not fully characterize the future change associated with the candidate action. For example, a candidate action that worsens a favorable environmental future may still yield a favorable outcome, while one that improves an unfavorable environmental future may still yield an unfavorable outcome. To analyze this issue, we formulate a Structural Causal Model (SCM) for action evaluation in WAMs. The SCM formalizes how action-induced and environment-driven changes jointly shape future evaluation, and reveals a backdoor path through which environmental evolution can also affect the future evaluation associated with a candidate action. Intervention and backdoor adjustment account for this path, providing a causal basis for characterizing the future change associated with each candidate action. Based on this analysis, we propose Causal Action-Gain World Action Model (CAG-WAM). For each candidate action, CAG-WAM predicts an action-conditioned future and an environmental future under the same current observation, and defines their evaluation difference as causal action gain (CAG) for characterizing the future change associated with the candidate action. During training, we incorporate CAG into Group Relative Policy Optimization (GRPO) to guide robot policy learning. During inference, we evaluate the initial action with CAG and, when needed, revise it using predicted future information. Experiments on multiple benchmarks support the effectiveness of CAG-WAM.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Manipulation, World Action Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UmmwOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper builds upon Cosmos Policy to improve future prediction-based policy optimization in World Action Models. The core motivation is that the absolute value of a predicted future may not accurately reflect the improvement caused by an action, as environmental evolution should also be considered. The method introduces two main modifications: (1) During training, the GRPO advantage is computed by subtracting the predicted future value under a stationary robot (HOLD) condition, instead of the group mean of candidate future values (CAG). (2) During inference, when CAG falls below a threshold, predicted future features are used to generate a revised action, which is then blended with the original action (Future Revision). Experiments cover RoboCasa, LIBERO, and real-world manipulation tasks, but the overall performance improvements are relatively limited, while inference latency is high.

Critical strengths
  1. Clear motivation and a coherent framework. The paper highlights the distinction between the absolute value of a predicted future and the improvement attributable to an action, and proposes using the predicted future under a HOLD condition (stationary robot) as a reference. This idea is applied to both policy optimization and inference-time action revision. The overall framework is clear and easy to follow.

  2. The experiments cover two simulation benchmarks, RoboCasa and LIBERO, as well as six real-world manipulation tasks. Ablation studies also demonstrate the individual performance gains from CAG-GRPO and Future Revision.

Critical weaknesses
  1. CAG essentially replaces the group-mean baseline in standard GRPO with a shared HOLD value, while removing group normalization. Since the HOLD value is independent of the sampled candidate actions, subtracting this baseline does not change the expected policy gradient under standard policy-gradient theory. Although this modification may affect finite-sample gradient estimation and PPO clipping, such differences primarily arise from the choice of advantage baseline rather than a new causal modeling mechanism. Therefore, the causal contribution implied by the term Causal Action Gain appears overclaim. Furthermore, according to Table 6, CAG-GRPO improves over standard GRPO by only +1.5 and +0.7 on RoboCasa and LIBERO, respectively. These gains are relatively small and may even fall within the variability commonly observed in robotic manipulation benchmarks.

  2. Future Revision uses CAG to determine whether an initial action requires correction, but the correction mechanism is largely heuristic. First, zaction−zHOLDz_{\mathrm{action}}-z_{\mathrm{HOLD}} only represents the feature difference between two predicted futures and does not necessarily indicate a direction toward a better action. Second, the newly generated action is blended with the original action using fixed weights, without reevaluating the quality of the resulting action. Therefore, there is no guarantee that the revised action is better. According to Table 6, Future Revision provides only +0.9 and +0.4 improvements on RoboCasa and LIBERO, respectively. These gains are marginal, while inference latency reaches 5.77 s/action, which is prohibitively expensive for robots requiring real-time control. In comparison, VLA models can achieve inference latency as low as 0.05 s.

Questions for authors

1. About CAG. As discussed in Weakness 1, CAG essentially replaces the group-mean baseline in standard GRPO with the HOLD value. Could the authors further explain the theoretical advantages of this replacement? Its unique contribution remains unclear from both the theoretical analysis and experimental results.

2. About Future Revision. As discussed in Weakness 2, several design choices in Future Revision lack sufficient justification, including using $z_{action}-z_{HOLD} $ to construct future feedback and directly blending the initial and revised actions. Why should these operations produce better actions? Additionally, please report the frequency of Future Revision, as well as changes in predicted values and actual success rates before and after revision.

3. Computational overhead. Please provide a direct comparison with the original Cosmos Policy under the same hardware and experimental settings, including training speed, inference latency, and GPU memory usage.

10 comments
AuthorsOpenReview ↗

Response for Reviewer Ummw

We sincerely appreciate Reviewer Ummw's thoughtful comments and the time and effort devoted to the review process. We hope that our responses can clarify the concerns.

Response for Weakness 1 & Question 1

We thank the reviewer for this thoughtful comment. CAG-GRPO builds on the GRPO optimization framework by introducing an action-evaluation signal derived from our causal analysis. In the SCM presented in Section 3, conditioning on the current observation blocks the identified backdoor path, leading to the relationship in Eq. (30). This relationship allows us to compare the expected outcomes of different control choices under the same observation. In CAG-WAM, we aim to measure how executing a candidate action changes the expected task outcome compared with issuing no additional manipulation commands. Since our policy uses incremental motion commands, HOLD represents this condition by setting all motion increments to zero while maintaining the current gripper state. Eq. (32) expresses the effect of the candidate action as the difference between these two expected future evaluations. CAG-WAM then uses paired-future prediction to approximate this effect, as described in Eqs. (31) and (35), and incorporates the resulting CAG into GRPO as the optimization advantage. Thus, the causal analysis guides the construction of the optimization signal, while GRPO provides the framework for using it in policy learning.

Regarding the theoretical distinction from standard GRPO, we agree that subtracting the shared HOLD evaluation does not change the relative ranking of candidate actions or the ideal expected score-function policy gradient relative to uncentered future evaluations, as discussed in Appendices C.2 and C.3. The key difference lies in how the optimization advantage is defined. Standard GRPO evaluates candidates relative to the sampled group, whereas CAG evaluates each candidate based on its expected future-evaluation change relative to the zero-increment condition. Consequently, CAG does not depend on the other candidates sampled in the group, and its sign indicates whether executing the candidate action is expected to be beneficial or detrimental to the task compared with issuing no additional manipulation commands. For example, when all sampled candidates are worse than the zero-increment condition, standard GRPO still assigns positive advantages to relatively better candidates, whereas their CAG values remain negative. Since group centering would cancel the shared evaluation term, as shown in Appendix C.2, we use uncentered CAG to preserve the sign and magnitude of the estimated action effect in policy optimization.

In practical finite-sample Flow-GRPO with PPO-style clipping, different advantage formulations can affect the actual policy updates. To assess the contribution of the proposed action-effect evaluation, we compare CAG-GRPO with standard GRPO, uncentered raw future evaluations, a constant baseline, and a current-state reference under matched training and evaluation settings. The results shown below provide empirical support for the contribution of the proposed optimization signal.

Advantage FormulationRoboCasa Avg. (%)LIBERO Avg. (%)
Standard GRPO71.898.7
Uncentered Raw Evaluation72.198.9
Constant Baseline71.998.5
Current-State Reference72.398.8
CAG-GRPO73.399.4

We also appreciate the reviewer's concern regarding the magnitude and variability of the reported improvements. To further examine performance consistency, we report more results, including their mean success rates and standard deviations.

MethodRoboCasa Avg. (Mean ± Std %)LIBERO Avg. (Mean ± Std %)
Standard GRPO71.76 ± 0.0898.68 ± 0.13
CAG-GRPO73.33 ± 0.0699.43 ± 0.09
AuthorsOpenReview ↗

Response for Reviewer Ummw

Response for Weakness 2 & Question 2

We thank the reviewer for this thoughtful comment. We would like to clarify the motivation behind Future Revision. Future Revision uses the difference between two predicted future representations as additional conditioning information for action generation. Specifically, CAG evaluates the initial action by comparing its predicted future with the future under zero-increment control. When the initial CAG falls below the revision threshold, we use the difference between these two future representations to construct feedback. This provides information about how executing the initial action changes the predicted outcome compared with issuing no additional manipulation commands. The feedback is then used as conditioning information by the same Action DiT to generate a revised action. The following experiments examine the effectiveness of this feedback in improving action generation.

Regarding action fusion, we combine the regenerated action with the initial action as Aexec=(1−λ)Ainit+λArevA^{\mathrm{exec}}=(1-\lambda)A^{\mathrm{init}}+\lambda A^{\mathrm{rev}}. This allows the regenerated action to adjust the initial policy output without completely replacing it. The coefficient λ=0.3\lambda=0.3 is selected based on the sensitivity analysis in Appendix I.5. We agree that the fusion does not guarantee an improvement for every action. The final fused action is not re-evaluated during online inference to avoid additional future prediction and evaluation. We instead examine its effect by comparing the initial and fused actions under matched simulator conditions.

To examine whether the future feedback improves action generation beyond simply generating another action, we compare Difference-Based Feedback with Re-generation Only and Future-Only Feedback (γ=0\gamma=0). All revision variants use the same trained policy, revision trigger, additional generation budget, and fusion rule. The results are shown below.

Revision MechanismRoboCasa Avg. (%)LIBERO Avg. (%)
Without Future Revision73.399.4
Re-generation Only73.399.4
Future-Only Feedback (γ=0\gamma=0)73.599.5
Difference-Based Feedback (Ours)74.299.8

These results provide empirical support for the contribution of difference-based future feedback beyond simply performing another action generation or using the initial predicted future alone.

We further examine whether Future Revision improves the final executed actions. We report the revision frequency and evaluate the initial and fused actions from the same simulator states where revision is triggered, using the same subsequent evaluation protocol. The predicted-value changes and success rates are measured on these triggered-state evaluations. The results are shown below.

BenchmarkRevision Frequency (%)Predicted Δ\DeltaValue (%)Success (Initial, %)Success (Fused, %)
RoboCasa10.9619.3054.4160.29
LIBERO18.1912.5859.977.3

These results provide further empirical support for the effectiveness of Future Revision in improving the predicted evaluation and actual task performance of the final fused actions.

Regarding computational cost, we agree that the additional benefit of Future Revision should be considered alongside its inference overhead. The latency reported in Appendix I.6 was measured through sequential execution of the complete inference components. To provide a direct comparison, we additionally benchmark the original Cosmos Policy and different CAG-WAM inference configurations on the same H100 GPU under matched settings. The latency is measured per action-chunk decision, with 16 low-level actions executed from each generated chunk. The results are shown below.

Inference SettingLatency (s/decision)Peak GPU Memory (GiB)
Original Cosmos Policy (Action Only)0.7455.433
CAG-GRPO Policy (Action Only)0.7365.433
CAG-WAM (Without Revision)1.3255.433
CAG-WAM (With Revision)1.8975.433

These results show that CAG-GRPO maintains nearly identical action-only inference latency to the original Cosmos Policy. The additional computation comes from paired-future evaluation and selectively triggered action regeneration, while the measured peak GPU memory remains unchanged. We consider the additional computational cost acceptable for the action-chunk-based manipulation setting studied here.

AuthorsOpenReview ↗

Response for Reviewer Ummw

Response for Question 3

We thank the reviewer for this suggestion. We provide a direct comparison of training speed, inference latency, and GPU memory usage under matched hardware and experimental settings. For training, we compare standard GRPO and CAG-GRPO to examine the additional cost introduced by the proposed optimization signal. We report the wall-clock time per update and peak GPU memory usage below.

Training MethodTraining Time (s/update)GPU Memory (GiB)
Cosmos Policy + Standard GRPO37.331.7
CAG-GRPO (Ours)40.832.3

Regarding inference, Appendix I.6 reports latency measurements obtained by sequentially executing the inference components, including initial action generation, paired-future prediction, future evaluation, and conditional action regeneration. This setting allows the computational cost of the complete inference process to be examined without overlapping computations. The reported latencies are 4.82 s without Future Revision and 5.77 s with Future Revision per action-chunk decision.

To provide a direct comparison of actual inference latency, we additionally report the original Cosmos Policy, the CAG-GRPO policy, and the complete CAG-WAM with and without Future Revision. The action-only CAG-GRPO policy does not require future prediction or evaluation during inference. In contrast, CAG-WAM without revision includes paired-future evaluation but does not perform additional action generation. All configurations use matched inference settings and timing protocols. The results are shown below.

Inference SettingLatency (s/decision)GPU Memory (GiB)
Original Cosmos Policy (Action Only)0.7455.433
CAG-GRPO Policy (Action Only)0.7365.433
CAG-WAM (Without Revision)1.3255.433
CAG-WAM (With Revision)1.8975.433

These results show that CAG-GRPO introduces a modest additional training cost while maintaining nearly identical action-only inference latency to the original Cosmos Policy. The additional inference computation mainly comes from paired-future evaluation and selectively triggered Future Revision, while the measured peak GPU memory remains unchanged. Since each decision generates an action chunk containing 16 executable low-level actions, we consider this additional computational cost acceptable for the action-chunk-based manipulation setting studied here, while recognizing that high-frequency real-time control may require stricter latency constraints.

Reviewer_ZvGfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes an interesting concept - what if we predict what changes a predicted action will bring to the world state in comparison of having the robot arm sit still (HOLD)? Can we model that action-gain with a causal model to enhance the WAM's capability in predicting a better future action?

However, the technical implementation of this concept, exhibits a few largely limitations.

Critical strengths
  • The idea of modelling an action-gain against a reference future state with a causal WAM to de-confound environment-induced changes from action-induced changes, and hence improve the performance, sounds novel and interesting.
  • The model is evaluated on two public benchmarks, RoboCasa and LIBERO, and has shown some performance improvement.
  • The appendix provides a careful discussion on the causal interpretation of the formulated model, especially regarding the limitations and implications by using HOLD as the reference future state.
Critical weaknesses
  • There is a loose connection between the causal analysis and the actual implementation. The "causal action gain" described is actually formulated as a subtracted, reference-relative contrast w.r.t. to HOLD. Conditioning on the current observation already blocks the identified backdoor path, so the subtraction doesn't provide additional deconfounding, not isolate the action-only component sufficiently. For example, when you have a rolling ball towards the edge, under HOLD the ball continues to roll and fall, under "catch" the ball will be stopped. "Environment factor" contributes differently in these two scenarios. What's measured by the subtraction is therefore a total effect of the choice, including action/environment interaction, which does not isolate the action-induced component of future change. This raises a lot of questions regarding the technical foundation of this paper.

  • The choice of HOLD as a reference action is not sufficiently justified. HOLD in many cases would be a harmful action for the task, and extrapolating the future from there doesn't seem to be a strong reference point for assessing the action gain. In fact I doubt it is adding much to using the "current state" for comparison. It would be interesting to see the difference in the experiments.

  • The experimental results show moderate benefits only over the strongest baseline methods, at a non-trivial efficiency cost.

Questions for authors
  • Did you compare with using current state instead of HOLD as the reference?
8 comments
AuthorsOpenReview ↗

Response for Reviewer ZvGf

We sincerely appreciate Reviewer ZvGf's constructive comments and the time and effort devoted to the review process. We will respond to each comment individually and hope our responses can address the concerns.

Response for Weakness 1

We thank the reviewer for this thoughtful comment. We would like to clarify how the causal analysis leads to the practical formulation of CAG-WAM. Our goal is to evaluate the expected change in task outcome associated with executing a candidate action.

Specifically, Eq. (2) uses backdoor adjustment to express the expected outcome of an action intervention by averaging over different observations. However, in robotic manipulation, the robot has already obtained its current observation before selecting an action. In the SCM presented in Section 3, the identified backdoor path runs through the current observation. Conditioning on this observation blocks that path, leading to the relationship in Eq. (30). This follows the same backdoor analysis as Eq. (2), but focuses on the current observation rather than averaging over different observations.

This relationship applies to different choices of the same action variable. We can therefore compare the expected outcomes of two choices under the same observation. Eq. (32) expresses the effect of choosing one over the other as the difference between their expected future evaluations. In CAG-WAM, we aim to measure how executing a candidate manipulation command changes the expected task outcome compared with issuing no additional manipulation commands. We therefore choose HOLD for this comparison. Since our policy uses incremental motion commands, setting all motion increments to zero while maintaining the current gripper state directly represents this condition. This connects the choice of HOLD to the action effect we aim to measure.

To estimate this effect, CAG-WAM predicts the futures under the candidate action and HOLD from the same observation and evaluates them over the same horizon. As described in Eq. (31), the learned world model approximates the future distributions under the corresponding control conditions. The expected evaluation difference over model-generated futures then approximates the action effect defined in Eq. (32), as expressed in Eq. (35). Thus, the backdoor analysis provides the causal basis for evaluating the effect of an action, while paired-future prediction provides a practical way to estimate it for policy optimization.

The reviewer's rolling-ball example illustrates this point. Under HOLD, the ball may continue rolling and fall, whereas a catching action may prevent it from falling. The difference between these outcomes measures how choosing the catching action changes the task result. As the reviewer notes, this includes the interaction between the action and the ball's motion. Eq. (32) includes this interaction in the overall effect of the action choice.

AuthorsOpenReview ↗

Response for Reviewer ZvGf

Response for Weakness 2

We thank the reviewer for this valuable comment. We would like to clarify why HOLD is suitable for the action evaluation considered in CAG-WAM. Our goal is to measure how executing a candidate action changes the expected future task outcome compared with issuing no additional manipulation commands. In our implementation, HOLD sets all motion increments to zero while maintaining the current gripper state. Since our policy uses incremental motion commands, HOLD directly represents this condition in robotic manipulation. This allows us to evaluate the effect of an additional manipulation command under a clearly defined control condition, consistent with the action effect formulated in Eq. (32).

HOLD may lead to unfavorable outcomes when a task requires active intervention. Its predicted future provides the comparison outcome under no additional manipulation commands, allowing us to evaluate the change brought by executing a candidate action.

The distinction from using the current state is also important. Current-state evaluation compares the candidate's predicted future with the present state, whereas HOLD allows us to compare the expected outcomes of two control choices over the same future horizon. In the rolling-ball example, the ball may still be on the table at the current moment but fall later without additional manipulation commands. Therefore, the predicted future under HOLD provides a more appropriate comparison for measuring the effect of executing the candidate action over that period.

To examine the influence of this choice, we compare HOLD with current-state evaluation and a Gaussian-sampled action reference under matched training and evaluation settings without Future Revision. The results shown below provide empirical support for using the predicted future under zero-increment control to evaluate the effect of candidate actions.

Reference FormulationRoboCasa Avg. (%)LIBERO Avg. (%)
Current-State Reference72.398.8
Gaussian Action Reference71.097.2
HOLD Reference (Ours)73.399.4
AuthorsOpenReview ↗

Response for Reviewer ZvGf

Response for Weakness 3

We thank the reviewer for this comment. We recognize the concern regarding the performance–efficiency trade-off. CAG-WAM improves the average success rate from 71.5% to 74.2% on RoboCasa and from 98.5% to 99.8% on LIBERO. Importantly, CAG-GRPO already achieves 73.3% and 99.4%, respectively, without requiring future evaluation during action-only inference.

The additional inference cost mainly comes from paired-future prediction and evaluation, while Future Revision performs another action generation only when the initial CAG falls below the revision threshold. The original latency measurements in Appendix I.6 were obtained through sequential execution of the complete inference components, reporting 4.82 s without revision and 5.77 s with revision per action-chunk decision. To provide a direct comparison, we additionally benchmark the original Cosmos Policy and different CAG-WAM inference configurations on the same H100 GPU under matched settings. The results are shown below.

Inference SettingLatency (s/decision)Peak GPU Memory (GiB)
Original Cosmos Policy (Action Only)0.7455.433
CAG-GRPO Policy (Action Only)0.7365.433
CAG-WAM (Without Revision)1.3255.433
CAG-WAM (With Revision)1.8975.433

The results show that CAG-GRPO maintains nearly identical action-only inference latency to the original Cosmos Policy. Including paired-future evaluation increases the latency to 1.325 s, while Future Revision adds 0.572 s when triggered, with the measured peak GPU memory remaining unchanged. Since Future Revision is selectively activated and each decision generates a chunk of 16 executable low-level actions, we consider the additional computational cost acceptable for the action-chunk-based manipulation setting studied here. These results provide further empirical support for assessing the reported performance improvements alongside their computational cost.

Reviewer_kdWsOpenReview ↗
Rating
3(+1)
Confidence
1(−2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAG-WAM, a framework for robotic manipulation that evaluates candidate actions by comparing their predicted futures against a reference future generated under a stationary HOLD condition. Motivated by a Structural Causal Model (SCM), the authors define the difference between the two predicted future evaluations as Causal Action Gain (CAG). This quantity is used as an uncentered advantage signal in GRPO-based policy optimization during training. During inference, CAG serves as a criterion for triggering an additional future-conditioned action generation step, followed by a weighted combination of the initial and revised actions.

The method is evaluated on RoboCasa and LIBERO, reporting average success rates of 74.2% and 99.8%, respectively, compared with 71.5% and 98.5% for the base world action model. Additional ablation studies, controlled environmental perturbation experiments, and real-world robot evaluations are provided. The proposed approach essentially relies on reference-based future evaluation and conditional action revision, while its causal interpretation depends on assumptions about the underlying world model and future evaluator.

Critical strengths
  1. The paper presents a relatively straightforward approach to evaluating candidate actions by comparing their predicted futures with a reference future. The overall framework is easy to follow.

  2. The paper includes experiments on RoboCasa and LIBERO, along with several ablation studies and additional evaluations. These results provide some empirical evidence of performance improvements over the base model.

Critical weaknesses
  1. The paper spends quite a lot of space discussing SCM, backdoor paths, and interventions in Section 3. However, Appendix C.1 explicitly states that CAG-WAM does not perform the marginal backdoor adjustment in Eq. (2). It also mentions that subtracting the environmental future evaluation does not provide additional deconfounding after conditioning on the current observation. This makes me question the role of the causal analysis in the actual method. The model basically compares two predicted futures under different action conditions. I am not convinced that this requires the causal formulation presented in the paper. There is also no direct experiment showing that the estimated CAG actually corresponds to the causal effect of an action.

  2. The environmental future evaluation is shared by all candidate actions, so subtracting it does not change the relative ranking of candidates. This is also acknowledged in Appendix C.2. Appendix C.3 further shows that the expected policy gradient remains unchanged when this shared term is used as a baseline. Given this, I am not sure how different the proposed CAG is from a standard action-independent baseline. Also, standard GRPO uses group-normalized advantages, while this paper directly uses uncentered CAG values. The improvement could simply come from using a different advantage formulation or from the clipping behavior during optimization. The current ablation only compares standard GRPO with CAG-GRPO. A comparison with uncentered raw future evaluations or other simple baselines is missing, making it difficult to identify what actually contributes to the improvement.

  3. In the main paper, the motivation is built around separating action-induced changes from environment-driven changes. However, Appendix G.5 explains that the environmental future is actually predicted under a stationary HOLD action. This is not the same as the environment evolving without robot action. HOLD is still a robot control condition, and the resulting future may include effects related to that condition. The authors also acknowledge in Appendix G.4 and G.5 that the method does not explicitly separate these two sources of change. So I find the explanation in the main paper somewhat misleading. The method is comparing the candidate action against HOLD, rather than directly isolating action-induced changes. I also did not find experiments checking how accurate the HOLD predictions are, or whether using a different reference action would change the results significantly.

  4. The future feedback is constructed by adding a scaled difference between the action-conditioned future representation and the environmental future representation. With the default setting of gamma equal to 1, this essentially doubles the initial future representation and subtracts the environmental representation. I do not see why this should produce a better condition for action generation. Appendix C.4 also admits that the difference in representation space does not necessarily point toward better future evaluations. I am also confused about how the action model is trained to use this feedback. The training section mainly discusses action generation from the current observation and task instruction, but during inference an additional future-feedback condition is introduced. It is not clear whether the model has been trained with this type of input, or how it is expected to handle it. The ablation study does show some improvement from Future Revision, but this does not rule out the possibility that the improvement simply comes from generating another action. Comparisons with future-only feedback and an additional generation step without the proposed difference-based feedback would be useful.

  5. The reported improvements are not particularly large. On RoboCasa, the average success rate increases from 71.5% to 74.2%, and on LIBERO it goes from 98.5% to 99.8%. LIBERO is already very close to saturation with the base model. I would like to see more evidence that these gains are consistent across different runs. More importantly, the experiments mainly report task success rates. They do not actually test whether CAG can estimate the effect of an action relative to HOLD. Since this is the main motivation of the paper, I think this is an important missing experiment. For example, the authors could execute a candidate action and HOLD from the same initial simulator state and compare the actual outcome difference with the predicted CAG. The real-world experiments are also not very convincing in their current form. Only CAG-WAM results are reported, without a base WAM comparison. It is therefore difficult to tell whether CAG is actually useful in these real-world settings.

  6. In Appendix I.2, the authors mention that the controlled environmental-dynamics experiments are conducted on four RoboCasa tasks, with 50 trials per task. If these are binary success/failure trials and the reported number is simply the average over four equally weighted tasks, the success rate should change in steps of 0.5 percentage points. However, Table 5 contains numbers such as 54.7%, 48.9%, and 57.3%. These numbers do not seem possible under the evaluation protocol described in the text. Maybe additional seeds or trials were used, but this is not explained. The authors should clarify how these results were obtained. This also raises concerns about the clarity of the experimental reporting.

Questions for authors
  1. In Appendix C.1, the authors state that CAG-WAM does not perform marginal backdoor adjustment and that subtracting the environmental future evaluation does not provide additional deconfounding. Then what is the actual role of the causal analysis in the proposed method? How is CAG different from simply comparing two action-conditioned predictions?

  2. Since the environmental future evaluation is shared across all candidate actions and does not change their relative ranking, why is it necessary to use HOLD as the reference? Have the authors tried using uncentered raw future evaluations or other simple baselines? I think these comparisons are important to understand where the improvement actually comes from.

  3. Appendix G.5 mentions that the environmental future is generated under a HOLD condition, rather than representing pure environmental evolution. How accurate are the world model's predictions under HOLD? Would the results change if another reference action were used? I would also like to know whether the authors have tested how well the predicted CAG matches the actual outcome difference between executing a candidate action and HOLD.

  4. How is the action model trained to use the future feedback introduced during inference? It is not clear from the training section whether this additional conditioning input is used during training. Also, why should the difference between two future representations help generate a better action? Could the authors compare this with simply performing another action-generation step without the proposed feedback?

  5. The improvements over the base model are relatively small, especially on LIBERO, where the baseline already reaches 98.5%. Are these improvements consistent across different random seeds? Also, why are there no baseline comparisons in the real-world experiments? Without these comparisons, it is difficult to judge how much the proposed method actually helps.

  6. In Appendix I.2, the authors state that four tasks are evaluated with 50 trials per task. However, the success rates reported in Table 5, such as 54.7% and 48.9%, do not seem consistent with this setup if the results are averaged over 200 binary success/failure trials. Were additional trials or random seeds used? Could the authors clarify how these numbers were calculated?

11 comments
AuthorsOpenReview ↗

Response for Reviewer kdWs

We sincerely appreciate Reviewer kdWs's valuable comments and the time and effort devoted to the review process. We will respond to each comment in turn, hoping that our responses can address the concerns.

Response for Weakness 1 & Question 1

We thank the reviewer for this thoughtful comment. We would like to clarify how the causal analysis is used in CAG-WAM. The statement in Appendix C.1 that CAG-WAM does not perform marginal backdoor adjustment refers specifically to the integral over the observation distribution in Eq. (2). Eq. (2) uses backdoor adjustment to express the expected outcome of an action intervention by averaging over different observations. In robotic manipulation, however, the robot has already obtained its current observation before selecting an action. Therefore, CAG-WAM uses the relationship derived from the same backdoor analysis for the current observation, rather than averaging over different observations.

Specifically, in the SCM presented in Section 3, the identified backdoor path runs through the current observation. Conditioning on this observation blocks that path, leading to the relationship in Eq. (30), which relates the expected outcome of executing a given action to the corresponding action-conditioned expectation under the current observation. This relationship applies to different choices of the same action variable. We can therefore compare the expected outcomes of two choices under the same observation. Eq. (32) expresses the effect of choosing one over the other as the difference between their expected future evaluations. In this way, the backdoor analysis provides the causal basis for evaluating how executing a candidate action changes the expected task outcome.

The practical CAG estimation follows this formulation. In robotic manipulation, we aim to measure how executing a candidate manipulation command changes the expected task outcome compared with issuing no additional manipulation commands. Since our policy uses incremental motion commands, we use HOLD to represent this condition by setting all motion increments to zero while maintaining the current gripper state. Starting from the same observation and task instruction, CAG-WAM predicts the futures under the candidate action and HOLD over the same horizon and evaluates them using the same future evaluator. As described in Eq. (31), the learned world model approximates the future distributions under the corresponding control conditions. The expected evaluation difference over model-generated futures then approximates the action effect defined in Eq. (32), as expressed in Eq. (35). Thus, the causal analysis provides the basis for measuring the action effect, while paired-future prediction provides a practical way to estimate it. The resulting CAG is used in both policy optimization and inference-time action revision.

Since CAG relies on predicted futures to estimate this action effect, we further examine how accurately it reflects the actual outcome difference. On RoboCasa and LIBERO, we start from the same simulator state, execute the candidate action and HOLD separately, and evaluate their outcomes over the same future horizon. We calculate the mean absolute error (MAE) between the observed evaluation difference and the predicted CAG. For comparison, we also report the error obtained by always predicting a difference of zero. The results are shown below.

BenchmarkCAG Estimation MAEZero-Difference Prediction MAE
RoboCasa0.0430.139
LIBERO0.0420.292

These results provide empirical support for using paired-future prediction to estimate the action effect defined through the causal analysis.

AuthorsOpenReview ↗

Response for Reviewer kdWs

Response for Weakness 2 & Question 2

We thank the reviewer for this valuable comment. We agree that subtracting the shared HOLD evaluation does not change the relative ranking of candidate actions or the ideal expected score-function policy gradient relative to uncentered future evaluations, as discussed in Appendices C.2 and C.3. In the SCM presented in Section 3, conditioning on the current observation blocks the identified backdoor path, leading to the relationship in Eq. (30). This relationship allows us to compare the expected future outcomes of different control choices under the same observation. Eq. (32) then expresses the effect of executing a candidate action as the expected change in task outcome compared with issuing no additional manipulation commands. CAG-WAM uses paired-future prediction to approximate this effect, as described in Eqs. (31) and (35), and incorporates the resulting CAG into policy optimization.

In robotic manipulation, our policy uses incremental motion commands. HOLD sets all motion increments to zero while maintaining the current gripper state, directly representing the condition without additional motion or gripper-change commands. Comparing the predicted futures under the candidate action and this condition over the same horizon allows us to estimate whether executing the candidate action is beneficial or detrimental to the task compared with issuing no additional manipulation commands. Standard GRPO constructs advantages relative to the sampled candidate group, whereas CAG evaluates each candidate independently of the other sampled actions. For example, when all candidates are worse than the zero-increment condition, standard GRPO still assigns positive advantages to relatively better candidates, whereas their CAG values remain negative. Since group centering would cancel the shared evaluation term, as discussed in Appendix C.2, we use uncentered CAG to preserve the sign and magnitude of the estimated action effect.

In practical finite-sample Flow-GRPO with PPO-style clipping, different advantage formulations can affect the actual policy updates. To examine whether the improvement comes from the proposed evaluation rather than simply changing the advantage formulation, we compare CAG-GRPO with standard GRPO, uncentered raw future evaluations, a constant baseline, and a current-state reference under matched training and evaluation settings. These comparisons examine the effects of group normalization, simple baseline subtraction, and the proposed future-evaluation signal. The results shown below provide empirical support for the contribution of the proposed action-effect evaluation to practical policy optimization.

Advantage FormulationRoboCasa Avg. (%)LIBERO Avg. (%)
Standard GRPO71.898.7
Uncentered Raw Evaluation72.198.9
Constant Baseline71.998.5
Current-State Reference72.398.8
CAG-GRPO73.399.4
AuthorsOpenReview ↗

Response for Reviewer kdWs

Response for Weakness 3 & Question 3

We thank the reviewer for this helpful comment. We would like to clarify the meaning and motivation of HOLD in CAG-WAM. In our implementation, HOLD sets all robot motion increments to zero while maintaining the current gripper state. In robotic manipulation, this represents the condition without additional commanded motion or gripper changes. It provides a meaningful control condition for evaluating how an additional manipulation command changes the expected future task outcome.

Our causal analysis provides the basis for this evaluation. Eq. (2) uses backdoor adjustment to express the expected outcome of an action intervention by averaging over different observations. However, the robot has already obtained its current observation before selecting an action. In the SCM presented in Section 3, the identified backdoor path runs through the current observation. Conditioning on this observation blocks that path, leading to the relationship in Eq. (30). This relationship applies to different choices of the same action variable, allowing us to compare their expected future evaluations under the same observation. Eq. (32) then expresses the effect of executing a candidate action as the expected change in task outcome compared with maintaining the zero-increment condition. This effect includes interactions between the action and environmental dynamics, as discussed in Appendices G.4 and G.5.

The choice of HOLD follows from this evaluation objective. Since our policy represents robot motion as increments, zero motion increments with an unchanged gripper state express the condition without additional manipulation commands. CAG-WAM predicts the futures under the candidate action and HOLD from the same observation and evaluates both over the same horizon. As described in Eq. (31), the learned world model approximates the future distributions under the corresponding control conditions. The expected evaluation difference over model-generated futures then approximates the action effect defined in Eq. (32), as expressed in Eq. (35).

To examine the accuracy of future prediction under HOLD, we compare its predicted future evaluation with the corresponding simulator result on RoboCasa and LIBERO. The prediction errors are shown below.

BenchmarkHOLD Evaluation MAE
RoboCasa0.05353
LIBERO0.03156

We further examine the influence of different comparison conditions by evaluating HOLD, current-state evaluation, and Gaussian-sampled action references under matched training and evaluation settings without Future Revision. The results are shown below.

Reference FormulationRoboCasa Avg. (%)LIBERO Avg. (%)
Current-State Reference72.398.8
Gaussian Action Reference71.097.2
HOLD Reference (Ours)73.399.4

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.