acceptodds
Under review as a conference paper at ICLR 2027

RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes

11 views

Abstract

Embodied coding agents can combine modular robot skills with frozen end-to-end policies, yet effective composition requires anticipating which policy family will succeed in the current physical state. We present **RoboAware**, which builds on coding agents' skill orchestration by learning only a state-conditioned responsibility coordinator from counterfactual outcomes. Inspired by the success of REPL, we propose the **P⁵** schema and formulate a hierarchical MDP based on it. P⁵ organizes skills uniformly into five semantic stages, defining where responsibility can be compared. To address the lack of counterfactual branch outcomes in existing work, we introduce **State-Locked Counterfactual Branching (SCB)**, which restores the same training state to generate and execute a code block from each admissible family, exposing outcomes that selected-branch experience leaves unobserved. Building on this, we propose **Execution-Aware Learning (EAL)**, which combines Monte Carlo tree search with Q-learning to distill these outcomes into family-conditioned values. At deployment, the coordinator selects the policy family according to observable context, and the frozen coding agent generates the next local code block. Comprehensive single-episode evaluations on 100 tasks show that RoboAware reaches a **77.0%** overall success rate, with SOTA averages of 90.0% on RoboSuite, 73.8% on diverse LIBERO-Pro task clusters, and 90.0% on challenging RoboTwin bimanual tasks, outperforming existing code-as-policy and VLA-harness baselines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Coding Agent, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_s5xQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RoboAware studies how embodied coding agents can select between modular robotic skills and frozen end-to-end policies. It introduces the P5 skill schema, State-Locked Counterfactual Branching (SCB), and Execution-Aware Learning (EAL). The approach uses same-state counterfactual executions to train a value-based coordinator that selects the policy family at semantic skill boundaries. Experiments are conducted on LIBERO-Pro, RoboSuite, and RoboTwin.

Critical strengths
  1. Same-state counterfactual branching addresses a genuine limitation of embodied policy coordination: an executed action does not reveal how an alternative policy would have performed from the same physical state. This makes the motivation for SCB compelling and gives EAL a meaningful training signal.

  2. The empirical gains are substantial, and internal ablations help isolate their sources. On RoboTwin, RoboAware achieves 90.0% success, compared with 53.8% without paired counterfactual outcomes and 41.9% using only modular execution. These comparisons provide stronger evidence than benchmarking against standalone policies alone.

Critical weaknesses

W1. Evaluation does not establish generalization to unseen task families.

The coordinator is trained using counterfactual branches collected from scene seed 0 and evaluated on seeds 1–10 of the same underlying task pool. While this protocol measures robustness to variations in initial conditions, it does not establish whether the learned responsibility coordinator generalizes to unseen tasks, manipulation primitives, or substantially different environments. Since the principal contribution is learning when different execution families should be selected, task-level generalization is essential for distinguishing transferable execution awareness from task-specific responsibility memorization.

W2. Strong dependence on privileged simulation and exact state restoration.

State-Locked Counterfactual Branching requires restoring identical physical states and obtaining privileged outcome information, including object-pose and perception-based reward signals. These capabilities are generally unavailable in physical robotic systems. Although the learned coordinator does not require such information at deployment, the paper provides no physical-robot validation or evaluation under approximate state restoration. Consequently, the applicability of the learning framework outside privileged simulators remains uncertain.

W3. Potential confounding factors in the counterfactual ablation.

The comparison between RoboAware and RoboAware-UO shows large improvements from using paired counterfactual outcomes. However, counterfactual branching may also change the amount, diversity, and quality of training experience available to the coordinator. The paper does not sufficiently isolate the benefit of same-state pairing from the benefit of collecting additional execution trajectories. A matched-interaction-budget comparison with unpaired but equally diverse trajectories would be particularly informative.

W4. The primary baseline comparison does not fully isolate coordinator quality.

RoboAware has access to a hybrid execution interface containing both modular and end-to-end policy families, whereas several baselines operate with more restricted action interfaces. Additionally, some baselines are evaluated without their usual cross-episode memory, and certain end-to-end policies are incompatible with the bimanual RoboTwin setting. Therefore, the aggregate performance advantage combines contributions from the available execution tools, hierarchical decomposition, and learned family selection. More matched-interface comparisons would strengthen the attribution to the proposed coordinator.

W5. Dependence on manually specified P5 stage contracts.

The P5 schema provides predefined semantic boundaries, typed inputs, postconditions, and admissible policy families. These structured interfaces are useful but require substantial domain knowledge and careful specification. The system's success may depend on the quality of these manually designed contracts rather than solely on learned execution responsibility. The paper does not evaluate robustness to incorrect stage boundaries, imperfect postconditions, or previously unseen skill schemas.

W6. Restricted family selection and limited interruption capabilities.

The coordinator chooses between only two broad execution families: modular and end-to-end. It does not explicitly optimize over multiple heterogeneous controllers with different capabilities. Furthermore, responsibility can change only at P5 decision boundaries, and some end-to-end invocations may span several stages before returning control. This limits its ability to detect and correct failures during long or contact-sensitive atomic executions.

W7. Limited characterization of computational and interaction costs.

The paper reports approximately 24 GPU-hours for coordinator training and an additional day for counterfactual data collection. However, counterfactual collection requires repeated execution of expensive coding-agent-generated programs and environment rollouts. A complete accounting of coding-model calls, simulation steps, execution latency, and total training cost would make it easier to assess whether the learned coordinator offers a favorable trade-off relative to simpler responsibility-selection methods.

Questions for authors
  1. How does the coordinator perform on completely unseen tasks or manipulation primitives, with task families held out during counterfactual data collection?

  2. How would EAL behave when exact simulator restoration or privileged pose-based reward signals are unavailable? Can training rely on approximate restoration or outcome-only feedback without substantially degrading performance?

5 comments
AuthorsOpenReview ↗

Point-by-point response to Reviewer s5xQ [1/2]

We sincerely thank the reviewer for the comments and review. We provide a point-by-point response to address your concerns below.

W1. Held-out-task generalization

Please see Held-out generalization for the three-level evaluation (within-domain RoboTwin held-out tasks, cross-benchmark LIBERO-Pro2, and real-world deployment). We will update the results in the revision.

W2.1 State restoration and binary outcome

We further test whether training itself can relax exact restore and dense rewards on RoboSuite, keeping the same EAL objective, model, and evaluation protocol. (1) Approximate restore: instead of exact simulator snapshot restore, we calculate and apply the EEF inverse transformation from the recorded state. As the surrounding environment may already have changed slightly, the branch start is only approximately the locked state. (2) Outcome-only reward: we keep the same training target but drop dense / pose-based reward terms, retaining only the binary terminal task outcome.

Training settingRoboSuite Success (%)
Exact restore + privileged reward (paper)90.0
Approximate restore87.5
Outcome-only reward83.0

Success rates remaining comparable on RoboSuite, supporting that EAL does not require perfect restore or dense privileged rewards to remain effective.

W2.2 Physical validation beyond simulation

EAL is designed for Sim2Real transfer: during training we deliberately exploit available privileged simulator information (exact state restoration, dense rewards, and counterfactual branching) to fit the coordinator. At deployment the frozen coordinator uses only onboard observations and never requires state restoration, privileged rewards, or further branching.

To validate deployment without privileges, we deploy the frozen coordinator without additional training on a real-world AgileX PiPER dual-arm System for a long-horizon compositional task: first reorient two beverage bottles lying on the table with both hands, then place them neatly and simultaneously into a plate. RoboAware exhibits execution-coordination behavior: modular executors handle approaching the bottles and the final placement onto one side of the plate, while E2E executors handle fine-grained grasping and in-air pose rotation to upright the bottle. This real-robot study supports that a coordinator trained under privileged simulation can operate without those privileges at deployment. Please refer to Section 3.4 in the revision for this real-world evaluation.

W3. Counterfactual pairing

Please see Counterfactual pairing for the matched-budget comparison of RoboAware and RoboAware-UO. As shown there, although Eq. (4) learns from individual transitions, paired collection exposes unsuccessful and nonpreferred branches that unpaired executed-only data miss, and the gains isolate the benefit of same-state counterfactual pairing.

W4. Hybrid interfaces, disabled memory, embodiment compatibility, and coordinator quality

The P5 stage decision boundary and hybrid modular and end-to-end interface are RoboAware's core agent-system contributions. As the pre-trained π0.5 model is not directly compatible with dual-arm setups, existing baselines cannot be used for RoboTwin. Benefiting from its flexible modular composition, RoboAware can adapt to more embodiment settings and a broader range of task scenarios.

We have conducted various ablation studies under shared infrastructure to isolate the contributions of available execution tools, hierarchical decomposition, and learned family selection, respectively. (Abbreviations: MO = modular only; EO = end-to-end only; TF = training free; UO = unpaired outcomes; FH = fixed handoff; ES = episode selection.)

First, MO and EO keep only one execution family and fall well below the hybrid interface, so both tool families are needed for broad coverage. Second, ES chooses a family once per episode and FH follows a fixed perceive/pre-manipulate to E2E handoff; both trail per-stage coordination, so fine-grained hierarchical decomposition at P⁵ boundaries matters. Third, TF skips coordinator training and UO trains without same-state pairing; both trail the full model, isolating the gain from learned family selection with paired counterfactual supervision.

VariantLIBERO-ProRoboSuiteRoboTwin 2.0
RoboAware-MO54.867.541.9
RoboAware-EO49.560.045.6
RoboAware-TF51.060.040.0
RoboAware-UO58.575.053.8
RoboAware-FH64.582.555.6
RoboAware-ES59.370.051.9
RoboAware73.890.090.0

To address the issue of cross-episode memory, please see Cross-episode memory for the Harness-VLA protocol and results. As reported there, RoboAware retains its advantage when the baseline can use accumulated task-specific experience, supporting the benefit of counterfactual coordination beyond execution memory.

AuthorsOpenReview ↗

Point-by-point response to Reviewer s5xQ [2/2]

W5. About P⁵ boundaries

As specified in Table 2, each modular API is assigned to exactly one P⁵ stage, making its intended stage boundary unambiguous. The E2E API is the explicit exception that can span several stages. Appendix B provides the execution steps, postconditions, and return conditions for each API. The admissible modular and π0.5\pi_{0.5}/FastWAM execution families are designed for broad coverage of object-centric manipulation, so these contracts are reusable across tasks. We will clarify this design and discuss stage-boundary re-estimation, imperfect postconditions, and previously unseen skill schemas as extensions for future work.

W6. Two-family selection and interruption during atomic execution

The modular family exposes a complete modular composition with separate perception, grasp-pose proposal, motion, and gripper components, while the coding agent dynamically selects an available end-to-end backend through in-context reasoning. Thus, the two-family abstraction organizes multiple execution capabilities. The retained end-to-end modules are screened offline for reliable closed-loop execution and provide internal feedback and correction during a call. Our coordinator learns when to schedule these capabilities to complete longer tasks and recover from failures. Responsibility is reconsidered when a call returns at a P⁵ decision boundary; finer-grained interruption within an atomic call remains an extension.

W7. Coding-model calls, simulation steps, latency, and total training cost

We report offline collection and training costs separately from deployment latency, including coding-model usage and simulator interactions.

StageMeasurementValue
Coordinator fittingHardware4 × A100, 80 GB each
Coordinator fittingWall-clock time~ 6 hours
Coordinator fittingGPU-hours~ 24
Counterfactual collectionWall-clock time on the same hardware pool~ 1 day
Full collection-and-training pipelineGPU-hours~ 120
Full collection-and-training pipelineCoding-model calls~ 4×10⁴
Full collection-and-training pipelineInput & output tokens~ 3×10⁸ & ~ 2×10⁷
Full collection-and-training pipelineSimulator control steps~ 3×10⁶
DeploymentCoordinator inference per decision (ms)~ 50
DeploymentMean episode execution time, RoboAware (s)~ 150
DeploymentMean episode execution time, training-free selection (s)~ 180

These measurements distinguish the one-time cost of counterfactual experience from deployment overhead. Deployment uses the frozen coordinator without further counterfactual branching.


Once again, thank you for your careful review. If most of your concerns are addressed, would you please consider increasing the recommendation of the paper?

Thank you for the detailed rebuttal and additional experiments. However, I am not convinced that the responses fully address my concerns. In particular, the generalization of the learned coordinator to genuinely unseen tasks and skills remains insufficiently established, and the independent contribution of same-state counterfactual pairing is not yet fully clear. Moreover, the method still relies heavily on manually defined P5 stage boundaries, and its robustness to imperfect task decomposition remains unexplored. Overall, while the additional results are encouraging, I do not believe they sufficiently address the limitations regarding generalization, methodological contribution, and practical applicability. Therefore, I maintain my original rating.

Reviewer_R2BEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks which modular API composition or a frozen end-to-end policy should control each stage of a robot skill. It fixes five stages per skill (P^5) and trains a single Q model, Qwen3.5-0.8B with a value head, to choose Mod or E2E at each stage boundary. The training data come from State-Locked Counterfactual Branching: the simulator restores the same snapshot and runs both families, so the dataset contains the outcome of the unchosen branch. A shallow UCB tree search collects these transitions; Q-learning fits them offline. On 100 tasks across LIBERO-Pro, RoboSuite, and RoboTwin 2.0, the method reaches 77.0% success against 57.0% for the best baseline. Ablations remove counterfactual branching, restrict to one family, fix the handoff point, and replace UCB with uniform allocation.

Critical strengths
  1. The paper proves its motivating claim rather than only its headline number. Training the same Q model on executed trajectories alone (RoboAware-UO) drops success from 73.8 to 58.5 on LIBERO-Pro and from 90.0 to 53.8 on RoboTwin. Selection bias from chosen-branch supervision is therefore real and costly, which is the paper's central thesis.
  2. The learning problem is small and fully attributable. Only a binary-action Q model is trained; the coding agent, APIs, and policies stay frozen, and deployment uses no simulator access, oracle reward, or search.
  3. The ablations cover the design space that matters. Group A shows neither family alone suffices (LIBERO-Pro 54.8 / 49.5 vs. 73.8). Group C shows a fixed handoff (64.5) and an episode-level choice (59.3) both lose to per-stage selection, which justifies P⁵ granularity. The backend × embodiment sweep (85–91% across 12 cells) shows the coordinator does not depend on one coding model or one robot.
Critical weaknesses
  1. No held-out tasks. Training and testing use the same 100 tasks and differ only in scene seed (App. C.1). The results show that the coordinator transfers across initial states, not that it has learned a general rule for family selection. The abstract and conclusion do not state this restriction.

  2. The baselines are weakened in ways that favor the method.

  • Harness VLA runs a LIBERO checkpoint on all three suites (20.0% on RoboTwin), while RoboAware's E2E route uses a RoboTwin checkpoint there.
  • Every memory-based baseline (CaP-X, RATS, ASPIRE) has its cross-episode memory disabled, removing the mechanism these methods are built around.
  • The Overall column averages only available columns, so baselines missing RoboTwin entries are scored on nine suites while RoboAware is scored on ten.
  • The modular grasp backend, NeuGraspNet+, was tuned on the benchmark (App. B.3), and its contribution is not separated from the coordinator's.

The paper should either compare apples-to-apples or state that the method's advantage holds only under these conditions.

  1. The method is a combination of known components, and the paper does not cite the closest RL prior art. e.g., Same-state branching from restored simulator snapshots is the vine estimator of TRPO (Schulman et al., 2015) and the core of Go-Explore (Ecoffet et al., 2021). The contribution is the combination within a frozen coding-agent harness, which is worthwhile but narrower than the framing implies.
Questions for authors
  1. Can you report results on a task split held out from SCB data collection, even for one LIBERO-Pro suite?
  2. Were the NeuGraspNet+ selection rules developed using any of seeds 1–10?
  3. What does REPL-style programming contribute to the design of P^5? The link is asserted in the abstract and Section 1 but not argued.
  4. Why fit Qθ with Bellman targets rather than regressing the tree returns G_i? No ablation compares the two.
4 comments
AuthorsOpenReview ↗

Point-by-point response to Reviewer R2BE [1/2]

We sincerely thank the reviewer for the thoughtful and encouraging feedback, especially for recognizing the motivation for counterfactual supervision and the focused learning problem. We are glad to provide a point-by-point response to address your concerns below.

Q1. Held-out-task generalization

Please see Held-out generalization for the three-level evaluation (within-domain RoboTwin held-out tasks, cross-benchmark LIBERO-Pro2, and real-world deployment). Thank you for distinguishing scene-seed from task-level generalization. We will report these held-out results and state the evaluation protocol clearly in the revised PDF (Section 3.4).

Q2. Cross-episode memory

We agree that enabling cross-episode memory for memory-based baselines matters for a fair comparison. As a concrete example, please see Cross-episode memory for the Harness-VLA protocol and results. We will clarify this protocol in the revised PDF. From the shared table, RoboAware retains its advantage when the baseline can use accumulated task-specific experience, supporting the benefit of counterfactual coordination beyond execution memory.

Q3. Harness-VLA and RoboAware

We clarify that Harness-VLA’s 20.0% average success rate is reported on RoboSuite, not RoboTwin. Neither Harness-VLA nor RoboAware leverages any model pre-trained on RoboSuite. To unpack RoboAware’s substantial advantage over Harness-VLA, we analyze their feature differences and identify that this gain primarily stems from a richer set of Executor Backends for orchestration, alongside more flexible switching between modular composition and end-to-end executors under our proposed P⁵ stage decision boundary.

FeatureHarness-VLARoboAware
Modular API Composition✓✓
Multiple Executor Backends✗✓
Responsibility Boundariesfixedflexible

Q4. Different numbers of suites in the Overall average

Thank you for identifying this issue. We recompute every method's average over the same nine columns: eight LIBERO-Pro splits and RoboSuite, excluding RoboTwin throughout. This recomputation targets the Code as policy and VLA with harness methods whose RoboTwin entries were previously unavailable (due to embodiment mismatch) or reported as 0% . RoboAware remains ahead of the strongest baseline under the common aggregation. We will report this comparison explicitly.

CategoryMethodOverall Success Rate (%)
Code as policyRATS11.0
Code as policyASPIRE63.3
VLA with harnessPigey41.7
VLA with harnessVLS32.6
VLA with harnessHarness-VLA50.0
VLA with harnessRoboAware75.6

Q5. NeuGraspNet+ tuning

We clarify that the NeuGraspNet+ selection rules were developed and tested exclusively on seed-0 scenes, with no use of seeds 1–10. We compare the two grasp backends within RoboAware while fixing the coordinator, remaining components, and test protocol. The full-task ablation quantifies the enhanced grasp backend's contribution separately from coordination, complementing its improved first-choice grasp success in the Panda screen.

MethodLIBERO-ProRoboSuiteRoboTwin 2.0
RoboAware-MO54.867.541.9
RoboAware-EO49.560.045.6
RoboAware-NeuGraspNet68.085.075.6
RoboAware73.890.090.0
AuthorsOpenReview ↗

Point-by-point response to Reviewer R2BE [2/2]

Q6. Relation to TRPO's vine estimator and Go-Explore

We thank the reviewer for relating our algorithm to these generative mechanisms. We will cite TRPO and Go-Explore and refine the contribution statement in the revision. We want to emphasize a key distinction is the setting: prior work focuses on exploration in discrete game simulators with discrete action spaces, whereas RoboAware introduces the P⁵ stage formulation to convert continuous robotic decision making into a discrete responsibility-orchestration problem, and uses counterfactual coordination for exploration and exploitation over these decision branches. This is the novelty we wish to emphasize.

Q7. How REPL-style programming motivates P⁵

The connection is the iterative cycle of reading the current state, executing a local program, and using the resulting feedback to plan the next step. P⁵ adapts this cycle to manipulation through Perceive, Propose, Pre-manipulate, Perform, and Post-manipulate. These stages expose the observations, candidate actions, preparation, execution, and follow-up needed for the coding agent to generate and revise local code blocks. Their semantic boundaries also provide explicit points at which execution responsibility can be reconsidered. We will make this connection clearer in the introduction.

Q8. Bellman targets versus direct regression of rollout returns

Direct regression uses Gi=r~i+Gi+1G_i = \tilde{r}_i + G_{i+1}, where Gi+1G_{i+1} follows the family choices actually executed in the rollout. The Bellman target, yi=r~i+(1−di)max⁡c′∈Ci+1Qθˉ(xi+1,c′)y_i = \tilde{r}_i + (1 - d_i) \max_{c' \in \mathcal{C}_{i+1}} Q_{\bar{\theta}}(x_{i+1}, c'), instead uses the estimated best admissible continuation, which need not match the sampled continuation. We compare these targets on RoboSuite with fixed data, model, and optimization budget.

Training TargetRoboSuite Success Rate (%)
Direct rollout-return regression72
Bellman target90

These results support Bellman updates in MCTS-guided EAL when limited samples make direct return regression more variable and less effective.


Once again, thank you for your careful review. If most of your concerns are addressed, would you please consider increasing the recommendation of the paper?

Thank you for your reply. The authors provide detailed responses and additional experiments, and I appreciate the clarification about Harness-VLA. I also apologize for the typo in my review (RoboTwin instead of RoboSuite).

The new results are convincing and address my concerns about generalization, baseline comparisons, and the method itself. I encourage the authors to include these results in the paper and clarify the differences between prior methods.

My main concerns and questions have been addressed, and I will maintain my recommendation.

AuthorsOpenReview ↗

Response to Reviewer R2BE

Thank you for the thoughtful follow-up and for taking the time to re-examine our responses and experiments. We are glad that the additional results and the Harness-VLA clarification address your concerns. As suggested, we will incorporate these results into the revised paper and more clearly distinguish RoboAware from prior methods.

Reviewer_AR3bOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RoboAware learns a coordinator that selects between modular robot skills and frozen end-to-end policies at semantic stage boundaries. State-Locked Counterfactual Branching evaluates alternative policy families from identical simulator states, and execution-aware Q-learning converts the collected transitions into deployment-time routing decisions. The method reports 77.0% overall success across 100 tasks, with evaluation on scene seeds excluded from coordinator training.

Critical strengths
  • The framework provides a concrete way to combine complementary execution capabilities while keeping the coding agent and low-level policies frozen. The separation between privileged training-time branching and observation-based deployment is clearly specified, and deployment requires neither simulator restoration nor online search.

  • The controlled routing ablations support the usefulness of adaptive coordination. On LIBERO-Pro, RoboAware achieves 73.8% success versus 54.8% for modular-only execution, 49.5% for end-to-end-only execution, and 64.5% for fixed handoff (Table 4). These comparisons share the underlying execution infrastructure and provide stronger evidence than the whole-system baseline comparisons alone.

Critical weaknesses
  • The main comparison does not adequately control for task-specific training experience. RoboAware collects privileged simulator experience on all 100 evaluation tasks using seed 0, whereas external baselines largely use frozen implementations with cross-episode memory disabled (Appendices C.1–C.2). Thus, the evaluation establishes generalization to new scene seeds within familiar tasks, and the reported superiority does not cleanly separate coordination quality from additional task-specific experience.

  • The central benefit attributed to same-state counterfactual pairing is not sufficiently isolated. RoboAware-UO removes branching, but the paper does not specify whether simulator interactions, retained transition counts, and family coverage are matched. Since Equation (4) learns from individual transitions without an explicit paired objective, broader exploration or additional data could explain part of the improvement. The matched-budget UCB-versus-uniform experiment addresses search allocation, but does not resolve this question about pairing.

Questions for authors

Can the strongest applicable baselines use the same seed-0 training experience, with their resulting memory or adaptation frozen during evaluation? Alternatively, how does RoboAware perform on held-out tasks?

Are RoboAware and RoboAware-UO matched in simulator interaction budget, training-set size, and family coverage? Does the advantage persist against an unpaired collection strategy with these factors controlled?

7 comments
AuthorsOpenReview ↗

Point-by-point response to Reviewer AR3b

We sincerely thank the reviewer for the careful review and valuable feedback. We provide a point-by-point response to address your concerns below.

Q1. Cross-episode memory

Thank you for raising this concern. Please see Cross-episode memory for the protocol, results, and conclusion: we enable Harness-VLA's cross-episode memory under the shared seed-0 / seeds 1–10 setup, and RoboAware retains its advantage when the baseline can use accumulated task-specific experience.

Q2. Counterfactual pairing

Please see Counterfactual pairing for the matched-budget comparison of RoboAware and RoboAware-UO. As detailed there, the two collections differ only in whether both family outcomes are observed from the same locked state, and the gains isolate the benefit of same-state counterfactual pairing.

Q3. Held out Generalization Performance

Please see Held-out generalization for the held-out results across RoboTwin, LIBERO-Pro2, and the real-world deployment. Please also refer to Section 3.4 in the revision.


Once again, thank you for your careful review. If most of your concerns are addressed, would you please consider increasing the recommendation of the paper?

Thank you for the additional experiments and detailed clarifications. The memory-enabled Harness-VLA comparison strengthens the evidence that RoboAware’s gains are not simply due to access to task-specific experience. The held-out-task evaluations also address my concern about generalization beyond different scene seeds. Together with the matched-budget counterfactual comparison, these results substantially alleviate my main concerns. Based on this additional evidence, I am now leaning toward weak acceptance.

One clarification remains regarding the counterfactual comparison: please specify whether the reported 10 trajectories, 250 simulator steps, and 100 transitions are counted per root, per task, or over another unit, and explain how the perturbed starting states for RoboAware-UO were sampled. Matching the total budgets is helpful, but differences in starting-state distributions could still affect attribution to same-state pairing. Including these details in the revised paper would make the comparison more convincing and reproducible.

AuthorsOpenReview ↗

Clarification on matched-budget units and UO starting-state sampling

We appreciate the reviewer’s positive stance given this additional evidence. We clarify the counting unit and the RoboAware-UO sampling rule below, and will include both in the revised paper.

1. Counting unit. Thanks for this question, which helps us clarify the counting unit. The figures presented in the common rebuttal under the “Counterfactual pairing” section use the LIBERO suite as a concrete example for coordinator training data collection. All three quantities are counted per task. For each training task, we keep N=10N=10 exploration trajectories. On LIBERO, a task has an average length of L=250L=250 MuJoCo simulation steps, corresponding to two P5\mathrm{P}^5 blocks and yielding 10 transitions per trajectory. This gives N×10=100N \times 10 = 100 retained transitions per task in the dataset. With N=10N=10 retained trajectories, RoboAware selects N/2=5N/2=5 locked states and executes both admissible families from each restored state (i.e., two counterfactual trajectories per locked state). In contrast, RoboAware-UO retains N=10N=10 unpaired executed trajectories.

2. UO starting states. For RoboAware-UO, starting states are sampled randomly from the same neighboring-state window Wi(Δ)\mathcal{W}_i(\Delta) used to construct SCB roots: non-terminal saved timestamps within Δ=10\Delta=10 MuJoCo steps of stage entry, where the corresponding stage-entry conditions are satisfied (see Appendix A.2). Each sampled state produces only one executed trajectory, without restoring to evaluate both admissible families at that identical state. By contrast, RoboAware locks a single state and observes outcomes from both families starting from that exact restored state. The two pipelines therefore share the same local starting-state neighborhood and the same overall compute budget; their core difference lies in whether both family outcomes are observed from a single locked state. Anchoring UO sampling to the same neighborhood as SCB roots ensures our comparison isolates the effect of same-state counterfactual pairing, rather than confounding it with differences in state sampling support.


We will explicitly document this counting unit and sampling rule in the revision. We hope explanation above can address your remaining questions. We would be happy to clarify further if any points remain unclear. Look forward to your reply!

Comments on the paper

3 comments
AuthorsOpenReview ↗

Shared clarifications on cross-episode memory, counterfactual pairing, and held-out generalization

We thank all reviewers for their valuable and careful comments. Here we extract several core shared questions for additional clarification.

Cross-Episode Memory

We enable Harness-VLA's cross-episode memory, accumulate experience from the same seed-0 training tasks, and freeze the resulting memory during evaluation on seeds 1–10 (10 tasks for each LIBERO-Pro split, 100 rollouts in total).

LIBERO-ProHarnessVLA-full (%)RoboAware (%)
Spat-T60.068.0
Spat-S48.068.0
Obj-T91.092.0
Obj-S59.086.0
Goal-T70.066.0
Goal-S54.084.0
L10-T37.064.0
L10-S45.062.0
Average58.073.8

From the table above, RoboAware retains its advantage when the baseline can use accumulated task-specific experience, supporting the benefit of counterfactual coordination beyond execution memory.

Counterfactual Pairing

We match the rollout budget and retained trajectory count while keeping the same modular and end-to-end families. The collections differ in whether alternative outcomes are paired at the same state.

Collection detailRoboAwareRoboAware-UO
Sampling strategyTwo counterfactual trajectories per locked stateExecuted trajectories from slightly perturbed states without pairing
Retained trajectories1010
Simulator steps250250
Retained transitions100100
Mod / E2E proportion (%)50 / 5051 / 49

Thus the two collections are matched in rollout count, simulator-step budget, retained transition count, and family coverage. They differ only in whether both family outcomes are observed from the same locked state. The table below reports success rates under the two collections:

MethodLIBERO-ProRoboSuiteRoboTwin
RoboAware-UO58.5%75.0%53.8%
RoboAware73.8%90.0%90.0%

Comparing RoboAware-UO and RoboAware, although Eq. (4) learns from individual transitions, paired collection exposes unsuccessful and nonpreferred branches that unpaired executed-only data miss, and the gains above therefore isolate the benefit of same-state counterfactual pairing.

Held-Out Generalization

BenchmarkEvaluation SetAverage Success Rate (%)
RoboTwinseen 16 tasks90.0
RoboTwinheld-out 34 tasks93.8
LIBERO-Pro2 (held-out)Spatial92.0
LIBERO-Pro2 (held-out)Object100.0
LIBERO-Pro2 (held-out)Goal76.0
LIBERO-Pro2 (held-out)Long74.0

In the current submission, RoboAware's SCB pool includes 16 carefully selected, challenging RoboTwin tasks, such as Hanging Mug and Move Pillbottle Pad, for coordinator training. We extend evaluation to three levels of generalization.

1. Within-domain task generalization: We evaluate the frozen coordinator zero-shot on the remaining 34 held-out RoboTwin tasks under seeds 1–10, without further training. RoboAware achieves 93.8% success (90.0% on the 16 seen tasks), supporting transfer of learned responsibility selection beyond the RoboTwin tasks used to collect counterfactual experience.

2. Cross-benchmark domain generalization: We directly deploy RoboAware on LIBERO-Pro2, a benchmark featuring runtime interventions. Notably, all tasks are held out, and the coordinator receives no further training. Even so, it achieves promising success rates ranging from 74.0% to 100.0% across the four splits.

3. Real-world generalization: Without additional training, we deploy RoboAware on a real-world AgileX PiPER dual-arm System for a long-horizon compositional task that reorients two beverage bottles and places them neatly into a plate; modular and E2E executors respectively handle approach/placement and fine grasping with in-air pose uprighting. Together, these evaluations provide evidence of task generalization, robustness under the tested runtime interventions, and practical real-world applicability. Please refer to Section 3.4 in the revision.

AuthorsOpenReview ↗

Rebuttal Summary

We thank the PCs, SACs, and ACs for careful reviews, and thank reviewers for their constructive feedback. They recognized the motivation for counterfactual supervision and the benefits of learning to coordinate complementary modular skills and end-to-end policies.

During the rebuttal, we added a memory-enabled Harness-VLA comparison, counterfactual pairing ablations controlling retained training data and starting-state sampling, and held-out evaluations on 34 RoboTwin tasks and LIBERO-Pro2. Two reviewers confirmed that these additions and clarifications largely addressed their main concerns. One raised their recommendation from weak rejection to weak acceptance, while the other maintained weak acceptance.

The third reviewer retained concerns about generalization, the contribution of counterfactual pairing, and predefined stage boundaries. We have since provided further clarifications connecting these concerns to the held-out results, pairing controls, and reuse of the same stage contracts across settings. We also highlighted experiments with approximate state restoration and outcome-only rewards, together with a zero-shot real-robot demonstration. These results strengthen the evidence for generalization and practical applicability.

We hope the AC will consider these additional results, clarifications, and the reviewers’ follow-up assessments.

AuthorsOpenReview ↗

Rebuttal Summary

We thank the PCs, SACs, and ACs for careful reviews, and thank reviewers for their constructive feedback. They recognized the motivation for counterfactual supervision and the benefits of learning to coordinate complementary modular skills and end-to-end policies.

During the rebuttal, we added a memory-enabled Harness-VLA comparison, counterfactual pairing ablations controlling retained training data and starting-state sampling, and held-out evaluations on 34 RoboTwin tasks and LIBERO-Pro2. Two reviewers confirmed that these additions and clarifications largely addressed their main concerns. One raised their recommendation from weak rejection to weak acceptance, while the other maintained weak acceptance.

The third reviewer retained concerns about generalization, the contribution of counterfactual pairing, and predefined stage boundaries. We have since provided further clarifications connecting these concerns to the held-out results, pairing controls, and reuse of the same stage contracts across settings. We also highlighted experiments with approximate state restoration and outcome-only rewards, together with a zero-shot real-robot demonstration. These results strengthen the evidence for generalization and practical applicability.

We hope the AC will consider these additional results, clarifications, and the reviewers’ follow-up assessments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.