acceptodds
Under review as a conference paper at ICLR 2027

Beyond Actions: Learning Future Operation Targets for Vision-Language-Action Models

Abstract

Actions specify what a robot should execute, but do not describe what those commands will achieve. Vision-language-action (VLA) policies nevertheless learn primarily by predicting action chunks from images, language, and state. We introduce Action–Future Coupled Effect Learning (AFCE), which makes a future-grounded physical Effect the policy's primary prediction target. An offline tokenizer constructs temporally ordered Effect tokens from demonstrated actions, realized robot responses, and sparse visual transitions. Reconstruction objectives encourage the representation to retain executable detail and observed physical change. A flow-matching VLA predicts Effects from current images, instruction, and robot state, while a state-conditioned decoder converts them into action chunks. Embodiment-specific interfaces allow single- and dual-arm tasks to share the Effect model. On DexJoCo, AFCE achieves overall success, improving over and UVT by 11.82 and 15.21 percentage points, respectively. It leads on all five bimanual tasks, highlighting the value of Effect supervision for coordinated dexterous manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Future Operation Representation, Latent Action Learning, Dexterous Manipulation, Robot Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XenNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduce Action-Future Coupled Effect Learning (AFCE) method, which makes a future-grounded physical effect the policy's primary prediction target. AFCE introduces Effect tokens as the primary prediction target for VLA policies. Offline tokenizers extract temporally ordered effects from demonstrations and robot responses. A state-conditioned decoder converts these effects into action chunks. The method achieves 50.97% success on DexJoCo, outperforming existing baselines. It also leads on all five bimanual tasks.

Critical strengths
  1. Well-motivated policy supervision. AFCE combines actions, realized robot responses, and visual transitions into an action-decodable target, incorporating physical consequences into policy learning without requiring future observations at inference.

  2. Promising bimanual results. AFCE achieves 37.33% bimanual success, outperforming the evaluated π₀.₅ (9.73%) and UVT (14.53%) baselines on all five tasks. These gains demonstrate its potential for coordinated manipulation.

Critical weaknesses
  1. Insufficiently specified controls limit attribution of the gains. The main comparison uses an officially released π₀.₅ multitask checkpoint, while the training configuration of the UVT baseline is insufficiently documented. It is therefore difficult to establish how closely optimization, task sampling, and adaptation settings are matched. Table 3 is valuable: its group means imply approximately 45.27% overall success for the action-only variant, versus 50.97% for full AFCE. However, it is unclear whether removing a modality also removes its reconstruction objective. If future-response or visual reconstruction remains active, the action-only variant would still receive future supervision. Moreover, the reported evaluation-seed variation does not establish robustness across independent training runs. These uncertainties materially affect how confidently the improvement can be attributed to the proposed future-grounded target.

  2. The executability diagnostic lacks a necessary reference. At the selected Effect dimension, ground-truth-Effect replay succeeds on only 25/110 training demonstrations, including 5/50 bimanual demonstrations. A raw-demonstration-action replay result under identical conditions is needed to interpret these numbers. Without that reference, it is difficult to determine whether failures arise primarily from representation and decoding errors or from the replay setup itself. The authors appropriately distinguish this diagnostic from closed-loop policy evaluation, but the central claim that the Effect preserves executable detail still needs stronger support. A controlled replay comparison would make this part of the contribution substantially more convincing.

Questions for authors

Clarification and targeted validation: In Table 3, does removing S or V remove only the encoder input, or also its reconstruction target and loss? Are the decoder architecture, Stage-II losses, and optimization settings otherwise unchanged? Can the full-AFCE versus action-only improvement be confirmed across independent training seeds? This would help establish the contribution of future supervision.

2 comments
AuthorsOpenReview ↗

Response to Reviewer XenN: Effect Construction, Matched Comparisons, and Replay Analysis

We thank the reviewer for the helpful questions.

W1 + Q1. Modality ablations and independent-run comparison. Removing SS or VV removes both the corresponding future-evidence input and its Stage-I reconstruction objective. The current robot state remains available to the action decoder in all variants; the Stage-II decoder, losses, and optimization settings are unchanged.

VariantEffect evidenceStage-I reconstruction
A-only latentAAction
w/o SA+VAction + Visual
w/o VA+SAction + State
Full AFCEA+S+VAction + State + Visual

The single-checkpoint modality ablation gives 45.27% overall for A-only latent and 50.97% for Full AFCE (+5.70 pp; +5.11 pp single-arm and +6.40 pp bimanual). To assess variation across training runs in the comparison with direct action prediction, we train Direct-A and Full AFCE using the same three training seeds:

SeedDirect-AFull AFCEΔ
4140.55%47.27%+6.73 pp
4239.76%50.97%+11.21 pp
4336.79%48.24%+11.45 pp
Mean ± SD39.03 ± 1.98%48.83 ± 1.92%+9.80 ± 2.66 pp

Full AFCE achieves higher overall success than matched direct-action prediction for each of the three training seeds. Direct-A and A-only latent are distinct controls: the former predicts actions directly, whereas the latter predicts an Effect constructed from action evidence only. We report the A-only latent result as a target-construction ablation, not as a multi-seed result.

W2. Raw-action replay reference. On the same 110 demonstration windows and initial states (H=30H=30), raw demonstrated actions succeed in 67/110 cases (60.9%), GT-Effects in 21/110 (19.1%), and predicted Effects in 10/110 (9.1%). The 25/110 result in Table 2 uses the Stage-I snapshot of the same decoder; the 21/110 result uses its jointly adapted Stage-II snapshot. In Effect replay, the Effect sequence follows demonstration time steps and the decoder uses the current robot state. This fixed-sequence diagnostic differs from the 50.97% closed-loop result, where the policy predicts Effects from current observations and replans. The gap from raw-action replay indicates remaining execution degradation in the Effect-to-action pathway; GT-Effects nevertheless yield higher replay success than predicted Effects under the same protocol.

Thank you for your questions about the ablations and replay evaluation.

  1. Modality ablations. Each ablation removes both the future modality input and its reconstruction loss. The decoder retains the current robot state.
  2. Controls. AFCE exceeded the action-only latent by 5.70 percentage points at seed 42 and outperformed Direct-A at all three training seeds.
  3. Replay reference. On the same 110 windows, demonstrated actions succeeded 67 times, ground-truth Effects 21 times, and predicted Effects 10 times.

As the discussion closes in a few hours, we would appreciate a brief reply on whether these clarifications resolve your concerns.

Reviewer_wFxMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Action–Future Coupled Effect Learning (AFCE), which trains a vision-language-action policy to predict a temporal Effect representation that couples demonstrated actions with their observed physical consequences. An offline tokenizer fuses action sequences, realized robot-state changes, and sparse visual transitions, with reconstruction objectives preserving executable action detail and future changes. A modified π0.5 policy predicts these Effects from current images, language, and robot state; a state-conditioned decoder maps them to action chunks. Future observations are needed only during training. On 11 simulated DexJoCo tasks, AFCE achieves 50.97% overall success versus 39.15% for π0.5 and 35.76% for UVT. It improves all five bimanual tasks, while its single-arm average remains slightly below π0.5.

Critical strengths
  1. The learning target is well motivated and connected to executable control. Distinguishing commanded actions from realized robot motion and environmental change gives the policy supervision about what an interaction achieves. Action reconstruction and jointly trained action readout constrain the Effect to retain control detail. The deployed policy uses current observations, so future evidence enriches supervision without becoming an inference requirement. This is a useful formulation for integrating physical consequences into VLA learning.

  2. The end-to-end bimanual results and modality ablations provide meaningful empirical support. AFCE improves bimanual success from 9.73% to 37.33% over π0.5 and leads on all five bimanual tasks. The full action, robot-response, and visual configuration also improves over the action-only latent variant by 5.11 percentage points on single-arm tasks and 6.40 points on bimanual tasks. These results support the combined supervision, although they do not establish a consistent independent benefit from each modality.

Critical weaknesses

The asymmetry between single-arm and bimanual performance remains insufficiently explained. The formulation applies to both configurations, but the gain over π0.5 is concentrated in bimanual tasks: single-arm success is 62.33% versus 63.67%, with a substantial Pinch Tongs regression from 43.33% to 5.33% (Table 1). But the proposed approach is supposed to be arm-agnostic. My concern is that the experiments do not identify what limits its benefit here. A lot of the bimanual gain is already present in that action-only variant. Controlled, task-level diagnostics are needed to separate Effect prediction, action decoding, and mixed-training effects. Without them, it remains unclear when the proposed target improves control and when it sacrifices precision.

Questions for authors
  1. Can you provide task-level ground-truth-Effect versus predicted-Effect decoding errors and replay results, particularly for Pinch Tongs? Does its regression arise during Effect reconstruction and decoding, policy prediction, or execution? The aggregate diagnostics do not resolve this distinction.

  2. Can you compare direct action prediction, the action-only latent variant, and full AFCE under matched training settings, including single-arm-only versus mixed single-arm/bimanual training? This would help explain the uneven gains and distinguish the contribution of future-grounded supervision from the latent interface and training mixture.

3 comments
AuthorsOpenReview ↗

Response to Reviewer wFxM: Precision Limitations, Replay Diagnostics

We thank the reviewer for highlighting the single-arm/bimanual asymmetry and the Pinch Tongs regression.

W1 + Q1. Task-level Effect decoding and replay analysis. We evaluate raw-action, GT-Effect, and predicted-Effect replay on the same 110 demonstration windows (10 per task) and initial states, with H=30H=30:

TaskRaw actionsGT-Effect replayPred-Effect replayPred. action MAEGT action MAE
Click Mouse10/103/105/100.09150.0256
Fold Glasses8/102/100/100.09300.0254
Hammer Nail7/103/101/100.09030.0246
Pick Bucket10/105/101/100.08190.0244
Pinch Tongs6/100/100/100.06360.0185
Water Plant10/106/102/100.06650.0208
Bimanual Assembly0/100/100/100.10300.0454
Bimanual Hanoi5/100/100/100.12430.0291
Bimanual Microwave10/102/100/100.10850.0269
Bimanual Photograph1/100/101/100.08440.0322
Bimanual Unlock iPad0/100/100/100.09740.0271
All67/11021/11010/110——

For Pinch Tongs, raw-action replay succeeds on 6/10 windows, whereas both Effect replays succeed on 0/10. In this task-level diagnostic, action MAE is 0.0636 for predicted Effects versus 0.0185 for GT-Effects. Separately, the teacher-forced, overlapping-window diagnostic in Appendix C gives Effect cosine similarity 0.982 and action MAE 0.0567 versus 0.0096 (5.9×). These measurements follow a different protocol from the task-level replay diagnostic above.

StageEffect cosinePred. action MAEGT action MAERatioPred. TCPPred. finger
Approach0.9650.10390.01159.1×0.05080.0689
Before contact0.9560.12790.01339.6×0.04500.0957
Finger closure0.9820.05880.01015.8×0.02570.0615
Manipulation0.9900.03270.00843.9×0.01920.0368
All0.9820.05670.00965.9×0.02840.0499

A separate dense evaluation around contact gives TCP errors of 0.0313/0.0041 for predicted/GT Effects at the annotated contact step.

The gap between 0/10 GT-Effect and 6/10 raw-action replay indicates execution degradation in the Effect-to-action pathway; the action-readout errors show that policy prediction introduces further error. The seed-42 Pinch Tongs ablation gives 13.33% for Effect-FM-only, 4.00% with auxiliary losses and a frozen decoder, and 5.33% for Full AFCE. These results suggest that the combined supervision/adaptation pathway improves aggregate performance but may compromise precision on this task. Matched Direct-A achieves 6.00% at seed 42; across three training seeds, Direct-A and AFCE average 13.11% and 8.89%, respectively.

AuthorsOpenReview ↗

Response to Reviewer wFxM: Controlled Ablations

W1 + Q2. Direct action, action-only Effect, and training-mixture controls.

MethodOverall (%)Single-arm (%)Bimanual (%)
Direct-A (3-seed mean ± SD)39.03 ± 1.9857.93 ± 8.1216.36 ± 6.59
Full AFCE (3-seed mean ± SD)48.83 ± 1.9263.52 ± 1.0331.20 ± 5.39
A-only latent (seed 42)45.2757.2230.93
Full AFCE (seed 42)50.9762.3337.33

Direct-A predicts actions directly; A-only latent predicts an Effect constructed from action evidence only. Direct-A and Full AFCE are evaluated across three independent training seeds; A-only latent uses one checkpoint trained with seed 42. Comparing it with Full AFCE at the same seed gives +5.70 pp overall, +5.11 pp single-arm, and +6.40 pp bimanual; the three-seed rows are used only for Direct-A versus Full AFCE. The three-seed comparison shows positive overall and bimanual gains over direct action prediction for all three runs; single-arm gains are not uniform. The A-only latent ablation does not establish robustness across training seeds. The policy-stage comparison uses the same π0.5\pi_{0.5} initialization, DexJoCo data and sampler, 60k updates, batch size 32, action horizon, and evaluation protocol; AFCE additionally trains the Stage-I tokenizer for 30k updates.

We further compare single-arm-only and mixed training with the same Stage-I Effect, decoder initialization, 60k updates, and training seed:

Training dataSingle-arm (%)Pinch Tongs (%)
Mixed (6 SA + 5 BA)62.33 ± 1.455.33 ± 2.31
SA-only (6 SA)60.33 ± 2.968.00 ± 2.00

Each training-mixture variant uses one training run (seed 42); mean ± SD is computed across three evaluation environment seeds. Under the fixed update budget, SA-only training exposes each single-arm sample more frequently but does not improve aggregate single-arm success (60.33% vs. 62.33%); Pinch Tongs success also remains low (8.00% vs. 5.33%). This suggests that competition from bimanual samples alone does not explain the single-arm/bimanual gap or the precision regression.

Thank you for highlighting the single-arm and bimanual performance gap.

  1. Pinch Tongs. Demonstrated actions succeeded on 6/10 replay windows, while ground-truth Effects succeeded on 0/10, indicating precision loss during decoding.
  2. Controls. Across three seeds, AFCE gained 14.84 percentage points over Direct-A on bimanual tasks and 5.59 on single-arm tasks. At seed 42, it also exceeded the action-only latent by 5.70 points overall.
  3. Training mixture. At seed 42, single-arm-only training yielded 60.33% single-arm success versus 62.33% with mixed training. Pinch Tongs success remained low in both settings.

With only a few hours remaining, we would be grateful to hear whether these comparisons address your questions.

Reviewer_v7mTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces AFCE, a two-stage VLA training scheme that constructs a future-grounded “Effect” target from demonstrated actions, realized robot response, and sparse visual transitions, then trains a policy to predict this Effect from current observations and decode it into an action chunk. On DexJoCo, AFCE reports 50.97% overall success versus 39.15% for a π0.5 checkpoint and 35.76% for UVT, with large gains on bimanual tasks but a notable regression on precision-sensitive single-arm tasks.

Critical strengths

The problem framing is interesting: separating “what the robot is asked to do” from “what physically happens” is a sensible way to think about supervision for manipulation policies.

The bimanual gains are substantial, and the paper does not hide the single-arm/precision regression; the failure-rate breakdown by task family is useful.

The two-stage design and embodiment-specific adapters are engineering-wise clean, and the deployment path avoids future observations, which is the right constraint.

Critical weaknesses

The causal claim is not established. Stage II jointly trains the decoder and applies a ground-truth-action auxiliary loss whose gradients reach the policy. The paper’s own diagnostics show that predicted-Effect decoding has 36.91 mm position error versus 8.30 mm for GT-Effect, and GT-Effect replay succeeds on only 25/110 episodes. If the Effect is this weak, why does closed-loop success improve so much? The likely answer is that the action auxiliary loss and joint decoder adaptation do much of the work, with the Effect acting as a regularizer rather than the mechanism. A same-architecture, same-budget direct-action baseline—and a predicted-Effect vs. GT-Effect closed-loop ablation—is missing.

Baseline fairness is unclear. The π0.5 number appears to come from an officially released multi-task checkpoint, while AFCE is trained for 60k updates on two nodes of GH200s with a mixed single-/dual-arm sampler. Were π0.5 and UVT trained with the same data, update budget, and tuning protocol? The enormous bimanual gap (37.33% vs. 9.73%) is exactly where data mixing and capacity could dominate, so this needs to be ruled out.

Statistical support is thin. Results use one trained checkpoint per method and three environment seeds; the reported standard deviations are across evaluation seeds, not training runs. That is enough for a benchmark report but not for a method-level claim, especially given the single-arm regression.

The precision regression is not a footnote. Pinch Tongs drops from 43.33% to 5.33%, and the seed-0 precision failure rate rises from 51.3% to 58.7%. The paper’s conclusion still emphasizes aggregate gains; the method is clearly not uniformly better, and the “Effect” bottleneck may be actively harmful for fine contact-rich control.

Questions for authors

Please specify the exact training data, update budget, and hyperparameter tuning for π0.5 and UVT, and confirm whether they were trained under the same protocol as AFCE.

Can you report a direct-action baseline with the same π0.5 backbone, same data, same 60k-update budget, and no Effect bottleneck? Without this, the headline gain is difficult to attribute.

How do you reconcile 25/110 GT-Effect replay success with 50.97% closed-loop success? Please clarify the replay protocol and report predicted-Effect vs. GT-Effect closed-loop results.

What is the measured contribution of the ground-truth-action auxiliary loss? If it is removed or heavily downweighted, does the bimanual advantage survive?

3 comments
AuthorsOpenReview ↗

Response to Reviewer v7mT: Matched Baselines, Training Stability

We thank the reviewer for the constructive comments.

W2 + Q1. Baseline protocols and training configuration. The π0.5\pi_{0.5} result in the main table uses the released DexJoCo checkpoint, which was not retrained under our 60k schedule; we therefore do not treat it as a matched-budget baseline. UVT follows its official training recipe and uses the same π0.5\pi_{0.5} backbone, 11-task DexJoCo dataset and sampler, and 60k policy-stage schedule as AFCE. The controlled Direct-A/AFCE comparison uses the same π0.5\pi_{0.5} initialization, images from three views, language, proprioception, 44-D actions, H=30H=30, batch size 32, and AdamW (LR 5×10−55\times10^{-5}, 10k warm-up, gradient clipping at 1.0) for 60k policy updates. AFCE adds 30k Stage-I tokenizer updates, so only policy-stage budgets match. We use the official UVT recipe and fixed Direct-A/AFCE settings without additional method-specific hyperparameter sweeps.

W1 + W3 + Q2. Matched Direct-A and training-run variation. Direct-A is the retrained π0.5\pi_{0.5} baseline that predicts actions directly from current observations, using the same policy initialization and policy-stage budget as AFCE. We train both methods using the same three training seeds:

Training seedGroupDirect-AAFCEΔ (pp)
41Single-arm67.00%64.00%−3.00
41Bimanual8.80%27.20%+18.40
41Overall40.55%47.27%+6.73
42Single-arm55.44%62.33%+6.89
42Bimanual20.93%37.33%+16.40
42Overall39.76%50.97%+11.21
43Single-arm51.33%64.22%+12.89
43Bimanual19.33%29.07%+9.73
43Overall36.79%48.24%+11.45
Mean ± across-run SDSingle-arm57.93 ± 8.12%63.52 ± 1.03%+5.59 ± 8.02
Mean ± across-run SDBimanual16.36 ± 6.59%31.20 ± 5.39%+14.84 ± 4.54
Mean ± across-run SDOverall39.03 ± 1.98%48.83 ± 1.92%+9.80 ± 2.66

Mean ± SD is computed across training runs before rounding. The AFCE run with seed 42 provides the checkpoint reported in Table 1 and achieves the highest overall success among these three runs. Overall and bimanual gains are positive for all three seeds; single-arm success decreases by 3.00 pp for seed 41.

W1 + Q3. GT-Effect replay versus closed-loop success. On the same 110 demonstration windows and initial states (H=30H=30), replay success is 67/110 (60.9%) for demonstrated actions, 21/110 (19.1%) for GT-Effects, and 10/110 (9.1%) for predicted Effects. The 25/110 result in Table 2 uses the Stage-I decoder snapshot; the 21/110 result uses its jointly adapted Stage-II snapshot. Effect replay follows the demonstration time steps and conditions decoding on the current robot state. By contrast, the 50.97% closed-loop result predicts Effects from current observations, executes an action prefix, and replans; demonstration GT-Effects may no longer align with the robot state once execution deviates from the demonstration. Replay is therefore not an estimate of autonomous policy success.

W4. Precision-sensitive regression. Pinch Tongs success is 5.33% for AFCE versus 6.00% for matched Direct-A at seed 42; across three training seeds, their means are 8.89% and 13.11%, respectively. The released π0.5\pi_{0.5} checkpoint achieves 43.33% under a different training protocol. In the seed-42 ablation, Effect-FM-only, frozen-decoder, and Full AFCE achieve 13.33%, 4.00%, and 5.33%, respectively, suggesting a possible precision trade-off associated with combined auxiliary supervision and decoder adaptation. Despite an Effect cosine similarity of 0.982, predicted-Effect action MAE is 0.0567 versus 0.0096 for GT-Effects (5.9×), with the largest gaps during approach and before contact (9.1×/9.6×). This highlights a precision limitation in action readout.

AuthorsOpenReview ↗

Response to Reviewer v7mT: Action Decoding

W1 + Q4. Action supervision and decoder adaptation. AFCE uses one action decoder, initialized from Stage I and jointly updated with the policy at 0.25× the policy learning rate. The GT-action loss (λGT=0.256\lambda_{\mathrm{GT}}=0.256) and finger-difference loss (λΔf=0.05\lambda_{\Delta f}=0.05) backpropagate through the decoder into the policy and are applied for flow times τ∈[0.2,0.7]\tau\in[0.2,0.7]. The same decoder is used at deployment; D0D_0 and D1D_1 in Appendix C denote its Stage-I and jointly adapted Stage-II snapshots for diagnostics.

ConfigurationGT-action + finger lossesDecoderOverall (%)Single-arm (%)Bimanual (%)
Effect-FM-onlyOffFrozen44.48 ± 1.9361.00 ± 2.4024.67 ± 2.66
Frozen decoderOnFrozen42.61 ± 1.1758.11 ± 1.9024.00 ± 1.06
Full AFCEOnUpdated50.97 ± 1.6562.33 ± 1.4537.33 ± 2.66

Without auxiliary losses, Effect-FM-only reaches 24.67% bimanual success, compared with 20.93% for matched Direct-A at training seed 42 (+3.73 pp). With the decoder frozen, enabling the losses alone does not improve this result (24.67% to 24.00%). Full AFCE jointly updates decoder and policy and reaches 37.33% (+13.33 pp over the frozen-decoder variant). The measured gain therefore supports the combined supervision/adaptation pathway, rather than establishing an independent benefit from either auxiliary loss. Each variant uses one training run (seed 42); values are mean ± sample SD across three evaluation environment seeds.

Thank you for your review. Our response addresses three points.

  1. Baseline and seeds. Both methods used 60k policy updates, with 30k additional tokenizer updates for AFCE. AFCE outperformed Direct-A at all three seeds, averaging gains of 9.80 percentage points overall and 14.84 on bimanual tasks.
  2. Replay. GT-Effect replay follows demonstration timing, while the closed-loop policy replans from current observations.
  3. Decoder and precision. At seed 42, updating the decoder raised bimanual success from 24.00% to 37.33%. Pinch Tongs remains a precision limitation.

With only a few hours left in the discussion, we would appreciate hearing whether these results address your concerns.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary Response to All Reviewers

We thank the reviewers for their constructive feedback. We added controlled experiments and clarified the training and evaluation protocols. Locations below refer to the revised manuscript.

Revisions to the Manuscript

PageLocationRevision
6Sec. 3.2Clarified that both action auxiliary losses update the same decoder and propagate gradients to the Effect policy; the same decoder is deployed.
7Sec. 4.1Distinguished the released π0.5\pi_{0.5} checkpoint from the retrained baseline (Direct-A). Specified matched policy-stage budgets, AFCE's additional 30k tokenizer updates, and seed 42 for the main AFCE results.
9Sec. 4.4; Table 3New: Three paired training seeds give 48.83 ± 1.92% overall for AFCE versus 39.03 ± 1.98% for Direct-A; paired improvement: 9.80 ± 2.66 pp.
9–10Sec. 4.5; Table 5New: Effect-FM-only, auxiliary losses with a frozen decoder, and joint optimization achieve 24.67%, 24.00%, and 37.33% bimanual success. This supports the combined supervision/adaptation pathway.
10Sec. 5Quantified the Pinch Tongs regression and refined the precision-control limitations.
16–18Appendix C; Tables C.1–C.2Specified diagnostic episodes 90–99 per task. Added a numerical readout table complementing existing Fig. C.3 and new Pinch Tongs contact-stage diagnostics; separated window-based and dense statistics.
18–19Appendix C; Table C.3New: Mixed versus single-arm-only training gives 62.33% versus 60.33% single-arm success and 5.33% versus 8.00% on Pinch Tongs.
21Appendix D.2Clarified default training seed and training-run versus evaluation-seed variability; corrected equation references.

Summary of Rebuttal Experiments and Analyses

The responses provide the following experiments and analyses, including expanded comparisons of existing results:

  • Matched baseline and training stability (v7mT Q2; wFxM Q2; XenN Q1). Direct-A and AFCE use matched initialization and 60k policy updates across seeds 41–43. AFCE improves mean overall success from 39.03% to 48.83% (+9.80 pp) and bimanual success from 16.36% to 31.20%. Overall/bimanual gains are positive in every run; single-arm success decreases by 3.00 pp for seed 41.
  • Action-only target control (wFxM Q2; XenN Q1). We clarify the existing A-only ablation and compare it with Full AFCE at seed 42: overall/single-arm/bimanual gains are +5.70/+5.11/+6.40 pp. We specify which evidence inputs and reconstruction objectives are removed and distinguish this single-run comparison from multi-seed robustness.
  • Auxiliary supervision and decoder adaptation (v7mT Q4). Effect-FM-only, auxiliary losses with a frozen decoder, and Full AFCE achieve 24.67%, 24.00%, and 37.33% bimanual success. Effect-FM-only also exceeds matched Direct-A at seed 42 (20.93%). Additional Pinch Tongs results are 13.33%, 4.00%, and 5.33%, respectively, revealing a possible precision trade-off (v7mT W4; wFxM Q1).
  • Training-mixture control (wFxM Q2). With the same initialization, training seed, and update budget, mixed versus single-arm-only training gives 62.33% versus 60.33% single-arm success and 5.33% versus 8.00% on Pinch Tongs. Removing bimanual data does not resolve the precision limitation.
  • Task-level execution replay (v7mT Q3; wFxM Q1; XenN W2). We report raw-action, GT-Effect/D1D_1, and predicted-Effect replay on the same 110 windows: 67/110, 21/110, and 10/110 successes, with per-task counts and action errors. Pinch Tongs gives 6/10, 0/10, and 0/10. These diagnostics are distinguished from Stage-I replay (25/110) and autonomous closed-loop evaluation.
  • Physical readout and contact precision (v7mT W4; wFxM Q1). We tabulate existing decoder-readout comparisons and add contact-stage analysis: predicted/GT action-MAE ratios reach 9.1× during approach and 9.6× before contact; a separate dense diagnostic gives contact-step TCP errors of 0.0313/0.0041. Matched Pinch Tongs comparisons also contextualize the regression: Direct-A/AFCE achieve 6.00%/5.33% at seed 42 and 13.11%/8.89% across three runs.

The added evidence supports consistent overall and bimanual gains across the evaluated training runs, while precision-sensitive manipulation remains unresolved.

Dear Reviewers,

Thank you all for your thoughtful feedback. With about a day left in the discussion period, we would appreciate it if each of you could take a look at our response to your review and let us know whether it addresses your concerns. We would be happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.