acceptodds
Withdrawn from ICLR 2027

FocusWAM: Adaptive Multi-View Future Prediction for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

11 views

Abstract

World action models (WAMs) use future prediction to guide robot manipulation, but the views needed to anticipate an interaction vary during task execution. Global prediction can miss fine-grained gripper–object dynamics, while predicting all views at every step incurs additional computational cost even when local details contribute little to control. We introduce FocusWAM, a world action model that uses global future features to select when to predict wrist views. It first anticipates scene-level dynamics and determines whether to activate wrist-view prediction for the upcoming action chunk. When activated, the local stream predicts wrist-view dynamics conditioned on global features. The resulting local features refine the global future representation. We supervise the refined representation with a global future prediction objective and use the global and local features to condition action generation. FocusWAM achieves 55.0% average success on RoboCasa GR1 and 50.40% on the randomized split of RoboTwin 2.0 after training on clean demonstrations only, together with 76.50% across four real-world tasks. Across the clean and randomized RoboTwin evaluations, FocusWAM improves average success over an independently trained always-on local policy from 65.0% to 68.3% while achieving a 1.50 inference speedup.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot, WAM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KzaYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FocusWAM focuses on the cost of predicting every camera view at each step of robotic manipulation. It keeps a global prediction stream active and selectively activates wrist-view prediction based on anticipated interactions. When activated, global features guide local prediction, while local features refine the global representation before action generation. Experiments on RoboCasa, RoboTwin, LIBERO, and real robots report improved task success and inference efficiency. Same-checkpoint comparisons further show that adaptive activation achieves a better success-latency trade-off than always-on local prediction.

Critical strengths

The main design choices show measurable benefits in the trained policy. On RoboTwin, adaptive activation improves success (68.3% vs. 65.0%) while reducing latency (177.8 vs. 267.5 ms). Disabling either cross-view connection also lowers success.

Critical weaknesses
  1. The same-checkpoint ablations limit comparisons with alternative designs. Tables 2-3 use the same trained checkpoint and only change the cross-view connections or local invocation rule at inference. During label-guided branch sampling, the local stream is enabled for 90% of positive-label chunks but only 10% of negative-label chunks (Appendix B.2). Thus, the lower success of the always-on local configuration may partly reflect this interaction-weighted training distribution. Similarly, bypassing global-to-local guidance or local-to-global refinement tests the trained policy's dependence on these modules, not their advantages over separately trained alternatives.
  2. The specific benefits of future prediction remain unclear. Table A3 compares current global image features from the frozen VAE encoder with global future features from the task-adapted, language-conditioned Video DiT. Thus, the improvement in local activation cannot be attributed to future prediction alone. Table 2 also bypasses local-to-global refinement at inference without isolating its auxiliary global future prediction objective LrL_r. These results do not isolate the benefits of future prediction for local activation or the added value of auxiliary future-prediction supervision for local-to-global refinement.
Questions for authors
  1. Can you train a separate always-on local model using the same data, supervision, and number of training updates? Does FocusWAM still achieve higher success than this retrained model?
  2. Can you compare the two gate inputs in Table A3 and periodic activation at the same local invocation rate? How do their success rates and latencies compare?
  3. Can you retrain FocusWAM without LrL_r while preserving the refinement? Please report task success and, where comparable, global future PSNR/SSIM.
  4. Does Table 4 use the same flow-prediction head and video-sampling procedure for global predictions with and without refinement? If not, how is the comparison controlled?
2 comments

We thank the reviewer for the concrete suggestions on independent training, invocation-rate matching, and refinement supervision. We address your four questions below.

Critical Weakness 1 and Question 1: Independently trained invocation-policy and structural ablations

The revised manuscript summarizes these independently trained results in Section 4.3 (page 8, lines 388–390 and 408–409). Appendix C.1 gives the training setup and Tables A8–A9 (pages 24–25, lines 1268–1298).

After independent training, always-on achieves 65.6%, compared with 68.3% for full FocusWAM, which also has lower latency. We separately retrained the ablated models in the original Tables 2 and 3, matching the full model's base initialization, training data, and update budget. Always-on enables local prediction throughout training and evaluation and retains the same global and local future-prediction supervision and refinement auxiliary objective. Each structural variant uses its specified architecture from the start of training. Removing refinement also removes Lr\mathcal L_r. Full FocusWAM uses the original reference model under the same training and evaluation protocol.

Corresponding to the original Table 2: Independently trained bidirectional-interaction ablations.

\hline \textbf{Variant} & \textbf{Guidance} & \textbf{Refinement} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} \cr \hline \text{w/o both} & \text{--} & \text{--} & 82.0 & 44.8 & 63.4 \cr \text{w/o global guidance} & \text{--} & \text{Yes} & 84.0 & 47.2 & 65.6 \cr \begin{array}{l}\text{w/o local-to-global} \cr \text{refinement}\end{array} & \text{Yes} & \text{--} & 84.6 & 47.4 & 66.0 \cr \textbf{FocusWAM} & \textbf{Yes} & \textbf{Yes} & 86.2 & 50.4 & 68.3 \cr \hline \end{array}$$ Both directions contribute under matched training budgets, and their combination achieves the highest success. **Corresponding to the original Table 3: Independently trained invocation-policy ablations.** $$\begin{array}{llrrrr} \hline \textbf{Variant} & \textbf{Local activation} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \textbf{Latency (ms)} \cr \hline \text{Global-only} & \text{Off} & 80.0 & 42.4 & 61.2 & 172.3 \cr \text{Always-on local} & \text{Always on} & 84.4 & 46.8 & 65.6 & 260.4 \cr \textbf{FocusWAM} & \textbf{Predicted} & 86.2 & 50.4 & 68.3 & 177.8 \cr \hline \end{array}$$ **FocusWAM exceeds the independently trained always-on model by 2.7 percentage points, while reducing mean latency from 260.4 to 177.8 ms.** The advantage therefore remains when always-on uses the same invocation rule during training and evaluation. ## Critical Weakness 2 and Question 2: Gate inputs and periodic invocation at the same invocation rate The revised manuscript summarizes this comparison in Section 4.3 (page 8, lines 409–410). Appendix C.3 gives the setup, Table A11, and analysis (page 25, lines 1318–1336). **We compared both gate inputs and periodic invocation with fixed policy weights and the same invocation rate (Active = 10.3%).** The two gates use the same classifier architecture and training setup, differing only in their input representations. The current-image gate threshold is calibrated on validation data and frozen during testing. Periodic invocation uses a fixed schedule. $$\begin{array}{lrrrrr} \hline \begin{array}{l}\textbf{Invocation rule /} \cr \textbf{input}\end{array} & \textbf{Active (\\%)} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \textbf{Latency (ms)} \cr \hline \text{Current-image gate} & 10.3 & 81.0 & 43.4 & 62.2 & 179.3 \cr \begin{array}{l}\textbf{Future-feature gate} \cr \textbf{(FocusWAM)}\end{array} & 10.3 & 86.2 & 50.4 & 68.3 & 177.8 \cr \text{Periodic invocation} & 10.3 & 80.6 & 43.0 & 61.8 & 178.8 \cr \hline \end{array}$$ **The future-feature gate exceeds the current-image gate by 6.1 percentage points and periodic invocation by 6.5 percentage points.** Mean latencies vary slightly from 177.8 to 179.3 ms but remain comparable. With the policy fixed and the invocation rate matched, the periodic control supports the value of state-dependent invocation timing. The two gates compare the complete representation interfaces in the original paper. The current-image gate reads frozen VAE observation features, whereas the future-feature gate reads task-adapted, language-conditioned Video DiT features. This comparison supports the complete future-feature interface, rather than isolating future-prediction supervision. Question 3 separately tests the refinement prediction objective.

Critical Weakness 2 and Question 3: Contribution of Lr\mathcal L_r with refinement retained

We have added the result with refinement retained and the auxiliary loss removed to the Local-to-global refinement paragraph in Section 4.3 of the revised manuscript (page 8, lines 380–381). Appendix C.4 adds the complete setup, analysis, and Table A12 (pages 25–26, lines 1340–1357). The supervision of the shared prediction head is clarified in Section 3.3 (page 5, lines 262–264) and Appendix B.3 (page 22, lines 1167–1168).

With the refinement architecture held constant, the full model with Lr\mathcal L_r achieves 68.3% average success, compared with 66.8% after retraining without this loss, a difference of 1.5 percentage points. Following your suggestion, the new ablation sets only λr=0\lambda_r=0, keeping the same base initialization, remaining training objectives, action paths, training data, and update budget as the full model. The full model uses λr=0.5\lambda_r=0.5. This ablation retains the original global and local future-prediction objectives Lg\mathcal L_g and Lw\mathcal L_w, the refinement architecture, and action supervision, removing only Lr\mathcal L_r. The separate no-refinement control removes both the module and this loss.

Within each model, global future predictions before and after refinement share the same flow-prediction head. Lg\mathcal L_g supervises predictions from the original global features, while Lr\mathcal L_r provides additional supervision on predictions from the refined features. Thus, when λr=0\lambda_r=0, the head continues to be trained by Lg\mathcal L_g. The two rows retaining refinement below evaluate the updated global features, while the no-refinement row evaluates global features without this update.

\hline \textbf{Variant} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \begin{array}{l}\textbf{Global PSNR} \cr \textbf{(dB)}\end{array} & \textbf{Global SSIM} \cr \hline \textbf{Full FocusWAM} & 86.2 & 50.4 & 68.3 & 22.58 & 0.817 \cr \text{w/ refinement, w/o}\\,\mathcal L_r & 85.4 & 48.2 & 66.8 & 21.90 & 0.796 \cr \begin{array}{l}\text{w/o local-to-global} \cr \text{refinement (independently} \cr \text{trained)}\end{array} & 84.6 & 47.4 & 66.0 & 21.86 & 0.793 \cr \hline \end{array}$$ **The first two rows compare models with the same refinement architecture, differing only in auxiliary predictive supervision.** Including $\mathcal L_r$ improves Clean and Rand. success by 0.8 and 2.2 percentage points, respectively, global future PSNR by 0.68 dB, and SSIM by 0.021. With the refinement architecture held constant, auxiliary predictive supervision improves both task success and global future prediction quality. ## Question 4: Flow-prediction head and video-sampling controls in Table 4 We have added a sentence at the end of the Table 4 caption in the revised manuscript, clarifying that both global-future entries use the same prediction head, video-sampling procedure, VAE, and metric computation (page 8, lines 420–422). The two global-future rows in the original Table 4 use the same trained model's global flow-prediction head, with the same video-sampling procedure, VAE encoder and decoder, and PSNR/SSIM computation.
Reviewer_RWZTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FocusWAM, a world action model that adaptively decides when to predict wrist-view futures. The main idea is to always run global (head-view) future prediction and use its features to gate whether wrist-view prediction is needed for the upcoming action chunk. When activated, the wrist stream is conditioned on global features, and its features in turn refine the global representation before action generation. The paper reports improvements on RoboCasa GR1, RoboTwin 2.0, and real-world experiments.

Critical strengths
  • The paper is generally well-written and easy to follow.
  • The paper deals with an underexplored apporach to improve the efficiecny of WAM inference and looks interesting.
  • The method that wrist views are mainly useful around grasping/placement and less so during transport is intuitive
  • Experiments are conducted under multiple benchmarks and real-world tasks.
Critical weaknesses

I'm slightly negative on the paper because the key claims rely on inference-time ablations of a single checkpoint. Specifically:

  • All variants in Tables 2 and 3 use the same checkpoint, which is trained with label-guided branch sampling. So if I understand correctly, The always-on setting is thus largely out-of-distribution from its training setup, and its lower performance may come from this mismatch rather than from a real benefit of selective prediction. I think separately trained global-only / always-on / without refinement variants should be compared.
  • DiT4DiT, the base model, is missing from the RoboTwin table. It is also unclear whether the gains over DiT4DiT come from the proposed components.
  • Real-world gains over DiT4DiT are 1–4 successes out of 50 per task, which looks quite marginal and likely within statistical noise.
Questions for authors

Please see my weaknesses

2 comments

We thank the reviewer for recognizing the efficiency motivation and evaluation across benchmarks. We address independently trained controls, the DiT4DiT baseline, and repeated real-world evaluation below.

Critical Weakness 1: Independently trained structural and invocation-policy ablations

We have added the independently trained structural ablation and always-on results to Section 4.3 of the revised manuscript (page 8, lines 388–390 and 408–409). Appendix C.1 adds the complete training setup, Table A8 for the structural variants corresponding to original Table 2, and Table A9 for the invocation policies corresponding to original Table 3 (pages 24–25, lines 1268–1298).

The independently trained always-on model achieves 65.6%, compared with 68.3% for full FocusWAM. Following your suggestion, we separately retrained the ablated models in the original Tables 2 and 3, matching the full model's base initialization, training data, and update budget. Structural variants use their respective architectures throughout training. Removing local-to-global refinement also removes its auxiliary loss Lr\mathcal L_r. Global-only does not train a local branch. Always-on enables local prediction during both training and evaluation and retains the same predictive supervision. Full FocusWAM uses the original reference model under the same training and evaluation protocol.

Corresponding to the original Table 2: Independently trained bidirectional-interaction ablations.

\hline \textbf{Variant} & \textbf{Guidance} & \textbf{Refinement} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} \cr \hline \text{w/o both} & \text{--} & \text{--} & 82.0 & 44.8 & 63.4 \cr \text{w/o global guidance} & \text{--} & \text{Yes} & 84.0 & 47.2 & 65.6 \cr \begin{array}{l}\text{w/o local-to-global} \cr \text{refinement}\end{array} & \text{Yes} & \text{--} & 84.6 & 47.4 & 66.0 \cr \textbf{FocusWAM} & \textbf{Yes} & \textbf{Yes} & 86.2 & 50.4 & 68.3 \cr \hline \end{array}$$ Removing global-to-local guidance, local-to-global refinement, or both connections yields 65.6%, 66.0%, and 63.4% average success, respectively, compared with 68.3% for the full model. These independently trained results support both directions within the evaluated setup, with their combination achieving the highest success. **Corresponding to the original Table 3: Independently trained invocation-policy ablations.** $$\begin{array}{llrrrr} \hline \textbf{Variant} & \textbf{Local activation} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \textbf{Latency (ms)} \cr \hline \text{Global-only} & \text{Off} & 80.0 & 42.4 & 61.2 & 172.3 \cr \text{Always-on local} & \text{Always on} & 84.4 & 46.8 & 65.6 & 260.4 \cr \textbf{FocusWAM} & \textbf{Predicted} & 86.2 & 50.4 & 68.3 & 177.8 \cr \hline \end{array}$$ **FocusWAM exceeds the independently trained always-on model by 2.7 percentage points, with 31.7% lower mean latency.** Always-on enables local prediction throughout both training and evaluation, directly addressing whether its lower success arose solely from forcing the branch on at inference. FocusWAM improves average success over global-only by 7.1 percentage points. ## Critical Weakness 2: Missing DiT4DiT baseline on RoboTwin and attribution of the gains We have added the controlled DiT4DiT comparison summary to the RoboTwin 2.0 paragraph in Section 4.2 of the revised manuscript (page 7, lines 339–340). Appendix C.2 adds the complete setup, Table A10, and the result analysis (page 25, lines 1302–1314). **We added a RoboTwin 2.0 DiT4DiT comparison under matched training and evaluation conditions.** Both methods use the same clean training split, camera inputs, image preprocessing, action interface, and training update budget, with an action horizon of 16 and four Action DiT sampling steps. Each task is evaluated for 100 episodes under each of the Clean and Rand. conditions. DiT4DiT retains its original architecture. $$\begin{array}{lrrr} \hline \textbf{Method} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} \cr \hline \text{DiT4DiT} & 82.0 & 45.2 & 63.6 \cr \textbf{FocusWAM} & 86.2 & 50.4 & 68.3 \cr \hline \end{array}$$ **FocusWAM improves average success by 4.7 percentage points**, with gains of 4.2 and 5.2 percentage points under Clean and Rand., respectively. This comparison shows higher observed success than the base model under the matched protocol. The preceding independently trained ablations further examine the roles of local prediction and refinement.

Critical Weakness 3: Reliability of the real-world improvement

We have clarified in Section 4.4 of the revised manuscript that the original table reports the initial evaluation batch (page 9, lines 452–453), and added the cumulative evaluation summary at the end of that section (page 10, lines 493–495). Appendix C.5, Tables A13 and A14, adds the new batch, cumulative results, and analysis (page 26, lines 1361–1383).

We conducted an additional 50 trials per task for both DiT4DiT and FocusWAM on all four tasks, bringing each method to 100 trials per task. Results for the additional batch are:

\hline \textbf{Method} & \begin{array}{l}\textbf{Knife} \cr \textbf{placement}\end{array} & \begin{array}{l}\textbf{Sausage} \cr \textbf{cutting}\end{array} & \textbf{Seasoning} & \textbf{Stirring} & \begin{array}{l}\textbf{Total} \cr \textbf{successes}\end{array} & \begin{array}{l}\textbf{Overall} \cr \textbf{success}\end{array} \cr \hline \text{DiT4DiT} & 43/50 & 26/50 & 37/50 & 34/50 & 140/200 & 70.00\\% \cr \textbf{FocusWAM} & 46/50 & 28/50 & 41/50 & 37/50 & 152/200 & 76.00\\% \cr \hline \end{array}$$ FocusWAM completes 12 more trials successfully in the additional batch. In the original batch, FocusWAM achieved 153/200 versus 143/200 for DiT4DiT, completing 10 more trials successfully. Combining both batches gives: $$\begin{array}{lrrrrrr} \hline \textbf{Method} & \begin{array}{l}\textbf{Knife} \cr \textbf{placement}\end{array} & \begin{array}{l}\textbf{Sausage} \cr \textbf{cutting}\end{array} & \textbf{Seasoning} & \textbf{Stirring} & \begin{array}{l}\textbf{Cumulative} \cr \textbf{successes}\end{array} & \begin{array}{l}\textbf{Overall} \cr \textbf{success}\end{array} \cr \hline \text{DiT4DiT} & 86/100 & 53/100 & 74/100 & 70/100 & 283/400 & 70.75\\% \cr \textbf{FocusWAM} & 92/100 & 57/100 & 82/100 & 74/100 & 305/400 & 76.25\\% \cr \hline \end{array}$$ **With evaluation expanded to 100 trials per task, FocusWAM completes 22 more trials successfully overall, with a 5.5 percentage point gain.** The cumulative task-level differences are 6, 4, 8, and 4 successes, with improvements on all four tasks in both batches. The additional evaluation reproduces the direction of the original improvement in another batch, providing further evidence for the real-world results.
Reviewer_TomQOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Recent World Action Models (WAMs) for robot manipulation use predictive visual representations to generate appropriate robot actions by forecasting future visual representations. Since robots commonly have multiple cameras, such as head-mounted and wrist-mounted cameras, recent WAMs commonly use multi-view information to generate robot actions. However, the views needed to anticipate an interaction can vary during task execution. For example, the wrist view usually helps reveal nearby geometry, but sometimes provides little information because of self-occlusion or the robot gripper’s fast movements. Thus, this paper raises the question of whether the wrist-view camera needs to be active at all times. Following this idea, the paper proposes a new WAM, named FocusWAM, which adaptively activates wrist-view prediction to refine the global future representation from the main visual stream (the head view) for robot manipulation. FocusWAM achieves a 55.0% average success rate on RoboCasa GR1 and a 76.50% success rate across four real-world tasks. Moreover, interestingly, FocusWAM improves the average success rate over always-on local prediction using the same checkpoint, from 65.0% to 68.3%, while achieving a 1.50× inference speedup.

Critical strengths

The proposed idea of adaptively using wrist-view prediction to refine the global future representation seems reasonable, and its effectiveness is well supported by experiments.

Critical weaknesses

This reviewer’s main concern is that selectively activating cameras only when their observations are useful is a well-established idea in active perception. Given prior work on active camera selection [1] and task-relevant active perception for robotic manipulation [2], the core idea of activating the wrist camera only when needed appears relatively intuitive. Therefore, the technical novelty of this paper seems limited.

References

[1] Wang, Boyu, Lihan Huang, and Minh Hoai. “Active Vision for Early Recognition of Human Actions.” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020.

[2] Xiong, Haoyu, et al. “Vision in Action: Learning Active Perception from Human Demonstrations.” 9th Annual Conference on Robot Learning.

Questions for authors
  1. How does the proposed method fundamentally differ from existing active perception and camera selection approaches, beyond applying selective camera activation to a world-action model? This reviewer thinks that simply adopting an idea from another related field and applying it to WAMs may not be a sufficient contribution to provide new insights to the ML community.

  2. The experimental comparison between FocusWAM and always-on local prediction is interesting because the proposed model achieves higher accuracy with lower latency. Could you provide further analysis of this result, including why and how this improvement occurs?

2 comments

We thank the reviewer for recognizing the motivation and experimental results of adaptive wrist-view future prediction. We have added citations to the two works you identified, Active Vision for Early Recognition of Human Actions and Vision in Action: Learning Active Perception from Human Demonstrations, in Section 2, Related Work, of the revised manuscript (page 3, lines 132–135), together with a comparison to FocusWAM. We address the methodological novelty and the success–latency results below.

Critical Weakness 1 and Question 1: Relationship to active perception and camera selection

FocusWAM studies how to construct and integrate multi-view future representations for action generation on demand, through adaptive local prediction, global-to-local prediction, and local-to-global refinement. We agree that using visual information according to task needs is an established principle in active perception. FocusWAM addresses a further question within WAMs: global futures provide scene and task context, while wrist futures provide local interaction details. How should these predictions inform each other and contribute to control during relevant action chunks?

Active Vision for Early Recognition of Human Actions selects camera observations under bandwidth and processing constraints to recognize human actions early. Vision in Action learns to control a head camera's viewpoint from human demonstrations to acquire task-relevant observations for manipulation. FocusWAM studies how to allocate future-prediction computation within a WAM: it invokes wrist prediction when needed, uses global predictive features to guide local prediction, and feeds local predictive features back to refine the global representation before action generation. The distinction therefore concerns both when to compute local futures and how global and local predictions interact to support control.

ViA also observes that adding wrist observations does not improve performance in its setting, illustrating that the benefit of local views depends on the perception and task conditions. FocusWAM further studies how to construct and exchange future representations on demand within a WAM. The policy has no separate camera-viewpoint control action. Selection applies to wrist future prediction and subsequent refinement, with two directions of interaction:

  1. Global-to-local prediction: Intermediate features from the global prediction stream condition wrist future prediction, supplying scene and target relationships beyond the wrist camera's limited field of view.
  2. Local-to-global refinement: Local predictive features update the global representation before action generation. An auxiliary global future-prediction objective supervises this update to incorporate local interaction details into the action-conditioning representation.

The distinction therefore also concerns how global context guides local futures and how predictive details refine the global representation used for control. Action generation reads hidden features trained through future prediction, without first generating and decoding complete future videos. This design links invocation timing to the use of predictive information: global context constrains local futures, and local interaction details then update the global future representation used for action generation. The original Tables 2 and 4 evaluate these connections through task success and future-prediction quality, respectively.

Question 2: Why does adaptive prediction improve success while reducing latency?

We have added the independently trained always-on comparison to the Adaptive view prediction paragraph in Section 4.3 of the revised manuscript (page 8, lines 408–409). Appendix C.1 describes the training setup (lines 1268–1273), while Table A9 and the following analysis report the complete invocation-policy results and phase-related interpretation (pages 24–25, lines 1287–1298).

The original Table 3 reports:

\hline \begin{array}{l}\textbf{Inference} \cr \textbf{configuration}\end{array} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \textbf{Active (\\%)} & \textbf{Latency (ms)} \cr \hline \text{Always-on local} & 83.6 & 46.4 & 65.0 & 100.0 & 267.5 \cr \textbf{FocusWAM adaptive} & 86.2 & 50.4 & 68.3 & 10.3 & 177.8 \cr \hline \end{array}$$ **The computational saving comes from fewer executions of local prediction and refinement.** The global stream remains active, while the local path runs for 10.3% of action chunks. Mean model inference latency falls from 267.5 to 177.8 ms, a 33.5% reduction and approximately 1.50× speedup. An independently trained always-on model, with local prediction enabled during both training and evaluation, achieves 65.6% at 260.4 ms, compared with 68.3% at 177.8 ms for FocusWAM. The advantage remains when the control uses an invocation rule consistent with its training. **We believe the success improvement reflects the changing relevance of wrist predictions across manipulation phases.** Near grasping or placement, the wrist view provides fine object–gripper geometry. During transport, object occlusion reduces the scene information visible from the wrist, while the global view retains object–target relationships. Invoking local prediction according to anticipated interactions concentrates its contribution to refinement on relevant action chunks. In the original paper, always-on achieves 65.0%, exceeding global-only at 59.6%, which supports the complementary value of local information. Adaptive invocation further achieves 68.3%, providing a better success–latency combination for this trained policy. This result is consistent with the interpretation based on phase-dependent information needs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.