- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
FocusWAM focuses on the cost of predicting every camera view at each step of robotic manipulation. It keeps a global prediction stream active and selectively activates wrist-view prediction based on anticipated interactions. When activated, global features guide local prediction, while local features refine the global representation before action generation. Experiments on RoboCasa, RoboTwin, LIBERO, and real robots report improved task success and inference efficiency. Same-checkpoint comparisons further show that adaptive activation achieves a better success-latency trade-off than always-on local prediction.
Critical strengths
The main design choices show measurable benefits in the trained policy. On RoboTwin, adaptive activation improves success (68.3% vs. 65.0%) while reducing latency (177.8 vs. 267.5 ms). Disabling either cross-view connection also lowers success.
Critical weaknesses
- The same-checkpoint ablations limit comparisons with alternative designs. Tables 2-3 use the same trained checkpoint and only change the cross-view connections or local invocation rule at inference. During label-guided branch sampling, the local stream is enabled for 90% of positive-label chunks but only 10% of negative-label chunks (Appendix B.2). Thus, the lower success of the always-on local configuration may partly reflect this interaction-weighted training distribution. Similarly, bypassing global-to-local guidance or local-to-global refinement tests the trained policy's dependence on these modules, not their advantages over separately trained alternatives.
- The specific benefits of future prediction remain unclear. Table A3 compares current global image features from the frozen VAE encoder with global future features from the task-adapted, language-conditioned Video DiT. Thus, the improvement in local activation cannot be attributed to future prediction alone. Table 2 also bypasses local-to-global refinement at inference without isolating its auxiliary global future prediction objective . These results do not isolate the benefits of future prediction for local activation or the added value of auxiliary future-prediction supervision for local-to-global refinement.
Questions for authors
- Can you train a separate always-on local model using the same data, supervision, and number of training updates? Does FocusWAM still achieve higher success than this retrained model?
- Can you compare the two gate inputs in Table A3 and periodic activation at the same local invocation rate? How do their success rates and latencies compare?
- Can you retrain FocusWAM without while preserving the refinement? Please report task success and, where comparable, global future PSNR/SSIM.
- Does Table 4 use the same flow-prediction head and video-sampling procedure for global predictions with and without refinement? If not, how is the comparison controlled?
2 comments
Critical Weakness 2 and Question 3: Contribution of with refinement retained
We have added the result with refinement retained and the auxiliary loss removed to the Local-to-global refinement paragraph in Section 4.3 of the revised manuscript (page 8, lines 380–381). Appendix C.4 adds the complete setup, analysis, and Table A12 (pages 25–26, lines 1340–1357). The supervision of the shared prediction head is clarified in Section 3.3 (page 5, lines 262–264) and Appendix B.3 (page 22, lines 1167–1168).
With the refinement architecture held constant, the full model with achieves 68.3% average success, compared with 66.8% after retraining without this loss, a difference of 1.5 percentage points. Following your suggestion, the new ablation sets only , keeping the same base initialization, remaining training objectives, action paths, training data, and update budget as the full model. The full model uses . This ablation retains the original global and local future-prediction objectives and , the refinement architecture, and action supervision, removing only . The separate no-refinement control removes both the module and this loss.
Within each model, global future predictions before and after refinement share the same flow-prediction head. supervises predictions from the original global features, while provides additional supervision on predictions from the refined features. Thus, when , the head continues to be trained by . The two rows retaining refinement below evaluate the updated global features, while the no-refinement row evaluates global features without this update.
\hline \textbf{Variant} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \begin{array}{l}\textbf{Global PSNR} \cr \textbf{(dB)}\end{array} & \textbf{Global SSIM} \cr \hline \textbf{Full FocusWAM} & 86.2 & 50.4 & 68.3 & 22.58 & 0.817 \cr \text{w/ refinement, w/o}\\,\mathcal L_r & 85.4 & 48.2 & 66.8 & 21.90 & 0.796 \cr \begin{array}{l}\text{w/o local-to-global} \cr \text{refinement (independently} \cr \text{trained)}\end{array} & 84.6 & 47.4 & 66.0 & 21.86 & 0.793 \cr \hline \end{array}$$ **The first two rows compare models with the same refinement architecture, differing only in auxiliary predictive supervision.** Including $\mathcal L_r$ improves Clean and Rand. success by 0.8 and 2.2 percentage points, respectively, global future PSNR by 0.68 dB, and SSIM by 0.021. With the refinement architecture held constant, auxiliary predictive supervision improves both task success and global future prediction quality. ## Question 4: Flow-prediction head and video-sampling controls in Table 4 We have added a sentence at the end of the Table 4 caption in the revised manuscript, clarifying that both global-future entries use the same prediction head, video-sampling procedure, VAE, and metric computation (page 8, lines 420–422). The two global-future rows in the original Table 4 use the same trained model's global flow-prediction head, with the same video-sampling procedure, VAE encoder and decoder, and PSNR/SSIM computation.
We thank the reviewer for the concrete suggestions on independent training, invocation-rate matching, and refinement supervision. We address your four questions below.
Critical Weakness 1 and Question 1: Independently trained invocation-policy and structural ablations
The revised manuscript summarizes these independently trained results in Section 4.3 (page 8, lines 388–390 and 408–409). Appendix C.1 gives the training setup and Tables A8–A9 (pages 24–25, lines 1268–1298).
After independent training, always-on achieves 65.6%, compared with 68.3% for full FocusWAM, which also has lower latency. We separately retrained the ablated models in the original Tables 2 and 3, matching the full model's base initialization, training data, and update budget. Always-on enables local prediction throughout training and evaluation and retains the same global and local future-prediction supervision and refinement auxiliary objective. Each structural variant uses its specified architecture from the start of training. Removing refinement also removes Lr. Full FocusWAM uses the original reference model under the same training and evaluation protocol.
Corresponding to the original Table 2: Independently trained bidirectional-interaction ablations.
\hline \textbf{Variant} & \textbf{Guidance} & \textbf{Refinement} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} \cr \hline \text{w/o both} & \text{--} & \text{--} & 82.0 & 44.8 & 63.4 \cr \text{w/o global guidance} & \text{--} & \text{Yes} & 84.0 & 47.2 & 65.6 \cr \begin{array}{l}\text{w/o local-to-global} \cr \text{refinement}\end{array} & \text{Yes} & \text{--} & 84.6 & 47.4 & 66.0 \cr \textbf{FocusWAM} & \textbf{Yes} & \textbf{Yes} & 86.2 & 50.4 & 68.3 \cr \hline \end{array}$$ Both directions contribute under matched training budgets, and their combination achieves the highest success. **Corresponding to the original Table 3: Independently trained invocation-policy ablations.** $$\begin{array}{llrrrr} \hline \textbf{Variant} & \textbf{Local activation} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \textbf{Latency (ms)} \cr \hline \text{Global-only} & \text{Off} & 80.0 & 42.4 & 61.2 & 172.3 \cr \text{Always-on local} & \text{Always on} & 84.4 & 46.8 & 65.6 & 260.4 \cr \textbf{FocusWAM} & \textbf{Predicted} & 86.2 & 50.4 & 68.3 & 177.8 \cr \hline \end{array}$$ **FocusWAM exceeds the independently trained always-on model by 2.7 percentage points, while reducing mean latency from 260.4 to 177.8 ms.** The advantage therefore remains when always-on uses the same invocation rule during training and evaluation. ## Critical Weakness 2 and Question 2: Gate inputs and periodic invocation at the same invocation rate The revised manuscript summarizes this comparison in Section 4.3 (page 8, lines 409–410). Appendix C.3 gives the setup, Table A11, and analysis (page 25, lines 1318–1336). **We compared both gate inputs and periodic invocation with fixed policy weights and the same invocation rate (Active = 10.3%).** The two gates use the same classifier architecture and training setup, differing only in their input representations. The current-image gate threshold is calibrated on validation data and frozen during testing. Periodic invocation uses a fixed schedule. $$\begin{array}{lrrrrr} \hline \begin{array}{l}\textbf{Invocation rule /} \cr \textbf{input}\end{array} & \textbf{Active (\\%)} & \textbf{Clean (\\%)} & \textbf{Rand. (\\%)} & \textbf{Average (\\%)} & \textbf{Latency (ms)} \cr \hline \text{Current-image gate} & 10.3 & 81.0 & 43.4 & 62.2 & 179.3 \cr \begin{array}{l}\textbf{Future-feature gate} \cr \textbf{(FocusWAM)}\end{array} & 10.3 & 86.2 & 50.4 & 68.3 & 177.8 \cr \text{Periodic invocation} & 10.3 & 80.6 & 43.0 & 61.8 & 178.8 \cr \hline \end{array}$$ **The future-feature gate exceeds the current-image gate by 6.1 percentage points and periodic invocation by 6.5 percentage points.** Mean latencies vary slightly from 177.8 to 179.3 ms but remain comparable. With the policy fixed and the invocation rate matched, the periodic control supports the value of state-dependent invocation timing. The two gates compare the complete representation interfaces in the original paper. The current-image gate reads frozen VAE observation features, whereas the future-feature gate reads task-adapted, language-conditioned Video DiT features. This comparison supports the complete future-feature interface, rather than isolating future-prediction supervision. Question 3 separately tests the refinement prediction objective.