- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces Seeing Farther to Act Now (SFAN), a world-action model that decouples the visual prediction horizon from the action horizon for robot manipulation. Motivated by the trade-off between broader task context and fine-grained local detail, SFAN uses sparsely sampled long-horizon futures to generate a primary action chunk and densely sampled short-horizon futures to predict residual corrections conditioned on that chunk. An auxiliary waypoint objective further supervises long-range task progression. Experiments on RoboCasa, composite RoboCasa365 tasks, and two real-world manipulation tasks report improved overall success rates over the evaluated baselines, while ablations support the complementary roles of long-horizon guidance and short-horizon refinement.
Critical strengths
A compelling idea supported by coherent method design and empirical evidence. I find the paper’s central argument convincing: the future information needed for a good action decision can extend beyond the current action horizon. SFAN implements this idea through a complementary design, using sparse long-horizon predictions to guide primary action generation and dense short-horizon predictions to provide residual corrections for precise execution. The experiments support this design choice: combining both pathways achieves 69% success on RoboCasa, compared with 66% and 63% for the short- and long-horizon variants alone, with further improvements reported on composite and real-world tasks. This alignment between motivation, method design, and empirical results is a key strength supporting my positive assessment of the paper.
Critical weaknesses
1. Insufficient isolation of the contribution of temporal complementarity. Although the horizon and fusion ablations provide useful evidence, SFAN also introduces an additional action model, staged residual training, and auxiliary waypoint supervision. Section 3.4 further mentions additional trajectories for residual learning without clarifying their quantity or whether the comparison methods receive equivalent data. A matched control retaining the full architecture, supervision, and training budget while using short-horizon futures in both pathways would more directly establish the benefit of combining temporal scales. Without such a comparison, the relative contributions of longer future context and the additional refinement machinery remain unclear, limiting confidence in the paper’s central methodological claim.
2. Unclear control implications of the inference overhead. Table 4 reports 1.77 seconds per action prediction for SFAN, compared with 0.70 seconds for Cosmos-Policy. However, the paper does not specify the control frequency, the number of actions executed before replanning, or whether inference overlaps with execution. These details determine the effective feedback rate and the delay between observation and action. Although the authors acknowledge the computational overhead, the reported success rates do not establish how the methods compare under the same timing constraints. This limits the evidence for SFAN’s practical advantage in responsive closed-loop manipulation.
Questions for authors
Q1. Fairness of training data and supervision. Section 3.4 mentions additional trajectories used during residual training. Could the authors clarify the amount of additional data and whether the baselines and ablation variants use equivalent training data? This is important for determining whether the reported improvements can be attributed to the proposed method rather than additional supervision.
Q2. Isolating the benefit of multi-timescale modeling. Could the authors provide a capacity-matched comparison in which both pathways use short-horizon predictions, while retaining the same architecture, training strategy, and auxiliary supervision? Although the existing ablations are informative, such a comparison would more directly establish whether the improvement comes from complementary temporal horizons rather than the additional refinement module.
Q3. Inference latency and closed-loop execution. Table 4 reports an inference latency of 1.77 seconds per action prediction, substantially higher than Cosmos-Policy (0.70 seconds). Could the authors clarify the actual control frequency, execution horizon, and whether inference is performed asynchronously? How does this latency affect closed-loop performance in the real-world experiments?
Dear Reviewers,
Please take time to review the responses from the authors and engage with them for further questions and comments if any.
Time left is not much and please do shortly.
Thanks
AC