- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies Late-Bound Robotic Manipulation and introduces MOSAIC, a benchmark, and ReWAM, a selective-memory model built upon LingBot-VA. MOSAIC uses shared interaction histories and execution-start states to evaluate how models utilize historical information under different subsequent instructions. ReWAM constructs learnable compact register representations for historical video chunks and selectively accesses detailed historical information through instruction-conditioned retrieval to reduce historical attention costs during prediction.
On RMBench, ReWAM achieves a mean success rate of 50.57%, compared with 55.00% for Dense and 44.71% for Local-A. On the three-target and six-target Store and Retrieve Blocks configurations of MOSAIC, it achieves 93% and 46% success. At 256 historical chunks, ReWAM also achieves a 3.06× model-side inference speedup over Dense.
Critical strengths
S1. Meaningful task formulation and benchmark design.
MOSAIC focuses on the practically relevant late-bound instruction setting in long-term robotic memory. Its shared-history protocol fixes the interaction history and execution-start state across different instructions, providing a more controlled setting for evaluating instruction-dependent use of historical information.
S2. Substantial and well-measured inference efficiency gains with long histories.
At 256 historical chunks, ReWAM reduces model-side inference latency from 27.47 s to 8.99 s. The measurements include denoising, retrieval, KV gathering, and intermediate cache operations, providing direct experimental support for the computational efficiency benefits of selective historical access.
Critical weaknesses
W1. Insufficient specification of the learning mechanism and validation of register-guided retrieval.
The paper describes how registers are constructed and used, and states that the original video and action prediction objectives are retained. However, the actual optimization process for the newly introduced registers is not sufficiently specified and requires further clarification.
Furthermore, the experiments primarily evaluate final task success without directly verifying whether the retrieval mechanism selects historical evidence relevant to subsequent instructions. The comparison between ReWAM and Local-A also changes both the historical representations and detailed-history access patterns, making it difficult to isolate the independent contribution of instruction-conditioned retrieval.
These limitations affect the reproducibility of the core method and leave the actual contribution of register-guided retrieval and the reasons for its performance variations insufficiently understood.
W2. Insufficient experimental coverage to substantiate the MOSAIC benchmark contribution.
MOSAIC is presented as a benchmark comprising seven task families and eight configurations, yet the paper reports manipulation results only for the three-target and six-target configurations of Store and Retrieve Blocks. The remaining six families are described and illustrated, but no evaluation results are reported for them.
Without evaluation results for the remaining six families, it is unclear how challenging these tasks are or whether they can effectively assess the intended memory capabilities and distinguish between different models. This lack of empirical evidence weakens MOSAIC's contribution as a benchmark.
Questions for authors
Q1. Could the authors clarify how the newly introduced register-based memory and retrieval mechanism is trained in practice, beyond stating that the original video and action prediction objectives are retained? Please explain how the register representations receive training signals and what role the discrete Top-K selection plays during optimization, with sufficient detail to make the method reproducible.
Q2. Could the authors provide direct evidence of retrieval quality, such as selected historical chunks under different subsequent instructions, recall of task-relevant historical evidence, or comparisons with alternative retrieval strategies under a matched historical reading budget?
Q3. MOSAIC defines seven task families, but only one is evaluated. Please report results for the remaining six families, including baseline comparisons, and analyze their difficulty and ability to evaluate different memory capabilities. If these evaluations are not available, please clarify which task families have been implemented and why they have not been evaluated.
Note: The six-target MOSAIC success rates are reported as 45% (ReWAM) and 55% (Local-A) in Section 5.3, but as 46% and 56% in Table 1. Please verify and ensure consistency.