- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SCAR studies how to train perceptual memory for vision-language-action policies. Building on a π₀.₅ policy with a separate memory expert, it combines three components: auxiliary prediction of structured textual descriptions of relevant past events and subtasks; temporal convolutions that compress 128 sampled frames into the memory-token budget of 32 frames; and a staged optimization schedule that initially assigns smaller learning rates to the pretrained policy than to the newly introduced memory pathway.
On the 16-task RoboMME benchmark, SCAR reports 48.0% average success, compared with 38.0% for FrameSamp+Expert, 43.7% for FrameSamp+Modul, and 45.5% for MemER. On four real-world tasks, it achieves 51.7%, versus 30.8% for FrameSamp+Expert. Component ablations investigate the auxiliary targets, temporal compression, and alignment strength.
Critical strengths
- A useful training recipe supported by substantial task-level evaluation.
The paper addresses an important problem: action prediction can provide a weak learning signal for information needed only at particular decision points. Its evaluation covers all 16 RoboMME tasks, including several forms of genuine partial observability, and four physical manipulation tasks. The gains extend beyond comparison with a memory-free policy: SCAR improves over both the closest architectural baseline and the stronger FrameSamp+Modul baseline on average. This makes the work potentially useful to practitioners building memory-augmented policies.
- The ablations provide meaningful evidence that training choices matter.
The comparisons go beyond simply adding a larger memory module. At 40k steps, combined summary/subtask supervision reaches 46.4%, compared with 36.9% for the action-only variant; temporal compression outperforms both the uncompressed 128-frame model and a parameter-matched 32-frame model; and intermediate alignment outperforms both freezing and immediate joint training.
Critical weaknesses
- The claimed isolation of memory supervision is contradicted by the attention graph, and the key auxiliary-loss ablation may also change information routing.
Section 4.1 states that preventing auxiliary tokens from attending to the current observation prevents the auxiliary task from using information available only in that observation. However, Figure 6 permits Prompt queries to attend to Image keys, and Auxiliary queries to attend to Prompt keys. Consequently, a multilayer transformer permits the information path
\rightarrow \text{Prompt}^{(l+1)} \rightarrow \text{Auxiliary}^{(l+2)}.$$ Memory queries also attend to Prompt keys, allowing current-image information to enter deeper memory representations indirectly. 2. The temporal-compression controls do not fully establish the claimed explanation beyond history and capacity. RQ3 includes 32F, 128F, 32F+Capacity, and 128F+TC. These comparisons show that the proposed module is useful, but the parameter-matched control receives only 32 frames, while the full model receives 128. Thus, the experiments do not exclude an interaction between additional visual evidence and additional nonlinear capacity. The missing comparison is a 128-frame control with comparable parameters and the same 512 output tokens—for example, fixed temporal pooling followed by a parameter-matched pointwise module. This would more directly test the value of learned temporal aggregation.Questions for authors
-
Clarification and validation of information isolation. Is Figure 6 the exact mask used for the reported results? How is the Image → Prompt → Auxiliary path prevented?
-
Clarification of the decisive supervision ablation. For Table 8’s Action-only result, was aux_lang_task disabled, or was its attention routing retained with only ()? Please provide the exact configuration.
-
Clarification and targeted compression control. What are the input-frame count, sampling stride, temporal coverage, spatial resolution, memory-token count, and parameter count for each Table 9 variant? Does a 128-frame, parameter-matched pooling/pointwise control retain the gap to temporal convolutions?
-
Clarification of statistical replication. Do the four “model seeds” correspond to independently trained models or repeated stochastic rollouts of the same checkpoint? For the real-world evaluation, please also state whether initial conditions were matched across methods.