- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Temporal Forcing, a framework for improving how vision-language-action models use observation history. The method compresses past observations into a small set of history tokens and integrates them into the VLA backbone through gated attention. During training, a pretrained StreamVGGT model provides history-conditioned geometric features to supervise historical states, changes between states, and the summarized history tokens, alongside current-frame alignment. The geometric teacher and alignment heads are not required at inference. Experiments report improvements over the base model from 96.6% to 98.8% on LIBERO, from 53.5% to 62.8% on RoboTwin 2.0, and from 6/30 to 13/30 successful executions on a real-robot multi-stage task. Ablations, history interventions, and representation probes further examine the contribution of the alignment objectives and how the model uses historical information.
Critical strengths
-
Rather than simply adding historical frames to a VLA model, the paper examines whether action supervision alone is sufficient for the model to retain and use important information from past observations. This question is directly relevant to manipulation under occlusion and multi-stage execution, and provides a sound motivation for additional representation supervision.
-
Compared with the same base model, it increases average success from 96.6% to 98.8% on LIBERO and from 53.5% to 62.8% on RoboTwin. Successful full-task executions also increase from 6 to 13 out of 30 real-robot trials. The appendix further reports an improvement from 75.0% to 77.8% on LIBERO-Plus. These results support the method’s effectiveness across different environments and task settings.
-
The authors use the same StreamVGGT architecture and pretrained weights to compare single-frame supervision with history-conditioned temporal supervision, reducing confounding differences between teacher models. They also disable history injection, modify historical inputs, and probe object and end-effector displacement to examine whether history affects action predictions and what information the model retains. Importantly, the displacement probes use recorded simulator states rather than teacher features as targets, providing an evaluation independent of the representation supervision used during training.
Critical weaknesses
-
In Section 4.3, adding history alone does not improve the baseline under the smaller training budget. However, under the larger budget in Table 1, the history-only variant outperforms the base model 97.4% vs. 96.6%. This suggests that the observed benefit of history depends on the training regime. The authors should clarify the scope of the statement in Section 4.3 and distinguish whether alignment primarily improves optimization efficiency, final performance after convergence, or both.
-
In the real-robot setting, the model uses a 2.8s history window without a persistent memory state that retains information beyond this window. Successful execution of a long-horizon task therefore does not, by itself, demonstrate retention of earlier task events. Moreover, the evaluation does not explicitly control for current-observation cues through cases where identical current observations require different actions depending on the preceding history. The qualitative example in Figure 8 also shows that the base model selects the correct second block but fails during placement, which is more directly indicative of an execution failure than a failure to remember task progress. While the results support improved multi-stage execution, they do not yet clearly distinguish the benefits of short-term temporal information from those of long-term task-progress memory.
Questions for authors
Please refer to the "Critical Weaknesses" for specific issues. If these can be resolved, I will raise the score.
4 comments
Response to Reviewer vob7 (Part 2/2)
Q2: In the real-robot setting, the model uses a 2.8 s history window without a persistent memory state that retains information beyond this window. The evaluation does not explicitly control for current-observation cues through cases where identical current observations require different actions depending on the preceding history. Fig. 8 is more directly indicative of an execution failure than a failure to remember task progress. The results do not yet clearly distinguish the benefits of short-term temporal information from those of long-term task-progress memory.
R2: We added the controlled experiment you asked for: a real-robot task in which visually indistinguishable current observations require different actions depending on the preceding history, so the current observation does not reveal the task state. TF succeeds in 44/60 trials, against 17/60 for the base model.
The task has observation aliasing by design. Three buttons (red, green, blue) are fixed on the table in front of the UR3, and the instruction gives the order in which to press them, e.g., "Press the buttons in the order red, green, blue. Press each button once." The buttons are purely mechanical and spring back after a press, so a pressed button is in the same state as an unpressed one, and nothing else in the scene changes. The image therefore depends only on the arm pose and does not show which buttons have been pressed.
Visually indistinguishable observations therefore occur at different stages of the task and require different actions. With the arm above the green button, the image looks the same before and after green is pressed: before, the correct action is to press; after, it is to move on to the next button. This occurs at every button, so each trial passes through such aliased states three times. The instruction does not resolve this: it gives the order, not how far the execution has advanced. Only the preceding observations do, and the press that separates the two states lies inside the 2.8 s window.
All models are fine-tuned on the same 96 demonstrations, and TF uses the same 2.8 s window as in the paper. Each model is evaluated for 30 trials on each of two orders (red → green → blue and blue → green → red); a trial succeeds only if all three buttons are pressed once in the instructed order.
| StarVLA-OFT (base) | History only (action loss) | Temporal Forcing | |||
|---|---|---|---|---|---|
| Successes | 11/60 | 27/60 | 17/60 | 21/60 | 44/60 |
| Success rate | 18.3% | 45.0% | 28.3% | 35.0% | 73.3% |
TF succeeds in 44/60 trials, against 17/60 for the base model and 27/60 for . History trained with the action loss alone reaches 21/60, close to the base model. Access to the past frames is not enough; the policy resolves the aliased states once its history representation is supervised. On LIBERO-Mem (R1), the advantage of TF likewise disappears when its history is removed at inference (79.0% → 20.5% success).
We agree that short-term information and long-term memory must be separated, and we distinguish three things. (1) Long-horizon execution: a task that lasts many steps; this is where the paper reports gains (LIBERO-Long, the hidden-placement task). (2) History use within a finite window: the policy conditions on the last 2.8 s; this is what TF improves and what the button task isolates. (3) Persistent memory across windows: retaining an event after it has left the window; TF has no such state, and we do not claim it. The supervision itself is not tied to the window: the teacher conditions each target on a longer causal context (Sec. 3.3; the whole episode prefix on the real robot, Appendix A), so the same objective applies to a longer window or a persistent state, which we leave to future work. The first two are compatible: a task may last a minute while the decision at hand depends on what happened in the last few seconds. We will state this scope in Sec. 1 and Sec. 5.
Hidden-placement task and Fig. 8: Fig. 8(b) shows a Stage-2 placement failure, as its caption states, and we read it as you do. The hidden-placement experiment establishes that the gain of TF appears in continuous multi-stage execution rather than in the isolated stages (Table 4); the button-pressing task complements it by fixing the current observation and varying only the progress. We will state both points in Sec. 4.4 and in the caption of Fig. 8.
We would be grateful if you could reconsider your assessment in light of these clarifications and new results. We welcome any further questions or discussion.
Thank authors for the rebuttal.
The author's reply addressed some of my concerns. So I decided to increase my rating.
We sincerely appreciate your careful evaluation of our rebuttal and your updated rating. Your comments have been instrumental in improving the paper, and we will incorporate the clarifications and new results discussed above into the revised version. Thank you for your support throughout the review process.
Response to Reviewer vob7 (Part 1/2)
Thank you for the encouraging assessment of the motivation, the controlled comparisons and the teacher-independent probe, and for two precise concerns. We respond to each below.
Q1: In Sec. 4.3, adding history alone does not improve the baseline under the smaller training budget. However, under the larger budget in Table 1, the history-only variant outperforms the base model (97.4% vs. 96.6%). The authors should clarify the scope of the statement in Sec. 4.3 and distinguish whether alignment primarily improves optimization efficiency, final performance after convergence, or both.
R1: Under the evaluated training budgets, Temporal Forcing (TF) improves both early training performance and the final measured checkpoint performance: it exceeds the final performance of the base and history-only models well before the end of training (by 10k of 20k steps on LIBERO-Mem, by 30k of 50k on LIBERO).
The sentence in Sec. 4.3, "History inputs alone do not improve the baseline", summarizes Table 2, the 10k-step protocol, where the history-only model is 2.5 points below the base model. Under the 50k-step protocol of Table 1 it is 0.8 points above, as you note. Under neither budget does history alone bring a clear improvement, whereas TF is above both models under both (93.6% vs. 90.7% and 88.2% at 10k steps; 98.8% vs. 96.6% and 97.4% at 50k). We will revise the sentence to "History inputs alone do not bring a clear improvement over the baseline" and report both budgets with it. The two protocols also differ in batch size, so they are not two points on one learning curve. The experiment below provides that curve.
To separate the two effects, we trained the base model, the history-only model (History Pathway, action loss only) and TF with the same recipe for 20k steps and evaluated checkpoints along training. We did this on LIBERO-Mem [1], a suite of 10 non-Markovian manipulation tasks (e.g., repeated pick-and-place, bowl swaps, occluded-target placement), because its performance is far from saturated. The history models use the 5.6 s window of Table 6. Each number is the mean of two training seeds, each evaluated on 20 unseen layouts per task (200 episodes per seed). We report the task success rate (all subgoals completed in the required order) and the subgoal completion rate of the benchmark paper (fraction of subgoals completed).
From 10k steps on, TF at every checkpoint is above the best checkpoint of either control at any step (39.0% success, 54.1% completion). At 20k steps it is 40.0 points of success rate and 32.0 points of completion above the history-only model. Thus, TF shows stronger early learning and higher performance at the final evaluated checkpoint. With the default 2.8 s window of the paper, TF reaches 73.5% success and 82.5% completion at 20k steps.
On LIBERO, under the 50k-step protocol of Table 1, TF is likewise above the history-only model at every evaluated checkpoint:
TF at 20k steps is above the history-only model at 30k and matches the base model's final 96.6%; TF at 30k is above the history-only model at 50k.
Is the history used at inference? On LIBERO-Mem at 20k steps, closing the injection gate at inference changes the history-only model by 2.5 points or less (success rate 39.0% vs. 36.5%, completion 54.1% vs. 52.6%) and reduces TF from 79.0% to 20.5% in success rate and from 86.1% to 26.1% in completion. This is the pattern of the 10k-step LIBERO result in Fig. 7(b), where closing the gate lowers TF by 5.4 points and leaves the history-only model unchanged (+0.3), now observed on a history-dependent benchmark.
We will add these learning curves and the efficiency analysis to Sec. 4.3, together with the revised sentence above.
[1] Chung, Nhat, et al. "Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective." AAAI 2026 (arXiv:2511.11478).