acceptodds
Under review as a conference paper at ICLR 2027

Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models

Abstract

Long-horizon robotic manipulation requires vision-language-action (VLA) models to track scene states and their evolution beyond the current observation. However, simply conditioning policies on observation history does not guarantee that the history is effectively utilized: action supervision constrains what the policy should do, but only indirectly constrains what its history representations should retain. To resolve this, we present Temporal Forcing, a 4D representation alignment framework that explicitly supervises latent temporal states and their transitions. Specifically, we first introduce a history pathway that compresses past observations into compact latent tokens. We then align these tokens and current-frame features with geometric targets from a pretrained 4D foundation model, providing direct supervision at both the state and transition levels. The 4D foundation model and alignment heads are used only for training-time supervision. Temporal Forcing improves average success from 96.6% to 98.8% on LIBERO, with the largest gain on LIBERO-Long (93.8% to 97.2%), and from 53.5% to 62.8% across twelve RoboTwin 2.0 tasks. Furthermore, Temporal Forcing increases full-task success from 20.0% to 43.3% on a physical multi-stage hidden-placement task. Controlled experiments show that 4D representation alignment is crucial for making observation history beneficial to the model. Code will be publicly available.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, 4D Representation Alignment, Long-Horizon Manipulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vob7OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Temporal Forcing, a framework for improving how vision-language-action models use observation history. The method compresses past observations into a small set of history tokens and integrates them into the VLA backbone through gated attention. During training, a pretrained StreamVGGT model provides history-conditioned geometric features to supervise historical states, changes between states, and the summarized history tokens, alongside current-frame alignment. The geometric teacher and alignment heads are not required at inference. Experiments report improvements over the base model from 96.6% to 98.8% on LIBERO, from 53.5% to 62.8% on RoboTwin 2.0, and from 6/30 to 13/30 successful executions on a real-robot multi-stage task. Ablations, history interventions, and representation probes further examine the contribution of the alignment objectives and how the model uses historical information.

Critical strengths
  1. Rather than simply adding historical frames to a VLA model, the paper examines whether action supervision alone is sufficient for the model to retain and use important information from past observations. This question is directly relevant to manipulation under occlusion and multi-stage execution, and provides a sound motivation for additional representation supervision.

  2. Compared with the same base model, it increases average success from 96.6% to 98.8% on LIBERO and from 53.5% to 62.8% on RoboTwin. Successful full-task executions also increase from 6 to 13 out of 30 real-robot trials. The appendix further reports an improvement from 75.0% to 77.8% on LIBERO-Plus. These results support the method’s effectiveness across different environments and task settings.

  3. The authors use the same StreamVGGT architecture and pretrained weights to compare single-frame supervision with history-conditioned temporal supervision, reducing confounding differences between teacher models. They also disable history injection, modify historical inputs, and probe object and end-effector displacement to examine whether history affects action predictions and what information the model retains. Importantly, the displacement probes use recorded simulator states rather than teacher features as targets, providing an evaluation independent of the representation supervision used during training.

Critical weaknesses
  1. In Section 4.3, adding history alone does not improve the baseline under the smaller training budget. However, under the larger budget in Table 1, the history-only variant outperforms the base model 97.4% vs. 96.6%. This suggests that the observed benefit of history depends on the training regime. The authors should clarify the scope of the statement in Section 4.3 and distinguish whether alignment primarily improves optimization efficiency, final performance after convergence, or both.

  2. In the real-robot setting, the model uses a 2.8s history window without a persistent memory state that retains information beyond this window. Successful execution of a long-horizon task therefore does not, by itself, demonstrate retention of earlier task events. Moreover, the evaluation does not explicitly control for current-observation cues through cases where identical current observations require different actions depending on the preceding history. The qualitative example in Figure 8 also shows that the base model selects the correct second block but fails during placement, which is more directly indicative of an execution failure than a failure to remember task progress. While the results support improved multi-stage execution, they do not yet clearly distinguish the benefits of short-term temporal information from those of long-term task-progress memory.

Questions for authors

Please refer to the "Critical Weaknesses" for specific issues. If these can be resolved, I will raise the score.

4 comments
AuthorsOpenReview ↗

Response to Reviewer vob7 (Part 1/2)

Thank you for the encouraging assessment of the motivation, the controlled comparisons and the teacher-independent probe, and for two precise concerns. We respond to each below.

Q1: In Sec. 4.3, adding history alone does not improve the baseline under the smaller training budget. However, under the larger budget in Table 1, the history-only variant outperforms the base model (97.4% vs. 96.6%). The authors should clarify the scope of the statement in Sec. 4.3 and distinguish whether alignment primarily improves optimization efficiency, final performance after convergence, or both.

R1: Under the evaluated training budgets, Temporal Forcing (TF) improves both early training performance and the final measured checkpoint performance: it exceeds the final performance of the base and history-only models well before the end of training (by 10k of 20k steps on LIBERO-Mem, by 30k of 50k on LIBERO).

The sentence in Sec. 4.3, "History inputs alone do not improve the baseline", summarizes Table 2, the 10k-step protocol, where the history-only model is 2.5 points below the base model. Under the 50k-step protocol of Table 1 it is 0.8 points above, as you note. Under neither budget does history alone bring a clear improvement, whereas TF is above both models under both (93.6% vs. 90.7% and 88.2% at 10k steps; 98.8% vs. 96.6% and 97.4% at 50k). We will revise the sentence to "History inputs alone do not bring a clear improvement over the baseline" and report both budgets with it. The two protocols also differ in batch size, so they are not two points on one learning curve. The experiment below provides that curve.

To separate the two effects, we trained the base model, the history-only model (History Pathway, action loss only) and TF with the same recipe for 20k steps and evaluated checkpoints along training. We did this on LIBERO-Mem [1], a suite of 10 non-Markovian manipulation tasks (e.g., repeated pick-and-place, bowl swaps, occluded-target placement), because its performance is far from saturated. The history models use the 5.6 s window of Table 6. Each number is the mean of two training seeds, each evaluated on 20 unseen layouts per task (200 episodes per seed). We report the task success rate (all subgoals completed in the required order) and the subgoal completion rate of the benchmark paper (fraction of subgoals completed).

Success rate / subgoal completion (%) at step5k10k15k20k
StarVLA-OFT (base)33.2 / 41.530.0 / 47.134.5 / 49.933.5 / 49.1
History only (action loss)19.8 / 32.332.0 / 44.933.8 / 51.139.0 / 54.1
Temporal Forcing41.0 / 50.850.5 / 63.568.5 / 79.079.0 / 86.1

From 10k steps on, TF at every checkpoint is above the best checkpoint of either control at any step (39.0% success, 54.1% completion). At 20k steps it is 40.0 points of success rate and 32.0 points of completion above the history-only model. Thus, TF shows stronger early learning and higher performance at the final evaluated checkpoint. With the default 2.8 s window of the paper, TF reaches 73.5% success and 82.5% completion at 20k steps.

On LIBERO, under the 50k-step protocol of Table 1, TF is likewise above the history-only model at every evaluated checkpoint:

LIBERO average success (%) at step20k30k50k
History only (action loss)94.295.697.4
Temporal Forcing96.797.998.8

TF at 20k steps is above the history-only model at 30k and matches the base model's final 96.6%; TF at 30k is above the history-only model at 50k.

Is the history used at inference? On LIBERO-Mem at 20k steps, closing the injection gate at inference changes the history-only model by 2.5 points or less (success rate 39.0% vs. 36.5%, completion 54.1% vs. 52.6%) and reduces TF from 79.0% to 20.5% in success rate and from 86.1% to 26.1% in completion. This is the pattern of the 10k-step LIBERO result in Fig. 7(b), where closing the gate lowers TF by 5.4 points and leaves the history-only model unchanged (+0.3), now observed on a history-dependent benchmark.

We will add these learning curves and the efficiency analysis to Sec. 4.3, together with the revised sentence above.

[1] Chung, Nhat, et al. "Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective." AAAI 2026 (arXiv:2511.11478).

AuthorsOpenReview ↗

Response to Reviewer vob7 (Part 2/2)

Q2: In the real-robot setting, the model uses a 2.8 s history window without a persistent memory state that retains information beyond this window. The evaluation does not explicitly control for current-observation cues through cases where identical current observations require different actions depending on the preceding history. Fig. 8 is more directly indicative of an execution failure than a failure to remember task progress. The results do not yet clearly distinguish the benefits of short-term temporal information from those of long-term task-progress memory.

R2: We added the controlled experiment you asked for: a real-robot task in which visually indistinguishable current observations require different actions depending on the preceding history, so the current observation does not reveal the task state. TF succeeds in 44/60 trials, against 17/60 for the base model.

The task has observation aliasing by design. Three buttons (red, green, blue) are fixed on the table in front of the UR3, and the instruction gives the order in which to press them, e.g., "Press the buttons in the order red, green, blue. Press each button once." The buttons are purely mechanical and spring back after a press, so a pressed button is in the same state as an unpressed one, and nothing else in the scene changes. The image therefore depends only on the arm pose and does not show which buttons have been pressed.

Visually indistinguishable observations therefore occur at different stages of the task and require different actions. With the arm above the green button, the image looks the same before and after green is pressed: before, the correct action is to press; after, it is to move on to the next button. This occurs at every button, so each trial passes through such aliased states three times. The instruction does not resolve this: it gives the order, not how far the execution has advanced. Only the preceding observations do, and the press that separates the two states lies inside the 2.8 s window.

All models are fine-tuned on the same 96 demonstrations, and TF uses the same 2.8 s window as in the paper. Each model is evaluated for 30 trials on each of two orders (red → green → blue and blue → green → red); a trial succeeds only if all three buttons are pressed once in the instructed order.

π0\pi_0π0.5\pi_{0.5}StarVLA-OFT (base)History only (action loss)Temporal Forcing
Successes11/6027/6017/6021/6044/60
Success rate18.3%45.0%28.3%35.0%73.3%

TF succeeds in 44/60 trials, against 17/60 for the base model and 27/60 for π0.5\pi_{0.5}. History trained with the action loss alone reaches 21/60, close to the base model. Access to the past frames is not enough; the policy resolves the aliased states once its history representation is supervised. On LIBERO-Mem (R1), the advantage of TF likewise disappears when its history is removed at inference (79.0% → 20.5% success).

We agree that short-term information and long-term memory must be separated, and we distinguish three things. (1) Long-horizon execution: a task that lasts many steps; this is where the paper reports gains (LIBERO-Long, the hidden-placement task). (2) History use within a finite window: the policy conditions on the last 2.8 s; this is what TF improves and what the button task isolates. (3) Persistent memory across windows: retaining an event after it has left the window; TF has no such state, and we do not claim it. The supervision itself is not tied to the window: the teacher conditions each target on a longer causal context (Sec. 3.3; the whole episode prefix on the real robot, Appendix A), so the same objective applies to a longer window or a persistent state, which we leave to future work. The first two are compatible: a task may last a minute while the decision at hand depends on what happened in the last few seconds. We will state this scope in Sec. 1 and Sec. 5.

Hidden-placement task and Fig. 8: Fig. 8(b) shows a Stage-2 placement failure, as its caption states, and we read it as you do. The hidden-placement experiment establishes that the gain of TF appears in continuous multi-stage execution rather than in the isolated stages (Table 4); the button-pressing task complements it by fixing the current observation and varying only the progress. We will state both points in Sec. 4.4 and in the caption of Fig. 8.

We would be grateful if you could reconsider your assessment in light of these clarifications and new results. We welcome any further questions or discussion.

Reviewer_vob7OpenReview ↗

Thank authors for the rebuttal.

The author's reply addressed some of my concerns. So I decided to increase my rating.

We sincerely appreciate your careful evaluation of our rebuttal and your updated rating. Your comments have been instrumental in improving the paper, and we will incorporate the clarifications and new results discussed above into the revised version. Thank you for your support throughout the review process.

Reviewer_jmjuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Temporal Forcing, a framework for improving how VLA models use observation history. It adds a History Pathway that compresses past frames into tokens and aligns the resulting representations with features from a pretrained causal 4D model. The training objectives supervise historical states, their transitions, and current-frame features; the 4D teacher is removed at inference time. The method improves success rates over its base model on LIBERO, RoboTwin 2.0, and a real-robot multi-stage manipulation task.

Critical strengths

1.The paper identifies a meaningful gap between providing observation history and learning a useful representation of it. Supervising both historical states and their transitions is a clear, well-motivated approach to this problem.

2.The controlled ablations help isolate the contribution of the proposed supervision. In particular, the history-only variant does not improve performance, while the full method does; the gate-closure and representation-probing analyses provide further evidence that the learned history affects action prediction.

Critical weaknesses

1.The main concern is the evaluation setup. The paper claims an innovation in temporal modeling, yet its benchmark evaluation is limited to LIBERO and RoboTwin-clean. These two benchmarks primarily test a model’s ability to fit the tasks and are very unlikely to involve tasks with substantial temporal demands. Memory-related VLA benchmarks should therefore be included to evaluate the effectiveness of the proposed method. Moreover, the ablations, especially “w/o 4D alignment” and “w/o history at inference”, should also be conducted on those memory benchmarks.

2.This work seems to be an incremental extension in the vein of Spatial Forcing. However, implicit representation alignment methods of this kind typically require expensive inference with an external teacher during VLA training stage, which can substantially increase training time and GPU memory usage. The training resources required by the proposed method should therefore be reported.

3.(Minor) Why does the History Pathway use a Q-Former to compress the representations? What would happen if the 16 learnable queries M_t directly extracted DINOv2 features through the Causal Transformer, without a Q-Former? Either a qualitative explanation or a quantitative experiment would be helpful.

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer jmju (Part 1/3)

Thank you for the positive assessment of the motivation and the ablations, and for the concrete requests. We address each below with new experiments.

Q1: The paper claims an innovation in temporal modeling, yet its benchmark evaluation is limited to LIBERO and RoboTwin-clean, which are very unlikely to involve tasks with substantial temporal demands. Memory-related VLA benchmarks should be included, and the ablations, especially "w/o 4D alignment" and "w/o history at inference", should also be conducted on those memory benchmarks.

R1: We added the memory benchmark LIBERO-Mem [1], with both requested ablations. Temporal Forcing (TF) reaches a 79.0% success rate and 86.1% subgoal completion, against 33.5% and 49.1% for the base model. LIBERO-Mem is a suite of 10 non-Markovian manipulation tasks (e.g., repeated pick-and-place, bowl swaps, occluded-target placement) in which the current observation does not determine the correct action. We train four models with the same recipe (20k steps, global batch size 64): the base model, the History Pathway trained with the action loss only, the History Pathway with current-frame alignment but without temporal alignment, and full TF. All history models use the 5.6 s window of Table 6 (K=7K=7). Each model is trained with two seeds, and each seed is evaluated on 20 unseen layouts per task; we report the mean (400 episodes per model). We report the task success rate (SR) and the subgoal completion rate (SC): an episode is successful when all subgoals are completed in the required order, and the completion rate, the metric of the benchmark paper (its Table 4), is the fraction of subgoals completed. For comparison we fine-tuned π0.5\pi_{0.5} on the same demonstrations, also with two seeds, and evaluated it with the same code.

Tasks: T1–T2, lifting an object and placing it back; T3–T6, the same repeated several times; T7–T8, swapping or rotating bowls between plates using an empty plate; T9–T10, placing an object in a basket and then moving a basket.

SR / SC (%)π0.5\pi_{0.5}StarVLA-OFT (base)w/o 4D alignmentw/o temporal alignmentTemporal Forcing
T190.0 / 90.090.0 / 90.070.0 / 70.077.5 / 77.5100.0 / 100.0
T290.0 / 90.067.5 / 67.562.5 / 62.567.5 / 67.577.5 / 77.5
T360.0 / 80.825.0 / 57.545.0 / 70.035.0 / 62.595.0 / 97.5
T425.0 / 53.350.0 / 70.060.0 / 75.860.0 / 74.280.0 / 84.2
T540.0 / 75.07.5 / 39.517.5 / 63.55.0 / 52.590.0 / 94.5
T615.0 / 61.12.5 / 41.420.0 / 51.10.0 / 45.465.0 / 84.3
T77.5 / 45.82.5 / 9.22.5 / 13.37.5 / 22.565.0 / 75.0
T812.5 / 35.00.0 / 7.55.0 / 21.25.0 / 21.225.0 / 54.4
T997.5 / 98.855.0 / 58.857.5 / 57.585.0 / 85.0100.0 / 100.0
T1097.5 / 98.835.0 / 50.050.0 / 56.290.0 / 92.592.5 / 93.8
Avg.53.5 / 72.933.5 / 49.139.0 / 54.143.2 / 60.179.0 / 86.1

TF has the highest average on both metrics: 79.0% SR and 86.1% SC, against 53.5% and 72.9% for π0.5\pi_{0.5}. On the six repetition and swap tasks (T3–T8), TF averages 70.0% SR, against 26.7% for π0.5\pi_{0.5} and at most 25.0% for the base model and the two ablations; on bowl swapping (T7), TF reaches 65.0% and no other model exceeds 7.5%. With the default 2.8 s window of the paper, TF reaches 73.5% SR and 82.5% SC, still above every other model in the table.

[1] Chung, Nhat, et al. "Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective." AAAI 2026 (arXiv:2511.11478).

AuthorsOpenReview ↗

Response to Reviewer jmju (Part 2/3)

R1 (continued): The two ablations you asked for are in the table below (LIBERO-Mem, averages over the 10 tasks, %; SR: success rate, SC: subgoal completion):

ModelHistoryLcur\mathcal{L}_{cur}Ltemp\mathcal{L}_{temp}SRSR, gate closedSCSC, gate closed
StarVLA-OFT (base)33.5n/a49.1n/a
w/o 4D alignment (history, action loss only)✓39.036.554.152.6
w/o temporal alignment✓✓43.245.560.157.9
Temporal Forcing✓✓✓79.020.586.126.1

(1) Without 4D alignment, the History Pathway trained with the action loss only reaches 39.0% SR and 54.1% SC, 40.0 and 32.0 points below TF. (2) With the gate closed at inference, TF drops from 79.0% to 20.5% SR and from 86.1% to 26.1% SC, providing evidence that its performance depends strongly on the history pathway. The two models without temporal alignment change by at most 2.5 points under this intervention. For TF, the drop is 5.4 points on LIBERO (Fig. 7(b)) and 58.5 points here, in line with the temporal demands of the two benchmarks.

The ranking also holds along training: evaluated every 5k steps, TF from 10k steps on (SC 63.5%, 79.0% and 86.1%) is above the best checkpoint of the base model (49.9%) and of the model without 4D alignment (54.1%).

On the real robot, we also added an ordered button-pressing task on the UR3. The buttons are purely mechanical and spring back, so a pressed button looks the same as an unpressed one, and visually indistinguishable observations require different actions depending on the preceding history. Over 60 trials per model (two orders, 30 trials each), TF succeeds in 44, the model without 4D alignment in 21, the base model in 17, and π0.5\pi_{0.5} in 27.

Q2: This work seems to be an incremental extension in the vein of Spatial Forcing. However, implicit representation alignment methods of this kind typically require expensive inference with an external teacher during the VLA training stage, which can substantially increase training time and GPU memory usage. The training resources required by the proposed method should therefore be reported.

R2: The distinction from Spatial Forcing lies in the supervised representation, not simply in using a 4D teacher. Spatial Forcing aligns the current-frame visual tokens; Temporal Forcing supervises history-conditioned latent states and their transitions. The 4D teacher is run once offline to precompute the targets, and is not executed during policy optimization or inference. This requires 10.9 GPU-hours of one-time preprocessing on LIBERO; online policy training increases step time by 21% and peak memory by 2.0 GiB per GPU.

We share Spatial Forcing's broader principle of geometric representation supervision, but address a different failure mode: a history pathway may receive observations while learning representations that are weakly used in subsequent decisions. Our objective therefore directly constrains the history features and their changes, whereas the current-frame alignment term plays a complementary role.

The evidence uses the same teacher and architecture. On LIBERO, history plus current-frame alignment with the causal 4D teacher yields 83.7%, below the base model (90.7%), while adding history-state and transition alignment yields 93.6% (Table 2, rows 1, 5 and 8; identical 10k-step protocol). On LIBERO-Mem, the corresponding task success rates are 43.2% and 79.0%. Replacing causal targets with framewise targets lowers the full model to 85.0% on LIBERO, while removing the transition loss reduces LIBERO-Long from 83.8% to 76.0%. Thus the evidence concerns where and how the representation is supervised, not the capacity of the pretrained geometry model.

What does the learned pathway contribute? A teacher-independent displacement probe improves from R2=0.44R^2=0.44 with action-only history training to 0.83 with Temporal Forcing. Closing the history gate at inference reduces LIBERO-Mem success from 79.0% to 20.5%, consistent with reliance on learned history features. We will state this distinction from Spatial Forcing explicitly in the Related Work.

AuthorsOpenReview ↗

Response to Reviewer jmju (Part 3/3)

R2 (continued), training resources: The cost of TF has two parts that should be kept apart: a one-time offline preparation of the 4D targets, and the online training of the policy. The 4D teacher appears only in the first part. It is never run during training or at inference.

(1) Offline preparation, once per dataset and reusable. StreamVGGT is run once over the training trajectories and its features are cached: 10.9 GPU-hours and 68 GiB for LIBERO (Appendix A), 14 GPU-hours for LIBERO-Mem. Every run on that dataset then reads the same cache; the main model and the ablations share it. For the main model this one-time cost is about 6% of the training compute.

(2) Online training. The main LIBERO model (50k steps, global batch size 128) trains in 22.4 hours on eight A100 GPUs. To compare models, we measured all of them under one setting (table below: 10k-step LIBERO protocol of Table 2; 4×A800, global batch size 64, bfloat16). TF increases the step time by 21% and the peak memory by 2.0 GiB per GPU over the base model. TF and the model without 4D alignment have the same step time: the alignment losses add no measurable time per step, and the 21% comes from encoding the past frames in the History Pathway.

ModelStep time (s)Peak memory per GPU (GiB)LIBERO avg. (%)Long (%)
StarVLA-OFT (base)1.8759.690.776.2
w/o 4D alignment (history, action loss only)2.2761.188.272.8
Temporal Forcing2.2761.693.683.8
Temporal Forcing w/o Q-Former2.3563.284.255.8

At inference the teacher and the alignment heads are removed; inference takes 58 ms per action chunk, against 45 ms for the base model (Sec. 4.4).

Q3: (Minor) Why does the History Pathway use a Q-Former to compress the representations? What would happen if the 16 learnable queries MtM_t directly extracted DINOv2 features through the Causal Transformer, without a Q-Former?

R3: We ran this ablation: without the Q-Former, the average success drops from 93.6% to 84.2% and Long from 83.8% to 55.8% (last row of the table in R2). In "w/o Q-Former", the 256 DINOv2 patch tokens of each past frame are linearly projected and fed directly to the same causal Transformer (3,584 temporal tokens instead of 28). The 16 readout queries, the gated injection and all losses are unchanged; the state and change losses use the mean of each frame's output tokens. This variant is trained under the protocol of Table 2 and has slightly higher step time and memory.

Why does the Q-Former help? The Q-Former compresses each frame into two tokens, which gives every history timestep a compact state. The state and transition losses act on these per-timestep states, and the causal Transformer and the readout queries operate over 28 tokens rather than several thousand. Without this compression, the per-timestep state is an average over 256 patch tokens, which is a weaker handle for the supervision.

We would be grateful if you could reconsider your assessment in light of these clarifications and new results. We welcome any further questions or discussion.

Reviewer_cYucOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a spatiotemporal perception module for VLAs to explicitly incorporate historical observations and 4D supervision. For each camera, the module extracts features from the past (K) frames using a frozen DINOv2 encoder and compresses them with a learnable Q-Former. Temporal offsets and camera identities are encoded, and 16 learnable queries aggregate the features into a fixed-length set of historical tokens. These tokens interact with the current main-camera image tokens through gated cross-attention before the first VLA decoder layer. Additionally, StreamVGGT is used to generate 4D supervision signals for training. The paper conduct experiments on RoboTwin, Libero and one real world manipulation tasks.

Critical strengths

Well practical motivation and the integration of 4D information is reasonable approach to enriching VLA training with geometric and temporal information. The paper is well written and clear to read for me.

Critical weaknesses
  1. The main concern is potentially redundant architectural design and many previous works have do similar integration into VLA to my knowledge. Using StreamVGGT to provide 4D supervision is a natural extension that can improve VLA performance. So the contribution seems incremental, as it mainly combines existing components (A+B) without introducing a fundamentally new idea.
  2. Simulation results seems little weak: 96.6% to 98.8% on LIBERO and 53.5% to 62.8% on RoboTwin 2.0. Furthermore, pi05 and MemoryVLA only on LIBERO, why not testing them on Robotwin and real-world experiments?
Questions for authors

See Critical Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer cYuc (Part 1/2)

Thank you for the careful review and for the positive comments on the motivation and the clarity of the paper. We address your concerns below, with new experiments.

Q1: The main concern is potentially redundant architectural design, and many previous works have done similar integration into VLA. Using StreamVGGT to provide 4D supervision is a natural extension that can improve VLA performance. So the contribution seems incremental, as it mainly combines existing components (A+B) without introducing a fundamentally new idea.

R1: The novelty lies in what and how we supervise, rather than which architectural components we combine. We acknowledge that the architecture uses established components. Our contribution is twofold. First, a finding: giving a VLA its observation history does not ensure that it learns or uses a representation of how the scene has evolved, even when its current-frame features are geometrically supervised. Second, a method that closes this gap: explicit supervision of history-conditioned latent states and their transitions.

History-aware architectures make past observations available, but availability does not specify which changes a compressed history representation should preserve. Current-frame geometric alignment supervises spatial features at the present observation; it does not directly constrain the intermediate history states or how those states evolve. Temporal Forcing (TF) formulates this missing constraint as causally conditioned latent-state alignment together with explicit latent-transition alignment: per-timestep history features match the 4D teacher's history-conditioned features, while differences between consecutive latent states match the corresponding teacher-feature differences. The teacher is used only to precompute training targets, not at inference.

We ran a controlled test of the proposed distinction. This is not simply a comparison between different teachers or model capacities. With the same history pathway, 4D teacher, backbone, and training protocol, history plus current-frame geometric alignment (the straightforward architectural combination) reaches 83.7% on LIBERO, below the base model (90.7%), whereas adding direct temporal-representation supervision reaches 93.6% (Table 2, rows 1, 5 and 8; 10k-step protocol). On LIBERO-Mem [1], which requires decisions dependent on observation history, the corresponding success rates are 43.2% and 79.0%, respectively (20k-step protocol). History trained through action loss alone reaches 39.0% on LIBERO-Mem. This isolates supervision of historical representations from simply providing history and supervising the current frame.

Why is this supervision temporal rather than ordinary feature distillation? Two aspects of our existing ablations are informative. First, with all losses and the network fixed, replacing causally conditioned targets with independently processed framewise targets lowers LIBERO success from 93.6% to 85.0% (Table 2, row 7 vs. row 8). The causal features show greater temporal contrast (0.49 vs. 0.10, Fig. 6). Second, removing the explicit transition loss lowers LIBERO-Long success from 83.8% to 76.0% (Table 2, row 9), while transition-only supervision already reaches a 93.2% average (row 10). This supports both causal target conditioning and explicit change supervision.

Does the learned history affect behavior? The teacher-independent motion probe improves from R2=0.44R^2=0.44 with action-only history training to 0.83 with Temporal Forcing (Table 3). Closing the history-injection gate at inference reduces LIBERO-Mem success from 79.0% to 20.5%, indicating substantial reliance on that pathway, whereas the action-only history model changes from 39.0% to 36.5%. In addition, on our real-robot ordered button-pressing task (see R2), visually indistinguishable progress states require different actions; Temporal Forcing succeeds in 44/60 trials versus 21/60 for action-only history and 17/60 for the base model. Together these results link the proposed supervision to representation quality and task behavior.

Spatial Forcing supervises current-frame visual representations; history-augmented VLAs emphasize providing and encoding temporal context. Temporal Forcing instead makes the history representation and its observed transitions the explicit objects of geometric supervision. The History Pathway is a compact carrier for this supervision, and it is not redundant: removing its Q-Former lowers the LIBERO average from 93.6% to 84.2% (new ablation). We will clarify this positioning in the revision.

[1] Chung, Nhat, et al. "Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective." AAAI 2026 (arXiv:2511.11478).

AuthorsOpenReview ↗

Response to Reviewer cYuc (Part 2/2)

Q2: Simulation results seem a little weak: 96.6% to 98.8% on LIBERO and 53.5% to 62.8% on RoboTwin 2.0. Furthermore, π0.5\pi_{0.5} and MemoryVLA are compared only on LIBERO. Why not test them on RoboTwin and in real-world experiments?

R2: LIBERO is close to saturation. Where there is headroom the gains are large, and they are largest on history-dependent tasks. We have also added π0.5\pi_{0.5} on every benchmark, and TF is higher on each.

BenchmarkStarVLA-OFT (base)π0.5\pi_{0.5}Temporal Forcing
LIBERO, average success (%)96.696.998.8
LIBERO-Long, success (%)93.892.497.2
RoboTwin 2.0, 12 tasks, success (%)53.558.862.8
LIBERO-Mem, success rate (%)33.553.579.0
LIBERO-Mem, subgoal completion (%)49.172.986.1
Real robot, ordered button pressing17/6027/6044/60

Relative to the base model, TF lowers the failure rate on LIBERO from 3.4% to 1.2%, with +3.4 points on Long. Beyond LIBERO, the gain is +9.3 points on RoboTwin 2.0 (Handover Block 0% → 44%, Handover Mic 39% → 79%), +45.5 points of success rate on LIBERO-Mem (new), and 17/60 → 44/60 on a new real-robot task; on the hidden-placement task of the paper it is 6/30 → 13/30. TF also improves zero-shot LIBERO-Plus from 75.0% to 77.8% (Table 8).

Comparison with π0.5\pi_{0.5}: In the paper, the RoboTwin comparisons use π0\pi_0 and DP3 (Fig. 3, Table 7) and the real-robot comparisons use OpenVLA and π0\pi_0 (Table 4). LIBERO numbers for π0.5\pi_{0.5} are the official openpi results. On RoboTwin 2.0 we follow the official RoboTwin recipe for π0.5\pi_{0.5} on the same twelve tasks and 50 clean demonstrations per task, and evaluate 100 episodes per task as in Table 7. On LIBERO-Mem and the real robot, π0.5\pi_{0.5} is fine-tuned on the same demonstrations and evaluated with the same protocol as TF.

The real-robot comparison uses a new task, ordered button pressing. Three buttons (red, green, blue) are fixed on the table, and the instruction gives the order in which to press them. The buttons are purely mechanical and spring back, so a pressed button looks the same as an unpressed one and the image does not show which buttons have already been pressed. A trial succeeds only if all three buttons are pressed once in the instructed order. All models are fine-tuned on the same 96 demonstrations and evaluated for 30 trials on each of two orders. TF succeeds in 44/60 trials, 17 more than π0.5\pi_{0.5} and at least twice as many as the base model (17/60), π0\pi_0 (11/60) and history trained with the action loss alone (21/60).

MemoryVLA: MemoryVLA is evaluated only on single-arm robots (WidowX, Google robot and Franka), and its released model predicts 7-DoF end-effector actions from a single third-person view, without wrist views or proprioception. RoboTwin 2.0 is bimanual, so running MemoryVLA there would require changing its action space and inputs rather than evaluating the released method. On the real robot, π0.5\pi_{0.5} could be added because a base checkpoint of the complete model, pretrained on more than 10k hours of robot data, is released with an official fine-tuning pipeline; we fine-tuned it on our task. MemoryVLA releases checkpoints for simulation benchmarks only: its real-robot checkpoints and data are not released, and its real-robot training script starts from the third-party CogACT checkpoint. Reproducing it on our UR3 was not feasible within the discussion period. We therefore compare on the benchmarks where MemoryVLA results are reported, and TF is higher on both: LIBERO (98.8% vs. 96.7%, Table 1) and zero-shot LIBERO-Plus (77.8% for TF, Table 8, vs. 70.2% for MemoryVLA and 73.1% for MemoryVLA++, as reported in the MemoryVLA++ paper).

We would be grateful if you could reconsider your assessment in light of these clarifications and new results. We welcome any further questions or discussion.

AuthorsOpenReview ↗

Additional result for Reviewer cYuc: MemoryVLA on LIBERO-Mem

Dear Reviewer cYuc,

In our response we explained why MemoryVLA could not be run on RoboTwin 2.0 or on our real robot. To still provide a direct comparison on a history-dependent benchmark, we have now trained MemoryVLA on LIBERO-Mem with its official code and its official LIBERO training recipe (20k steps), on the same demonstrations and with two seeds, and evaluated it on the same unseen layouts as the other models (400 episodes per model):

LIBERO-MemStarVLA-OFT (base)π0.5\pi_{0.5}MemoryVLATemporal Forcing
Success rate (%)33.553.546.879.0
Subgoal completion (%)49.172.970.186.1

We would be grateful to know whether our response and these results address your concerns, and we are happy to clarify anything further before the discussion period closes.

AuthorsOpenReview ↗

A kind follow-up before the discussion closes

Dear Reviewer cYuc,

As the discussion period closes in a few hours, we would like to kindly follow up. In our response we addressed each of your questions, and we added a direct comparison with MemoryVLA on LIBERO-Mem (46.8% success, against 79.0% for Temporal Forcing). We have also run a new controlled experiment on the difference from Spatial Forcing: a model that keeps everything in Temporal Forcing and removes only the change loss reaches 54.5% success on LIBERO-Mem, against 79.0% for the full method and 43.2% without any temporal alignment (two-seed means).

If these address your concerns, we would be sincerely grateful if you would consider updating your rating. If anything remains unclear, we would be glad to clarify it right away.

Thank you very much for your time.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision

Dear Reviewers and Area Chair,

We have uploaded a revised version of the paper. New or changed text is marked in blue. The changes are:

  1. Sec. 4.3 (new), Memory-Dependent Tasks: results on LIBERO-Mem, with comparisons to π0.5\pi_{0.5} and MemoryVLA and the ablations of Temporal Forcing on this benchmark (Table 2). (Reviewers jmju and cYuc)
  2. Sec. 4.4: the sentence on history inputs is revised and now reports both training budgets (Reviewer vob7); representation-level measurements of state and transition alignment are added.
  3. Sec. 5: the scope of the method is stated; it operates on a bounded history window, and persistent memory beyond the window is left to future work. (Reviewer vob7)
  4. Related Work: the difference between pointwise alignment and the change loss is stated explicitly. (Reviewers jmju and cYuc)
  5. Appendix A: training cost and the ablation without the Q-Former (Table 12). (Reviewer jmju)
  6. Appendix B: learning curves on LIBERO-Mem (Table 7) and on LIBERO (Reviewer vob7), and a weight-matched control for the change loss.
  7. Appendix C: a new real-robot task with observation aliasing, ordered button pressing (Table 11); the roles of the two real-robot tasks are stated there and in the caption of Fig. 8. (Reviewers vob7 and cYuc)

The title, the abstract, the method and all previously reported results are unchanged. Four tables were added (Tables 2, 7, 11 and 12). Because the new Sec. 4.3 and Tables 2 and 7 precede existing material, the former Secs. 4.3 and 4.4 are now Secs. 4.4 and 4.5, the former Tables 2 to 5 are now Tables 3 to 6, and the former Tables 6 to 8 are now Tables 8 to 10. Section and table numbers in our responses to the reviewers refer to the original submission; figure numbers are unchanged.

Thank you again for your time and feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.