acceptodds
Under review as a conference paper at ICLR 2027

Imagine the Future, Internalize the Gist: Efficient VLA Reasoning via Internalized Spatiotemporal Imagination

Abstract

Vision-language-action (VLA) models increasingly incorporate intermediate reasoning to improve robotic manipulation, yet existing approaches primarily reason about observed states without explicitly anticipating future scene evolution. Extending such reasoning to explicit future rollouts at every inference step, however, introduces substantial computational overhead. We propose IG-VLA, a VLA reasoning framework that enables models to imagine the future and internalize the gist. Our Latent Spatiotemporal Reasoning learns to imagine task-relevant future scene evolution directly in visual representation space, guiding action prediction without costly pixel-level video generation. To further reduce inference overhead, we introduce Scene Gist Memory, which internalizes reasoning-derived scene–behavior associations into a compact Scene Gist Token, preserving the benefits of future reasoning while bypassing explicit future imagination at inference. Extensive experiments on LIBERO, LIBERO-Plus, and VLABench demonstrate the effectiveness and efficiency of IG-VLA. On the LIBERO-Plus Language suite, both the reasoning and gist policies outperform the strongest baseline by nearly 6% in success rate. The gist policy also achieves up to speedup over baselines, reducing inference latency from 1,081 ms to 169.5 ms per action chunk on a single NVIDIA A6000 GPU. These results demonstrate that future spatiotemporal reasoning can be effectively internalized for efficient VLA deployment.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Efficiency, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_BrYYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IG-VLA, built on ACoT-VLA. During training, a latent reasoning module predicts future SigLIP feature residuals to guide action generation. This pathway is then distilled into a compact Scene Gist vector predicted from the current context alone, so inference skips the future rollout. The paper evaluates on LIBERO, LIBERO-Plus, VLABench and one real UR3 task, and reports results on par with or slightly above ACoT-VLA.

Critical strengths
  1. The idea of training with future imagination and deploying without it is practical and well motivated by deployment cost. The latency breakdown in Figure 4 makes clear where the time goes.

  2. Predicting latent residuals instead of full future features is a sensible design. It focuses supervision on scene change and avoids pixel generation.

  3. The evaluation covers three benchmarks plus a real robot test. The VLABench comparison with ACoT-VLA uses identical training data and schedule. The gist policy matches the reasoning policy almost exactly on all benchmarks, so the distillation step itself works.

Critical weaknesses
  1. The paper lacks two baselines: the same backbone trained with identical data and steps but without the reasoning module, and a gist policy trained without the reasoning teacher. Without them, the gains over ACoT-VLA cannot be attributed to future reasoning rather than extra training, extra parameters, or the auxiliary loss acting as a regularizer.

  2. The improvements are small. The margin over ACoT-VLA is 0.4 points on LIBERO and 0.5 points on the LIBERO-Plus average, and the VLABench progress score is lower than ACoT-VLA (46.11 vs 47.40). The one large gain, on the LIBERO-Plus Language suite (79.7 to 85.6), is left unexplained, and it is unclear why visual future prediction would help specifically under language perturbation.

  3. The efficiency claim uses the wrong reference. Speedups are measured against the paper’s own reasoning policy, and against world action models whose latencies are copied from the ImageWAM paper rather than measured on the same stack. According to Figure 4, the gist policy is essentially the ACoT-VLA pipeline plus a 0.13 ms predictor.

  4. The method does not match the motivation. The introduction describes imagining futures under candidate actions and choosing among their consequences. The module actually regresses a single deterministic future with no action conditioning and no selection. “Scene Gist Memory” also stores nothing across time or episodes.

  5. Method details are missing or inconsistent. Several components are never described: the implicit action reasoner, the coarse and fine action modules, and the auxiliary pixel reconstruction head. Equation 2 and Algorithm 1 disagree on whether the policy receives the residual or the absolute latent.

  6. In Figure 5, raising the cosine loss weight from 0.3 to 0.5 drops LIBERO-Spatial from 98.6 to 37.0, an implausibly sharp collapse that points to training instability and needs multiple seeds. Residual versus absolute latent prediction is not ablated.

  7. The real-world evidence is weak. With one single-arm task and 16 trials per method, the reported 62.5, 56.2 and 50.0 differ by only one or two successes. The paper also does not say how the reasoning module was trained on the 77 demonstrations.

Questions for authors
  1. What is the performance of the identical backbone without the reasoning module, and of a gist policy trained without the reasoning teacher, under the same training budget?

  2. What are the latencies of ACoT-VLA and pi0.5 measured on your A6000 setup?

  3. Is there direct evidence that the policy uses the predicted futures? For example, how does performance change when they are zeroed or replaced with futures from another episode?

  4. Why does the gain concentrate on the LIBERO-Plus Language suite?

9 comments

[Controlled Ablations to Isolate the Contribution of Future Reasoning]

We thank the reviewer for requesting these controls. We compare the following variants under the same π0.5/ACoT-VLA backbone and initialization. The matched ACoT-VLA control omits future-feature prediction, latent future reasoning, and Gist distillation. The auxiliary-future-prediction-only control retains future-feature prediction training but removes future/Gist inputs from the action path. The Gist Policy without the reasoning teacher uses dataset ground-truth actions for supervision.

MethodSpatialObjectGoalLongAvg.
Matched ACoT-VLA (w/o Future Reasoning or Gist)99.0098.0098.0093.0097.00
Auxiliary Future Prediction Only (w/o Reasoning Input or Gist)98.6099.6097.2094.2097.40
Scene Gist Policy (w/o Reasoning Teacher; GT Action Supervision)98.2099.6098.4095.8098.00
Scene Gist Policy (Ours, Full Reasoning-Teacher Distillation)99.40100.0098.2097.8098.90

Against the 97.00% matched ACoT-VLA control, the full Scene Gist Policy reaches 98.90% average success (+1.90 points) and improves LIBERO-Long from 93.00% to 97.80% (+4.80 points). Compared with auxiliary future prediction only, the gains are +1.50 points on average and +3.60 points on LIBERO-Long. Reasoning-teacher distillation further improves over direct ground-truth action supervision by +0.90 points on average and +2.00 points on LIBERO-Long.

[About Performance Gains]

While the improvements on the near-saturated LIBERO benchmark are modest, IG-VLA achieves substantial gains on challenging instruction-related evaluations. Specifically, it improves success rates on LIBERO-Plus Language from 79.7% to 85.6% (+5.9 points). On VLABench Semantic-Instruction, IG-VLA improves both Intention Score (56.8 to 65.2, +8.4 points) and Progress Score (39.6 to 48.49, +8.89 points). Moreover, IG-VLA achieves a higher overall VLABench Intention Score (66.68 vs. 63.50), demonstrating stronger performance in instruction-sensitive manipulation scenarios.

While the overall VLABench Progress Score is slightly lower (46.11 vs. 47.40) due to minor trade-offs on standard non-paraphrased subtasks, IG-VLA achieves higher overall Intention Score (66.68 vs. 63.50) and large gains on Semantic-Instruction (+8.40 Intention, +8.89 Progress), showing clear advantages when accurate instruction grounding is required.

[Language Perturbation Robustness]

The improvement on LIBERO-Plus Language (+5.9 points) can be explained by the task-grounded guidance provided by future visual reasoning. Prior work [1] demonstrates that aligning language with desired visual changes supports generalization to unseen instructions.

Moreover, [2] finds that 80–96% of failures under language paraphrasing arise from planning-level trajectory divergence rather than low-level execution errors. This finding highlights the importance of accurately understanding task goals and grounding them in the physical environment.

Motivated by these findings, IG-VLA predicts fine-grained future visual representations to guide action generation, potentially reducing sensitivity to surface-level linguistic variations while preserving task-relevant scene dynamics. This provides an explanation for the larger improvement on the Language suite.

[1] Myers V, He A W, Fang K, et al. Goal representations for instruction following: A semi-supervised language interface to control. Conference on Robot Learning, PMLR, 2023: 3894–3908.

[2] Kim et al. LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models. EMNLP 2026.

[Additional WAM Latency Measurements]

We agree with the reviewer that the cross-paper speedup ratios in the submission used latencies quoted from the ImageWAM paper rather than measured on our stack, and that the 1,081 ms figure belonged to Fast-WAM-IDM rather than Fast-WAM itself. Below, we compare our paper's latency numbers (169.5 ms for Scene Gist Policy and 420.7 ms for Reasoning Policy, measured on a single NVIDIA RTX A6000 GPU with batch size 1 and BF16 precision) against same-GPU measurements of Fast-WAM and ImageWAM using the variant names from ImageWAM Table 11:

MethodImplementationMean (ms)
π0.5Uncompiled (BF16)138.85
ACoT-VLAUncompiled (BF16)171.58
IG-VLA Scene Gist Policy (Paper)Uncompiled (BF16)169.50
IG-VLA Reasoning Policy (Paper)Uncompiled (BF16)420.70
Fast-WAMPrefix Only180.60
Fast-WAM+ Compiled89.30
ImageWAM-FLUX.2 4BPrefix Only225.60
ImageWAM-FLUX.2 4B+ Action Loop Compile206.70
ImageWAM-FLUX.2 4B+ Image Prefill Compile184.10
ImageWAM-FLUX.2 4B+ Action Static Graph183.60

Measured on the same GPU, our uncompiled Scene Gist Policy (169.5 ms) is on par with ACoT-VLA (171.6 ms) and prefix-only Fast-WAM (180.6 ms), and faster than the tested ImageWAM configurations (183.6–225.6 ms), while being slower than compiled Fast-WAM (89.3 ms). We will therefore remove the 6.38×, 1.78×, and 1.55× cross-paper speedup claims from the abstract, introduction, and Table 4, and retain the internal speedup relative to our explicit Reasoning Policy (420.7 ms to 169.5 ms, 2.48× speedup).

[Clarification of Future Imagination and Scene Gist Memory]

We thank the reviewer for highlighting the need to clarify the connection between our motivation and implementation.

Future Imagination vs. Candidate-Action Evaluation. Our central motivation is to enable VLA policies to anticipate future environmental changes before generating actions, rather than relying solely on current observations and language instructions. IG-VLA realizes this through a latent spatiotemporal reasoning module that predicts a sequence of future visual representations conditioned on the current observation and task instruction. These predicted representations subsequently guide action generation.

Importantly, our method does not explicitly enumerate candidate actions or select among simulated outcomes. Instead, it learns a task-conditioned future trajectory that provides intermediate guidance for action prediction. This design avoids the additional computational overhead of multi-candidate future simulation while retaining the benefits of explicit future reasoning.

Scene Gist Memory vs. Episodic Memory. Scene Gist Memory is designed to internalize future reasoning knowledge into a compact representation, rather than maintain an external episodic memory bank. During training, the Scene Gist Compressor distills reasoning-enhanced representations into a compact Gist Token, and the Scene Gist Encoder learns to recover this representation directly from current observations and instructions.

Consequently, the learned Gist can provide future-informed guidance at inference without explicitly generating the full future trajectory. The term "memory" refers to this learned, parameterized internalization of future reasoning knowledge, not the storage or retrieval of historical episodes. We will clarify this distinction in the revised manuscript.

[Clarification of Method Components and Future Latent Representations]

We thank the reviewer for identifying these missing details and the inconsistency between Eq. (2) and Algorithm 1. We will revise the method and implementation sections to describe all components explicitly.

Our architecture is built directly on the ACoT-VLA action backbone. The coarse action module, also referred to as the Explicit Action Reasoner (EAR), uses a flow-matching head to generate a coarse action trajectory that serves as an explicit action-level plan. The Implicit Action Reasoner (IAR) extracts a compact latent action prior from the intermediate VLM representations. The fine action module is the final flow-matching action expert, which predicts the executable action chunk conditioned on the visual-language context and the coarse and implicit action-reasoning signals. These components are inherited from ACoT-VLA. Our contribution is the future-reasoning module and its distillation into the Gist policy. We will make this architectural dependency and the roles of EAR and IAR explicit.

The auxiliary pixel reconstruction head is used only during Stage-1 future-feature training. It encourages the predicted future representation to retain spatially meaningful visual information by reconstructing the corresponding future RGB observation. This auxiliary head is not used during action fine-tuning or inference and therefore introduces no inference-time overhead.

Clarification of the future latent representation. The action policy directly consumes the predicted future latent residuals (Δz^t,k(\Delta\hat{z}_{t,k}), rather than reconstructed absolute future latents. These residuals represent the anticipated changes in the visual scene and provide future-aware guidance for action generation. We apologize for the inconsistent notation between Eq. (2) and Algorithm 1. We will carefully check the entire manuscript and revise the equations, algorithm, figures, and accompanying text so that they consistently reflect the actual implementation.

[Training Stability and Residual vs. Absolute Future Prediction Ablation]

We thank the reviewer for raising this concern. We agree that the sharp decrease in LIBERO-Spatial success at $\lambda_{\mathrm{dir}}=0.5$ warrants an examination of training robustness.

We repeated the λdir=0.5\lambda_{\mathrm{dir}}=0.5 Stage-1 training with two additional random seeds, keeping the initialization, data, batch size, optimizer, and training schedule unchanged. All three runs completed 20k training steps, and their loss curves closely overlap throughout training. The following table reports the loss metrics from the completed 20k-step runs:

SeedTotal LossLatent MSECosine LossCosine Similarity (k=10)Reconstruction LossPredicted Transition Norm
420.16050.02740.24590.80840.05268.4618
430.15590.02680.23770.81850.05228.6714
440.15780.02650.24490.81010.05188.5736

The three runs converge to nearly identical total losses, latent-regression losses, cosine alignment, reconstruction losses, and predicted transition norms. We observe no loss divergence, exploding gradients, or latent-scale drift across the three seeds. These results indicate that the λdir=0.5\lambda_{\mathrm{dir}}=0.5 setting does not cause anomalous or seed-specific instability during Stage-1 future-representation training.

Due to the computational cost of completing the subsequent Stage-2 action training and full LIBERO rollout evaluation, we were unable to finish end-to-end policy evaluations for all three seeds within the rebuttal period. We therefore report the completed multi-seed Stage-1 analysis as an initial robustness check. We will complete the matched Stage-2 training and downstream evaluation for the additional seeds and report the resulting mean and variance in the revised paper.

Regarding residual versus absolute latent prediction, our method predicts the latent transition (Δzt,k=zt+k−zt(\Delta z_{t,k}=z_{t+k}-z_t), rather than directly predicting the absolute future latent (zt+k(z_{t+k}). This residual parameterization removes much of the static observation content and focuses the prediction objective on task-relevant temporal changes. This is the design motivation, rather than evidence of empirical superiority over absolute latent prediction. We agree that the original submission did not include a complete matched residual-versus-absolute downstream ablation. We will add this full ablation in the revised version.

Reviewer_YU4hOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents IG-VLA, a framework for efficient reasoning in VLAs. The authors first train a VLA to “imagine” the future and then internalize this reasoning into a more efficient representation to reduce inference costs. The method has two stages. First, a reasoning policy is trained to predict changes in the SigLIP embeddings of future frames from the current image and instruction, and the predicted future features are fused into the action expert. The reasoning policy is then frozen, and its reasoning output is distilled into a Scene Gist vector predicted from the prefix features by an MLP. At inference time, the model does not need to predict the future and only uses the gist policy. The experiments compare the method with existing VLAs on LIBERO, LIBERO-Plus, VLABench, and real-robot tasks. The paper also reports inference latency and hyperparameter ablations.

Critical strengths

This paper addresses an important problem. Efficient VLA reasoning is an important topic. The proposed method first learns to predict future visual features during training and then distills this reasoning into a lightweight Scene Gist. At inference time, the model does not need to explicitly predict the future, making it faster while maintaining the same success rate.

Critical weaknesses
  1. Some parts of the writing are unclear and potentially misleading. For example, the GPU time breakdown in Fig. 4 includes the IAR and EAR components. To my understanding, these are components of ACoT-VLA, but the authors do not explain this clearly in the implementation details. The method section also does not state that the entire architecture is directly based on ACoT-VLA, and IAR and EAR are not defined. It is also unclear whether the pretrained weights are taken directly from ACoT-VLA or trained by the authors using the same architecture.
  2. Following the previous point, if the architecture directly uses ACoT-VLA, the improvements on all three benchmarks are very limited, and some scores are even lower than those of ACoT-VLA. The ablations also do not directly evaluate the addition of “future imagination,” which is the central idea of this paper. This weakens the credibility of the proposed method.
Questions for authors
  1. Are the model architecture and weights directly based on ACoT-VLA? Which checkpoint was used to initialize training for the simulation experiments? Were the ACoT-VLA results in the tables reproduced using the same initialization and training setup?
  2. In Appendix A.2, the authors state that the three methods in the real-robot experiments “share the same fine-tuning configuration (..., pi05 base initialization, ...) and differ only in the method itself.” Could the authors explain how these modules were initialized?
  3. Under the same setup, what is the score if only the future-feature prediction loss is added, while the imagination module is removed at inference time and no gist is used? What is the result if the gist is trained directly using ground-truth actions without using the reasoning policy as the teacher?
  4. What is the latency of ACoT-VLA on the same GPU under the same evaluation setup?
5 comments

[Performance Improvements and Ablation of Future Imagination]

We address the two parts of this concern through matched training controls and direct interventions on the predicted futures.

First, we evaluate the following variants under the same π0.5/ACoT-VLA backbone and initialization. The matched ACoT-VLA control omits future reasoning and Gist distillation. The auxiliary-future-prediction-only control retains future-feature prediction training, but the action policy receives neither predicted future features nor a Gist representation, and the imagination module is disabled at inference. We also compare direct ground-truth action supervision for the Gist Policy against our full reasoning-teacher distillation.

MethodSpatialObjectGoalLongAvg.
Matched ACoT-VLA (w/o Future Reasoning or Gist)99.0098.0098.0093.0097.00
Auxiliary Future Prediction Only (w/o Reasoning Input or Gist)98.6099.6097.2094.2097.40
Scene Gist Policy (w/o Reasoning Teacher; GT Action Supervision)98.2099.6098.4095.8098.00
Scene Gist Policy (Ours, Full Reasoning-Teacher Distillation)99.40100.0098.2097.8098.90

The full Scene Gist Policy achieves 98.90% average success, compared with 97.00% for matched ACoT-VLA, 97.40% for auxiliary future prediction only, and 98.00% for direct ground-truth action supervision. On LIBERO-Long, the corresponding scores are 97.80%, 93.00%, 94.20%, and 95.80%.

Second, we conduct inference-time interventions on LIBERO by zeroing, replacing, or shuffling the predicted future representations while keeping the trained Reasoning Policy unchanged.

MethodSpatialObjectGoalLongAvg.
Original Futures99.4099.8099.2097.2098.90
Zero Futures97.6099.4098.2096.0097.80
Same-Task, Different-Episode Futures99.2099.4098.2096.0098.20
Different-Task Futures99.2099.6098.2096.0098.25
Shuffled Future Sequence98.6099.4098.2097.2098.35

Zeroing future representations reduces average success from 98.90% to 97.80% (-1.10 points), with a larger drop on LIBERO-Long (-1.20 points). Replacing futures with those from other episodes or tasks, or disrupting their temporal order, also consistently degrades performance. These results provide direct evidence that the action policy utilizes the predicted future representations and benefits from their contextual and temporal information.

[Clarification of Module Initialization in Real-Robot Experiments]

We thank the reviewer for requesting this clarification. In our real-robot experiments, “π0.5 base initialization” means that all three methods identically initialize their shared components of the SigLIP visual encoder, Gemma VLM backbone, and base flow-matching action head from the same public π0.5 checkpoint, and are fine-tuned using the same configuration. The π0.5 baseline directly uses this initialization without additional modules. For both the ACoT-VLA baseline and our method, the compatible projection and time-embedding layers of the EAR are warm-started from the corresponding layers of the pretrained action head, while the IAR parameters that have no π0.5 counterpart are randomly initialized. Our method uses the same initialization and additionally introduces the latent video reasoning module, including its input/output projections and horizon embeddings, which are also randomly initialized because no corresponding parameters exist in π0.5. All remaining parameters that cannot be mapped from the π0.5 checkpoint are initialized from a zero-mean Gaussian distribution with a standard deviation of 0.02. Thus, the three methods share the same pretrained initialization and fine-tuning recipe and differ only in their method-specific modules. We will clarify these details in Appendix A.2.

[Ablation of Auxiliary Future Prediction and Reasoning-Teacher Supervision]

We evaluate these variants under the same π0.5/ACoT-VLA backbone and initialization. The matched ACoT-VLA control omits future reasoning and Gist distillation. The auxiliary-future-prediction-only control retains future-feature prediction training, but the action policy receives neither predicted future features nor a Gist representation, and the imagination module is disabled at inference. We also compare direct ground-truth action supervision for the Gist Policy against our full reasoning-teacher distillation.

MethodSpatialObjectGoalLongAvg.
Matched ACoT-VLA (w/o Future Reasoning or Gist)99.0098.0098.0093.0097.00
Auxiliary Future Prediction Only (w/o Reasoning Input or Gist)98.6099.6097.2094.2097.40
Scene Gist Policy (w/o Reasoning Teacher; GT Action Supervision)98.2099.6098.4095.8098.00
Scene Gist Policy (Ours, Full Reasoning-Teacher Distillation)99.40100.0098.2097.8098.90

The full Scene Gist Policy achieves 98.90% average success, compared with 97.00% for matched ACoT-VLA, 97.40% for auxiliary future prediction only, and 98.00% for direct ground-truth action supervision. On LIBERO-Long, the corresponding scores are 97.80%, 93.00%, 94.20%, and 95.80%.

[Inference Latency Comparison]

Consistent with the latency breakdown in the paper (measured on a single NVIDIA RTX A6000 GPU with batch size 1 and BF16 precision), our Scene Gist Policy achieves 169.5 ms per action chunk compared with 420.7 ms for the Reasoning Policy (2.48× speedup, a 59.7% reduction). Under the same hardware and precision setup, we additionally measure the baselines:

MethodMean (ms)
π0.5138.85
ACoT-VLA171.58
IG-VLA Scene Gist Policy (Paper)169.50
IG-VLA Reasoning Policy (Paper)420.70

Because the predicted Scene Gist vector gtg_t directly replaces the Implicit Action Reasoner (IAR) output at inference (Sec. 3.1), the Scene Gist Policy bypasses both the multi-step future rollout and the IAR extractor, achieving comparable latency to ACoT-VLA (169.50 ms vs. 171.58 ms) while eliminating the overhead of explicit future reasoning.

[Clarification for our model architecture, checkpoint and results]

We thank the reviewer for identifying this ambiguity. We agree that the relationship to ACoT-VLA and the roles of IAR and EAR were not stated clearly enough. Our method adopts ACoT-VLA as its action-reasoning backbone and adds the proposed latent future-imagination module and its interface to the action policy. In this backbone, the Explicit Action Reasoner (EAR) predicts a coarse action plan, while the Implicit Action Reasoner (IAR) uses learnable queries to extract intermediate reasoning features from the VLM and condition the final fine-action prediction. Thus, IAR and EAR are inherited components rather than contributions introduced in this work.

Figure 4 reports an end-to-end latency decomposition of the policies. The Reasoning Policy includes the inherited EAR and IAR components together with our future-imagination module. At inference, the Scene Gist Policy uses the predicted Gist vector gtg_t to replace the IAR output and therefore bypasses both the IAR extractor and the multi-step future rollout, while retaining the remaining action-generation pathway. The breakdown accounts for the full inference cost; EAR and IAR are inherited from ACoT-VLA rather than newly proposed components. We will clarify these execution paths and label the inherited components explicitly in the figure and caption.

Regarding initialization, we use the ACoT-VLA architecture but do not initialize the reported models from an already fine-tuned ACoT-VLA policy checkpoint. The shared SigLIP encoder, Gemma backbone, and base flow-matching action head are initialized from the public π0.5 base checkpoint and fine-tuned by us. Following the ACoT-VLA initialization scheme, the EAR is warm-started from the corresponding projection and time-embedding layers of the base action head, whereas the IAR parameters are randomly initialized. Our latent future-prediction module has no counterpart in π0.5 or ACoT-VLA and is also randomly initialized. We will add these definitions, the architectural relationship to ACoT-VLA, and the exact initialization procedure to the method and implementation sections.

The ACoT-VLA entries in our main comparison tables are taken from the corresponding settings reported in the original ACoT-VLA paper, rather than from our reproduction. The ACoT-VLA results in the LIBERO-Plus SFT table correspond to its published supervised fine-tuning setting. We also reproduced ACoT-VLA using its released architecture and training protocol, but obtained slightly lower performance than the published results. We therefore compare against the stronger published results to avoid overstating our improvement. We will revise the table captions and implementation details to clearly distinguish reported results from our reproduced and matched-budget results, and to specify the checkpoint used to initialize each simulation experiment.

Reviewer_R9unOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes IG-VLA, a framework that enhances conventional Vision-Language-Action models with future spatiotemporal reasoning. Specifically, it introduces a latent reasoning module that predicts future visual representations to guide action generation, avoiding expensive pixel-level video generation. To improve inference efficiency, the authors further propose Scene Gist Memory, which distills reasoning-enhanced representations and action predictions into a compact token, eliminating the need for explicit future rollouts during inference. The method is evaluated on LIBERO, LIBERO-Plus, VLABench, and a real-world robotic manipulation task. Experimental results demonstrate competitive performance compared with existing VLA baselines, with modest improvements in task success rates and notable gains in inference efficiency. In particular, the distilled policy achieves 169.5 ms per action chunk while largely preserving the performance of the full reasoning policy.

Critical strengths

Efficient reasoning distillation. The proposed Scene Gist Memory effectively reduces inference overhead while largely preserving the performance of the full reasoning policy. Specifically, inference latency decreases from 420.7 ms to 169.5 ms per action chunk, with competitive task success rates. This demonstrates the practical value of the proposed distillation framework for efficient VLA deployment.

Broad experimental evaluation. The authors evaluate their method across multiple simulation benchmarks, including LIBERO, LIBERO-Plus, and VLABench, as well as a real-world robotic manipulation task. These experiments cover different manipulation settings and evaluation conditions, providing relatively broad empirical support for the effectiveness and applicability of the proposed approach.

Critical weaknesses

Limited novelty and unclear distinction from existing WAM approaches. The core idea of incorporating future visual prediction into action learning has already been explored in existing World Action Models (WAMs), such as Fast-WAM and ImageWAM. While IG-VLA introduces Scene Gist Memory for efficient inference, the paper does not clearly establish how its future reasoning mechanism fundamentally differs from these prior approaches. Moreover, the predicted future representations are not explicitly conditioned on candidate actions, making it unclear to what extent the model performs actual future reasoning rather than simply predicting future visual features. These concerns limit the perceived novelty of the proposed framework.

Insufficient ablations and analysis of the proposed training framework. The proposed framework involves multiple modules, training stages, and loss functions, but their individual necessity is not sufficiently validated. In particular, it remains unclear whether the performance improvements come from future reasoning, additional supervision, or action distillation. Furthermore, the paper provides limited evidence that the learned future representations and Scene Gist Tokens are informative and effectively utilized by the action policy. Potential representation collapse or feature bypass cannot be ruled out, especially since the Gist Policy retains access to the original visual-language context. More comprehensive loss ablations, representation analysis, and comparisons with simpler training alternatives would be important to substantiate the proposed design.

Questions for authors
  1. Could the authors clarify the key technical differences between IG-VLA and existing WAM approaches, particularly Fast-WAM and ImageWAM? What distinguishes the proposed future reasoning mechanism from conventional future visual representation prediction?

  2. Could the authors provide more comprehensive ablations on the necessity of the proposed loss functions and training stages? In particular, how does the method perform with simpler alternatives, such as auxiliary future prediction without Gist distillation or direct action distillation without future reasoning?

  3. How do the authors verify that the predicted future representations and Scene Gist Tokens contain meaningful information and are actually utilized by the action policy? For example, could the authors evaluate the effects of removing, replacing, or perturbing the Gist Tokens to rule out potential representation collapse or feature bypass?

6 comments

We thank the reviewer for the constructive feedback and for recognizing the practical value of Scene Gist distillation and the breadth of our experimental evaluation. We address the critical weaknesses and questions below.

[Distinction from Fast-WAM and ImageWAM]

We thank the reviewer for raising this important point. While Fast-WAM and ImageWAM also leverage future visual information for action learning, the fundamental distinction lies in whether future scene evolution is explicitly modeled as a fine-grained reasoning trajectory and directly used to guide action generation.

Specifically, Fast-WAM eliminates explicit future imagination during its efficient inference pathway and directly predicts actions from learned representations. ImageWAM leverages future image-editing representations to guide action prediction, but does not explicitly construct a fine-grained spatiotemporal reasoning trajectory. In contrast, IG-VLA explicitly predicts a sequence of future latent visual states, capturing intermediate scene evolution rather than relying solely on implicit future knowledge or endpoint-oriented representations. These predicted states are directly incorporated into our Reasoning Policy, providing fine-grained temporal and spatial guidance for action generation.

Furthermore, IG-VLA introduces a complementary Scene Gist Policy that internalizes the reasoning-derived behavioral knowledge into a compact token, bypassing explicit future rollouts when computational efficiency is prioritized. Therefore, our contribution is not merely incorporating future prediction into VLA, but establishing a unified framework that supports both explicit fine-grained latent future reasoning and its efficient internalization for action generation.

[Additional ablations and analysis of the proposed training framework]

We conduct additional LIBERO ablations on the training objectives, Gist Token, and future-reasoning supervision.

(1) Action Distillation and Gist Alignment.

MethodSpatialObjectGoalLongAvg.
Gist Policy (Action + Alignment)99.40100.0098.2097.8098.90
Gist Policy (Action only)99.2099.8098.2096.4098.40
Gist Policy (Alignment only)99.2099.4097.8097.2098.40
Action Distillation w/o Gist98.8099.4098.4095.0097.90

The complete objective achieves the highest average success rate (98.90%), outperforming either individual objective (98.40%, +0.50 points). Removing the Gist Token reduces performance from 98.90% to 97.90% (-1.00 point), particularly on LIBERO-Long (97.80% vs. 95.00%, a 2.80-point gain with Gist), demonstrating the benefits of Gist-based reasoning internalization beyond direct action distillation.

(2) Ground-Truth vs. Reasoning-Teacher Supervision.

Under the same backbone and initialization, we compare ground-truth action supervision against full reasoning-teacher supervision:

MethodSpatialObjectGoalLongAvg.
Gist Policy (Ground-Truth Action Supervision)98.2099.6098.4095.8098.00
Gist Policy (Reasoning-Teacher Supervision, Ours)99.40100.0098.2097.8098.90

Reasoning-teacher supervision improves the average success rate from 98.00% to 98.90% (+0.90 points), with a substantial gain on LIBERO-Long (95.80% to 97.80%, +2.00 points). These results confirm that distilling behavioral guidance from the explicit future-reasoning teacher outperforms direct ground-truth action supervision.

(3) Auxiliary Future Prediction without Inference-Time Gist.

We compare the following variants under the same π0.5/ACoT-VLA backbone and initialization. The auxiliary-future-prediction-only control retains future-feature prediction training, but the action policy receives neither predicted futures nor a Gist representation. The matched ACoT-VLA control omits future-feature training entirely.

MethodSpatialObjectGoalLongAvg.
Matched ACoT-VLA (w/o Future Reasoning or Gist)99.0098.0098.0093.0097.00
Auxiliary Future Prediction Only (w/o Reasoning Input or Gist)98.6099.6097.2094.2097.40
Scene Gist Policy (w/o Reasoning Teacher; GT Action Supervision)98.2099.6098.4095.8098.00
Scene Gist Policy (Ours, Full Reasoning-Teacher Distillation)99.40100.0098.2097.8098.90

(4) Ablations of the Future-Prediction Losses.

We further remove individual Stage-1 future-prediction objectives. Since these variants optimize different total objectives, their total losses are not directly comparable. We instead report the corresponding diagnostic terms, averaged over training steps 19-20k.

MethodLatent MSECosine LossMean Cosine SimilarityReconstruction LossPredicted Latent Norm
Future prediction without cosine loss0.0173N/A0.82980.04549.32
Future prediction without pixel loss0.01730.16540.8346N/A9.35
Future prediction without MSE loss0.11370.15100.84900.044321.60

The target latent norm is approximately 11.1. Removing MSE slightly improves directional cosine similarity, but increases the diagnostic latent MSE by approximately 6.6× and causes the predicted latent norm to grow to 21.6, nearly twice the target norm. This indicates that cosine supervision alone leaves the magnitude of the latent transition underconstrained and permits a scale-degenerate solution. In contrast, removing the directional cosine loss preserves stable magnitude learning but slightly weakens cosine alignment, showing that it provides a complementary directional constraint. Removing pixel reconstruction also preserves stable latent optimization, suggesting that pixel supervision serves as an auxiliary representation objective rather than being required for convergence.

These results also support our choice of �0��0�. The MSE term anchors the magnitude of the predicted latent transition, whereas the directional cosine term refines its orientation. Near convergence, the raw cosine loss is approximately one order of magnitude larger than the MSE loss; weighting it by 0.1 therefore gives the two terms comparable effective contributions. Removing MSE produces severe magnitude drift, while removing the directional term modestly reduces cosine alignment. Together, these observations support using a balanced combination of magnitude and directional supervision rather than relying on either objective alone.

[Scene Gist Token Utilization and Representation Analysis]

We conduct additional token intervention experiments and representation probing on LIBERO to verify that Scene Gist Tokens encode meaningful information and are utilized by the action policy.

(1) Scene Gist Token Interventions.

InterventionSpatialObjectGoalLongAvg.
Original Gist99.40100.0098.2097.8098.90
Random Token98.8099.6097.8097.6098.45
Same-Task, Different-Episode Gist99.0099.6098.2096.4098.30
Zero Gist98.6099.4098.2096.4098.15
Different-Task Gist97.6099.6097.6096.0097.70

Removing or replacing Scene Gist Tokens consistently degrades average success rates across all perturbations: 98.90% to 98.45% (-0.45 points) with random tokens, 98.30% (-0.60 points) with same-task different-episode Gist, 98.15% (-0.75 points) when zeroed, and 97.70% (-1.20 points) under different-task replacement (with LIBERO-Long dropping from 97.80% to 96.00%). These results confirm that Scene Gist Tokens retain task-relevant information that actively contributes to action generation.

(2) Scene Gist Representation Probing.

We further evaluate the information encoded in Scene Gist representations using linear probes for task classification and gripper-coordinate prediction.

RepresentationTask Probe Acc.Gripper Probe R²
Predicted Scene Gist39.375%0.3792
Teacher-produced Scene Gist58.750%0.4761
Current Visual-Language Features99.375%0.9478

Both predicted and teacher-produced Gist Tokens retain decodable task and spatial information, providing evidence against complete representation collapse. The higher probe scores of the original visual-language features are expected, as Gist Tokens compress reasoning-derived information into a compact representation rather than preserving all scene details.

Together, these results demonstrate that Scene Gist Tokens retain meaningful task and spatial information and contribute to action generation.

Comments on the paper

2 comments

Thank you for the follow-up questions. We address each point below.

[Training settings and the matched ACoT-VLA baseline]

Yes, IG-VLA is trained with the same setting as ACoT-VLA's released training protocol. However, in our ablation table on Scene Gist, the "Matched ACoT-VLA" row is not our reproduced ACoT-VLA model. Specifically, it is obtained by removing the Reasoning Policy and the Scene Gist module from an IG-VLA model trained under this same setting. Here, "matched" means that removing the future reasoning and Scene Gist modules from our full IG-VLA pipeline leaves only the ACoT-VLA backbone. It refers to architectural equivalence rather than an exact reproduction of ACoT-VLA using all the original training configurations and hyperparameters reported in its paper. We adopt this ablation design to more clearly isolate and analyze the individual contributions of future reasoning and Scene Gist while keeping the underlying backbone identical across all variants.

Under this controlled ablation setting, "Matched ACoT-VLA" which removes our two proposed modules leads to a performance drop on LIBERO-Long , highlighting the effectiveness of future-informed representations, particularly for long-horizon manipulation tasks, where the importance of future reasoning has also been demonstrated in prior works [1, 2]. Thus, this comparison is actually an internal ablation comparing IG-VLA with and without its proposed components. We apologize for the confusion caused by our earlier wording.

[Starting point of the ground-truth-supervised Gist Policy]

Ground-truth-supervised Gist Policy training starts from the Stage-1 Reasoning Policy, the same starting point as our full method, rather than from a model that never saw that objective. We use this setup so that both variants start from the same learned future representations and differ only in the supervision signal (ground-truth actions vs. the reasoning teacher).

Starting from a model that has never been trained with the future reasoning objective would introduce two variables of previously learned future reasoning knowledge and the type of action supervision signal, which would make it difficult to isolate the contribution of reasoning-teacher supervision

[Speedup reference]

We sincerely thank the reviewer for pointing this out. In the final version, we will revise all speedup-related statements throughout the manuscript to explicitly specify the models and baselines for each reported speedup, ensuring clarity and avoiding potential confusion.

Thank you again for your careful reading and helpful suggestions.

[1] Tian, Y., Yang, S., Zeng, J., Wang, P., Lin, D., Dong, H., & Pang, J. (2025). Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation. In Proceedings of the International Conference on Learning Representations (ICLR).

[2] Lin, M., et al. (2025). HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Models. arXiv preprint arXiv:2512.09928.

Dear Reviewer,

Thanks for your comments and questions.

We have provided further clarifications and detailed responses to your additional questions. As the discussion deadline approaches, we would sincerely appreciate your feedback and remain happy to address any remaining concerns.

Thank you again for your time and consideration.

Best regards,
Paper 36635 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.