acceptodds
Withdrawn from ICLR 2027

Structured Latent Reasoning for Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) models can benefit from intermediate reasoning, but explicit chain-of-thought reasoning incurs substantial autoregressive decoding overhead in closed-loop control. Existing latent reasoning methods reduce this cost through continuous latent representations, yet leave the structured reasoning roles of explicit traces implicit. We propose Structured Latent Reasoning (SLR), which organizes latent representations into semantically distinct reasoning units corresponding to different forms of intermediate reasoning and supervises them through auxiliary text decoding. Because the relevance of these units varies across manipulation transitions, we further introduce transition-adaptive learning, which reweights their supervision according to relevance signals from the action policy. At inference, SLR operates directly on the structured latent units, avoiding autoregressive reasoning generation. SLR achieves 73.7% average success on SIMPLER, improving over LaRA-VLA by 4.9 points, and reaches 79.0% average task completion on real-robot tasks while maintaining gains under distribution shift. SLR runs at 341 ms per control step, combining the structured semantics of explicit reasoning with the inference efficiency of latent reasoning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action models, Latent reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.60 over 5 reviews, above 10% of the others.
Reviewer_6HrGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Structured Latent Reasoning (SLR), which organizes continuous latent tokens into semantically explicit reasoning units and supervises them with an auxiliary text decoder strictly during training. It introduces Transition-Adaptive Learning (TAL) to dynamically reweight each unit's loss based on its relevance to the action policy at each control step. Experiments demonstrate that SLR achieves a 73.7% success rate on SIMPLER and 79.0% on real-robot tasks with strong OOD generalization, while running at 341 ms/step—over 16× faster than explicit CoT.

Critical strengths

The paper’s primary strength lies in its Structured Latent Reasoning (SLR) framework, which elegantly solves the trade-off between reasoning quality and execution speed. This design preserves structured intermediate semantics while achieving an inference latency of just 341 ms/step (over 16× faster than explicit text CoT), making it genuinely viable for real-time robotic control. SLR also demonstrates exceptional out-of-distribution generalization, proving that structured latent representations boost both control performance and policy robustness.

Critical weaknesses
  • W1. Overclaim on “Consistent Improvements” Across Sub-Tasks. The paper claims consistent improvements over existing baselines. However, in the SIMPLER evaluation (Table 1), SLR’s overall lead over LaRA-VLA (73.7% vs. 68.8%) is driven almost entirely by a single task—Stack block (54.2% vs. 25.0%). On Put spoon and Put carrot, SLR actually underperforms LaRA-VLA (83.3% vs. 95.8% and 58.3% vs. 62.5%, respectively). The claim of “consistent improvements” is therefore not supported by the per-task results and should be revised to reflect the task-dependent nature of the gains.

  • W2. Limited Flexibility and Scalability of the Fixed Latent Architecture. The model uses a rigid, hardcoded 8-slot latent architecture with a fixed semantic order. This static capacity may be suboptimal for simpler single-step primitives, where many slots could be redundant, or for highly complex multi-stage tasks, where eight slots may be insufficient. The paper does not provide ablations on the number of latent units or their ordering, nor does it discuss how the architecture could be adapted to tasks of varying complexity. This limits the claimed generality of the approach.

Questions for authors
  • Q1. Localized Performance Gains. In Table 1, SLR's margin over LaRA-VLA (73.7% vs. 68.8%) is almost entirely driven by a single sub-task (Stack block: 54.2% vs. 11.5%), while SLR underperforms LaRA-VLA on Put spoon and Put carrot. Could the authors clarify why the performance gains are so highly localized to block stacking?

  • Q2. Robustness to Annotation Noise. SLR relies on a complex synthetic pipeline (Qwen, Grounding DINO, OWL-ViT) to generate targets for all 8 slots. Given that synthetic labels inherently contain bounding-box noise or LLM hallucinations, how robust is SLR to label noise during training?

  • Q3. Completeness and Justification of the 8-Slot Design. The architecture hardcodes a fixed 8-slot topology with pre-assigned semantics. Could the authors provide theoretical or empirical justification for the completeness and necessity of this specific 8-slot breakdown, rather than leaving the latent space unconstrained or dynamically sized?

Minor:

  • Figure 2 retains a visible red spell-check squiggly line under text.
Reviewer_ybXrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper adds eight latent positions to a VLA built from Qwen3-VL-4B and a DiT flow-matching action head. Each position is tied to one Embodied-CoT field (task, plan, object boxes, subtask reasoning, subtask, motion reasoning, motion command, gripper positions). During training, a frozen Qwen language-model stack with a learned input projection decodes each position into its own field. After 30k steps with uniform per-unit losses, training restarts from that checkpoint and weights each unit's loss by the action head's detached cross-attention to that unit. The decoder is removed at inference.

The paper reports 73.7% average success on four SIMPLER WidowX tasks versus 68.8% for LaRA-VLA (Table 1), 79.0% mean graded completion on four real-robot tasks versus 71.0% for π0.5 (Figure 3), gains under object substitution and instruction paraphrase (Figure 4), and 341 ms per action chunk (Figure 7).

In the ablations, uniform per-unit supervision reaches 67.7%, about one point above the action-only model, and the remaining gain appears with adaptive weighting (up to 73.7%).

Critical strengths
  1. The weighting scheme is simple and comes with a well-designed control. Weighting each unit's text loss by the shared attention needs no extra model and keeps the total supervision weight fixed.

  2. The real-robot study includes distribution shift and is documented in detail. 4 WidowX tasks with different demands are tested in distribution and under 2 shift axes, with 60 trials per method per axis. Appendix C gives the partial-credit rubrics, the exact paraphrased prompts, the held-out object substitutions, and the full annotation pipeline. SLR has the highest score on every task and on both shift axes against π0.5\pi_{0.5}, StarVLA and ECoT.

Critical weaknesses
  1. The fixed-role structure is not shown to help performance. In Table 2 (p. 8), action-only is 66.67%, adding latent tokens gives 65.63%, and adding per-unit text decoding gives 67.71%. Role-bound decoding therefore adds about one point over the action-only model and stays below LaRA-VLA (68.8%). There is no run in which the same eight latents are supervised with the concatenated reasoning text without role binding, which is the comparison contribution 1 needs.

  2. How much does the structured reasoning really help? The claim in Section 4.2(3) that SLR beats LaRA-VLA "because of its structured reasoning" also lacks support. The paper's own action-only model (66.67%) is already within about two points of LaRA-VLA, whose own no-CoT baseline is 55.2% (LaRA-VLA Table 9). It could be doubted that most of the cross-paper gap reflects different base models and training recipes. The authors should clarify whether any unstructured-supervision run exists; if not, contribution 1 and Section 4.2(3) should be restated.

  3. The real-robot study does not test the claim it is used for, and the baseline conditions are not given. Contribution 3 and the conclusion claim consistent gains over existing latent-reasoning baselines in and out of distribution, but Figures 3-4 contain no latent-reasoning method. The starting checkpoints, inputs and fine-tuning steps of π0.5\pi_{0.5}, StarVLA and ECoT are not reported, so part of the 63.0 →\rightarrow 79.0 gap over StarVLA may come from in-domain pretraining.

  4. Cross-system numbers are used for mechanism claims, and the Table 1 protocol is mixed. Section 4.2(1) concludes from 73.7% versus 20.1% that "the gain comes from the latent interface rather than the reasoning content". ECoT is built on OpenVLA with discrete action tokens, and the paper's own action-only model already scores 66.67%, 46.6 points above ECoT, without any reasoning. The same applies to Section 4.2(2): action-only is above F1 (59.4%) and UD-VLA (62.5%). The remark in Section 4.4 that explicit textual reasoning is "particularly fragile" rests on one ECoT implementation whose real-robot training is not described. External Table 1 entries are taken from published papers, and LaRA-VLA's 68.8% was measured on 24 rollouts per task in its paper, while Section 4.1 states 96 rollouts for the evaluation.

  5. Figure 7 contradicts its own measurement claim. Section 5.4 states that every latency in the figure was measured on one RTX 4090. ThinkAct (7,513 ms) and Fast-ThinkAct (805 ms) are identical to Table 5 of the Fast-ThinkAct paper. The same two values appear in Figure 8 of LaRA-VLA, whose caption attributes them to an NVIDIA A100. A fresh measurement on different hardware does not reproduce two published timings at the reported precision. The abstract and the introduction use this pair to claim that SLR is "over 2× faster than Fast-ThinkAct." LaRA-VLA and ECoT in the same figure (253.12 ms, 5,460.32 ms) differ from LaRA-VLA's A100 numbers (135 ms, 4,434 ms), so the figure is not one table copied from a single paper. Under an explicit claim that the whole figure was measured on one RTX 4090, two entries match an A100 table exactly. That is a false description of the experiment. I need the measurement record for every bar. Until that record is produced, I reject the speed claims. This discrepancy also makes me doubt the other numbers reported in the paper.

  6. The positioning against prior latent and auxiliary-reasoning work is too broad. Section 2 states that existing latent methods do not explicitly preserve distinct semantic reasoning roles. LaRA-VLA uses a Qwen3-VL backbone and a DiT action expert, trains on structured CoT with subtask, motion-reasoning and target-object components, and reports that latents of different components form separate clusters (its Figure 7). Fast-ThinkAct (Huang et al., 2026) already decodes latents to text with a training-only verbalizer. ECoT-Lite (Chen et al., 2025), which Section 2 describes as generating text before acting, trains with reasoning as an auxiliary target and acts without reasoning at test time; it is the closest precedent for the "+text decoding" row. The actual new elements are fixed per-role slots with decoding supervision kept throughout training, and the attention-based weighting. In addition, the eight latents are produced sequentially through a learned projection (Appendix E), so "avoiding autoregressive reasoning generation" should read as avoiding text decoding.

  7. Minors. The λ\lambda in the sentence after Eq. 6 (L233) is undefined. The same line (L233) "Eq. equation 6" must be a typo. The backbone is called Qwen3-VL-4B-Instruct in Section 4.1 and Qwen3-VL-4B-Instruct-Action in Appendix E.

Questions for authors
  1. Does a run exist in which the same eight latents are supervised with the concatenated reasoning text and no role binding? If not, how should contribution 1 and Section 4.2(3) be restated?

  2. For flat, random and shuffled (Figure 5) and the "+text decoding" and "Uniform" rows (Tables 2-3): did each start from the same 30k checkpoint with the same optimizer restart, step count and checkpoint rule as SLR? How many rollouts per task was each evaluated on, and what does SLR score on seed 0 alone? The action-only row uses half the batch size of the other rows; was that the only difference in its optimization setup?

  3. How were the 45k checkpoint and β\beta = 0.1 chosen without a validation split? If other checkpoints of the SLR and shuffled runs were saved, what are their SIMPLER averages?

  4. For π0.5\pi_{0.5}, StarVLA and ECoT on the real robot: what were the starting checkpoints, inputs (cameras, proprioceptive state) and fine-tuning steps? Did any of them receive Bridge pretraining comparable to SLR's 30k checkpoint?

  5. For each external row in Table 1, what is the source paper and the number of rollouts per task? Which sentences in Section 4.2(1), 4.2(2) and 4.4 still hold once those rows are read as system-level comparisons?

  6. Provide the measurement record for every bar in Figure 7: hardware, model, and whether the number was copied.

Reviewer_3DLjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Structured Latent Reasoning (SLR), a framework that aims to preserve the semantic structure of explicit CoT reasoning while avoiding autoregressive reasoning generation at inference time. SLR organizes latent representations into semantically distinct reasoning units and supervises each unit using an auxiliary text decoder. The paper further introduces Transition-Adaptive Learning, which adjusts the supervision weights of individual reasoning units at each transition based on cross-attention from the action policy. SLR achieves 73.7% average success on SIMPLER and 79.0% average task completion on real-world manipulation tasks.

Critical strengths
  • Important problem formulation and promising empirical results.

This paper addresses an important challenge in VLA models: incorporating intermediate reasoning while avoiding the computational overhead of explicit CoT generation. The proposed framework uses auxiliary supervision for latent reasoning units only during training, enabling efficient action generation without explicit reasoning at inference time. The consistent performance improvements on SIMPLER, real-world manipulation tasks, and OOD evaluations demonstrate the practical potential of the proposed approach.

  • Well-designed ablation studies supporting transition-adaptive learning.

This paper compares uniform, random, shuffled, and transition-adaptive supervision weighting strategies. In particular, the 4.9-point improvement over shuffled weighting suggests that aligning supervision weights with the current transition is more effective than merely assigning non-uniform weights. This comparison provides meaningful empirical evidence supporting the effectiveness of transition-adaptive learning, one of the paper's main technical contributions.

Critical weaknesses
  • Insufficient differentiation from closely related work.

The paper does not adequately clarify its technical contributions relative to LaRA-VLA, one of the most closely related prior methods. LaRA-VLA also internalizes structured textual and visual CoT into continuous latent representations and reports semantic separation among different reasoning components. Therefore, the paper's characterization of existing latent reasoning methods as leaving reasoning structure implicit requires a more careful discussion. In particular, the authors should clarify how explicit unit-wise supervision and transition-adaptive learning differ from the mechanisms employed in LaRA-VLA and other related approaches. This distinction is important for assessing the novelty of the proposed method.

  • Insufficient evidence isolating the contribution of semantic structuring.

The component ablation in Table 2 shows that unit-wise text supervision provides a relatively modest improvement of approximately 2.1 pp, whereas transition-adaptive learning yields a further improvement of approximately 6.0 pp. However, the paper does not compare the proposed role-specific supervision against an unstructured latent reasoning approach with an equivalent auxiliary text supervision objective. Consequently, the benefit of explicitly structuring latent reasoning units is not clearly isolated. Furthermore, the representation analyses in Figure 6 are primarily qualitative and do not directly establish that individual reasoning units play distinct functional roles in action generation. While the experiments provide evidence for the effectiveness of transition-adaptive learning, further validation is needed to substantiate the contribution of structured latent reasoning itself, which is central to the paper's claims.

Questions for authors
  • Novelty relative to LaRA-VLA. LaRA-VLA also internalizes structured textual and visual CoT into continuous latent representations and reports semantic separation among reasoning components. Could the authors clarify the key technical distinction between the proposed role-specific latent units and the latent reasoning representations learned by LaRA-VLA? In particular, what capabilities or properties does SLR introduce that are not already captured by LaRA-VLA?

  • Contribution of explicit semantic structuring. Table 2 shows that adding unit-wise text supervision improves performance, but it does not isolate the benefit of assigning distinct semantic roles to individual latent units. What evidence supports the claim that explicit semantic structuring, rather than auxiliary text supervision itself, contributes to the performance improvement? Are there any controlled comparisons or additional analyses that help distinguish these effects?

Additional Comments

  • Mathematical formulation in Section 3. Equation (1) appears to be mathematically incorrect, as it equates a marginal action distribution with a factorization of the joint distribution over the latent state and action. The latent state should therefore be marginalized out. The authors should revise Equation (1) to ensure mathematical consistency. Additionally, the statement following Equation (6) refers to an undefined parameter, lambda. This should also be corrected.

  • Citation of LaRA-VLA. LaRA-VLA has already been published at ICML 2026 (PMLR, Vol. 306, pp. 5409–5426). Since the conference proceedings were available before the ICLR 2027 submission, the authors should cite the published conference paper rather than the arXiv preprint.

  • Presentation of Figure 2. An unintended red underline appears beneath DiT, and the alignment of some text labels should be checked for consistency.

Reviewer_ay5MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Structured Latent Reasoning (SLR) for VLA The method assigns different semantic roles to latent tokens and supervises them using a training-only text decoder. It further introduces transition-adaptive learning, which uses attention from the action policy to dynamically weight the supervision of different reasoning units.

Critical strengths
  1. The method combines semantic supervision with latent reasoning, avoiding expensive autoregressive text generation during inference. This is a reasonable design for robot control.
  2. The comparisons with uniform, random, and shuffled weighting provide evidence that transition-dependent supervision can improve policy performance. The real-robot experiments are also valuable.
Critical weaknesses
  1. The central motivation is not sufficiently supported. The paper argues that existing latent reasoning methods lack explicit semantic structure, which limits their effectiveness. However, the absence of explicit role assignments does not necessarily mean that latent representations cannot learn such structure implicitly. The paper provides no direct evidence that existing latent reasoning methods actually suffer from this limitation. Fig6 only analyzes the proposed SLR representations, without examining whether similar semantic differentiation emerges in existing methods. More importantly, Tab2 shows that structured text supervision improves success from 65.6% to only 67.7%, while adaptive weighting further increases it to 73.7%. This raises the question of whether the main contribution is structured latent reasoning or simply a better auxiliary supervision strategy. Without a controlled comparison that separates semantic structuring from the additional text supervision, the central claim remains insufficiently validated.

  2. The empirical advantages are less convincing than the overall results suggest. Although SLR achieves the highest average score on simpler, it underperforms LaRA-VLA on two of the four tasks, with a substantial portion of the improvement coming from block stacking. The comparison also relies on previously published baseline results rather than matched training conditions. The real-robot evaluation involves only 10 trials per task and reports partial-credit completion scores rather than binary task success. Furthermore, SLR has higher inference latency than LaRA-VLA (341 ms vs. 253 ms). These limitations make it difficult to assess whether the additional complexity consistently translates into better performance.

Questions for authors
  1. Can the authors provide evidence that existing latent reasoning methods do not already learn semantically differentiated representations? A direct representation analysis of an unstructured latent baseline would help establish the proposed motivation.

  2. Can the authors compare SLR against a baseline with the same reasoning annotations, auxiliary decoder, and training budget, but without explicitly assigning semantic roles to individual latent units? This would clarify whether the improvement comes from semantic structuring or additional supervision.

  3. Can the authors provide targeted interventions on individual reasoning units, such as masking or replacing them, to demonstrate that they have distinct functional roles in action prediction?

  4. Can the authors report variability across evaluation runs and clarify how the measured 341 ms inference latency is reconciled with the reported 5 Hz control setting?

Reviewer_E4UKOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper frames the main problem that they are addressing: "Can latent reasoning retain the structured reasoning of explicit CoT while preserving its inference efficiency?" More specifically, they argue explicit CoT provides structured reasoning but is slow. Latent reasoning is fast but unstructured. SLR claims to get the benefits of both by instilling structured to the otherwise unstructured latent tokens. In addition, they introduced a transition-adaptive weighting that scales each token's training loss by how much the action head attends to it. SLR is evaluated on SIMPLER against 1. no-reasoning 2. explicit reasoning 3. latent reasoning baselines, supplemented by a few real-robot experiments, weighting ablations and inference latency comparisons.

Critical strengths

None

Critical weaknesses
  1. The paper's problem statement rests on unproven assumptions. The paper ask whether latent reasoning can retain the structured reasoning of explicit CoT while preserving its inference efficiency. This framing depends on several premises, none of which is argued or tested. Since none of these premises are validated by experiments, the paper's central question is never established.
  • It assumes structured reasoning is better than unstructured reasoning: The paper assumes without evidence that adding structure to reasoning rather than implicit latent result in a better model. It's own ablation also showed that adding structure alone adds no performance difference (67.7% for structured vs 65.6% for unstructured latents).
  • It assumes implicit latent reasoning loses certain valuable gains compared to explicit reasoning: No analysis has shown that unstructured latents are worse than structured reasoning. In fact in Table 1 lara-vla outperforms all explicit reasoning methods. The paper's novelty rests on the fact that it retains the speed of implicit latent reasoning models while avoiding the flaws of it. However, if it cannot prove that implicit latent reasoning is worse than explicit reasoning, and that it is also slower than implicit reasoning models (compared to lara-vla) then the paper has no argument for its value.
  • It assumes latent reasoning is unstructured: while latent reasoning can be unstructured, the model should still learn useful structure implicitly but perhaps not in an interpretable form. The paper never compares hand-designed decomposition vs letting the model learn its own implicit structure from the same supervision
  • It confuses explicit with structured: Explicitness is more to do with whether reasoning traces are decodable into text, whereas structure concerns whether it is decomposed into certain roles. These are orthogonal.
  1. The evaluation results are weak and doesn't justify the paper's claims. The validation results are misattributed at various points.
  • ECoT comparison is misleading: In Section 4.2 Table 1, 53.6 point gain is attributed to the latent interface rather than the reasoning content. However ECoT uses a completely different backbone and action head (Prismatic-7B with discretized autoregressive action tokens) vs SRL's Qwen3-VL-4B with a DiT-B flow matching head. Moreover, in section 5.1 Table 2, the paper's own action-only baseline without reasoning (66.7%) already beats ECoT (20.1%) which contradicts the 53.6% gain that is attributed to the latent interface, rather this gap reflects the architecture not the interface.
  • The adaptive-weighting gain is confounded: SLR’s 6 point gain over uniform supervision (67.7 → 73.7 in Section 5.1 Table 2) coincides with three other changes. A restarted optimizer and learning rate schedule, extra training steps, and a checkpoint picked from a longer run (App. E). The comparison is also uneven. Only SLR is evaluated over four seeds (App. B), while the ablation numbers are consistent with a single seed. The gain may come from the training schedule rather than the weighting itself.
  • The core comparison with implicit latent reasoning models are barely ran under controlled conditions: LaRA-VLA is the only latent reasoning baseline which is in Section 4.2 Table 1, which SLR marginally improved ~5%. Section 4.3 Real Robot eval and Section 4.4 OOD experiments include 0 latent baselines.
  • The validation results are not convincing on the current benchmark setup in both simulation and real world: Simulation only covers 4 SIMPLER tasks. The real-robot results in only use 10 trials per task and the OOD results 15 per task, which 1. is subject to noise in the real world 2. uses arbitrary designed task completion scores.
  • The paper doesn’t show its latents are actually structured. Its evidence is qualitative: attention heatmaps and from training episodes and selected decodings in Section 5.3. Decodability shows each token contains its assigned content, but not that tokens are distinct from one another or that the policy relies on that structure. The authors themselves note attention is not a causal measure (App. F). A quantitative test, such as masking individual tokens at inference, is missing. Just as the paper does not show that existing latent reasoning lacks structure, it does not show that SLR’s latents possess it.
Questions for authors
  1. Under matched backbone, action head, training schedule, and checkpoint selection, how does SLR compare with unstructured latent reasoning, eg., the same eight latent tokens supervised with the full concatenated reasoning text rather than role-partitioned fields? Please report multiple seeds with confidence intervals. This directly tests whether structure helps, which is the paper’s central premise.
  2. Does masking or ablating individual latent units at inference selectively degrade the corresponding behavior, e.g., masking the gripper unit hurting grasping? This tests whether the learned latents are actually structured.
  3. Did the uniform supervision and alternative weighting variants in Table 2,3 receive the same optimizer and LR restart, total training steps, checkpoint selection procedure, and number of evaluation seeds as the reported SLR result? If not, the adaptive weighting gain may be an artifact of the training schedule.
  4. Were all latencies in Figure 7 measured by the authors on the same RTX 4090 with the same settings, or are some (e.g., ThinkAct, Fast-ThinkAct) taken from prior work?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.