acceptodds
Under review as a conference paper at ICLR 2027

Latent Memory Palace: Reasoning for Control as Autoregressive Variational Inference

Abstract

Human decision-making is highly flexible – some actions are taken immediately; others require longer deliberation. Language models have exhibited a similar capacity for adaptive “reasoning.” However, transferring this capability to continuous control policies has been challenging, as directly reasoning in language space may lack the granularity for spatial understanding and precise motions. In this work, we show that reasoning for control policies can emerge by organizing information in an autoregressive latent space reminiscent of a memory palace, where retrieval is iterative and adaptive. Our method, Latent Memory Palace (LMP), formulates reasoning as variational inference with an autoregressive latent distribution. We derive a latent-space reinforcement learning technique to tractably optimize its variational lower bound. The resulting policy, LMP-, achieves strong empirical performance in simulation and real-world domains while exhibiting interpretable, adaptive allocation of test-time compute. We further show that the same framework yields a variable-length action tokenizer, LMP-, which significantly improves the performance of downstream autoregressive policies. Together, these results present a new perspective on latent reasoning for control through the lens of variational inference.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generative Control Policies, Latent Reasoning, Variational Inference, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bY8EOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LMP, which formulates iterative, adaptive "reasoning" for robot control as variational inference with a variable-length autoregressive latent variable: an encoder (posterior) maps observation–action pairs to latent token sequences terminated by EOS, and a decoder reconstructs actions; the ELBO is optimized via latent-space RL (a PPO-style clipped surrogate with a stepwise KL decomposition), and a variance-decay schedule over latent length induces compression and adaptive allocation of compute. The framework instantiates both a policy (LMP-π) and a variable-length action tokenizer (LMP-tok). On real-robot DROID and simulated LIBERO/D3IL/RoboMimic, LMP-π outperforms Diffusion Policy, OpenVLA, VAE-policy and VQ-BeT baselines; LMP-tok outperforms VQ-VAE, FAST, and OAT tokenizers under the same downstream autoregressive policy. Analyses show variable-length latents and compression both contribute, that more latent steps express more behavioral modes, and that latent length negatively correlates with local action variance (e.g., gripper movements use fewer steps).

Critical strengths
  • Clean and principled formulation: casting adaptive computation as variational inference over variable-length discrete latents is elegant, and the variance-decay compression schedule yields genuinely adaptive latent lengths with a clear mechanism (each additional step must "pay for itself" via sharper reconstruction).
  • Strong and unusually fair empirical evaluation: real-robot plus three simulated benchmarks, two instantiations from one framework, and tokenizer comparisons that control for the downstream policy; ablations isolate the contributions of variable length, compression strength, and vocabulary size.
  • Insightful mechanistic analysis: test-time truncation experiments, mode-visualization, and the latent-length vs. action-variance correlation clarify what the latent sequence actually encodes, going beyond headline numbers.
Critical weaknesses
  • The "reasoning" framing overstates the mechanism: the paper itself shows that LMP does not perform iterative refinement — longer latents simply enumerate more behavioral modes. The memory-palace/reasoning analogy may mislead readers about what is being learned; the contribution is more accurately a variable-length latent VAE policy.
  • The RL-based optimization is acknowledged to be fragile and hyperparameter-sensitive, but the paper gives no training-cost or stability comparison against the much simpler baselines (diffusion policy, VQ-BeT); scalability beyond 1B models and multi-task datasets remains unclear.
  • Statistical and evaluation rigor is limited: DROID results are 20 trials without variance/CIs, LIBERO comparisons are marginal (0.933 vs 0.909 overall), the "bottom-10 tasks" framing is a post-hoc lens, and no significance testing is reported.
  • The tokenizer evaluation uses a single downstream policy class and reports no latency/compression-rate analysis (average token length, decoding overhead), limiting the practical case for LMP-tok over FAST/OAT.
Questions for authors
  • Since latent steps encode modes rather than iterative refinement, in what sense is this "reasoning"? Can you show a task where the latent sequence exhibits temporal/planning structure (e.g., plan-then-act) rather than mode enumeration, or otherwise clarify the analogy?
  • What is the training cost (GPU-hours, buffer size, PPO hyperparameter sensitivity) of latent-space RL relative to the baselines, and are the DROID baselines tuned with a comparable budget? Can you report per-task variance/CIs?
  • Would a simpler fixed-length VAE with a larger codebook recover most of the gap, i.e., how much of the benefit comes from variable length per se versus the latent formulation plus compression?
  • For LMP-tok: what are the average sequence lengths and decoding latencies compared to FAST/OAT, and does the tokenizer remain effective on high-frequency or dexterous control (e.g., 30Hz+ tasks)?
3 comments

We thank the reviewer for the positive review and for finding our formulation "clean and principled," our experiments "strong and unusually fair," and our analysis "insightful." We address the questions below.

(W1, Q1) Reasoning overstates the mechanism. Really a variable-length latent VAE policy. How to reconcile? Show task where latent sequence exhibits temporal / planning, or clarify the analogy.

We clarify that LMP does not follow the conventional interpretation of reasoning in a reinforcement learning setting, where more reasoning leads to higher return. We consider the behavior cloning setting, and here more reasoning corresponds to a better fit of the expert distribution. However, whether or not this directly translates to higher success depends on the property of the task. For example, in the DROID real-world tasks where recovery is important, more reasoning steps tend to improve performance (Figure 9 (b)). We additionally find that on a canonical multimodal benchmark, Push-T, LMP shows a strong correlation between the number of test-time reasoning steps and the coverage score. The mean coverage score and standard deviation over 3 seeds are shown in the following table, and we have added these results to Figure 9 (b) of the revised paper.

Max Latent Steps0246810121416
Push-T coverage score0.51 +- 0.070.64 +- 0.050.80 +- 0.040.92 +- 0.030.95 +- 0.010.96 +- 0.010.97 +- 0.010.97 +- 0.010.96 +- 0.01

(Q3) Compare to fixed length VAE policy with a larger codebook

We train 1-step LMP with larger codebooks of size 16, 256, and 4096, matching the capacity of a vocabulary of 4 over 2, 4, and 6 steps, respectively. The following table shows mean success +- std over 3 seeds on robomimic square. We find that while vocab size 16 and 256 follow their autoregressive counterparts closely, vocab size of 4096 shows regression (compared to the closest neighbor from Figure 9.c, i.e. 4x8=65536). This suggests that the autoregressive latent space does more than provide capacity: it facilitates learning structured codes to populate the latents (e.g., because of shared prefixes).

Capacity16 (4^2)256 (4^4)4096 (4^6)65536 (4^8)
1-step LMP0.76 +- 0.030.81 +- 0.030.75 +- 0.02-
Autoregressive LMP (Vocab 4)0.77 +- 0.010.80 +- 0.03-0.82 +- 0.03

(W4, Q4) Tokenizer no report of average token length and decoding overhead.

We report the average token length, decoding latency, and total inference latency in the following table (evaluated on a single H200 with batch size 1). We find that, on average, LMP tokens are shorter than OAT tokens and decode faster. FAST produces the longest codes but decodes the fastest because it does not go through a learned decoder. Combining token generation and decoding, LMP and OAT have comparable overall latency. FAST's longer token sequences outweigh its faster decoding, resulting in the highest overall latency.

TokenizerVocabRobomimic token lengthRobomimic decode latency (ms)Robomimic total latency (ms)LIBERO-90 token lengthLIBERO-90 decode latency (ms)LIBERO-90 total latency (ms)
LMP-tok647.671.4128.47.341.4125.3
OAT-64648.001.6728.68.001.6828.7
OAT-100010008.001.6828.78.001.6828.7
FAST102421.170.08577.013.320.08348.1

(W3, Q2) DROID no variance / CI. LIBERO gains are marginal.

We report the DROID results with confidence intervals as well as the per-trial outcomes.

DROID Results with Confidence Interval

The following table shows success rate over 20 trials, with 95% Wilson confidence intervals. Bold marks methods whose success rate lies within the 95% confidence interval of the best method. We further provide p-values from paired McNemar tests of LMP-π against each baseline. Against OpenVLA, LMP-π is significantly better (paired McNemar p < 0.05) on every measure except clean-table ≥1/3. Against DP, the per-task difference is significant on marker-mug (p = 0.031) and clean-table ≥2/3 (p = 0.008), borderline on peg-hole (p = 0.057) and clean-table 3/3 (p = 0.065), and not significant on block-bowl (p = 0.27). Pooling the paired trials across the four tasks, LMP-π succeeds where DP fails on 34 trials versus 7 the other way, which is statistically significant (p = 2.5 × 10^-5).

block-bowl (zero-shot)marker-mug (zero-shot)peg-hole (finetuned)clean-table ≥1/3 (finetuned)clean-table ≥2/3 (finetuned)clean-table 3/3 (finetuned)
OpenVLA0.30 [0.15, 0.52]0.00 [0.00, 0.16]0.30 [0.15, 0.52]0.80 [0.58, 0.92]0.30 [0.15, 0.52]0.05 [0.01, 0.24]
DP0.40 [0.22, 0.61]0.25 [0.11, 0.47]0.30 [0.15, 0.52]0.80 [0.58, 0.92]0.55 [0.34, 0.74]0.25 [0.11, 0.47]
LMP-π0.65 [0.43, 0.82]0.55 [0.34, 0.74]0.70 [0.48, 0.85]0.95 [0.76, 0.99]0.95 [0.76, 0.99]0.60 [0.39, 0.78]
p vs. DP0.270.0310.0570.250.0080.065
p vs. OpenVLA0.0390.0010.0210.380.00020.001

Per-trial DROID results

1 = success, 0 = failure. For clean-table, the entry is the number of objects cleared out of 3.

TaskMethod1234567891011121314151617181920
block-bowlOpenVLA10001010010110000000
DP10001000110010100110
LMP-π11001011011101011011
marker-mugOpenVLA00000000000000000000
DP01100000000000100110
LMP-π01110000111010100111
peg-holeOpenVLA00111010010001000000
DP00000011110000101000
LMP-π10011110111111010101
clean-tableOpenVLA2/31/31/32/31/33/31/31/31/30/30/32/32/31/30/31/30/32/31/31/3
DP3/33/31/31/30/31/31/31/30/32/30/32/32/33/32/33/33/30/32/32/3
LMP-π2/33/33/33/30/33/33/33/32/32/32/33/33/33/33/33/32/32/33/32/3

We have added the CI and per-trial results as Tables 6 and 7 in the revised paper.

We emphasize that while the aggregate LIBERO results may seem marginal, the individual task results in Figure 12 show that DP success rates taper off significantly on the low-success tasks. The bottom-10 result is intended to convey this in a tabular format.

(W2, Q2) No training-cost or stability comparison against baselines.

Training on DROID for 150k gradient steps took ~28h for LMP and ~21h for DP on 32xH100s. Training on single-task Robomimic Square for 100k gradient steps took ~6.5h for LMP and ~2.5h for DP on 4xA100s. On the tokenizer side, training on LIBERO-90 for 100k gradient steps took 2.5h for LMP-tok on 1xA100 and 2h for OAT on 1xA100. VQ-VAE took 0.5 h, and FAST took 1.5 min on CPU. LMP is generally slower than baselines because of the explicit latent-space rollout and buffer collection overhead.

Thank you for your response, I will keep my score.

Reviewer_sgy9OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes a way for robot policies to “reason” before acting without using natural language. Instead of mapping an observation directly to an action, the policy generates a variable-length sequence of discrete latent tokens and decodes it into an action chunk. The method formulates this process as variational inference, using a decoder variance schedule to encourage longer sequences when their improved action likelihood justifies the cost, and optimizes the latent sequences with an RL-style objective. Its contributions are the general framework (LMP), a control policy (LMP-π) that adaptively allocates test-time computation, and an action tokenizer (LMP-tok) derived from the same framework. Experiments on real-world DROID and simulated LIBERO, D3IL, and RoboMimic benchmarks show that LMP-π improves on established baselines in the reported evaluations, particularly on difficult multitask problems. Also, LMP-tok substantially outperforms competing tokenizers in downstream policy learning. Ablations support the contributions of variable-length latents and the length penalty, while further analyses suggest that additional latent steps primarily expand behavioral diversity and that computation varies with action characteristics. The authors acknowledge that the sampling-based training procedure is sensitive to hyperparameters and susceptible to latent collapse.

Critical strengths
  1. LMP-π generates variable-length sequences of discrete latent tokens, terminating through a learned EOS decision. A decoder variance schedule penalizes longer sequences unless improved action reconstruction justifies the additional computation. A PPO-style objective optimizes latent inference within an imitation-learning framework. Unlike ECoT's explicit language-based reasoning traces and RD-VLA's threshold-based stopping rule, LMP learns latent computation directly for action prediction. On DROID, LMP-π outperforms Diffusion Policy and OpenVLA across all six reported task measures, with notable gains on more complex tasks.
  2. LMP-π performs strongly across simulated benchmarks, including LIBERO-90, where it achieves 93.3% overall success and 64.5% on its ten lowest-scoring tasks, compared with 46.3% for Diffusion Policy. Beyond the control policy itself, LMP-tok substantially outperforms VQ-VAE, FAST, and OAT in downstream policy learning when evaluated with the same autoregressive policy architecture. Ablations further support the contributions of variable-length latents and compression. These results strengthen the case that the framework offers benefits beyond a single task or model instantiation, supporting its broader relevance to robot learning.

Together, these results provide substantive evidence that the method's design choices translate into practical performance gains.

Critical weaknesses

The empirical advantage over existing generative policies is not uniform, and its statistical reliability is insufficiently characterized. The OpenVLA baseline is a custom 1B Llama 3.2 implementation without vision-language pretraining; although intended as a parameter-matched control, this limits the comparison’s representativeness of the published model. More importantly, Appendix D.2 shows that Diffusion Policy achieves comparable single-task performance; it numerically exceeds LMP-π on five of ten D3IL and RoboMimic tasks and ties on two, with average success rates of 0.73 versus 0.76 on D3IL and 0.85 versus 0.83 on RoboMimic. Although LMP-π performs better on the reported DROID and LIBERO-90 aggregate metrics, the DROID evaluation uses only 20 trials per task and reports no uncertainty estimates. Consequently, the evidence supports the effectiveness of LMP-π but less clearly establishes the consistency and statistical reliability of its advantage over existing policies.

Questions for authors

DROID results are averaged over 20 trials per task, but Table 1 does not report uncertainty estimates. Could the authors provide per-trial success outcomes and confidence intervals for the comparisons between LMP-π, Diffusion Policy, and OpenVLA, ideally accounting for the shared initial configurations? Could they also quantify uncertainty for the test-time truncation results used to support the recovery-behavior interpretation? This helps clarify how robust the real-world performance differences are.

3 comments

We thank the reviewer for the positive review and for finding our experiments provide "substantive evidence that the method's design choices translate into practical performance gains." We address the questions below.

(W1, Q1) DROID results have insufficient statistical reliability. Provide per-trial success outcomes and confidence intervals.

We report the DROID results with confidence intervals as well as the per-trial outcomes.

DROID Results with Confidence Interval

The following table shows success rate over 20 trials, with 95% Wilson confidence intervals. Bold marks methods whose success rate lies within the 95% confidence interval of the best method. We further provide p-values from paired McNemar tests of LMP-π against each baseline. Against OpenVLA, LMP-π is significantly better (paired McNemar p < 0.05) on every measure except clean-table ≥1/3. Against DP, the per-task difference is significant on marker-mug (p = 0.031) and clean-table ≥2/3 (p = 0.008), borderline on peg-hole (p = 0.057) and clean-table 3/3 (p = 0.065), and not significant on block-bowl (p = 0.27). Pooling the paired trials across the four tasks, LMP-π succeeds where DP fails on 34 trials versus 7 the other way, which is statistically significant (p = 2.5 × 10^-5).

block-bowl (zero-shot)marker-mug (zero-shot)peg-hole (finetuned)clean-table ≥1/3 (finetuned)clean-table ≥2/3 (finetuned)clean-table 3/3 (finetuned)
OpenVLA0.30 [0.15, 0.52]0.00 [0.00, 0.16]0.30 [0.15, 0.52]0.80 [0.58, 0.92]0.30 [0.15, 0.52]0.05 [0.01, 0.24]
DP0.40 [0.22, 0.61]0.25 [0.11, 0.47]0.30 [0.15, 0.52]0.80 [0.58, 0.92]0.55 [0.34, 0.74]0.25 [0.11, 0.47]
LMP-π0.65 [0.43, 0.82]0.55 [0.34, 0.74]0.70 [0.48, 0.85]0.95 [0.76, 0.99]0.95 [0.76, 0.99]0.60 [0.39, 0.78]
p vs. DP0.270.0310.0570.250.0080.065
p vs. OpenVLA0.0390.0010.0210.380.00020.001

Per-trial DROID results

1 = success, 0 = failure. For clean-table, the entry is the number of objects cleared out of 3.

TaskMethod1234567891011121314151617181920
block-bowlOpenVLA10001010010110000000
DP10001000110010100110
LMP-π11001011011101011011
marker-mugOpenVLA00000000000000000000
DP01100000000000100110
LMP-π01110000111010100111
peg-holeOpenVLA00111010010001000000
DP00000011110000101000
LMP-π10011110111111010101
clean-tableOpenVLA2/31/31/32/31/33/31/31/31/30/30/32/32/31/30/31/30/32/31/31/3
DP3/33/31/31/30/31/31/31/30/32/30/32/32/33/32/33/33/30/32/32/3
LMP-π2/33/33/33/30/33/33/33/32/32/32/33/33/33/33/33/32/32/33/32/3

We have added the CI and per-trial results as Tables 6 and 7 in the revised paper.

(Q1) Uncertainty quantification for the test-time truncation results in the real world. Clarify robustness of truncation results.

We report the CI for test-time truncation results in the following table.

Task / Max Latent Steps0816
block-bowl0.55 [0.34, 0.74]0.65 [0.43, 0.82]0.65 [0.43, 0.82]
marker-mug0.40 [0.22, 0.61]0.55 [0.34, 0.74]0.55 [0.34, 0.74]

The results do not establish statistical significance. To reliably show the effect of test-time truncation on multimodality, we additionally evaluate on a canonical multimodal benchmark, Push-T. Results are shown in the following table (3 seeds, mean +- std). We find that truncating the latent steps significantly lowers the Push-T coverage score. We have added the Push-T latent truncation results to Figure 9 (b) in the revised paper.

Max Latent Steps0246810121416
Push-T coverage score0.51 +- 0.070.64 +- 0.050.80 +- 0.040.92 +- 0.030.95 +- 0.010.96 +- 0.010.97 +- 0.010.97 +- 0.010.96 +- 0.01

Thank you for your response. I appreciate your acknowledgment that the DROID truncation results are not statistically significant. The Push-T results support the mode-coverage claim, whereas the explanation of recovery behavior on DROID should be framed as a hypothesis. Please ensure that Sec. 4.3 reflects this distinction.

I would also request that the main text state that DP performs comparably on single-task benchmarks and that the OpenVLA baseline is a non-pretrained reimplementation. With these revisions, I consider my concerns adequately addressed, and I'll increase the soundness score to 3.

Thank you for your feedback! We will incorporate these suggestions into the final revision.

Reviewer_6CG3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Latent Memory Palace (LMP), a framework that formulates adaptive reasoning for robotic control as variational inference over variable-length autoregressive latent sequences. The proposed method uses a posterior-prior architecture and PPO-style optimization to learn latent representations of expert actions, with a length-dependent decoder variance encouraging adaptive computation. Experiments across simulated and real-world robotic manipulation tasks demonstrate competitive policy performance, while the resulting action tokenizer achieves substantial improvements over existing tokenization methods.

Critical strengths
  1. The paper provides a principled formulation of latent reasoning through autoregressive variational inference. The combination of a learned posterior, prior, and variable-length latent sequences offers a flexible framework for modeling continuous robotic actions.
  2. The method demonstrates promising empirical performance across multiple simulated and real-world benchmarks.
Critical weaknesses
  1. The adaptive stopping mechanism is not fully convincing. The decoder learns an additional variance scale, so the stopping behavior is not determined by latent length alone. Table 3 shows that stronger compression improves performance, but this could simply be a regularization effect.
  2. It's unclear how much of the improvement actually comes from latent reasoning. Figure 6 compares against fixed-length models with different representation capacities. Also, the authors acknowledge that longer latent sequences mainly capture more behavioral modes rather than refine decisions.
  3. LMP-tok uses a vocabulary of 64, while OAT and FAST use around 1,000, making token prediction potentially easier for the downstream policy.
Questions for authors
  1. Figure 5 reports only success rates, without reconstruction errors or average token lengths.
  2. The paper claims adaptive test-time computation, but only reports latent sequence lengths. Could the authors provide actual inference latency and FLOPs compared with Diffusion Policy, and show whether adaptive lengths offer a better success-versus-compute trade-off?
2 comments

We thank the reviewer for the positive review and for finding our method "principled", "offering a flexible framework," our experiments "promising." We address the questions below.

(W3) Tokenizer dimension mismatch

We thank the reviewer for pointing out this mismatch. FAST and OAT used their default vocabulary sizes of 1024 and 1000 respectively, while LMP used a vocabulary size of 64. This indeed introduces an additional confound of vocabulary size. To remove this confound, we run an additional set of comparisons to OAT tokenizers instantiated with sequence length 8 and vocabulary size 64, matching the LMP config. We find that the downstream policy performance for OAT-64 is marginally better than OAT-1000 (0.48 vs 0.47, 0.33 vs 0.31), but still below LMP-tok. This shows that the lower performance of OAT is not a result of a larger vocabulary. We do not compare to FAST-64 because 64 tokens is fewer than the number of quantization bins for FAST (81), and reducing the quantization bins would lead to much longer codes. We have updated Table 9 in the paper with the new OAT results.

TaskVQVAEFASTOAT-1000OAT-64LMP-tok
robomimic-lift0.82 ± 0.060.98 ± 0.020.97 ± 0.020.90 ± 0.051.00 ± 0.00
robomimic-can0.32 ± 0.080.57 ± 0.120.57 ± 0.080.72 ± 0.060.93 ± 0.02
robomimic-square0.02 ± 0.020.42 ± 0.100.32 ± 0.080.32 ± 0.030.67 ± 0.10
robomimic-tool-hang0.00 ± 0.000.00 ± 0.000.00 ± 0.000.00 ± 0.000.17 ± 0.02
robomimic average0.290.490.470.480.69
libero-soup-cheese0.18 ± 0.050.22 ± 0.130.15 ± 0.040.17 ± 0.160.92 ± 0.05
libero-mug-mug0.28 ± 0.020.10 ± 0.040.23 ± 0.020.22 ± 0.100.88 ± 0.06
libero-moka-moka0.00 ± 0.000.02 ± 0.020.17 ± 0.080.20 ± 0.000.63 ± 0.08
libero-bowl-drawer0.02 ± 0.020.37 ± 0.090.60 ± 0.070.52 ± 0.080.75 ± 0.08
libero-book-caddy0.23 ± 0.060.40 ± 0.070.42 ± 0.100.53 ± 0.150.57 ± 0.06
libero average0.140.220.310.330.75

(Q1) Tokenizer reconstruction errors and average token lengths

We report the reconstruction error and average token lengths of LMP-tok and baselines in the following table. We find that LMP-tok in fact has the highest action reconstruction error. However, its downstream policy has the highest success. This suggests that the structure of the LMP tokens makes it easier for the downstream policy to learn.

TokenizerVocabRobomimic mean MSERobomimic mean token lengthLIBERO-90 mean MSELIBERO-90 mean token length
LMP-tok640.00717.670.00347.34
OAT-64640.00438.000.00288.00
OAT-100010000.00288.000.00128.00
FAST10240.000421.170.000313.32

(W1) Adaptive stopping not convincing. Decoder learns an additional variance. Stronger compression improves performance, but could be regularization.

The decoder variance schedule sets an upper bound on the variance, while the learnable scale provides additional flexibility. In practice, we find that the effective variance at convergence still follows a decreasing trend. The following table compares the max and average effective variance of the LIBERO-90 policy.

Termination step246810121416
Max σ² (x10^-4)73.639.821.511.76.33.41.91.0
Mean Effective σ² (x10^-4)4.273.002.081.681.251.070.780.51

We agree that stronger compression improving performance could be a regularization effect. Indeed, stronger compression forces the model to explain actions with fewer tokens, encouraging information sharing. This is similar to how weight decay regularizes by fitting the data with a simpler model class. We emphasize that compression is compatible with other forms of regularization. For example, our training runs use weight decay of 1e-6.

(W2) Unclear how much improvement comes from latent reasoning. Figure 6 only compared against fixed length models with different representations.

In Figure 9.c, we ablate the training-time latent size, and find that LMP with fewer reasoning steps at small vocabulary size performs worse. However, this could merely be a result of capacity reduction. To test this, we train 1-step LMP with larger codebooks of size 16, 256, and 4096, matching the capacity of a vocabulary of 4 over 2, 4, and 6 steps, respectively. The following table shows average success over 3 seeds on robomimic square. We find that while vocab size 16 and 256 follow their autoregressive counterparts closely, vocab size of 4096 shows regression (compared to the closest neighbor from Figure 9.c, i.e. 4x8=65536). This suggests that the autoregressive latent space does more than provide capacity, but facilitates learning structured codes to populate the latents (e.g., because of shared prefixes).

Capacity16 (4^2)256 (4^4)4096 (4^6)65536 (4^8)
1-step LMP0.76 +- 0.030.81 +- 0.030.75 +- 0.02-
Autoregressive LMP (Vocab 4)0.77 +- 0.010.80 +- 0.03-0.82 +- 0.03

(Q2) Claim test-time computation but only report latent sequence lengths. Provide actual inference latency and FLOPs compared with diffusion policy.

We report the inference latency and FLOPs of LMP-π compared to DP. On a single H200 with batch size 1, the LIBERO-90 LMP-π checkpoint averages 10.2 latent steps with inference latency of 75.1 ms and 517 GFLOPs. DP with 10 DDIM steps has an average inference latency of 72.8 ms and 531 GFLOPs. This is expected because both models are instantiated with matching parameter counts and have about the same number of model calls. At comparable latency and with fewer FLOPs, LMP-π outperforms DP across the 90-task LIBERO evaluation (0.933 vs 0.909 overall, 0.645 vs 0.463 on bottom-10 tasks).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.