acceptodds
Under review as a conference paper at ICLR 2027

Keeping Memory In-Context:Noise-Matched Training for Long-Horizon Robot Control

17 views

Abstract

Long-horizon robot control requires acting on information that has left the current observation. Memory-augmented vision–language–action policies address this by storing past observations and retrieving them into the policy's context, and recent work has steadily improved what is stored and how it is retrieved. Such systems nevertheless fall far short of what the same policy achieves when the relevant memory is handed to it directly. We show that much of this gap lies not in the memory but in its use: the memory does reach the policy, but the policy was trained on memory in a form different from the one it receives at deployment, and so it does not read it. We propose , which closes the training–deployment mismatch behind this read-out gap by training the policy to read the memory it will be given at deployment. reaches 42.0 on with the same backbone as the benchmark's own memory mechanisms, matching or exceeding the per-task scores of the strongest published configuration on 7 of its 16 tasks, reaches the best published M(1) average on (83.8), and on two physical arms raises success from 25.0 to 47.5 on a retrieval task and from 36.7 to 66.7 on an occlusion task. Whether a policy uses memory is decided less by how much is stored than by how it was trained to read it.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action, robotic manipulation, memory, in-context learning, imitation learning, long-horizon manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_wDWROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that memory-conditioned VLA policies underuse memory because they are trained on annotated context but deployed on context written by a learned predictor, therefore suffer from a distribution shift. The authors add two components: 1. Noise-matched training; during training, the ground-truth subgoal is corrupted with wrong subgoals, empty context and coordinate jitter, at rates measured on the deployed predictor. For frame memory, NMT means training under the deployment frame-selection rule. 2. Counterfactual read-out probing (CRP). It measures how much the actions change under a memory swap, and is used for checkpoint selection. Results are shown on RMBench, RoboMME, a channel-corruption study and two real-robot tasks.

Critical strengths
  • NMT makes the policy robust to a corrupted or uninformative memory channel. It scores 28.8 vs. 16.9 when 75% of episodes carry a wrong subgoal, and 18.1 vs. 6.9 with a placeholder. This is the claim the evidence supports cleanly, and it matters for any system whose memory module can fail.
  • The frame-layout result is a useful caution. The same weights score 42.2 under their training layout and 24.5 under another. Comparisons of frame-memory mechanisms are therefore sensitive to the frame-selection rule, not only to the memory design.
Critical weaknesses
  • NMT is not shown to help at deployment, which is the paper’s central claim.
    • With the deployed predictor, the clean-trained and NMT policies score the same (27.5, 44/160; Table 3). Gains appear only under synthetic corruption of the channel.
    • None of the headline results isolate NMT:
      • RoboMME uses frame memory, with neither the predictor nor ν. Its 42.0 equals the 42.2 the same weights reach under their own training layout.
      • RMBench has no with/without-NMT comparison.
      • On hardware, the clean-context policy appears only in the probe, with no success rate. The frame variant, which uses no NMT, matches the NMT variant (19/40 on ARX).
    • The evidence supports robustness to a corrupted memory channel, not a closed read-out gap.
  • Neither of ROBIN’s distinguishing ideas is shown to matter.
    • There is no comparison of noise matched to the predictor’s error rates against generic context noise or dropout.
    • There is no comparison of ρ-based checkpoint retention against success-only selection.
    • Without these, ROBIN reduces to context-noise augmentation. Its robustness benefit is what the cited noise-injection and conditioning-dropout work (Laskey et al., 2017; Ho & Salimans, 2022) would lead one to expect, so the contribution beyond them is not established.
Questions for authors
  • Why do the two policies “coincide by construction” with the deployed predictor (l.394–395)?
  • What is the clean-trained ARX policy in Fig. 3c? Why would it ignore an always-correct context, when in Table 3 the clean-trained policy follows its context closely?
2 comments
AuthorsOpenReview ↗

Response to Reviewer wDWR

We thank Reviewer wDWR for the sharp, focused critique and for highlighting both our channel-corruption robustness and frame-layout findings. We ran every requested control under a matched 10k-step budget from the released checkpoint (all manuscript revisions are highlighted in blue).

W1 — Does NMT help at deployment? NMT prevents catastrophic collapse when the memory channel fails while maintaining statistical parity on live online predictors. As you note in your first strength, resilience to a failing memory module matters for any learned predictor. Our new paired panels establish three facts:

  1. Resilience under channel failure: Annotated-context BC falls to 6.88% on an uninformative placeholder and 2.50% under 100% wrong subgoals, whereas NMT reaches 18.12% on the placeholder (+11.25, p = 1.4e-03) and ~15% under 100% corruption (+12 points across two seeds; Table 3(a); Common Responses 1 and 4).
  2. Driven by counterfactual wrong subgoals: Neither clean continuation nor abstention-only dropout reproduces this protection on either seed (Table 3(b); Common Response 2).
  3. Clean-vs-online trade-off: Weighing context against vision trades ceiling on a noiseless oracle (69.38% vs 86.25%; Sec. 4.3; App. I, Fig. 11) but incurs no statistically significant loss on live online predictors—matching annotated training on validation (27.50 vs 27.50), leading nominally on unseen test scenes (20.31 vs 17.81, 320 paired episodes, p = 0.26), and remaining at parity across three weaker predictors (App. I, Table 10; Common Response 3). Because real predictor errors occur in runs (78% follow-on error rate; App. J, Table 11), such failure stretches arise in real rollouts.

W2 — Which results isolate NMT. Common Response 5 (now opening Sec. 4) maps every table to its isolating comparison. On subgoals, Table 3 and four controls isolate Eq. 6 (Common Responses 1–2). On frames, Table 2's greyed row isolates training under the deployed rule on identical weights (24.5 → 42.0, 14/16 tasks up), recovering the 42.2 of the original layout at 1/8th training cost while placing 10 vs 2 of 32 frames in the last 30 steps and setting Counting SOTA (k6P1 W3). RMBench compares full systems under official protocol (best M(1) mean 83.8 vs 79.0). On ARX hardware, both ROBIN variants lift memoryless π0.5 from chance (25.0) to 47.5, while Figure 3(c) directly contrasts clean-context BC against ROBIN (0.0097 noise floor vs 0.045; Q2).

W3 and the requested ablations. Every requested ablation is now run under the same 10k budget from the released checkpoint (Table 3(b); App. H, Tables 7–9; Common Response 2): Eq. 6 is tested directly against clean-context training with an identical pipeline, which it beats by +11.88 points at 100% corruption (p = 6.6e-05) and by +13.12 on a second seed; it beats context dropout by +10.62 (p = 1.5e-03; +9.38 on the second seed), with both margins reproducing on a second NMT training run in the identical control pipeline (+8.12, p = 7.2e-03; +6.88, p = 0.035). Both fourfold continuation and generic 15%/15% wrong-subgoal noise retain the gain (14.38 against 15.62, p = 0.85), showing that counterfactual wrong-subgoal exposure—rather than exact rate matching—is the active mechanism, with empirical predictor rates providing a tuning-free default; and validation success vs CRP select the same 25k checkpoint (App. K, Table 12; dQu3 Q1). Crucially, this separates NMT from conditioning dropout (Ho & Salimans, 2022), where empty slots mark absence explicitly and leave the policy credulous to wrong subgoals, and from DART (Laskey et al., 2017), which perturbs expert actions rather than retrieved context.

Q1 — Why do the two policies "coincide by construction"? That phrasing was imprecise and is replaced in Sec. 4.3 with the paired episode breakdown (App. I, Fig. 11; Common Response 3): equal 27.50 totals, 28 of 160 episodes decided differently (14:14, p = 1.00), reflecting the dose-response crossover band rather than construction.

Q2 — What is the clean-trained ARX policy in Fig. 3c, and why does it ignore context unlike Table 3's clean policy? It is π0.5 fine-tuned on the ARX demonstrations with annotated subgoals in its prompt, without NMT (now stated in Figure 3's caption). The difference stems from data incentives: on RoboMME, the current view never identifies the target, forcing BC to read the subgoal (86.25 clean vs 2.50 wrong). On ARX demonstrations, memory matters only in a brief initial window (removing the prefix raises ROBIN's action error by 38.6% over steps 1–30 vs 2.7% full-episode; App. C), giving standard BC negligible gradient on the memory slot—so swapping memory moves its actions by just 0.0097 (noise floor) vs 0.045 for ROBIN.

AuthorsOpenReview ↗

brief check-in

Dear Reviewer,

As the discussion period draws to a close, we wanted to share a brief three-line summary of the controls run directly from your review checklist, in case a quick index is helpful alongside our full reply above:

  • Deployment behaviour & clean-to-corrupted trade-off (W1, Q1; Sec. 4.3, Table 3a, App. I Fig. 11 & Table 10): We replaced "coincide by construction" with the exact paired breakdown (14:14 discordant on validation; 20.31% vs 17.81% on 320 unseen test episodes, p = 0.26; parity across three weaker predictors) and plotted the full clean-to-corrupted curve (trading 69.38% vs 86.25% noiseless ceiling for 15.00% vs 2.50% under channel failure and 18.12% vs 6.88% on placeholders).
  • Matched 10k-step ablations & checkpoint selection (W3; Table 3b, App. H Tables 7–9, App. K Table 12): Under an identical 10k pipeline from the released checkpoint, NMT beats clean-context continuation (+11.88 at 100% corruption, p = 6.6e-05; +13.12 on seed 7) and context dropout (+10.62, p = 1.5e-03), both reproduced in a 2nd NMT training run; generic 15%/15% and 4× rates confirm counterfactual wrong-subgoal exposure is decisive (with empirical rates as a zero-tuning default); and validation success selects the same 25k checkpoint across all 8 candidates.
  • Channel isolation & ARX clean-BC baseline (W2, Q2; Sec. 4, Table 2, Fig. 3c, App. C): Sec. 4 now opens with a table isolating each channel's intervention, and Fig. 3c / App. C clarify why clean-context ARX BC collapses to the 0.0097 CRP noise floor (vs 0.045 for ROBIN) when memory matters only in the initial 30 steps.

Please let us know if any question remains open or if we can clarify anything further before the discussion window closes. Thank you again for your sharp and helpful feedback!

Reviewer_dQu3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the training–deployment distribution mismatch in memory-enhanced VLA models for long-horizon robotic manipulation. The authors propose ROBIN, a framework designed to improve how VLA policies utilize memory information during inference. The method consists of two main components: (1) noise-matched training (NMT) and (2) counterfactual read-out probing (CRP). The method is evaluated on RMBench, RoboMME, and two real-world robotic platforms. The results suggest that matching the memory distribution between training and deployment can improve robustness and encourage memory-dependent behavior.

Critical strengths
  • The paper identifies the training–deployment distribution gap in memory-conditioned VLA models, which is an important issue worth investigating, particularly when memory information is generated by imperfect predictors during deployment.
  • The authors conduct experiments on two simulation benchmarks and two real-world robotic platforms, providing evidence of the potential usefulness of the proposed approach.
Critical weaknesses
  • The scope and generalizability of the study should be better clarified. There are multiple approaches to incorporating memory into VLA models, including explicit memory storage and retrieval, visual token compression, and recurrent hidden states. To my understanding, this paper mainly focuses on memory mechanisms where historical information is explicitly retrieved or represented as additional context for the policy. Although alternative memory architectures are mentioned in the related work, the scope of the proposed approach and its applicability to other memory mechanisms are not sufficiently discussed.
  • The paper is difficult to follow in several places, partly because important concepts are introduced without sufficient background or intuitive explanations. For example, the terms "noise-free" and "piecewise constant" (line 43) are used before their meanings in the context of memory-conditioned policies are clearly established.
  • The practical relevance of the demonstrated training–deployment mismatch is not sufficiently justified. The frame-memory experiments demonstrate a substantial performance degradation when different frame-sampling strategies are used during training and inference. However, this distribution shift appears to be artificially introduced, since the same frame-sampling strategy could straightforwardly be used in both stages. It is therefore unclear whether this experiment reflects a realistic challenge encountered in memory-enhanced VLA deployment or simply demonstrates the expected sensitivity of a learned policy to changes in its input distribution.
  • The memory predictor is insufficiently evaluated, and its training–deployment distribution shift is not adequately investigated. The memory predictor plays an important role in NMT, as its estimated error distribution determines how the memory inputs are perturbed during policy training. However, the paper provides only limited evaluation of the predictor. Appendix B reports overall confusion and abstention rates, but a more comprehensive evaluation of subgoal prediction accuracy, spatial localization error, and temporal consistency would be valuable. More importantly, the memory predictor itself may experience a training–deployment distribution shift. During deployment, the predictor receives observations and historical trajectories generated by the learned VLA policy, which may differ from those in expert demonstrations.
Questions for authors
  • How is counterfactual read-out probing used for policy selection? The paper introduces read-out probing to measure the sensitivity of predicted actions to changes in memory inputs. However, it is unclear how this metric is combined with task success or other criteria to select the final policy checkpoint.
  • How are historical visual observations integrated into the VLA backbone? The paper states that multiple historical frames are processed through the existing image pathway of the pretrained pi0.5 backbone. However, the precise mechanism for combining visual features across different timestamps is unclear.
  • How are the 48k DAgger-style corrective samples collected, and how much do they contribute to predictor performance? Appendix B mentions additional on-policy corrective samples for training the memory predictor, but the collection protocol is unclear. Which policy generates these samples, and how are the corresponding ground-truth subgoals obtained?
2 comments
AuthorsOpenReview ↗

Response to Reviewer dQu3

We thank Reviewer dQu3 for recognising the training–deployment memory gap and our multi-platform evaluation, and for the constructive questions below (all manuscript revisions are highlighted in blue).

W1 — Scope and generality. We explicitly state in the revised Abstract, Sec. 1, and Sec. 5 that ROBIN targets retrieved in-context memory (textual subgoals or selected frames). Both components operate on the context interface rather than backbone internals: NMT models errors on context writes, while CRP requires only substituting memory inputs at fixed observation, extending naturally to latent memory banks or recurrent states in future work.

W2 — Clarity. "Noise-free" (always correct) and "piecewise constant" (changing only at phase boundaries) are now defined at first use in Sec. 1, contrasted against live predictor outputs, and Secs. 3–4 are reorganised cleanly around the two memory channels (Common Response 5).

W3 — Practical relevance of frame-layout mismatch. This mismatch arises whenever a pretrained policy is paired with a memory selector whose index schedule differs from pretraining—standard whenever a new retrieval/sampling rule is attached to an existing VLA checkpoint, or when comparing memory modules whose selection schedules differ alongside their architectures. Its impact is large: changing only the rule on fixed weights drops success by 17.7 points (42.2 → 24.5; and 40.8 → 15.5 on a second checkpoint; Sec. 4.2, Table 2), exceeding the spread among the top three published systems in Table 2 (38.0, 42.4, 44.5). Post-training for 10k vs 80k steps under the deployed rule recovers 42.0 and sets Counting SOTA.

W4 — Comprehensive predictor evaluation and distribution shift. Across 29,076 unseen-scene rollout records (App. J, Table 11; full breakdown in CRCB Q2), template accuracy is 71.39% overall and 80.14% inside the 30-step memory-dependent window, with 10.3 px median localisation error (30.1 px mean). Three additional diagnostics characterise its behaviour:

  • Temporal consistency (29,076 records across 1,153 rollouts, logged every 16 steps): Predicted segment counts match ground truth (median ratio 1.00). Within a constant true phase, predictions switch 12.9% of the time, with A→B→A flicker in 3.7% of triples; 65% of true phase changes are detected at the first logged step (90th-percentile lag 32 steps). On the deployed validation panel, the median episode has 3 predicted segments (matching held-out demos) with 3.7% flicker.
  • Distribution shift: As you anticipate, the predictor shifts from demonstrations to rollouts: trained on demonstrations alone, it scores 84.2% on held-out demonstrations vs 70.8% on policy rollouts of unseen scenes, a 13.4-point drop (see Q3 for corrective data).
  • Recoverability: A diagnostic probe on episode-disjoint splits reaches 31.2% (per-task majority), 52.5% (current frame only), and 60.4% (memory + current frame)—confirming memory contributes +7.9 points to subgoal identification.

Q1 — CRP in checkpoint selection vs success-only selection. Candidates are filtered by validation success and ranked by CRP so checkpoints scoring without using memory are rejected. Across all eight checkpoints of the run on the 160-episode validation panel under the deployed predictor (App. K, Table 12):

step2.5k5k10k15k20k25k30k40k
success (%)15.6216.8823.7524.3822.5027.5024.3825.00

Validation success itself peaks at 25k, coinciding with the deployed checkpoint, so reported scores do not depend on CRP selection (25k vs runner-up 40k: +2.50, 11:7 discordant, p = 0.48; per-checkpoint ρ values were not archived separately for this sweep, and CRP's main empirical role is diagnosing shortcutting as on ARX in Fig. 3c).

Q2 — Integrating historical frames into π0.5. Historical frames pass through the pretrained vision encoder and enter as memory tokens with sinusoidal timestamp embeddings, doubling max sequence length (App. M). A structured attention mask prevents current-view vision tokens from attending to memory tokens while allowing action tokens to attend to both, preserving pretrained visual features.

Q3 — Collection and impact of the 48k DAgger corrective samples. We roll out the policy online, record visited trajectory features, and label each step with the simulator's exact oracle subgoal on validation scenes (disjoint test-split verification in Sec. 4.1, Table 3a; App. J, Table 11). On unseen-scene rollouts, corrective samples narrow the demo-to-rollout gap from 13.4 to 9.1 points and raise rollout accuracy by 0.55 points (70.84 to 71.39)—confirming that predictor data collection alone leaves a substantial deployment error rate that NMT is needed to handle.

AuthorsOpenReview ↗

brief checkin

Dear Reviewer, As the discussion window nears its close, we wanted to provide a brief three-point map of how the revised manuscript addresses your questions, for quick reference alongside our response above:

  • Scope, clarity & frame-mismatch relevance (W1–W3, Q2; Abstract, Sec. 1, Sec. 3–5, Table 2, App. M): We explicitly scoped ROBIN to retrieved in-context memory, defined "noise-free" and "piecewise constant" at first use, detailed π0.5 historical-frame token/mask integration (App. M), and verified frame-rule mismatch on two checkpoints (42.2 → 24.5 and 40.8 → 15.5, exceeding published mechanism spreads and recovered in 10k vs 80k steps).
  • Unseen-scene predictor evaluation & distribution shift (W4, Q3; Sec. 4.1, App. J Table 11): Across 29,076 unseen-scene rollout records, we added memory-window accuracy (80.14% vs 71.39% overall), localisation error (10.3 px median), temporal consistency (3.7% flicker, median segment ratio 1.00), demo-to-rollout shift (84.2% → 70.8%, narrowed by DAgger), and disjoint feature probing (+7.9 pts from memory).
  • Checkpoint selection (Q1; App. K Table 12): Across all eight candidate checkpoints (2.5k–40k), validation success under the deployed predictor peaks at the same 25k checkpoint (27.50%), confirming our headline results do not rely on custom selection, while CRP diagnoses memory-ignoring shortcutting on ARX (Fig. 3c).

Please let us know if you have any remaining questions or if any further clarification would be useful before the discussion period ends. Thank you again for your constructive review!

Reviewer_k6P1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

The paper studies how memory-augmented vision-language-action (VLA) policies use historical information during robot control. The authors propose ROBIN, which combines Noise-Matched Training (NMT) and Counterfactual Read-out Probing (CRP). NMT augments ground-truth symbolic subgoals with errors sampled according to the deployed memory predictor's error rates. For visual memory, the authors train the policy using the same historical frame-selection rule used at deployment. CRP measures changes in predicted actions when memory inputs are replaced while keeping the current observation fixed. ROBIN achieves 68.1% success on RMBench and 42.0% on RoboMME. The authors also report improved robustness under synthetic memory corruption and experiments on two physical robot arms.

Critical strengths
  1. The RoboMME experiments show that changing the temporal layout of historical frames reduces success from 42.2% to 24.5%. Fine-tuning under the new layout recovers performance to 42.0%. This is an interesting observation that highlights the importance of consistent memory representations during training and deployment.

  2. The authors conduct controlled corruption experiments and demonstrate substantial improvements under severely unreliable symbolic memory. The physical robot experiments also show improvements over memory-less policies on retrieval and occlusion tasks. These results provide useful empirical evidence about memory-conditioned robot control.

Critical weaknesses
  1. NMT provides no improvement under normal deployment conditions.

The main concern is Table 3, where NMT and clean-context training both achieve 27.5% success under the actual deployed predictor. Improvements appear primarily under artificially intensified corruption. This raises questions about the practical benefit of matching predictor errors during training. Comparisons with standard context dropout, generic noise augmentation, and training directly with predicted subgoals are missing.

  1. The noise distribution is insufficiently justified.

NMT matches the frequencies of three error categories, but the proposed operator does not account for temporal correlations, systematic subgoal confusions, or dependencies between errors and observations. Incorrect subgoals are sampled uniformly, which may differ considerably from realistic predictor behavior. The authors should demonstrate whether matching the actual predictor statistics provides any advantage over simpler corruption strategies.

  1. The symbolic and visual memory procedures are poorly unified.

For symbolic memory, NMT introduces artificial errors into ground-truth subgoals. For visual memory, the procedure consists primarily of aligning training and deployment frame-selection rules. The paper provides little justification for treating these substantially different interventions as a unified method. Furthermore, the strongest RoboMME experiment recovers performance after deliberately changing the frame layout, reaching approximately the same success as the original configuration. The additional benefit of the proposed training procedure remains unclear.

  1. CRP provides insufficient evidence of correct memory usage.

CRP measures action differences after replacing the memory input. A policy may produce substantially different actions without interpreting the memory correctly. The paper does not evaluate whether counterfactual memory substitutions lead to the appropriate actions. The contribution of CRP to checkpoint selection is also unclear, as there is no direct comparison with conventional selection based on validation success.

  1. Limited improvements over existing memory methods.

ROBIN achieves 68.1% on RMBench compared with 67.8% for EventVLA, with worse performance on M(n) tasks. On RoboMME, it achieves 42.0% compared with 44.5% for FrameSamp-Modul. These results provide limited evidence of a consistent improvement over existing approaches. The RMBench experiments also use per-task continuation and inference-time sampling for selected tasks, complicating attribution of the gains. The physical robot experiments lack a direct comparison between clean-context training and NMT.

  1. Limited applicability to other memory architectures.

The paper evaluates symbolic subgoals and selected historical frames. Many modern memory-augmented VLA policies use recurrent hidden states, learned memory tokens, or latent memory banks. It is unclear how NMT and CRP would operate with these representations, especially when memory states evolve recurrently and depend on the entire observation history. The generality of the proposed approach remains untested.

  1. Presentation and clarity.

The paper requires considerable effort to follow, particularly in the method and experimental sections. Some explanations are repetitive and overly abstract, making it difficult to identify the main technical ideas. The relationship between symbolic subgoal corruption and visual frame alignment could be explained more clearly, and several important implementation details are deferred to the appendices. A more concise presentation with concrete examples and a clearer connection between the proposed methods and experiments would improve readability and help readers assess the contributions.

Questions for authors
  1. Why does NMT provide no improvement under the actual deployed predictor in Table 3? Have the authors compared it with standard context dropout, generic noise augmentation, or training directly on predictor-generated subgoals?

  2. How closely does the synthetic corruption distribution match the predictor's actual errors? In particular, how are temporal correlations and systematic subgoal confusions handled? How sensitive are the results to the corruption probabilities?

  3. What is the advantage of the visual frame-alignment procedure over training with the intended frame layout from the beginning? Why is the alternative layout preferable when the original configuration already achieves comparable success?

  4. How does CRP verify that a policy uses memory correctly? Could the authors evaluate counterfactual histories with known correct actions and report whether CRP-based checkpoint selection improves success over standard validation-based selection?

  5. How would ROBIN work with recurrent memory, learned memory tokens, or latent memory banks? Can the authors provide evidence that the approach generalizes to memory mechanisms beyond symbolic subgoals and historical frame selection?

  6. How does NMT handle cases where corrupted memory makes the expert action impossible to infer? Since Eq. 7 retains the original expert action as the training target, could this introduce conflicting supervision and encourage the policy to ignore memory or exploit dataset biases?

  7. How are the training and validation data separated? Section 4.1 and Appendix B indicate that both the 48000 DAgger-style corrective samples and the controlled ablation panel come from the validation split. Are the episodes and environment seeds used for predictor training disjoint from those used for evaluation and checkpoint selection?

  8. How much of the reported performance can be attributed to ROBIN? On RMBench, the improvement over EventVLA is only 0.3 percentage points, while ROBIN uses per-task continuation and inference-time sampling for selected tasks. Could the authors provide comparisons under matched training and inference budgets, as well as a direct clean-context versus NMT comparison on physical robots?

5 comments
AuthorsOpenReview ↗

Response to Reviewer k6P1 (1/2) [W1–W3, Q1–Q3]

We thank Reviewer k6P1 for the detailed, constructive evaluation and for noting both our frame-layout finding and robustness under corrupted memory. Below we address every weakness and question directly with new controls and measurements (all manuscript revisions are highlighted in blue).

W1, Q1 — Deployed predictor tie; dropout, generic noise, and training on predicted subgoals.

  • Why the two policies tie on validation: NMT trains the policy to weigh its subgoal against visual evidence, trading ceiling on an unattainable noiseless oracle (69.38% vs 86.25%; Sec. 4.3; App. I, Fig. 11) for +12.50 points when the channel fails (15.00% vs 2.50%) while remaining at statistical parity across live online predictors—27.50 each on validation, 20.31 against 17.81 on unseen test scenes (p = 0.26), and no significant loss across three weaker predictors (App. I, Table 10; Common Response 3). The synthetic per-query curves cross at 20%–35% corruption, qualitatively matching the regime of the predictor's 28.6% unseen-scene template error.
  • Why channel failure is realistic: On unseen rollouts, a wrong prediction is followed by another 78% of the time (vs 29% base rate; App. J, Table 11), producing long runs of wrong subgoals within real episodes; NMT lifts 100%-wrong episodes from 2.50% to 15.00% and cuts placeholder degradation from 6.88% to 18.12% (Table 3(a)).
  • Dropout vs generic noise: NMT beats context dropout by +10.62 at 100% corruption (p = 1.5e-03; Table 3(b), App. H; Common Response 2), whereas dropout fails to separate from annotated training. Fixed-rate 15%/15% noise with wrong subgoals retains the gain (14.38 against 15.62, p = 0.85), showing that counterfactual wrong subgoals—rather than exact rate matching—drive robustness.
  • Training directly on predicted subgoals: On episodes the predictor was trained on, it is 98.6% correct, against 71.4% on unseen scenes—a channel 27 points cleaner than deployment, which collapses toward clean continuation (3.75% at 100% corruption). Sampling from Eq. 6 exposes the policy to failure modes without requiring held-out rollouts.

W2, Q2 — Realism of Eq. 6: systematic confusions, temporal persistence, and rate sensitivity. Eq. 6 models both properties (clarified in Sec. 3.2–3.3):

  • Systematic confusions & persistence: Wrong templates come from Z\mathcal{Z}, each prompt's table of same-task phases (115 prompts), and draws are keyed by (episode, subgoal segment) so errors persist across the segment rather than redrawing per frame.
  • Match to real predictor errors (unseen rollouts, App. J, Table 11; dQu3 W4): Real errors are strongly persistent (78% follow-on rate vs 29% base rate) and broad (246 distinct pairs; the ten most frequent cover 46%). A majority (57%) name another phase of the same episode—37% of confusions name the preceding phase, against 16% for the following phase—matching Eq. 6's same-task design.
  • Sensitivity: Robustness is invariant across rates (8%/5%, 35%/10%, 15%/15%; App. H, Tables 7–8; Common Response 2) and across test-error structures: whether test corruption is drawn independently at every policy query with out-of-task templates, or uses same-task templates held within a segment (App. H, Table 9), the margin is the same (13.12 against 1.25, p = 6.6e-05; Common Response 1).

W3, Q3 — Unifying both channels, and what frame alignment buys. Both channels instantiate one principle—train on the deployment read-out context (Sec. 3.2, Sec. 4.2, Table 2):

  • Unification: On symbolic subgoals, deployment differs from annotation via predictor errors (closed by Eq. 6); on visual frames, via the frame-selection schedule T(t) (closed by post-training under the deployed T(t)).
  • Magnitude & cost (Q3): Holding weights fixed, changing only T(t) drops success from 42.2 to 24.5 (p = 4e-22), and on a second checkpoint from 40.8 to 15.5 (800 paired episodes, 229:27, p = 5e-41, 15/16 tasks lower). Alignment recovers 42.0 (p = 6e-22) in 10k steps—1/8th of full 80k training—letting pretrained checkpoints adopt new memory schedules cheaply.
  • Why the deployed layout: It places 10 of 32 frames in the last 30 steps (vs 2 for uniform sampling) while bounding the max gap at 26 steps (App. E), setting new SOTA on PickXtimes (94.0) and SwingXtimes (98.0).
AuthorsOpenReview ↗

Response to Reviewer k6P1 (2/2) [W4–W7, Q4–Q8]

W4, Q4 — Correct memory use vs action sensitivity; checkpoint selection.

  • Sensitivity vs semantic correctness: CRP (ρ) is a rollout-free test of causal action sensitivity—a necessary condition that diagnoses memory-ignoring shortcutting (such as clean-context BC on ARX at the 0.0097 noise floor vs 0.045 for ROBIN)—whereas semantic correctness is evaluated via rollouts: 86.25% task success with the true subgoal vs 2.50% with a wrong one, and ROBIN dropping from 27.50 to 20.62 when its predictor loses history (p = 0.035).
  • Checkpoint selection: Across all 8 checkpoints (2.5k–40k; dQu3 Q1, App. K), validation success already peaks at 25k (27.50%), agreeing with the deployed checkpoint, so headline scores do not rely on custom selection.

W5, Q8 — Gains over prior work and attribution.

  • Benchmarks: Gains concentrate where distant memory is decisive. On RMBench, ROBIN achieves the best published M(1) mean (83.8 against 79.0), best 9-task mean (68.1), and #1 on 5/9 tasks—largest on single-event recall (Observe&Pick 38 against 21, Swap-T 96 against 87, Press Button 8 against 3). Neither task marked † (inference sampling) is one of ROBIN's five task wins, and all rows use the official 50-demo / 100-rollout protocol. On RoboMME (same π0.5 backbone), ROBIN sets SOTA on PickXtimes (94.0 against 92.7), SwingXtimes (98.0 against 92.0), and the Counting suite (66.0 against 65.2), tying #1 on VideoPlaceOrder (36.0).
  • Attribution: Table 2's greyed row isolates frame alignment on identical weights (24.5 → 42.0, 14/16 tasks up). On ARX hardware, Figure 3(c) compares clean-context BC directly against ROBIN: without NMT, action sensitivity stays at the 0.0097 noise floor (4.6× below ROBIN's 0.045; wDWR Q2), leaving the policy invariant to recalled targets where memoryless π0.5 scores 25.0% vs ROBIN's 47.5%.

W6, Q5 — Recurrent or latent memory. We explicitly scope the paper (Abstract, Sec. 1, Sec. 5) to retrieved in-context memory (subgoals or selected frames). Both tools act on the context interface: NMT requires an error model on context writes, while CRP requires only swapping memory states at fixed observation (applicable to latent banks or recurrent states in future work).

W7 — Presentation. We reorganised the manuscript around the two channels, opening Sec. 4 with the channel-mapping table (Common Response 5), moving Eq. 6's template/persistence rules and per-sample T(t) construction into Sec. 3.2–3.3, and defining all flagged terms at first use.

Q6 — Supervision under corrupted memory. Under abstention, acting from the current view is well-posed; under confusion, the expert action remains the desired target when a subgoal is wrong. Three results confirm this does not teach the policy to ignore memory: (i) CRP moves ROBIN's actions 4.6× further than clean BC (Figure 3(c)); (ii) degrading predictor memory lowers ROBIN's task success, from 27.50 through the deployed predictor to 20.62 when that predictor loses its history (6:17, p = 0.035); and (iii) quadrupling the confusion rate changes no cell across seven corruption levels (Common Response 2).

Q7 — Train/validation separation. Tables 1–2 and Table 3's synthetic corruption rows never query the predictor. The 48k DAgger samples come from validation scenes and touch only the deployed-predictor rows; re-evaluating those rows on the strictly disjoint test split confirms the finding: 20.31 for ROBIN vs 17.81 for annotated training across 320 paired episodes on all 16 tasks (23:15 discordant, p = 0.26; Table 3(a), Sec. 4.1).

AuthorsOpenReview ↗

brief checkin

Dear Reviewer,

As the discussion period draws to a close, we wanted to share a concise quick-reference to the controls and analyses added for your review, in case it helps streamline your check of our detailed two-part response above:

  • Deployment trade-off, disjoint test split & controls (W1, Q1, Q7; Sec. 4.3, Table 3, App. H–I, Fig. 11): Weighing subgoals against vision trades noiseless-oracle ceiling (69.38% vs 86.25%) for channel-failure resilience (15.00% vs 2.50% at 100% wrong; +11.88 over matched clean continuation, +10.62 over context dropout, replicated across seeds and a 2nd NMT run) while maintaining statistical parity across live online predictors—including 20.31% vs 17.81% on 320 paired episodes of the strictly disjoint test split (p = 0.26).
  • Eq. 6 realism & same-task test errors (W2, Q2, Q6; Sec. 3.2–3.3, App. H Table 9, App. J Table 11): Across 29,076 unseen-scene rollout records, real predictor errors show 78% follow-on persistence and 57% same-episode phase confusions; under segment-held same-task test confusions, NMT retains the exact same advantage (13.12% vs 1.25%, p = 6.6e-05), while still depending on predictor memory online (27.50% → 20.62% without history, p = 0.035).
  • Channel unification, CRP selection & scope (W3–W7, Q3–Q5, Q8; Sec. 3–4, Table 2, Fig. 3c, App. K Table 12): Changing only the frame schedule drops fixed weights 42.2 → 24.5 and 40.8 → 15.5 across two checkpoints, recovered in 10k (1/8th) steps; validation success peaks at the same 25k checkpoint across all 8 candidates; and we explicitly scope ROBIN to retrieved in-context memory with a channel-isolation table opening Sec. 4.

Please let us know if anything remains unclear or if we can clarify any point further before the discussion window closes. Thank you again for your thorough feedback!

Reviewer_CRCBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ROBIN explores context augmentation and alignment to improve memory use in π₀.₅ robot policies. It combines noise-matched training for textual subgoals, aligned historical-frame selection for visual context, and counterfactual memory probes for checkpoint selection. Validation spans RMBench, RoboMME, controlled context perturbations, and two real-robot tasks, reporting competitive task performance and improved robustness to corrupted context.

Critical strengths
  1. Counterfactual probing of memory dependence. CRP tests whether memory affects policy actions by substituting the memory while holding the current observation fixed. Together with the frame-layout interventions, the experiments show that historical context can influence both action predictions and task success. This provides a useful diagnostic beyond success rate alone for assessing whether a policy uses its memory inputs.

  2. Robustness to unreliable subgoals. Under controlled interventions, ROBIN outperforms the clean-context baseline. This provides concrete evidence that context corruption during training can improve resilience to memory-channel failures.

Critical weaknesses
  1. Limited novelty For textual subgoals, replacing, withholding, and perturbing the context according to predictor error rates appears to be a form of context regularization, yet the paper does not compare it with standard alternatives such as context dropout or fixed-noise augmentation. For historical-frame inputs, the central prescription is to match frame sampling between training and deployment, which should ordinarily be part of a consistent experimental setup. The additional methodological contribution of either approach is therefore insufficiently established.

  2. Mismatch between method and evidence. The method devotes substantial attention to textual subgoal construction and noise-matched training, while the main benchmark results in Tables 1 and 2 use historical-frame inputs. These results do not directly validate predictor-specific subgoal regularization. The paper needs clearer separation of the two configurations and corresponding ablations to establish which mechanisms support its claims about improved historical memory utilization.

  3. effectiveness of historical memory utilization. The textual interface decouples historical understanding from action generation through predicted subgoals. However, these predictions are not consistently reliable, So, whether the predictor is effectively using the available historical information? NMT makes the downstream action policy more tolerant of incorrect subgoals, but this does not resolve the concern about inadequate memory utilization in the upstream prediction stage or establish that the system uses history more effectively.

  4. computational cost. Under RMBench’s fixed-offset sampling, consecutive inputs such as [0, t−300, t−60, t−30, t−15, t] and [0, t−299, t−59, t−29, t−14, t+1] share only the anchor frame, limiting straightforward reuse of the previous input’s computation. Caching historical visual features could reduce encoding costs but introduces storage requirements and does not necessarily eliminate subsequent context computation. Without details on token counts, caching, memory consumption, and inference latency, the practical cost of the multi-frame approach remains unclear.

Questions for authors
  1. Does matching the predictor’s error statistics when training provide additional benefits over simply regularization (simply perturbing or replacing subgoals with proper probabilities, et, al) ?

  2. What is the subgoal prediction accuracy, particularly at decision points that require historical information? Since historical understanding and action generation are decoupled through predicted subgoals, this would help assess how effectively the predictor uses memory.

  3. Could the authors clarify how historical-frame contexts are constructed during training and inference? Specifically, during training, are historical-frame contexts assembled independently according to T(t) for each sample, or maintained incrementally through a streaming cache? How is insufficient history handled at the beginning of an episode? During inference, how are action-chunk execution, observation collection, and replanning coordinated to ensure that the frames required by T(t) are available and that context construction remains consistent with training? Does satisfying this sampling schedule impose significant constraints on the execution cadence?

  4. Could the authors explicitly specify the memory context used in each table, experiment, and ablation—historical image frames, textual subgoals, or both? A concise mapping would help clarify which results support each implementation and methodological claim. I found this correspondence difficult to follow in the current presentation.

5 comments
AuthorsOpenReview ↗

Response to Reviewer CRCB (1/2) [W1–W3, Q1–Q4]

We thank Reviewer CRCB for highlighting CRP and our channel-corruption study as strengths, and for the constructive questions on ablations, predictor memory use, and inference cost. Revisions in the updated PDF are highlighted in blue.

W1, Q1 — Is NMT more than context regularisation; does matching error statistics help? We ran all requested controls under a matched 10k-step budget from the released checkpoint (Table 3(b); App. H, Tables 7–9; Common Response 2). Abstention-only context dropout recovers only 2.50 of the operator's 13.12 points at 100% corruption (p = 0.29 vs annotated training); the remaining +10.62 comes from showing counterfactual wrong subgoals (22:5, p = 1.5e-03, replicated on a second seed, and +6.88, p = 0.035 in a second NMT run in the identical control pipeline). Clean-context continuation is beaten by +11.88 (p = 6.6e-05; +13.12 on a second seed). Thus NMT is not reducible to standard dropout regularisation: withholding marks absence explicitly, leaving the policy credulous whenever a wrong subgoal is present. On Q1, what must match deployment is the presence of counterfactual wrong subgoals, whereas exact rate matching is not required—both fixed-rate 15%/15% wrong-subgoal noise (14.38 against 15.62, p = 0.85) and a fourfold 35%/10% continuation retain the gain—so Eq. 6's empirical rates provide a principled, zero-tuning default.

For historical frames, holding both mechanism and weights fixed, changing only the selection rule drops success by 17.7 points (42.2 → 24.5; Sec. 4.2, Table 2)—exceeding the spread among the top three published configurations in Table 2 (38.0, 42.4, 44.5). Post-training under the deployed rule recovers 42.0 in 10k steps (1/8th of full 80k training) and sets SOTA on RoboMME's Counting suite (66.0; k6P1 W3).

W2, Q4 — Which memory context each result uses. Common Response 5 maps every table to its memory channel and isolating comparison, and the revised paper opens Sec. 4 with this table: Eq. 6's subgoal operator is isolated by Table 3 and Common Response 2's controls; frame-rule alignment is isolated by Table 2's paired before/after row (24.5 → 42.0).

W3 — Does the predictor use the history it is given? Yes, and downstream task success depends on it. Replacing the predictor's history stream with the current frame drops unseen-scene accuracy from 71.4% to 59.5% and costs annotated-context training 11.25 points of task success (27.50 → 16.25, p = 1.2e-04; App. I, Table 10; Common Response 3). Moreover, the predictor is more accurate inside the memory-dependent window than outside it (Q2). NMT acts on the complementary downstream stage, making the policy tolerant of residual errors.

Q2 — Subgoal prediction accuracy, especially at memory-dependent decisions. Evaluated on 29,076 on-policy records from unseen scenes, split by whether the step falls in the first 30 steps after execution begins (where the target must be recalled from memory):

template accuracynlocalisation error (median)(mean)
Memory-dependent window80.14%1,59613.8 px30.4 px
Rest of episode70.88%27,48010.2 px30.1 px
All71.39%29,07610.3 px30.1 px

Template selection is higher inside the memory-dependent window than outside it (80.1 vs 70.9), with comparable localisation error (median 13.8 vs 10.2 px on 256 px images, means equal). Errors occur throughout the episode with heavy-tailed localisation tails, matching Eq. 6's confusion branch and multi-scale jitter (±8 / ±24 / ±64 px; App. J, Table 11).

Q3 — How are frame contexts constructed at training and inference? Contexts are assembled independently per sample from the index rule T(t), not a streaming cache: each sample carries (episode index, step index, execution start index) and the buffer materialises the exact frames T(t) selects. Early in an episode, T(t) keeps frame 0 as the anchor and zero-pads/masks unused slots. At inference the identical rule runs on the live buffer, and because T(t) is a pure function of step index t, it places no constraint on action-chunk cadence: after executing a chunk of any length, frames are simply re-selected at the new step index (Sec. 3.2).

AuthorsOpenReview ↗

Response to Reviewer CRCB (2/2) [W4]

W4 — Computational cost. The context budget is fixed at compile time and does not grow with episode length, on either channel:

memory tokens addedhow it scales
Frame channel512 (32 frames × 16 tokens, one view, token dim 2048)fixed budget; the selection rule always returns at most 32 frames however long the episode
Subgoal channel+64 text tokens (max token length 64 → 128)one subgoal line, fixed

There is no KV cache that grows over an episode and no retrieval index to search. Each frame is encoded once, when it arrives, and its pooled features are cached; at every step the selection rule only chooses which cached frames enter as memory tokens, so consecutive steps never re-encode history, whatever offsets they use. The added attention is also restricted — current-view vision tokens do not attend to memory tokens, only the action tokens do. We measured both channels against the same backbone served without the memory stack, on one A100 at batch 1 (JAX preallocation off in every arm, so peaks are real):

per policy stepmemoryless backbone+ memoryΔ
frame channel, 512 tokens (3 episodes × 1,312 frames)97.98 ms128.78 ms+30.79 ms (+31%)
subgoal channel, +64 text tokens97.76 ms101.79 ms+4.03 ms (+4%)

The frame channel's cost does not grow with episode length: with fewer than 200, 200–500, 500–900 and 900–1,312 frames in the buffer, model latency is 128.68 / 128.74 / 128.89 / 128.80 ms, so a step late in a long episode costs what an early one costs. Encoding is paid once per frame, 5.65 ms, and is flat as well; the feature cache it fills grows host memory by 0.84 GB over a 1,312-frame episode (0.26 GB for the memoryless arm). Peak device memory is 16870 MiB against 8644 MiB for the frame channel and 8656 MiB (+8) against 8648 MiB for the subgoal channel. The frame measurement serves the released frame-sampling checkpoint, which has our frame policy's memory configuration and token budget and differs only in which frames the rule selects. App. M (Table 14) reports these measurements in full.

AuthorsOpenReview ↗

Brief check-in

Dear Reviewer,

As the discussion period draws to a close, we wanted to provide a quick three-line index to the new experiments run specifically for your questions, in case it saves you time reviewing our detailed response above:

  • Dropout, generic noise & rate sensitivity (W1, Q1; Table 3b, App. H): Under an identical 10k-step pipeline from the released checkpoint, NMT beats clean continuation (+11.88 at 100% corruption, p = 6.6e-05) and context dropout (+10.62, p = 1.5e-03, replicated on a 2nd seed and 2nd NMT run), while fixed-rate 15%/15% and 4× (35%/10%) wrong-subgoal recipes retain the gain—making empirical predictor rates a tuning-free default.
  • Upstream predictor history use & accuracy (W3, Q2; App. I Table 10, App. J Table 11): Removing predictor history drops unseen-scene accuracy from 71.4% to 59.5% and task success from 27.50% to 16.25% (p = 1.2e-04); across 29,076 unseen-scene rollout records, accuracy is higher inside the 30-step memory-dependent window (80.14%) than outside (70.88%), with 10.3 px median localisation error.
  • Channel mapping, frame buffer & inference cost (W2, W4, Q3, Q4; Sec. 3.2, Sec. 4, App. M Table 14): Sec. 4 now opens with an explicit channel-mapping table; frame contexts are assembled per sample with no execution-cadence constraint; and per-step latency is constant in episode length (+4.03 ms subgoals, +30.79 ms frames across 1,312-step episodes).

Please let us know if any question remains unclear or if any further detail would be helpful before the discussion window closes. Thank you again for your time and constructive review!

Comments on the paper

8 comments
AuthorsOpenReview ↗

Common Response (1/7): Executive Summary & Index

We thank all four reviewers for their constructive feedback. In response we trained three new control policies and a second NMT policy from the released checkpoint and ran more than forty new evaluation panels, each paired on the same 160 episodes (substantive manuscript revisions are highlighted in blue). Below is an executive summary, detailed in the Common Response and followed by point-by-point replies.

1. What NMT buys: resilience under channel failure with online-predictor parity (k6P1 W1, wDWR W1). A policy trained on annotated context obeys its subgoal unconditionally: success tracks the channel from 86.25 down to 2.50 as the subgoal goes from correct to wrong, and to 6.88 on an uninformative placeholder. Under NMT, the same backbone weighs the subgoal against visual observations—cutting placeholder degradation (18.12 vs 6.88, +11.25, p = 1.4e-03) and keeping 15.00 under 100% wrong subgoals, six times as much (+12.50 and +11.88 on two evaluation seeds, p ≤ 3.1e-04; Common Responses 1 and 4). While trading ceiling on an unattainable noiseless oracle (69.38 vs 86.25; App. I, Fig. 11), this incurs no significant loss across live online predictors: parity on validation (27.50 each, Table 3), nominal lead on unseen test scenes (20.31 against 17.81, 320 paired episodes, p = 0.26), and parity across three weaker predictors (Common Response 3).

2. Where the gain comes from: counterfactual wrong subgoals rather than abstention-only dropout (CRCB W1, Q1; k6P1 W1, W2; wDWR W3). All controls train from the released checkpoint under a 10k-step budget (Common Response 2). Clean-context continuation gives no protection: NMT beats it by +11.88 (p = 6.6e-05), and by +13.12 on a second seed. Conditioning dropout fails as well, because withholding marks absence explicitly while leaving the policy credulous to wrong subgoals: NMT beats dropout by +10.62 points (p = 1.5e-03, replicated on a second seed), and both margins reproduce on a second NMT run in the identical control pipeline (+8.12, p = 7.2e-03; +6.88, p = 0.035). Thus counterfactual wrong-subgoal exposure—not exact rate matching—is decisive: empirical predictor rates (8%/5%, a tuning-free default), a 4× continuation (35%/10%), and generic 15%/15% noise all retain the gain at 100% corruption (+8.12 to +13.12 over annotated-context training, every p ≤ 4.4e-03).

3. Which result isolates each mechanism (CRCB W2, Q4; wDWR W2; k6P1 W3). Both channels instantiate one principle—train the policy on the context distribution it will read at deployment. Common Response 5 maps every table to its channel and isolating ablation: Eq. 6's noise operator on textual subgoals (Table 3 and four controls), and post-training under the deployed frame-selection rule on visual frames (same weights before and after: 24.5 → 42.0 on RoboMME, 14 of 16 tasks higher, in 1/8th of full training steps).

Requested by the reviewsResultWhere
NMT against clean-context training with an otherwise identical pipeline (wDWR)NMT +11.88 at 100% corruption and +13.75 at 75%, replicated on a second seed and on a second training runCommon Response 2
Context dropout (CRCB W1, k6P1 W1, wDWR W3)NMT +10.62 over dropout, replicated on a second seed and on a second training run; dropout does not separate from annotated-context trainingCommon Response 2
Generic or fixed-rate noise; sensitivity to the rates (CRCB Q1, k6P1 Q2, wDWR W3)three wrong-subgoal recipes at different rates all keep the gain: no level to tuneCommon Response 2
Train/validation separation (k6P1 Q7)conclusion unchanged on a held-out test split, 320 paired episodesCommon Response 3
Predictor accuracy where memory is needed, localisation, temporal consistency, its own shift (CRCB Q2, dQu3 W4)measured on 29,076 records from unseen scenesCRCB Q2, dQu3 W4
A test corruption with the structure of the predictor's own errors (k6P1 Q2)same-task wrong subgoals, kept within a segment: NMT 13.12 against 1.25 (p = 6.6e-05)Common Response 1
A counterfactual with a known correct action (k6P1 Q4)86.25% with the correct subgoal, 2.50% with a wrong onek6P1 W4
CRP against success-only checkpoint selection (k6P1 Q4, dQu3 Q1, wDWR W3)both select the same checkpointdQu3 Q1
Inference cost (CRCB W4)+4.03 ms per step for the subgoal channel, +30.79 ms for frames, constant in episode lengthCRCB W4
Statistical reliabilitya second evaluation seed; a second NMT training run; an archived panel re-run (p = 1.00)Common Responses 2 and 4

All per-episode records are released with the revision.

AuthorsOpenReview ↗

Common Response (2/7): CR1 — Channel-Failure Robustness

1. Robustness when the memory channel fails, paired on the same episodes

Table 3's corruption rows are re-run so that every comparison is paired on the same 160 validation episodes, for the predictor's measured rates (8% confusion / 5% abstention) and for a fourfold recipe (35% / 10%). Each time the policy is queried, the subgoal is replaced with the stated probability by a wrong one (a generic "pick up the object" at a random image location; the "out-of-task" rows of the revised Table 3). This test corruption differs from the operator's own, which draws other phases of the same task and holds a draw within a subgoal segment, so the comparison measures transfer to errors the policy was not trained on.

Queries corruptedAnnotated contextNMT 8%/5%vs annotatedNMT 35%/10%vs annotated
75%16.8828.75+11.88 (37:18, p = 0.014)30.00+13.12 (29:8, p = 7.5e-04)
100%2.5015.00+12.50 (22:2, p = 3.6e-05)10.62+8.12 (16:3, p = 4.4e-03)

All four margins are significant, and Common Response 4 repeats both rows under a second evaluation seed: at 100% both margins reproduce within 1.25 points and stay significant. (Protocol note: auditing the evaluation harness showed that in subgoal-injection mode the corruption flag was drawn per policy query rather than latched across an episode—making the submission's two 75% rows two runs of the same per-query process, Common Response 4—and full 160-episode re-evaluation updates the incomplete 92-episode 100% baseline cell from 3.1% to 2.50%.)

The margin is the same under the operator's own kind of error. Most of the deployed predictor's confusions name another phase of the same episode (57% on unseen scenes; k6P1 W2), so we repeated the 100% row with wrong subgoals of that kind: another phase of the same task at a random location, kept until the annotated phase changes (for instructions with no entry in the operator's table, 15-20% of draws, the generic wrong subgoal is used). Annotated-context training succeeds in 1.25% of episodes and the deployed NMT policy in 13.12% (+11.88, 21:2, p = 6.6e-05). The fourfold recipe keeps 10.00 (+8.75, 16:2, p = 1.3e-03), and the two 10k-step NMT training runs of Common Response 2 keep 12.50 and 10.62 (+11.25, 19:1, p = 4.0e-05; +9.38, 17:2, p = 7.3e-04). The revised Table 3 carries this row.

A policy trained on annotated context has no alternative to following its subgoal: when the channel emits an uninformative placeholder it drops to 6.88% (whereas NMT substantially cuts this degradation to 18.12%, +11.25, 24:6, p = 1.4e-03; for reference, the memoryless backbone scores 17.9% on the test benchmark), and when the subgoal is wrong at every query of an episode it succeeds in 2.50% of episodes while NMT keeps six times that. Episodes of this kind occur with a real predictor: on scenes it has not seen, a wrong prediction by the deployed predictor is followed by another 78% of the time against a 29% base rate (dQu3 W4), so individual episodes run on a wrong subgoal for long stretches even when average accuracy is high. The 100% row isolates them.

AuthorsOpenReview ↗

Common Response (3/7): CR2 — Controls & Ablations

2. Where the gain comes from: counterfactual wrong subgoals rather than abstention or extra training

Three controls, under a matched 10k-step budget. Starting from the released symbolic-grounded-subgoal checkpoint, each control arm trains for 10k steps in a single verified pipeline, alongside a second NMT policy (second run) trained from the same checkpoint in that exact pipeline, and is evaluated beside the 10k-step checkpoint of our main NMT run. Clean continuation is the clean-context training with an otherwise identical pipeline that Reviewer wDWR asked for: the context is left as annotated. Context dropout switches off the confusion branch of Eq. 6 and moves its probability to abstention (13% in total, the operator's combined rate; the imprecision branch is unchanged), so the only difference from NMT is whether wrong subgoals are ever shown. Generic noise is the fixed-rate augmentation of CRCB W1 and k6P1 W1: 15% of subgoals are replaced by one drawn uniformly from all tasks' subgoals (a pool of 121) and 15% are withheld, redrawn independently for every training sample.

Queries corruptedannotated context+ clean continuation+ context dropout+ generic noise 15%/15%+ NMT 8%/5%+ NMT 8%/5%, second run
75%16.8815.0021.8832.5028.7523.75
100%2.503.755.0014.3815.6211.88

Paired on the same 160 episodes, at 100% corruption:

@100%Δdiscordantp
dropout − annotated+2.506:20.29
operator − annotated+13.1225:41.0e-04
operator − dropout+10.6222:51.5e-03
operator − clean continuation+11.8821:26.6e-05
generic − clean continuation+10.6221:49.1e-04
generic − dropout+9.3821:65.9e-03

The extra fine-tuning is not the source. The clean continuation stays with annotated-context training at both levels (15.00 against 16.88, and 3.75 against 2.50; p ≥ 0.67). Against it NMT gains +11.88 at 100% corruption (21:2, p = 6.6e-05) and +13.75 at 75% (30:8, p = 4.7e-04).

Withholding the subgoal is not the source. Context dropout recovers 2.50 of the operator's 13.12 points and is not separable from annotated-context training (p = 0.29). The remaining +10.62 is carried by showing wrong subgoals, at p = 1.5e-03. At 75% the ordering is the same: dropout +5.00 (20:12, p = 0.22), the operator +11.88 (32:13, p = 6.6e-03), the operator over dropout +6.88 (25:14, p = 0.11).

Both contrasts replicate on a second evaluation seed. We re-ran the annotated, clean-continuation, dropout and operator arms at both levels under seed 7:

seed 7annotated+ clean continuation+ dropout+ operator 10k+ operator 25k
75%17.5014.3816.8828.7526.25
100%3.751.255.0014.3815.62
seed 7Δdiscordantp
@75% dropout − annotated−0.6216:171.00
@75% operator − annotated+11.2531:139.6e-03
@75% operator − dropout+11.8832:136.6e-03
@75% operator − clean continuation+14.3833:106.1e-04
@100% dropout − annotated+1.254:20.69
@100% operator − annotated+10.6220:34.9e-04
@100% operator − dropout+9.3818:31.5e-03
@100% operator − clean continuation+13.1223:21.9e-05

The operator's level is the same on both seeds at 75% (28.75 and 28.75) and within 1.25 points at 100%. Dropout is the arm that moves: its +5.00 at 75% becomes −0.62 on the second seed, and at neither cell on neither seed does it separate from annotated-context training. The operator's margin over dropout is +6.88 and +11.88 at 75%, +10.62 and +9.38 at 100%, and reaches p < 0.01 in three of those four cells. Its margin over the clean continuation is +13.75 and +14.38 at 75%, +11.88 and +13.12 at 100%, with p < 0.001 in all four cells, and on neither seed does the clean continuation separate from annotated-context training.

They also reproduce on a second training run. The last column of the first table is NMT trained a second time from the released checkpoint, in the same pipeline as the three controls. It does not separate from the first run at either level (20:28, p = 0.31; 12:18, p = 0.36), and at 100% corruption it repeats all three contrasts: +9.38 over annotated-context training (18:3, p = 1.5e-03), +8.12 over the clean continuation (17:4, p = 7.2e-03) and +6.88 over dropout (17:6, p = 0.035). At 75% it is ahead of the clean continuation by +8.75 (28:14, p = 0.044).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.