acceptodds
Under review as a conference paper at ICLR 2027

Minimal Recurrent Behavioral Memory for Imitation under Partial Observability

Abstract

What is the least recurrent memory needed to reproduce a specified expert under partial observability? The instantaneous requirement is the conditional entropy of the expert's behavioral quotient, but recurrence must also preserve distinctions that future observations will not restore before use. We characterize this minimal recurrent behavioral memory by a compatibility relation: under transitivity its classes attain the exact minimum, while the general case is an entropy minimization over closed compatible state assignments, with exact certificates on finite instances. A sole-carrier measurement protocol separates behavioral sufficiency, excess code rate, and information carried by observations or other memory paths; experimental bit requirements refer to the induced symbolic behavioral model under the stated occupancy. Across manipulation tasks, learned code rates remain near zero- and two-bit requirements as hidden modes grow to , and anticipatory memory follows a requirement despite zero instantaneous demand during waiting. Learning this representation remains difficult: event-agnostic future-behavior supervision yields sufficient seeds with one frozen configuration and improves the longest-horizon pixel setting from to sufficient held-out seeds (closed-loop success from to ). On unmodified community benchmarks, the protocol certifies delay-independent requirements, which sufficient codes match at mid-delay. The supervision aids commitment but can induce predictive surplus; annealing it lets imitation and rate training reduce that surplus, separating the information-theoretic target from the ability to learn it.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
imitation learning, partial observability, memory, representation learning, rate-distortion, information bottleneck, state abstraction, recurrent policies, behavioral memory, robot manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FnUaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Minimal Recurrent Behavioral Memory (MRBM), which takes a theoretical approach to questioning what is needed for memory and context in robot learning tasks. It proposes a series of proofs that seeks to find the lower bound to the information that is needed for a given task, under certain assumptions. At heart, the paper decomposes memory into two components, in my mind loosely equivalent to short term and long term memory: the current class GE,tG_{E, t} stores memory required by the immediate action, and the behavioral memory \Upgammat\Upgamma_t stores information that should persist further into the future.

Critical strengths

This paper is well motivated, and I really like it's problem formulation. I takes a very principled decomposition to what is the minimum of memory we need to store to maximally perform in a task that requires such memory. In other words, minimize storage (bitwise information) while maximizing sufficiency (to solve the task). I believe this alone sets forth a clear framework for how to think about both memory and context in learning.

The idea of incorporating future observations into consideration for what should be learned is a strong point -- this makes a lot of sense intuitively, as some memory information may only become clear once we see the full task through and is appropriately highlighted as an important point throughout.

Critical weaknesses

While the paper presents a nice theoretical formulation, it's applications are unclear. The writing is filled with empirical numbers and readouts that seem rather specific to the task at hand (ln364, ln349-350, ln320-321 for example). Indeed, it is impossible to match theory with practice always precisely, but in such cases, I am not fully convinced by the experiments that a) the set of assumptions used for the proof is actually necessary in practice, and b) more broadly, that the proposed method has practical applications to real-world robotics.

Put differently, I think the paper's framework and positioning is strong; however, the design decisions and instantiations make me question it's applicability and relevance. I feel that the whole paper is motivated from a simple robotics example, but the method is extremely overfit to solving that example in particular. It is not clear to me as the reader how this method would extend to tasks that have more information requirements, for example beyond 2 bits. The experimental evidence does not demonstrate the broad applicability nor robustness needed to convince me that this framework is viable and practical for applications of memory in robotic tasks. For longer horizon tasks and real-world image observations, it is unclear to me how such a method would cleanly extract out the bits of information to preserve, propagate, and match. The appendix has a thorough number of proofs, though they do not contribute to me being convinced of the general practicality of this method.

Overall, the lack of comparison and connection to other baselines on real-world robotics tasks makes me question whether this framework sees any practical benefit over existing memory architectures. I strongly encourage the authors to perform more real-robot experiments, on harder, long-horizon tasks, and compare against more learning-based methods such as [1] and prove practical and empirical gain. If this is infeasible, I at least encourage authors to do more thorough investigation/comparison to improve the practicality of the proposed framework.

[1] Gated Memory Policy, https://arxiv.org/abs/2604.18933

Questions for authors
  • How does this method extend to real-world applications, where information bits possibly extend beyond 2->1->0? Can the authors please detail out a realistic setting in which their framework, or a learning-based approximation in high-dimensional space, would be practically usable without needing to manually define constraints/information? Which parts of the framework would scale, and which would be less scalable?
  • I am concerned that the theoretical establishments and assumptions limit the broad applicability and practically of the method. Specifically, if the authors want this to work in robotics, can the authors please walk through a concrete example of how their method would apply in a long-horizon task? What is a task where the assumptions hold, and what is a task where the assumption would break? Which components are more brittle than others?
  • As an overall note, I would suggest the authors not dive so deeply into jargonic terms so early, in both appendix, and especially in the introduction. It is extremely unclear what a lot of phrases mean in the introduction, thus hindering the understanding of the paper. For example, please see ln 46-48 in the introduction. Further, in the teaser Figure 1, "recurrent target that accounts for future observations" is a significant part of the design, yet this phrase is hard to parse. Rate level meanings and significance should also be explained earlier, if the introduction is chosen to start so technical.
  • Relatedly, can the authors please clarify what the main takeaway from each table and figure is? It's uncelar which is the baseline, and what the numbers mean relative to performance. It's difficult to parse what is actually helping.
  • I also think figures and tasks would benefit from visual grounding -- could the authors please explain more why the task they chose is appropriate for what they want to demonstrate?
4 comments

Thank you for asking for clearer practical uses, evidence beyond two bits and our own tasks, and a comparison with explicit-history policies. The revision adds public pixel benchmarks and real-robot recordings, and explains the results through three questions: what must be remembered, what was learned, and how much excess code rate remains?

W2 / Q5 — Beyond two bits and our own tasks.

On unmodified MIKASA-Robo RememberColor3/5/9, the finite task models require 1.585, 2.322 and 3.170 bits through a five-step delay. A scripted simulator expert supplies demonstrations. Recurrent learners receive images and proprioception, without previous-action inputs, and use task-informed future-action supervision and image translation augmentation (Section 4.2; Appendix E.6).

  • RC3: 5/5 seeds pass the memory diagnostic. Four retain 1.585 bits and one 1.517; held-out action MSE is 0.0018–0.0044. In the rate-penalty sweep, three runs match both retained information and code rate to the 1.585-bit requirement.
  • RC5: 5/8 seeds pass, retaining 2.302–2.322 bits at code rates of 2.58–3.11, with held-out MSE 0.0025–0.0044. This demonstrates sufficient memory above two bits, with measurable surplus.
  • RC9: 0/4 seeds pass. The best retains 2.231 of 3.170 bits despite a code rate of 3.339 bits. The 0.939-bit shortfall shows why code rate alone cannot establish that the required information was learned.

Passing means meeting the empirical diagnostic, not exact zero error. Information and sufficiency use all recorded episodes; errors use the held-out 20%. Appendix E.6 reports all seed counts, the rate sweep and schedule comparisons. The original three-bit readout result also remains: 23/24 runs pass at K = 24, with rates of 2.99–3.14 bits.

W1 / Q1 / Q2 — Practical use, longer horizons, and components.

The requirement provides an offline reference for choosing memory capacity and distinguishing missing task information from excess rate. Section 7 now gives the workflow: specify the expert and task model; certify what must persist and for how long; measure the learned memory; then evaluate control in closed loop. Enumeration is confined to certification. Learning and deployment do not enumerate histories; exact computation requires an enumerable model or a separate bound construction. Section 5 measures how longer delays reduce learning reliability at fixed requirements.

Appendix B.11 now accounts for all studies' metered compute, approximately 1,272 GPU-hours. Table 3 states the assumptions. Appendix E.9 connects the component controls to their consequences: a persistent hidden state can bypass the measured code; auxiliary prediction can retain surplus; and the smallest theoretically adequate codebook may train unreliably. These results separate the memory requirement from learning reliability and control performance.

W3 — GMP and community real-robot data.

On the public place-back recordings, an eight-direction model requires three bits for median intervals of 8.0 and 12.9 seconds. These include 5.5 and 10.6 seconds with no instantaneous demand. Finer position partitions change the level while preserving the retention profile in the tested models (Appendix E.7).

The released GMP gate opens in 82% of carry frames and 4–7% of frames in the middle of the saucer phase, although both phases require retaining position. GMP keeps history when the gate closes: the gate measures access, while the certificate measures information that must remain available. This explains their complementary roles without interpreting a closed gate as lost memory (Section 6). Appendix E.8 likewise includes the history channel when assessing explicit-history policies; its local-attention experiments do not identify a minimum window.

Small learners retain 1.2–1.4 of the three required bits on these recordings. We have not evaluated hardware closed-loop performance or jointly trained GMP with the certificate.

Q3 / Q4 — Presentation and figure takeaways.

The abstract and introduction now start with a robot remembering and later forgetting a cue. Tasks are explained at first mention, and captions state the findings and compared quantities. Formal terms follow the motivating example. The general response lists all revision locations.

Hi Authors, I appreciate your careful response and clarifications to my question. To be concise -- I am still not convinced my core concern on practicality is addressed.

Taking the authors' response specifically in response to W1/Q1/Q2 (2nd response): "Section 7 now gives the workflow: specify the expert and task model;". If my understanding is correct, this doesn't feel too scalable. In a real-world visuomotor task, defining the relevant behavioral distinctions, what information is available in the observations, and when that information will become available again seems to require substantial task-specific knowledge, is quite the undertaking.

This abstracts away a lot of the decision making and difficulty of the problem, and primes the robot in terms of what it needs to remember. It remains unclear to me how one would construct such a model for a more complex task, a long horizon task, or even a language-specified task (e.g. open-vocabulary).

I understand that enumeration is confined to offline certification, rather than learning or deployment. However, my concern is broader than the computational cost of enumeration. It is whether the model needed for certification can be constructed faithfully in realistic settings, and whether the resulting certificate translates into a useful decision about the learned policy. For example, the three-bit experiments show that a theoretically sufficient codebook does not necessarily support reliable learning. Thus, while the requirement provides a lower bound on capacity, I am not yet convinced that it provides actionable guidance for choosing a memory architecture or capacity in practice.

Nonetheless, I find the theoretical formulation interesting. However, the clarifications mainly explain how to apply the framework under a specified task model, but my concern is upstream -- the specification of the task model seems like a hard assumption that undermines this method's practicality, especially as diagnostic. I don't believe my concerns were sufficiently resolved, and will maintain my recommendation.

Reply to the follow-up (1/2)

Thank you for the follow-up. New Appendices E.10–E.12 address how a task model can be constructed from recordings, how that construction affects policy diagnosis, and how visual history measurements can guide a design choice. Analysis plans were recorded in advance and procedures frozen before confirmation; the supplement documents development changes and reuse of recordings.

1. A finite model built without simulator labels. For the ten low-dimensional recordings of Appendix B.2 we rebuilt the task model from measured observations and expert commands, retaining the recorded cue and phase channels. Behavioral distinctions are clusters of the commanded waypoint and observation classes are clusters of the measured end-effector position, fitted on half of the episodes and frozen; the solver is unchanged. Three recordings served for development and the other seven were run once.

Requirement. Of 261 steps, 253 give exactly the requirement of the labelled model, and across the 70 waiting steps the largest difference is 0.002 bit. Three approach steps in the two tasks with four grasp classes differ by two bits.

When a class becomes observable again is inferred from the recorded observations under the declared tolerance. In readout-3, the labelled model requires one bit at steps 11–13; the reconstruction requires one bit at step 12 and three bits at steps 11 and 13.

The recovered distinctions. At 226 of the 261 steps, observation, behavior and memory classes coincide. The differences arise from unresolved arm sag in Task A (28 steps, below the 5 cm tolerance), a one-step shift in grasp timing in the four-grasp-class tasks (6 steps), and the expert's random hover height becoming visible (one step).

Diagnoses of trained policies. Without retraining, we re-scored all 1,032 eligible saved policies, excluding eight engineering trials by a pre-specified rule. Under the main-text sufficiency gate, 1,029 receive the same verdict (475 pass both, 554 fail both). The three changes occur in the 16-class task: of four runs passing under the labelled model, one passes when the reconstruction tests grasp one step later. Task A verdicts remain unchanged, although second-wait code rates rise by up to 1.0 bit when sag is not counted as observed. Appendix E.10 reports the per-task statistics and held-out variant.

Scope and sensitivity. The cue and phase channels are instrumented inputs and episodes are time-aligned. Results change when the tolerance moves from 5 cm to 2 or 10 cm. At 512 hidden masses, the fitted model covers only 5% of held-out sequences, limiting that dataset's independent validation. On the development recording of readout-2, 11 of 40 verdicts change for the same one-step reason.

2. Visual demonstrations without phase, delay or color labels. On RememberColor with three colors, a frozen procedure receives only camera frames, end-effector position and expert commands. It locates the first step at which commands differ and measures how much earlier frames improve held-out prediction there. Developed on one recording, it was run once on two recordings with new scenes and delays it was not given (20 and 12 steps), locating the first differing step at 26 and 18. Squared error fell from 0.099 to 0.015 and from 0.106 to 0.019: reductions of 0.084 [0.079, 0.088] and 0.087 [0.083, 0.092], and 0.077 on both recordings across unseen arm poses. Gains concentrate in the first five cue frames; later single frames give −0.001 to 0.000. With histories exchanged between episodes, gains are at most 0.0005 in magnitude.

Reply to the follow-up (2/2)

A design choice made before training. A fixed rule applied only to training episodes of the 12-step recording selected 14 frames, reaching the latest frame with a sufficiently large positive prediction gain (Appendix E.11). The same network was trained with 7, 14 and 18 visible frames, two seeds each. Held-out squared command error at the first differing step is 0.132, 0.017 and 0.016. Seven and 14 frames differ by 0.115 [0.109, 0.122], and 14 and 18 by 0.0004 [−0.0008, 0.0016]. Intervals resample held-out episodes after averaging the two seeds. This supports the selected window in an offline comparison. Intermediate windows, closed-loop success and computational savings were not measured.

3. A lower bound from the observation–action law. The best of three pixel discretizations covered 64% of held-out histories at the first differing step; none yielded a reliable finite model. Proposition 6 (Appendix E.12) gives a lower bound without constructing a finite partition. It sums the earlier history's additional predictive information at each future decision, conditioning only on observations and actions available by that decision. It applies to any exact recurrent realization in Definition 2.

Under the ten low-dimensional benchmark models' enumerated joint distributions, this bound equals the certificate at every step: 261 confirmation and 104 development steps. On pixels, a separate measurement gives held-out log losses of about 0.14 bit with cue frames and 1.58 bits without them when predicting clustered action labels. The 1.44-bit improvement measures the earlier history's contribution for the fitted predictors; it is not a certified memory bound. The five-color losses are 0.61 and 2.32 bits.

4. Scope. The low-dimensional experiments connect data-built models to the paper's requirement and policy diagnosis, including changed verdicts. The visual experiments recover behavior-relevant timing and guide a history-window choice before training. Their evidence covers one benchmark with one cue and one decision; automatic construction for long-horizon or open-vocabulary tasks remains untested.

5. Testing the model against recordings. The memory diagnostic exposed a one-step pixel-capture offset that correct placement alone missed (Appendix E.1): the model required one bit through a wait, while every seed carried zero bits and still placed correctly. This prompted a direct observation probe and correction of the recording timing.

6. What the certificate adds once a model exists. The requirement and the sufficiency check distinguish capacity, content and memory paths.

  • Capacity. A three-bit requirement rules out K < 8 for exact reproduction. At K = 8, 10, 12, 16 and 24, the numbers passing our empirical gate out of 24 are 0, 1, 5, 15 and 23; passing runs carry 2.99–3.14 bits. The requirement sets the feasible floor, and the sweep measures how capacity above that floor affects learning reliability under the tested configuration.
  • Content. On RC9 the code carries 3.34 bits against a requirement of 3.17 but retains 2.23 bits of color. Exceeding the rate floor does not ensure the required content.
  • Location. A code at 1.06 bits under a two-bit requirement, with high task success, motivates checking memory outside the measured code; success alone does not locate it.

These measurements distinguish cases that task success alone cannot separate. The certificate is not given to the policy. Public-benchmark learners use task-informed supervision (Appendices E.6–E.7). In Section 5, the frozen event-agnostic configuration passes on 36/40 seeds; neither its base objective nor its auxiliary supervision uses latent labels or event annotations.

Reviewer_ezfAOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper is about how a policy can imitate an expert policy with limited information. Concretely, the paper answers the question: how many bits of additional information does the learner need to match the expert. The instantaneous requirement is the entropy of the expert’s “behavioral quotient” conditioned on the observation. Recurrent policies have a different entropy requirement which is given by entropy of the “class variable” (capital gamma) conditioned on the current observation. The authors prove that the entropy requirement of the recurrent policy is greater than or equal to the entropy requirement of the policy that receives the full history.

The authors test their theory by training imitation policies whose memory is given by a discrete code. Its entropy can be directly measured and compared against theory. On simulated tasks it is shown that the empirical measurements match theory.

Critical strengths

The main theoretical results that I checked appear to be correct (Theorem 1 and Theorem 2). The empirical result (Figure 3) also supports the authors theory.

Critical weaknesses

The main weakness of this work is its presentation. The paper uses a significant amount of jargon in the abstract and introduction that is either never defined or defined later in the paper. For example in the abstract the following non-self-explanitory terms are used:

  • Specified expert
  • Behavioral quotient
  • Sole-carrier measurement protocol
  • Closed compatible state assignments
  • Behavioral sufficiency
  • Zero instantaneous demand
  • Additionally, “task A” is referenced in the main text before the description of what that means.

The empirical results are based on a symbolic partition of the observations that are chosen by the authors. An infinite number of partitions exist and while demonstrating that theoretical results hold on one partition supports the authors results it is not sufficient.

Questions for authors

Clarification request: What is the intended use of this result beyond the benchmarks in the paper? Clarification request: How much do the results depend on the partition of the observations that the authors chose?

2 comments

Thank you for highlighting the early terminology and the dependence on partition choices. We have rewritten the opening around a concrete memory task and added an observation-convention study in Appendix B.2.

W1 — Terminology and task definitions.

The abstract starts with a robot retaining a cue while waiting. It can forget the cue after its last use, or earlier if the cue will be shown again before use. This example introduces the theoretical question before the formal terms. The introduction explains Task A, readout-2 and A′ at first mention; Figure 1 explains the 2 → 1 → 0-bit profile directly. Section 3 then asks whether the required information is present, how much surplus remains, and whether another memory path carries information.

W2 / Q2 — What fixes the grouping, and what can change it?

Task A provides a concrete distinction. With 512 hidden modes, changing the sag and nuisance symbols raises the strong-congruence comparison from 3.06 to 8.95 bits at grasp. This comparison also preserves distinctions in observation supports. Our behavioral requirement stays zero because those distinctions do not change the expert's grasp choice (Appendix B.2).

For a specified expert and observation process, the theoretical grouping follows from behavior and reachability. Current classes group states with the same observation and expert action law. Recurrent compatibility also compares future behavior after every continuation reachable from both histories. Under the stated assumptions and transitivity, these classes give the exact minimum and an attaining recurrent representation. Beyond transitivity, the general formulation supplies bounds and compatible assignments, including matching finite-instance certificates. This defines a behavioral requirement rather than assigning the entropy of an arbitrary partition to memory.

Applying the theory to data still requires observation symbols, action granularity, model support and an evaluation distribution. The new study tests ten datasets under fourteen convention types, yielding 126 applicable combinations. Valid tested conventions that preserve the behaviorally used information leave the requirement at the reported steps unchanged to 10⁻³ bit.

Changes to task information have interpretable effects. Hiding a visible decision class increases memory demand at that decision. Showing the latent again at grasp reduces the earlier waiting requirement from 3 to 2 bits in readout-3 and from 2 to 1 in pI, while leaving their later waiting requirements unchanged. Thus future observations can reduce what must be stored before a cue's last use. Invariance to arbitrary partitions would miss this dependence on the information available to the policy.

Appendix B.2 separately reports invalid models and non-transitive cases, including the overly fine 1 cm visibility threshold. These checks establish consistency within the model; they do not establish complete fidelity to raw observations. The real place-back data provide a complementary sensitivity check: finer position partitions raise the bit level, while all tested partitions preserve the need to retain information between lift-off and return (Appendix E.7).

Q1 — Intended use.

The requirement gives a reference for interpreting learned memory. On public MIKASA-Robo RC5, five of eight recurrent learners using task-informed supervision pass the diagnostic, retaining 2.30–2.32 bits at code rates of 2.58–3.11. The relevant information has been learned, with measurable surplus (Appendix E.6). Section 7 now explains how to specify the model, certify its requirement, compare a learned memory, and evaluate control separately. Table 3 collects the assumptions; the general response lists the revision locations.

We have added an analysis directly addressing your question about how the partitions are chosen and how the conclusions depend on that choice. Appendix E.10 constructs task models from the recordings and checks whether they give the same memory requirements and diagnoses of trained policies.

The observation and action symbols are built by clustering measured end-effector positions and commanded waypoints with one declared tolerance (samples closer than 5 cm are linked into the same cluster), while retaining the recorded cue and phase channels and gripper commands. Clusters are fitted on half the episodes and frozen, without reading simulator class, mass or symbolic-action labels. Given these symbols, the behavior and memory classes are computed by the unchanged solver; they are not chosen by us. Three datasets served for development; the other seven were processed once with the frozen procedure.

On the same recorded episodes, the reconstructed model and the paper's labelled model give identical requirements at 253 of 261 steps, and the 70 waiting steps differ by at most 0.002 bit. At 226 steps, all three partitions—observation, current behavior and memory—coincide. We also re-scored all 1,032 eligible saved policies for these seven datasets without retraining: 1,029 receive the same pass/fail verdict under the paper's sufficiency gate. For 704 runs, this follows from matching classes at the tested steps. In Task A, however, the classes differ at 28 steps and all 224 policy verdicts remain unchanged.

The changed verdicts are informative too. On the 16-class task, the reconstructed model tests the grasp decision one step later: three of the four policies that pass under the labelled model fail under the reconstruction. The same timing difference changes 11 of 40 verdicts on the development dataset readout-2. These cases show where the diagnosis depends on the abstraction, and are reported alongside the agreements.

The reconstruction still assumes instrumented cue and phase channels, time alignment and a declared tolerance; changing the tolerance changes some results. At 512 hidden masses, only 5% of held-out sequences occur in the fitted model, so that dataset cannot stand alone. The evidence supports construction and checking of a model from measurements in these tasks, with explicit sensitivity to modeling choices. It does not establish a unique partition or invariance to arbitrary partitions.

For your question about intended use, Appendix E.11 adds a separate example on visual demonstrations. Without phase, delay or color labels, we measure how much each earlier frame improves held-out prediction of the expert's command at the first step where commands differ across episodes. A rule fixed in advance and applied only to training episodes then selected a 14-frame history window before any policy was trained. With the same network and training budget, two seeds per window, held-out command errors at that step were 0.132, 0.017 and 0.016 for 7, 14 and 18 frames. This supports choosing a history range from the measurement; it is an offline prediction result, not a certified bit budget or closed-loop success.

Reviewer_FrcKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper considers how much minimal recurrent memory an agent requires to reproduce an expert under partial observability when the future observations act as side information. Further, annealing future action supervision allows the paper to mitigate challenges arising from learning in long horizon setups.

Critical strengths
  1. By phrasing what is critical for imitation, i.e. remembering what changes future actions without paying trivial costs for memorizing the world which doesnt inherently help with the goal, the paper focuses its effort on characterizing memory towards the end goal of interest (imitation).
  2. The paper rigorously considers audit rigor which helped catch certain types of bugs; more over the paper has a clear scope that enables it to present ablations detailing what memory is required as opposed to getting into the question of answering why those neural networks struggle to learn etc.
Critical weaknesses
  1. The theory appears to require grouping all possible hidden states which seems like it doesnt help guide how we can apply some of these results to guide practical scenarios all that well, i.e. the theory is fairly removed from practice.
  2. One other weakness is that passing the memory test doesnt imply good controller behavior (cf. the physical weighting task)
Questions for authors

none apart from the weaknesses mentioned above.

3 comments

Thank you for recognizing the measurement discipline and asking us to clarify its practical role. The revision explains how an offline memory requirement helps distinguish missing information, excess code rate, and control failures, with new public-data examples.

W1 — Grouping hidden states and practical applicability.

The certifier enumerates a finite symbolic task model offline. Learning and deployed inference do not enumerate histories. Once the expert behavior, observations and model support are specified, the certificate gives a common reference for comparing learned memories. Exact computation requires an enumerable model or a separate bound construction (Sections 3 and 7).

The new results illustrate three useful outcomes:

  • A learned memory reaches the requirement. On unmodified MIKASA-Robo RC3, 5/5 recurrent learners pass the diagnostic; three additional rate-penalized runs match the 1.585-bit requirement in both retained information and code rate.
  • The required information is present, with surplus. On RC5, 5/8 learners pass, retaining 2.302–2.322 bits at code rates of 2.58–3.11. These public pixel experiments use task-informed supervision and image translation augmentation (Section 4.2; Appendix E.6).
  • The code has enough rate but lacks task information. On RC9, the best learner uses 3.339 bits of code rate but retains only 2.231 of the required 3.170 bits. The resulting 0.939-bit shortfall cannot be diagnosed from code rate alone.

Public real-robot recordings extend the measurement to longer intervals: 1,226 and 288 retained place-back episodes yield an eight-direction requirement of three bits lasting median intervals of 8.0 and 12.9 seconds. Small learned memories retain 1.2–1.4 bits, giving a quantified offline gap (Appendix E.7).

Appendix B.2 examines ten datasets under fourteen observation-convention types. It separates stability when task information is preserved from changes in memory demand when cues are hidden or shown again. Table 3 collects the assumptions needed to interpret the certificate and learned-code comparison.

W2 — Memory sufficiency and successful control.

We agree that passing the memory diagnostic does not guarantee closed-loop success. A pass establishes retention of the required distinctions under the evaluated expert distribution, using the stated empirical threshold. Executing accurate actions and producing the necessary observations remain separate requirements.

The weighing intervention makes this distinction concrete. Expert-led observation generation raises grasp-side accuracy from 0.30–0.32 to 0.93–0.99, while closed-loop execution remains unreliable. This supports retention and use of the cue under controlled observations and identifies sensitivity to sensing and control as a remaining issue (Section 7; Appendix E.3).

The diagnostic is therefore useful alongside control evaluation: it distinguishes a representation that lacks task information from one that retains it under the tested observations but fails during execution. Section 7 now makes both evaluations explicit. The general response lists the changes to the exposition and the added evidence.

Reviewer_FrcKOpenReview ↗

Response to author rebuttal

Thank you for your clarifications. Given the limited confidence I have with respect to my review, I will retain my score as is.

Thank you for reading our response and for letting us know. If any specific point remains unclear, we would be glad to clarify it before the discussion closes.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of revisions

We thank the reviewers for asking us to clarify the practical use, dependence on modeling choices, and evidence beyond the original tasks. The central contribution is a minimum memory requirement that accounts for future observations. Under the stated assumptions and transitivity, we characterize that minimum and construct a recurrent representation that attains it; more general finite models admit bounds and, in some cases, matching certificates.

The added experiments connect this target to learned memories: RC3 reaches the certified code rate, RC5 retains sufficient information above two bits with surplus, and RC9 and public real-robot recordings expose measurable information shortfalls. The revisions retain the original theoretical development and experimental structure.

Guide to the revised paper

  • Abstract, Section 1 and main captions: introduce the cue–wait–decision example before formal terms; explain tasks at first mention and state figure takeaways.

  • Section 3 and Appendix B.2: separate the behavioral definition from modeling choices; test ten datasets under fourteen observation-convention types (126 applicable combinations), including the 512-mode Task A counterexample and future re-observation effects.

  • Section 4.2 and Appendix E.6: add unmodified MIKASA-Robo pixel tasks, per-seed results, the RC3 rate sweep and RC9 schedule comparisons; identify task-informed supervision and image augmentation explicitly.

  • Sections 4.2 and 6; Appendices E.7–E.8: analyze public real-robot recordings, the released GMP gate and explicit-history readers, distinguishing access to history from retained information.

  • Section 7, Table 3 and Appendix E.9: give a practical workflow, collect the assumptions, and explain component controls and the distinction between memory diagnostics and closed-loop performance.

  • Reproducibility statement and Appendix B.11: describe the supplementary code and results across all studies, and update metered compute by study to approximately 1,272 GPU-hours and 714 CPU-only tasks.

Our individual responses give the evidence for each reviewer's questions.

We have added Appendices E.10–E.12 to address how task models can be constructed from data and how the diagnostics can inform a practical choice. The main text is unchanged from our previous revision.

  • E.10: Task models reconstructed from low-dimensional recordings without simulator class labels, and comparisons of the resulting memory requirements and diagnoses of saved policies, including cases where the verdict changes.
  • E.11: Behavior-relevant temporal dependence estimated from visual demonstrations, and an offline evaluation of a history window selected before policy training.
  • E.12: A prediction-based lower bound from the observation–action law, exact checks under the benchmark models, and separate visual log-loss measurements. The visual losses are not certified memory requirements.

The updated supplement includes the analysis scripts, result files, protocol and deviation records (extensions/PROTOCOLS.md), and scripts that recompute the reported table values. Our individual follow-ups discuss the evidence relevant to each reviewer's questions.

Correction to Table 15. The two system-identification rows reported sufficiency under the relaxed gate of Appendix B.6 rather than the full gate of Section 3. We have aligned these rows with the full gate (M = 32 / 128 / 512): readout-2, 4/8, 6/8, 1/8 → 2/8, 0/8, 1/8; readout-3, 6/8, 0/8, 0/8 → 4/8, 0/8, 0/8. Rates, closed-loop success and the behavioral-learner counts in Table 15 are unchanged. The table caption and supplement README now state the gate used. Appendix B.6 also distinguishes these rows from the readout-3 codebook sweep, whose reported full-gate counts are unchanged. The supplement's result files are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.