- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
BIRD conditions an action-chunk policy on a recurrent representation of observation history. A frozen MAE encodes images, and a GRU integrates visual features, proprioception, and the previous action. A discrete stochastic latent augments the recurrent state, and cross-attention reads recent belief representations. An auxiliary decoder predicts future visual tokens during training and is omitted at deployment. Across six simulated 2D tasks, BIRD reports 74% mean peak success versus 44% for the strongest baseline. The system improves control on tasks that require memory. However, future prediction provides no clear added benefit over current-feature reconstruction, and its two-step prediction objective does not establish how cues needed for later decisions are retained. Evaluation remains limited to a small 2D suite, leaving applicability to more demanding robot manipulation uncertain.
Critical strengths
-
BIRD achieves strong results on tasks that require memory. It reaches 73% success on transient-cue Push-Box, 100% on Counting, and 87% on Drawer, versus 51%, 31%, and 47% for the strongest baseline on each task (Figure 4).
-
Auxiliary visual supervision substantially improves the policy. Removing this loss lowers mean success from 80% to 58% on the partially observable pair and from 65% to 49% on the fully observable pair (Table 1). This supports training recurrent features with both visual and action supervision.
-
Attention over past beliefs improves memory-dependent control. On the partially observable ablation pair, the readout raises mean success from 61% to 80% compared with conditioning only on the current recurrent state (Table 1). This supports retaining access to earlier beliefs alongside the current state.
Critical weaknesses
-
The added benefit of future prediction is unclear. In Table 1, reconstructing current visual features achieves comparable or better performance. At the tested horizon, the results support auxiliary visual supervision, but do not establish what predicting the future adds.
-
The predictive objective does not directly address the stated problem of retaining information for later decisions. It predicts visual features only two steps ahead, while the action head predicts 16-step chunks. A cue needed later, such as which drawer to choose, may be irrelevant to the next two observations. With the intervening expert actions also provided, the predictor can succeed without retaining that cue. Accurate short-term prediction therefore does not establish that the objective encourages the long-term memory used to motivate it.
-
The evaluation leaves applicability to more realistic manipulation uncertain. All six tasks use simulated 2D manipulation with images and 2D positional actions. Transient cues, counting, and hidden drawer contents provide meaningful controlled memory tests. However, there is no 3D or real-robot evaluation to show whether the learned memory remains useful under richer visual and physical complexity, including changing viewpoints, occlusion, and object interactions.
Questions for authors
-
Given the comparable performance of current-feature reconstruction, what benefit motivates retaining future prediction in the final method?
-
How does the two-step prediction objective, given the intervening expert actions, encourage retention of cues needed only later? What evidence shows that it teaches this form of memory?
-
Can you provide evidence that the memory benefit extends to more realistic 3D or real-robot manipulation? Please explain the visual, physical, and memory challenges covered, so the evidence can be related to the intended robotics setting.
4 comments
How does the two-step prediction objective, given the intervening expert actions, encourage retention of cues needed only later? What evidence shows that it teaches this form of memory?
The motivation is shown in Eq. (15). The MIKASA-Robo results show improved information retention, as demonstrated by the probe (0.99 vs 0.36 on ShellGame-Touch, Appendix C) and MIKASA drops (Table 4).
Can you provide evidence that the memory benefit extends to more realistic 3D or real-robot manipulation? Please explain the visual, physical, and memory challenges covered, so the evidence can be related to the intended robotics setting.
We show in the MIKASA-Robo environments that our method is able to solve 3D POMDPs covering visual, physical, and memory challenges including wrist camera views, occlusion, and 3D control. For real-robot evaluations, we cover real images, contact dynamics, counting, and cube-under-cup tasks. Generally we see that in imitation learning 2D environments such as pushT are useful in designing performant visual imitation learning algorithms.
References
[1] Cherepanov, Egor, et al. "Memory, benchmark & robots: A benchmark for solving complex tasks with reinforcement learning." arXiv preprint arXiv:2502.10550 (2025). [2] Gangwani, Tanmay, et al. "Learning belief representations for imitation learning in pomdps." uncertainty in artificial intelligence. PMLR, 2020.
Dear Reviewer,
Thank you for your efforts in reviewing this submission. As the discussion period is approaching its end, I would like to kindly remind you to read the authors' rebuttal and engage in the discussion. Your input is important for reaching a well-informed final assessment.
Thank you for your time and cooperation.
Dear reviewer, as the discussion period comes to an end, please let us know if there are any further questions or if anything in our rebuttal is unclear.
We thank the reviewer for the time and feedback and answer the questions below:
The future observation prediction is motivated by maximizing mutual information with the POMDP state as in equation 15 and as shown in [2]; we also ablate this on the MIKASA-Robo [1] environments and find that it helps retain past information.
No forward prediction on MIKASA-Robo (success, mean ± std over 3 seeds; Table 4)
Does the memory keep the cue? We show the cue (e.g. which cup hides the ball), then hide it, and check whether it can still be read out of the model's recurrent state with a simple linear classifier. The arm is kept still, so the cue cannot be inferred from the image or the arm. Accuracy shortly after the cue is hidden (chance is 0.33; Appendix C):
The two losses serve different purposes, the auxiliary objective is only there as a proxy of maximizing the mutual information between the latent state and the true state of the POMDP, i.e., the loss helps us maintain needed information from past observation in the latent state of the POMDP. On the other hand the action chunking loss is only a BC loss concerned with future actions. We show in the ablations on MIKASA-Robo that this improves task memory. We have added evaluation on the MIKASA-Robo benchmark and on real world experiments.
In addition to our 2D results, we show our performance on the 3D MIKASA-Robo environment and on real world experiments in Fig. 4, Table 1 and Fig. 5.
MIKASA-Robo (success, mean over 3 seeds, 300 episodes each; Fig. 4)
Real robot, UR5 (success rate; Table 1)
Beyond the motivation in Eq. (15), the benefit shows on MIKASA-Robo. Removing forward prediction drops success from 0.97 to 0.42 on ShellGame-Touch, from 0.97 to 0.70 on RememberColor3 and from 0.91 to 0.83 on ShellGame-Pick (Table 4, Appendix D). A linear probe with the arm held still, so that neither the image nor the arm carries the cue, shows the cue fading much faster from the recurrent state without forward prediction, e.g. 0.99 against 0.36 on ShellGame-Touch three steps after the cue is hidden (Appendix C).