Remember Smarter: Visual History Compressor and Hyperbolic Experience Space for Robotic Memory

Zero-shot transfer in complex long-horizon manipulation depends on reusing successful experiences acquired while executing simpler subtasks. This setting requires a policy to retain visual context throughout the current trajectory and retrieve relevant experience from other trajectories. However, many VLA models use only a bounded set of recent visual frames and lack an explicit mechanism for reusing successful prior trajectories. Mamba scales linearly with sequence length and is therefore well suited to compressing visual histories, while hyperbolic space can represent hierarchical relations among related experiences. Based on these properties, we propose Remember Smarter (RS), a module that adds a visual history compressor and a hyperbolic experience memory to existing VLA models. The visual branch uses a bidirectional spatial Mamba and a causal temporal Mamba to summarize patch sequences from multiple camera views. Residual cross-attention adds this summary to the VLM states used for action generation while preserving the original sequence of visual tokens. The experience branch encodes final-layer VLM states from successful simple-subtask trajectories into a Poincaré VAE latent space and organizes the resulting codes with a HypHC-inspired binary topology. Retrieved experiences are converted into prompt tokens for action generation. When integrated with ${\pi }_0$, RS improves the overall success rate on LIBERO-Plus by 16.9 percentage points over the base policy. RS also improves real-robot performance, including zero-shot transfer to task compositions that require maintaining object-name grounding, coping with visual occlusion, or recognizing state changes that are visually subtle or unobservable, such as button presses.