Imagining Before Acting: Neuro-Inspired Counterfactual Critiques for Robot Learning

Before taking action, humans and animals constantly simulate hypothetical actions: What if we did it differently instead? Modern learning systems for robot action execution, however, mostly test the effects of actions only post hoc, by trial and error, which is costly, slow, and sometimes unsafe. We suggest a neuro-inspired architecture that provides a pre-action counterfactual critique mechanism to the robot policy, based on computational models of interaction between the prefrontal cortex and hippocampus during mental simulation. A learned world model generates short imagined rollouts for the proposed action as well as sampled counterfactual alternatives, which are organized into a structured replay buffer by a hippocampal-inspired module, and then evaluated by a prefrontal-inspired arbiter, which signals to accept/revise the proposed action through a calibrated signal before execution. On a simulated six-task manipulation suite, ICC-Net outperforms strong model-based baselines by 12-20 percentage points in terms of the mean task success rate, more than 30% fewer occurrences of unsafe actions, and attaining a fixed performance level with approximately 40% fewer interactions with the environment. All the results of these experiments in this paper are generated from a numerical simulation of the proposed architecture and baselines, not from actual robot testing or GPU-scale training; for this reason, this paper reports these results as a feasibility study and provides a detailed description of what is required for empirical validation.