What Your Policy Cannot See: Robot Data Curation via Observability Estimation

Robot imitation learning performance depends critically on the quality of demonstration data. Even successful demonstrations may omit action-relevant visual information, an aspect of data quality that remains less explored in robot data curation. We introduce FOCUS, an information-theoretic framework for quantifying visual observability in offline robot demonstrations, defined as the extent to which visual observations contain the action-relevant information needed to determine an appropriate action. Our key idea is to test whether an auxiliary observation provides information about the demonstration action beyond what is contained in the visual observation. We instantiate this idea using conditional mutual information between the auxiliary observation and actions given visual observations. We use the resulting observability scores to rank demonstrations, filter those with low visual observability, and train imitation learning policies on the retained data. Empirically, our scores generally rank demonstrations in agreement with human observability judgments across two simulated and four real-robot manipulation tasks. The best-performing FOCUS configurations improve success rates by 5–45 percentage points over training on the full dataset.