Information-theoretic empowerment enhances healing following damage in self-organizing neural cellular automata.

A core problem in modern artificial intelligence research is building systems that are robust to the unexpected injuries and assaults that inevitably come from being embedded in a complex and unpredictable environment. Prior research has shown that distributed healing capabilities can be trained into ensembles of self-organizing neural cellular automata (NCAs), but the underlying mechanisms of how the system learns to heal remain unclear. Here we show that this innate healing capacity can be augmented and tuned by the inclusion of information-theoretic auxiliary loss functions: additional metrics that guide the formation of emergent structure but are independent from the actual task being performed. We explored two primary auxiliaries: the time-delayed mutual information between the state of the system before damage and after healing, and the empowerment. We report that including both auxiliaries improves the capacity of NCAs to heal following random damage above and beyond the performance achieved using the typical training regime. Furthermore, we find that this effect is most pronounced when the NCA attempts to fit complex, high-entropy targets, as well as improving the ability of individual cells to represent their position in space. This work uses efficiently and fully-differentiable estimators, and validates the potential of information-theoretic auxiliary functions as active channels with engineering potential. Finally, these results point towards a future high part-count machines and/or computers that can sustain and recover from unexpected internal damage or external surprise.