SynMorph: Learning to Build Task-Adaptive Spatiotemporal Models from Synaptic Primitives

In real-world spatiotemporal learning applications, a single observation system often involves diverse learning tasks with heterogeneous objectives and data patterns. Researchers still need to manually design task-specific models according to data characteristics, even when tasks share the same spatiotemporal observations but require different input-output mappings, resulting in considerable engineering efforts. Existing architectures addressing task diversity mainly rely on fixed-structure adaptation, such as transfer learning, or architecture search strategies that lack structural inheritance and composition. Inspired by synaptic diversity in biological brains, we propose Synaptic Morphogenesis (SynMorph), a task-adaptive framework that learns to construct spatiotemporal models from heterogeneous synaptic primitives according to task-specific data patterns. Specifically, SynMorph first generates task encoding by jointly characterizing task types and spatiotemporal patterns. It then constructs a neuron-level meta-synaptic pool inspired by diverse synaptic mechanisms and adaptively selects synaptic primitives to compose task-specific networks through a shared Deep Q-Network (DQN) controller. The decision process is guided by network states and an overall reward function balancing model complexity and prediction performance. Finally, the composed architecture is trained end-to-end for the target task. Extensive experiments on diverse datasets involving regression and classification tasks demonstrate the effectiveness and interpretability of SynMorph, achieving up to 7.0% improvement over the strongest existing approaches on environment datasets. SynMorph provides a neuro-inspired perspective for task-adaptive spatiotemporal learning by uncovering the potential dependencies among data patterns, task requirements, and model architectures, which can be viewed as `the model of model'.