Multi-Level Semantic Skills for Transfer: Exploration, Control, and Adaptation across Discrete and Continuous Domains

Hierarchical methods in reinforcement learning (RL) offer a natural substrate for transfer: a skill can keep the same semantic role even when the state representation, action space, or surrounding task changes. We exploit this principle by adapting a recently introduced multi-level RL framework, where learned policies become higher-level actions and can be reused through task-specific implementations. We develop three progressively broader forms of transfer. First, semantically meaningful skills learned in one task are reused to guide exploration and accelerate learning in related tasks. Second, hierarchical knowledge is carried from discrete state and action spaces to continuous states and continuous control by replacing lower-level implementations while preserving higher-level semantic structure. Third, learned hierarchical relationships are adapted to apparently different domains by separating reusable semantic structure from target-specific choices, including recursive decompositions. Experiments across navigation, control, and structured computation demonstrate the resulting gains in learning and adaptation when semantic structure is shared. Because these skills admit natural-language descriptions, they also provide an interpretable interface for communication, coordination, and cooperation in agentic systems.