Separating Semantic Applicability from Execution Competence for Transferable Rule-Guided Hierarchical Reinforcement Learning

Rules provide interpretable guidance for hierarchical reinforcement learning, but transferring them requires distinguishing whether a rule applies from whether the corresponding skill can be executed successfully. These two quantities can change independently when task descriptions or low-level policies change. We propose a framework that separates semantic applicability from execution competence in rule-guided skill selection. A semantic evaluator assesses whether observed evidence supports, contradicts, or leaves a rule’s preconditions unresolved, while a separate competence estimator learns from actual skill outcomes. A high-level selector integrates these signals with task-value estimates, retaining deterministic checks for explicit constraints and a fallback for insufficient evidence. This separation allows execution feedback to update competence estimates without treating every failed action as evidence against a rule’s applicability. We evaluate the framework under independently controlled changes in semantic descriptions and skill competence, using matched interaction budgets and shared low-level training improvements. Experiments show higher task returns and more efficient adaptation than HRL-ID and a recent strong reinforcement learning baseline. Ablations attribute these gains to the explicit separation of applicability and competence rather than to a stronger semantic model or additional training data. These findings support separating semantic and execution uncertainty as a practical principle for transferring rule-guided hierarchical policies.