- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes the Lexical Policy Network, a language-conditional architecture for multi-task reinforcement learning. A pretrained text encoder maps task descriptions to contextual embeddings, and a gated MLP generates softmax weights over multiple subpolicies, performing operations that are a weighted sum of their outputs. Subpolicies and gates are trained end-to-end using SAC. Separate settings freeze pretrained subpolicies and train gates on composite tasks. Experimental results show that LEXPOL achieves significant improvements over baselines on MT10 and MT50.
Critical strengths
-
At both reported training budgets, LEXPOL achieves higher average success rates than the multi-task baselines evaluated on MetaWorld MT10 and MT50. Combining this with CARE further improves the performance of 2M steps. The standard error results for ten seeds provide useful evidence of their empirical competitiveness.
-
Ablation changes the number of expert policies and compares six pretrained language encoders. Other experiments examined freezing versus joint training policies, integration with CARE, and MiniGrid combined courses. This coverage helps evaluate the architecture for different configurations and training setups.
Critical weaknesses
-
The method is insufficiently specified, and its stated architecture is inconsistent with the interpretation of the main composition experiment. Section 3.1 and Figure 1 define the gate as a function only of the language context, while Section 2 states that this context is fixed for an entire episode. Thus, for a fixed task and fixed model parameters, the weights do not depend on the current state. Nevertheless, in my understanding, Figure 2 tests how the gating network selects between two already pre-trained policies when retraining the gating network on a new composite task, which does not follow from the stated architecture that the dominant gate weight should be the same throughout the state space for the same task.
-
The experiments do not isolate the claimed contributions of language semantics, dense policy composition, or generalization, and the literature analysis is inadequate. Language only conditions the gate, so task IDs or learned task embeddings could provide the same task-disambiguation signal for the fixed training tasks. The composition experiments also involve additional training on the new tasks, rather than demonstrating generalization to unseen instructions or compositions without further learning. Missing top-1 routing and inference-cost comparisons, together with limited discussion of recent related work, further weaken the justification for the proposed design.
-
Algorithm 1 is inadequate as a training specification: it omits the training losses and does not explain how the blended action distribution, log-probabilities, or gradients are computed under SAC, and it uses the notation and inconsistently. Figure 2's caption refers to top/bottom images even though the panels are left/right, and Table 2 misspells MOORE as MOOORE. The manuscript also lacks an AI-use disclosure section.
Questions for authors
-
Does the implemented gate take only , or does it also receive state, history, or task-progress information? Is the task description fixed throughout a composite-task episode? Please give the exact quantity visualized in Figure 2 and explain how its spatially varying colors arise under the architecture in Section 3.1. How is progress through "red goal, then blue goal" represented?
-
How does LEXPOL perform when the language input is replaced by one-hot task IDs, learned/random task embeddings, or shuffled descriptions under otherwise identical settings? Do semantic similarities between task-description embeddings correspond to similarities in the learned gate weights or reusable skills?
-
What success rate and inference cost are obtained when the gate selects its top-1 expert before any actor evaluation, compared with the current dense weighted-action method? Could you compare this approach with executing only the top-1 policy to assess the performance-cost trade-off?
-
What motivated the selected policy counts in Table 8 and Table 9? It seems that the increase of policy count does not contribute to a increase performance on MT10, how does a k = 10 model, especially with task-based routing, compare with ten separately trained SAC agents under explicitly matched interaction budgets?
-
Since LEXPOL uses language instructions to condition policy selection, have you evaluated its generalization to OOD tasks or unseen task compositions without additional training?
-
Can you compare full-policy and internal-module routing with matched parameter counts and budgets?