- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper frames lifelong structural adaptation as an active-span adequacy problem: whether the currently exposed portion of a frozen pretrained skill space is what limits performance, as distinct from task novelty or low return. For this, EGSG uses persistent skill projection residuals as a signal of where the span may be insufficient, and expands the span only when held-out execution confirms the proposed direction helps. EGSG outperforms fixed-capacity, alternative growth, and lifelong RL baselines on Meta-World with few structural updates.
Critical strengths
-
The paper addresses a meaningful question: diagnosing when the representation, rather than optimization or exploration, is the bottleneck in lifelong RL. The approach is also structurally efficient, with no growth in trainable parameters, few structural updates and low validation overhead.
-
The added directions improve matched success from 0.23 to 0.50 on held-out requests before any further policy learning, showing that the gain comes from the added direction and not only from continued training.
Critical weaknesses
-
EGSG depends heavily on the source tasks, since it can only use behaviors the frozen source decoder can already produce. The split is chosen by the authors, and incoming tasks are largely covered by the source (Table 7: 95.4% of incoming expert variance at K=8; Fig. 1: 3 of 5 incoming tasks already solved at K=4). It is unclear how EGSG performs when incoming tasks are less covered by the source, which limits the applicability of this approach.
-
The paper describes the old span as preserved after growth, but the same request can still produce a different executed skill once a new direction is added. Since G_inv and the skill decoder are frozen, forgetting might mainly occurs in the high-level policy, which is shared across tasks and trained with 50% rehearsal. Rehearsal is shared across all EGSG-family controls, but none of the experiments isolates whether the structure itself prevents forgetting. The paper should compare EGSG without rehearsal, and a no-projection baseline (K=16) trained with the same rehearsal data.
-
Proposing where to expand the span and then verifying it before acceptance is essentially about sampling (or search) efficiency for verification, but this efficiency gain is not measured. According to the paper, validation costs only ~0.3% of the task budget per candidate, so a brute-force best-of-N baseline that validates many PCA or random directions and commits the best one also can be a good option. The paper should demonstrate its efficiency advantage over this.
Additional feedback
- State the frozen and trained components in the main text, since G_inv being frozen is not very clear. Cite GPM, SPiRL, and HIRO. Clarify the Fig. 1 vs. App. F.2 inconsistency for button-press.
Questions for authors
Please see the weaknesses above.